For the next ten years, touch is all that matters, says Google. It's reversing its decision to add the Pointer Events standard to Chrome, meaning web developers lose what could be their best chance to get a single API for handling touch, mouse and pen input.
In January the Blink team said Pointer Events -- a technology originally built by Microsoft and now a W3C standard -- was a priority for 2014. Now it's off the agenda. Dropping support for it isn't an attack on Microsoft; Google is, as always, just being expedient and pragmatic. But its choice to stick with Apple's model for touch input development is going to cause problems for web developers, who will now be faced with writing far more code to deal with mouse, pen and other inputs. And it highlights the drawbacks in Google's approach to performance.
The origins and promise of Pointer Events
Just last month Microsoft announced that so many mobile web pages are written so they only work properly on iPhones, that the Windows Phone version of IE 11 will pretend to be Safari. That includes supporting the elderly Touch Events model for scrolling, clicking, and zooming with your fingers -- a model that Apple first came up with in 2007 when the iPhone came out.
But there are problems with Touch Events: It doesn't always work well when you have a trackpad as well as a touch screen, and it has a checkered history of standardization. When the W3C starting building a standard based on it, Apple not only declined to join in, but in 2011 it also claimed it had patents covering those touch principles. Although that's since been resolved, building a standard on top of Touch Events (which Apple wasn't involved with) didn't deal with the biggest problem -- that it only covers touch, not any of the other ways you might interact with a web page.
In 2012, Microsoft teamed up with Mozilla to take the Pointer Events technology it built for IE10 to the W3C. This is single API for handling mouse, touch and pen input (and maybe more) so developers would only have to write code once, instead of for all the different ways users might interact with a page.
Pointer Events was initially popular: It might hold the record for fastest-approved API. In January, the Google Chrome team who created Blink -- Google's fork of the the widely used WebKit rendering engine -- said Pointer Events were a priority for 2014. The Blink team even got as far as checking in code for a key CSS property that Pointer Events relies on, which has also been in the Firefox Nightly builds since version 28.
So why did they change their mind?
One reason is the same lazy behavior by web developers that meant Microsoft had to add Touch Events support to Windows Phone.
"Pointer events would likely never supplant touch events on the web (especially without support from Safari)," Google's Rick Byers explained. Plus he said Pointer Events wouldn't let you use scrolling to do other things on the page at the same time (like pulling down from the top of a web page to refresh it).
But Google's most important objection will probably be performance: The time it takes to find out if you're touching or clicking or writing on a web page.
"The hit testing model required by pointer events imposes a non-trivial performance penalty (hit test on every movement event) that neither Android, iOS or touch events has. We're not willing to add any feature that increases the web's performance disadvantage relative to native mobile platforms," Byers said.
But IE 11 has pretty zippy performance and it has Pointer Events. How did Microsoft manage that?
Hardware acceleration, Jacob Rossi of the Internet Explorer Developer Platform Team told CITEworld.
"Our experience has shown panning and zooming to be the most performance critical of touch interactions and Pointer Events offers deliberate design considerations that enable browsers to hardware accelerate those interactions. This is a key reason why IE is able to deliver industry-leading touch response time, latency, and frame rate."
Hardware acceleration is one reason why Microsoft didn't create a version of IE 9 for Windows XP, which doesn't have the Direct2D technology that IE uses to hardware accelerate everything from rendering fonts to displaying images -- including speeding up touch. Because Google supports Windows XP -- not to mention Vista and Mac OS X -- it can't use the same efficient hardware acceleration to speed everything up.
What about other performance issues apart from moving around and zooming?
"Pointer Events also includes additional features, notably element transition events and automatic hit-testing, which enable developers to more easily build complex UI," said Rossi. "Our testing indicates that these features typically attribute just a fraction of a millisecond per event. Developers tell us they value these advanced capabilities of Pointer Events so we will continue to work with the industry and community on moving the web forward."
The reason Google started putting Pointer Events support into Blink was that developers were asking for a better way to handle mouse and touch together. That's still the case, Rossi told us, so Pointer Events is still important.
"We believe a fundamental design principle of the web is that it has the ability to be accessible by anyone. Developers continue to tell us it's hard to write to both Touch and Mouse Events together. Further, we think the web platform should expose hardware to its fullest, allowing developers to innovate and differentiate. In support of this, we'll continue to work on Pointer Events because most developers tell us they prefer it conceptually."
Microsoft, Google, and web standards dance
You might know the apocryphal Chinese curse, "may you live in interesting times." But fewer people are familiar with the other two passive-aggressive wishes that go with it: "may you come to the attention of your superiors" and "may you get what you desire."
Web developers and browser makers who used to wish that Microsoft would pay attention to web standards certainly got what they wished for. In the old days, IE didn't implement anything new -- including new standards. Since the change of attitude with IE 9, IE has implemented standards if they're widely used, stable and don't seem fundamentally flawed to Microsoft. Thanks to the new continuous development policy for IE, Microsoft can support new standards without waiting to ship IE 12. That means IE is doing better at staying current with standards, when they fit those three criteria.
The Web Audio API, for controlling sound from a browser, is still at the working draft stage at the W3C, but it's already in other browsers and the IE team is working on adding it. But Web MIDI, which proposes to implement the MIDI protocol used by electronic devices to communicate with one another, is at an earlier draft stage and isn't implemented in any other browsers. So it's listed at status.modern.ie as "not currently planned."
Microsoft is even making its own proposals at the W3C, some of which have been enthusiastically adopted -- especially the performance-related interfaces that let web developers understand more about how a page loads.
Microsoft's newly polished enthusiasm for standards is an obvious contrast to Apple, which hasn't taken proprietary Safari features that many web developers use to any standards body (or indeed the FaceTime protocol -- despite early promises), hasn't produced a WebRTC implementation (so far the only WebRTC discussion Apple has joined in was the debate over whether V8 or H.264 should be the standard codec) and hasn't contributed much to recent attempts to standardise touch interactions. At the 2014 WWDC Apple finally announced it will support the IndexedDB storage API in Safari; that's a standard that's been in other browsers since Firefox 4, Chrome 11, and IE 10.
Google's approach to standards looks like expedience and pragmatism. When the shadow DOM specification for displaying HTML 5-standard web components (like the video player) was still in development but the Blink team was close to shipping their implementation, Google's Tab Atkins pointed out "whatever API gets shipped will be frozen almost immediately".
That wasn't meant to be an ultimatum or to suggest Blink wouldn't support an agreed standard later, "just the reality that whatever syntax we choose here is likely to freeze quickly as soon as it's shipped. Even if we want to, it will be hard or impossible to change in the future." On the Blink team, "freeze" means "something that's no longer changeable (or at least is probably too difficult to change) due to compat pain."
That doesn't sound too different from Microsoft's tendency to bend over backwards to keep compatibility for enterprise customers -- for decades in some cases -- and the way those customers demand Microsoft let them turn off the feature in IE that blocks out-of-date versions of Active X controls like Java. The customer's always right.
But Google and Microsoft are taking very different approaches to dealing with the compatibility issues with touch on the web.
Touch isn't everything
Even Google thinks we'll need generic pointer events that can handle multiple kinds of input in five or ten years' time. But web developers who look beyond the dominance of the smartphone and tablet need to deal with multiple inputs now. As Xamarin's Nat Friedman is fond of pointing out, mobile means wearables and smart watches as well as phones. And web developers want what they build to work well on devices like Xbox One and the Amazon Fire TV -- and on the millions of notebooks with trackpads and mice in the market, and on new tablets with pens like the Surface Pro 3. They want it to be easy to integrate with Cortana voice control or and Kinect gesture control even gadgets like Thalmic's Myo armband that offer muscle control.
Google's decision means that's more work for web developers, as it forces them to write multiple versions of the same code.
They'll need to support Pointer Events, and whatever Google adds to Touch Events to support different input messages, and maybe Android's Motion Events if they aren't the same as Blink's version. Even though Rossi promises that Microsoft will "continue our interoperability commitment through work in the Touch Events Community Group, including taking into consideration any marked improvements to the Touch Events API that get implemented for iOS and Android devices," that's still more for web developers to keep track of.
They'll be using feature detection (or more likely sniffing for browser user agents) to decide which of those events you get on your device, and using libraries to fill in the gaps, which means worse browser performance than a built-in browser API like Pointer Events. More complicated code doesn't just mean more work for web developers. It means more chances for code to go wrong and give you a bad user experience.
Web developers already have a problem making web sites that handle touch and mouse input together. The reason Touch Events support is only on Windows Phone and not in Windows 8.1 is that the IE team found that if they turned on Touch Events in the desktop browser, your mouse or trackpad would stop working on "around 10% of top sites."
That's because those web developers assumed you would never have touch and a mouse at the same time -- but nearly all new Windows notebooks today have both. Pen and voice and gesture control aren't common yet, but we already have the Surface Pro and Galaxy Note phones and Adobe's Ink pen and the Jot pen for iPad, Cortana, Xbox Kinect, Siri and Myo (not to mention a Honeywell thermostat you can talk to, to turn up the heating).
The PC isn't growing much but it isn't dying off either. The future doesn't only look like touch- - unless you're Google and all you see is Android. It would be nice if the web was ready for that future, and Google isn't helping.