The rumour that Apple is developing an AR headset (or AR glasses) isn't new; claims that the company had its sights set on such a device have been circulating for a few years now. But things got interesting in 2017 when the announcement of ARKit at WWDC made it clear that Apple is deeply interested in this area. And no wonder: with Google and Microsoft actively developing AR devices, it seems like a party Apple can't afford to miss.
It also hasn't escaped our notice that Apple's latest smartphones are well-equipped for AR, thanks to bionic chipsets, which enables AR processing.
Apple certainly looks like it is gearing up to be a big player in the AR field, but what exactly does it have up its sleeve? Is Apple making a pair of AR glasses? And what would such a pair of glasses look like? In this feature, we look at all the latest Apple AR rumours, patents and possible release date details, including the news that iOS 13 will include support for AR glasses.
What is augmented reality?
First, a quick refresher on terms, as many get AR and VR confused. VR headsets are mounted on the head in a similar way to Ski goggles, and completely block your view of the outside world. The VR headset tracks your head movement, and the 3D image displayed inside the headset moves accordingly. This makes it appear as if you're wholly inside a 3D 'virtual' world.
Augmented reality, on the other hand, is where the glasses are see-through and you can still see the world around you, but an image is displayed in front of your eyes.
AR hit the headlines in recent years, first thanks to Google Glass (which displays 2D images) and more recently with a headset developed by Microsoft called HoloLens that embeds 3D images in the world around you.
Microsoft's HoloLens glasses are already available for developers - although anyone can buy one. However, it costs the astronomical price of $3,000, or $5,000 for the Commercial Suite which includes enterprise features. There's a newer HoloLens 2 coming later in 2019 with upgraded features, but it's still business-focused.
As for Google Glass, development of the consumer-focused product was halted back in 2015, but Google revealed a new-and-improved Google Glass Enterprise Edition 2 in 2019.
Apple is known to be a trailblazer, but it's still part of the Silicon Valley tech industry (albeit a more secretive one). If Google and Microsoft are all working on AR products, you can bet your bottom dollar that Apple also has an AR headset prototype in its labs.
Plus, Apple CEO Tim Cook has gone on record discussing the big possibilities he sees for AR, as we discuss below.
However, it's an industry that is in its infancy. With that in mind, when can we expect to see Apple's AR headset or glasses?
Apple AR headset: Release date
Assuming that Apple is working on an AR headset or AR glasses (we look at the evidence that Apple is working on an AR headset below) when is such a device likely to launch?
We don't think Apple will release its AR glasses in 2019, but a 2020 release date is looking increasingly likely. The original report of a 2020 release date came way back in November 2017 via Bloomberg, but it has been backed up by reputable sources in the months and years since.
An April 2018 cnet report suggested that 2020 would be the launch date, according to 'sources'.
Then, in March 2019, Apple guru Ming-Chi Kuo reported that Apple's first AR product is set to go into mass production as soon as Q4 2019, with a deadline of Q2 2020. He also detailed features of the headset, but we'll go into that in more detail below.
However long we have to wait, Loup Ventures analyst Gene Munster believes that augmented reality will be the next big thing for Apple.
He also expects that Apple will release Apple Glasses in late 2021, a year later than Kuo's predicted release date. It's something he says we aren't yet ready for, "but eventually we will be".
Apple's AR device is here already!
We don't actually have to wait for Apple to launch an AR headset. The company already sells an AR device - the iPhone.
For most people, their first taste of AR came when they started playing the popular Pokemon Go app on their iPhone in the summer of 2016.
Now, thanks to the TrueDepth camera on the iPhone X and XS, a bionic chipset, and iOS 12 - which Apple describes as "the world's largest augmented reality platform," the iPhone is taking augmented reality to the next level.
The iPhone X's AR credentials weren't a surprise, a Bloomberg report back in March 2017 claimed that AR would be introduced on the iPhone first, with glasses coming later down the line.
That report claimed that there are "hundreds of engineers devoted to the cause" including a number of people from Apple's iPhone camera team working on AR-related features for the iPhone.
The Bloomberg report was supported by a claim from a Business Insider source (in November 2016) that Apple has teams from several acquired startups working on iPhone AR, and this reflects Apple's short-term desire to put AR into the hands of consumers.
By adding AR capabilities to the iPhone Apple allows consumers to point the phone at a real-world object, like a statue or a car, and be offered contextual information overlaid onto the real world.
Integrating AR into a smartphone that millions of people already have introduces the concept of AR, and may make them more likely to buy Apple-made glasses once they go on sale.
Speaking of which...
Claims that an Apple AR headset is in the works at Apple stretch back some years and a number of reports have appeared presenting evidence that the project is well underway at Apple. In fact, the project even has a code-name - T288.
There have been rumours that Apple has been making AR/VR related hires some time. In November 2016 a Business Insider source claimed that Apple has hired an expert in head-mounted displays (HMDs).
In November 2016, a Bloomberg report claimed Apple had ordered a small number of "near-eye displays" for internal testing displays.
Citing people with knowledge of the project, in a March 2017 report, the FT claimed that Apple had started to assemble a team of augmented reality experts a year previously. Apparently, more resources have been devoted to the AR project than Apple's rumoured driverless car.
Then in April 2017, more evidence of some sort of AR project came from leaked Apple employee injury reports obtained by Gizmodo. While they're mostly about mundane workplace injuries like cafeteria burns, there were two that stand out:
One involved someone testing an unknown prototype device who "experienced discomfort in her eye and said she was able to see the laser flash at several points during the study." She was later referred to an optometrist.
Another involved an employee who "reported eye pain after working with a new prototype, though it may be associated with use".
While these could reflect either an AR or VR device, an internal source told Gizmodo that the injuries were likely linked to an AR prototype Apple is working on, speculating that it could be "something like glasses with an overhead display".
A newer Bloomberg report, from November 2017, claimed there is a team of several hundred engineers working on AR-related projects, according to its sources.
The team, led by Mike Rockwell, who previously ran engineering at Dolby Lab, is working on several hardware and software projects under the umbrella code name of 'T288', according to Bloomberg.
The team also worked on ARKit. The tool that lets software developers build AR apps for iPhone and iPad. We discuss ARKit in more detail below.
Back in May 2015 Apple acquired a German company. Metaio was a relatively well-known company with regards to augmented reality, powering many of the popular AR applications being used today - including Ikea's virtual catalogue, and Ferarri's AR showroom app.
Since being bought out by Apple, Metaio has gone quiet - heading to its website will offer you a brief overview of the company, a contact email address and not much else. Mysterious.
Apple has confirmed the acquisition of Flyby Media, a company well-known for developing technology that allows smartphones to 'see' the world around them. Notably, the company has worked with Google in the past to develop the image-recognition abilities found in Google's Project Tango.
Though the website is now down (typical of an Apple acquisition), Flyby Media described itself as "dedicated to building new technology that can elevate, rather than replace, our real-world experiences." and boasted a team with knowledge in the areas of large-scale SLAM, indoor navigation, sensor fusion, image recognition as well as 3D tracking.
Apple at CES
Another Bloomberg report, in January 2018, presented more evidence that Apple has plans in the AR space. The publication claims that, despite not exhibiting at the show, Apple was present at CES 2018, with execs meeting suppliers to discuss AR glasses, according to people familiar with the meetings.
There is yet more evidence that Apple is working on something in the AR space - this time coming from Apple itself.
Apple CEO Tim Cook said in an August 2016 interview with The Washington Post that Apple is "doing a lot of things" in the augmented reality space. He didn't stop there though, as the Apple boss went on to claim that he thinks of it as a "core technology" for the company.
"I think AR is extremely interesting and sort of a core technology," Cook stated during the interview. "So yes, it's something we're doing a lot of things on behind that curtain we talked about."
Following on from these comments, Cook took part in a rare interview session at Sen. Orrin Hatch's Utah Tech Tour in Salt Lake City in October 2016 where he again discussed augmented reality, although this time he gave us a better idea on what Apple may be working on.
"AR I think is going to become really big," said Cook. "VR, I think, is not gonna be that big, compared to AR … How long will it take? AR gonna take a little while, because there's some really hard technology challenges there. But it will happen. It will happen in a big way. And we will wonder, when it does [happen], how we lived without it. Kind of how we wonder how we lived without our [smartphones] today."
It's interesting that Apple is focusing more on augmented reality than virtual reality, although it does make sense - especially with the booming popularity of apps that feature the technology like Pokemon Go, and the introduction of ARKit which makes it much easier for developers to offer AR capabilities in their apps.
Cook sees AR as becoming something that a huge part of the population will use on a daily basis, going on to state that he thinks "that a significant part of the population, of developed companies, and eventually all countries, will have AR experiences every day".
Speaking to The Independent during a trip to the UK in February 2017, Cook said that he prefers AR over VR because it "allows individuals to be present in the world but hopefully allows an improvement on what's happening presently."
Considering that VR seems to provide the more immersive and enjoyable experience of the two platforms at the moment, why does Cook believe that AR is the future? People don't want to be cut off from the world, apparently.
"Most people don't want to lock themselves out from the world for a long period of time and today you can't do that because you get sick from it," he said, discussing virtual reality. "With AR you can, not be engrossed in something, but have it be a part of your world, of your conversation. That has resonance."
Cook even went as far as to liken it to the smartphone, and how that has changed the world. "I regard it as a big idea like the smartphone" he claimed. "The smartphone is for everyone, we don't have to think the iPhone is about a certain demographic, or country or vertical market: it's for everyone. I think AR is that big, it's huge."
Tim Cook also addressed AR and ARKit during the Q3 2017 earnings call. When asked a question about how people will be using ARKit, Cook replied "I could not be more excited about AR and what we're seeing with ARKit in the early going." He continued to explain the variety of ways he has already seen AR used, and that he thinks "AR is big and profound and this is one of those huge things that we'll look back at and marvel on the start of it."
Yet more evidence that Apple is interested in AR: Apple announced ARKit as part of iOS 11 at WWDC 2017 and was updated as part of iOS 12. ARKit allows developers to offer high-end augmented reality capabilities in their apps without having to develop the AR system, and improves existing AR solutions too - for example, Pokemon GO running ARKit looks much smoother than it did previously.
Per Apple's ARKit page on its website, it claims that ARKit is a "cutting-edge platform for developing augmented reality apps for iPhone and iPad" and it offers "powerful capabilities for positional tracking and scene understanding".
It makes sense for Apple to offer AR as part of the iPhone and iPad experience, as it's a device already in the hands of millions of consumers. It also encourages developers to create AR apps that could in future be ported to an AR headset, if that's the route that Apple decides to go down.
Plus, it makes iOS the biggest AR platform in the world. Not too bad for emerging technology!
Interestingly, iOS 13 could introduce support for stereo AR glasses. That's according to a 9to5Mac report, which claims that along with AR glasses support, iOS 13 will offer advanced AR features like the ability to detect human poses and the ability to work with a touch-pad controller.
We probably won't see a consumer product on stage at WWDC 2019, but it looks like Apple is preparing developers for something big...
Apple's AR headset: Price
We expect that when it launches Apple's AR headset will have a high price, as is Apple's custom. But we hope that the price will be a little more down-to-earth than that offered by the competition currently.
As we said above, Microsoft's HoloLens glasses cost $3,000 (approx £2,193), or $5,000 (approx £3,655) for the Commercial Suite which includes enterprise features.
As for Google Glass, the Google Glass Enterprise Edition costs $1,828 (approx £1,336).
Apple's AR headset: Design
An iPhone Hacks report in June 2017 claimed that Apple's smart glasses will have a unique design, according to their source.
Their insider provided details on the design - featuring a bone conduction earpiece, a microphone, an accelerometer and a capacitive frame for navigation and interaction, the glasses sound impressive.
Some creative people have already come up with potential designs for Apple's glasses. For example, this YouTube video (which enthusiastically claimed that the specs would launch back in 2018) has some neat designs.
Apple's AR headset: Specs
We don't know much about the technologies being used in Apple's AR specs (get it) but there are a few rumours circulating that could give an indication about what to expect.
According to a cnet report in April 2018, Apple's AR headset will offer an 8K display for each eye.
The 8K displays will make the VR and AR images look more lifelike and will help avoid the nauseous feeling that can be the result of using AR/VR headsets.
An older report from June 2017 suggested that the smart glass being used in Apple's AR headset was being manufactured using a Kopin NED Acetate frame.
There were also claims that Apple is looking to sell its AR glasses with prescription lenses or even polarised lenses in partnership with Zeiss.
A cnet report suggests that the AR glasses won't require a computer or smartphone to run.
Cnet's source claimed that the headset will require a dedicated box powered by a custom Apple processor to which it will connect via a "high-speed, short-range wireless technology". The box will use the wireless technology 60GHz WiGig, according to cnet's source. The 60GHz WiGig signal cannot typically penetrate walls but it can propagate off reflections from walls, ceilings, floors and objects using beam-forming. This may not matter if the headset is used in just one location.
This does mean that users won't be required to install various base stations around a room to enable to headset to determine their location (as is the case with HTC Vive). Everything will be included in the one box and headset.
According to cnet, the box will feature a 5-nanometer processor. Apple is designing the chip in-house and the chip will be similar to the chips Apple is said to be developing for its Macs, with a switch from Intel planned for 2020.
However, that conflicts with the more recent claims of Ming-Chi Kuo, a man with a proven record of accurate predictions. Kuo states that the first-gen AR glasses will be heavily dependent on the iPhone, with the AR glasses essentially only providing the display. That leaves the iPhone to handle the computing, rendering, internet connectivity and location services from the users pocket - hinting that, like with the Apple Watch, Android users won't be able to get in on Apple's AR fun.
While we assume that all this computing will be done wirelessly, it's not explicitly mentioned in Kuo's report.
If the iPhone handles all the heavy lifting, Apple can create a headset that is light and stylish, but there are also concerns about the impact to the battery life of your iPhone if it has to power the AR experience.
Apple's AR headset: Features
Bloomberg's sources claim that the glasses will wirelessly connect to an iPhone and will display not only information but images and other data directly in front of the wearer's field of view.
KGI analyst Ming-Chi Kuo believes that Apple could integrate AR to redefine key product lines. For example, an AR-powered interface could change the way that users interact with small-screen devices like the Apple Watch, eliminating issues with clunky controls.
Apple's AR headset: Software
A new operating system, dubbed rOS (r for reality), is being built for the AR headset. It is based on iOS, according to Bloomberg.
rOS will power the AR headset. Apparently, the 'rOS' software group is headed by Geoff Stahl, formerly a software manager for games and graphics at Apple.
As for how you will get apps, Apple has discussed pairing the headset with its own version of the App Store, where users would be able to download content, just as they do with the iPhone, Watch, Apple TV and Mac, according to Bloomberg.
Apple AR Patents
In January 2017, Apple was granted a pair of patents that could give us a glimpse at the company's AR aspirations. Patent no. 9,560,273 details the hardware framework of an AR device that can understand its environment thanks to enhanced computer vision capabilities, while patent no. 9,558,581 details the method of overlaying virtual information on a physical environment.
Both patents were filed back in 2015 by German AR specialist Metaio shortly before Apple acquired the firm, and transferred to Apple in November 2016.
Patent 273 describes a "wearable information system having at least one camera" but goes further and discusses the possibilities of having more cameras, a screen, a UI and even internal components dedication to computer vision. The patent details a head-mounted display as the primary platform for AR, but it also suggests that a smartphone (aka the iPhone) could serve as a decent stand-in.
However, on the whole, the patents deal more with object recognition than it does the visualisation of the AR data. The issue of object recognition is a barrier in the AR industry on the whole, as existing solutions require a large amount of power and thus, aren't ideal for real-world use.
In Apple's case, the 'invention' maintains a low-power scanning mode for the majority of its operation, with high-power modes triggered in relatively short bursts.
Then, in May 2017, a patent surfaced depicting the head-mounted display of Apple's upcoming augmented reality headset. The patent details a device and techniques for projecting a source image in a HMD comprised of left and right displays, suggesting that they may look more like standard glasses than Google's failed Glass.
In the filing, Apple notes that some HMDs can be used to view a transparent image imposed upon a real-world view. Sound familiar to augmented reality? We think so too.
Another Apple patent filing, discovered in July 2017, describes how a pair of AR-enabled glasses could interact with a smartphone. The user is said to be able to interact with the objects that appear on the glasses using their smartphone. Of course, the phone would need to be equipped with the necessary hardware and software to be compatible with the glasses, and we're unsure if the tech is already featured on the iPhone 7 or not.
Another concept in the patent describes how the user can interact with objects that appear on the AR glasses by pointing at them - a similar mechanic is used by Microsoft in its HoloLens glasses, so it's not a far-fetched idea. The patent claims that it'll be possible via an array of cameras attached to the glasses, as the cameras will map and track the user's finger as it moves towards the on-screen object.
However, a patent published in Q3 2018 suggests that the AR headset could be controlled via a touch-enabled device like the iPhone. As well as depicting a person wearing a headset and interacting with a touchscreen device, the patent specifically details how "the user interacts with the GUI in part through stylus and/or finger contacts and gestures on the touch-sensitive surface."
With more advanced patents appearing, it suggests that Apple is taking augmented reality very seriously, and we might be seeing Apple's AR-enabled device on the market sooner than originally thought.
KGI analyst Ming-Chi Kuo told investors that he believes that Apple's track record of delivering innovative and high-quality user experiences will help the company move into - and succeed - in the AR market.