Apple recently unveiled the Vision Pro, a groundbreaking device likely to open new possibilities in VR/AR applications. Stuffed with innovations, it is turning it into what is possibly the most advanced personal electronics device eve. Apple has filed 5,000 patents to make the Vision Pro happen.
So what is the Vision Pro?
The Vision Pro is a new type of computer, worn on the face and is based on spatial computing. It blends real experience with digital content, turning computing into a pervasive environment.
It builds on Apple’s expertise in hardware, software and services that it has built over the past four to five decades. The Apple silicon that powers the device represents years of development. It offers a fully immersive 3D interface controlled seamlessly by eyes, hands and voice, reflecting the UI development across the last four to five decades.
Sleek and slim, it features a controller-free design that relies on a system of cameras and sensors to track finger and hand movements.
Worn like ski goggles, you see a 3D interface of your current environment with no limits on the size of the perceived viewing area. There is no need for a desk, or lap, to place the device. You can sit in a café booth or small space like an airplane seat and watch a Hollywood blockbuster.
The computer senses what you are interested in and who you are interacting with by tracking your eye movements and then looks at your hands to determine what you want to do. The eye tracking feature tracks your eyes and when you look at an app, it will automatically open it.
Apple silicon
Powering this super sophisticated device is the dual chip system comprising the Apple M2 processor and the new R1 chip. The M2 runs the headset’s operating system and applications. The R1 chip is specifically created to process data from the headset’s 12 cameras, five sensors and six microphones. It only takes a speedy 12 milliseconds, faster than the blink of an eye, to process the data between the time an action is detected by the sensors and the time it is displayed on the headset’s screen.
This seamlessly integration of data by the R1 chip provides the smooth and immersive experience, removing the motion sickness challenge users often feel when they use other VR headsets.
UHD resolution and eye tracking
The computer’s output is projected directly into a user’s eyes with two tiny high-resolution displays. In fact, the user sees two ultra high definition screens provided by a dual micro-oled display with a total of 23 million pixels. This is akin to watching a 4K TV in each eye.
For people who use spectacles, special inserts from Zeiss will enable them to use the Vision Pro.
Watching movies and playing games can be immersive experiences. The headset will automatically adjust the screen to provide the ultimate cinematic environment by dimming the surrounding light and using spatial audio.
To open files, the eye movements are tracked. When the eyes glance at an app and the eye tracking sensor understands what you want to do and opens the app. Bring your thumb and a finger together to select items and flick to scroll. To type, you can speak or use the virtual keyboard in front of you.
AI embedded in the Vision Pro let users scan their faces to create a digital persona for FaceTime and Zoom calls. Using machine learning the digital persona mimics face and hand movements to offer a more realistic and immersive experience.
To ensure that users do not look like zombies walking around with a headset that has a blacked out slab across it, a feature called EyeSight displays your eyes when it senses that someone has entered your field of vision.
There is also an upgraded spatial audio called audio ray tracing which matches the sound output to your surroundings by analysing the features and materials of your space.
VisionOS – the new operating system for the Visual Pro – has a security system called Optic ID, which uses iris scans for authentication. The data is encrypted and works with Secure Enclave, the subsystem integrated into Apple system-on-chips. However, any data collected when users are intereacting with apps and websites will stay private, said Apple.
A leapfrog in XR development
Several years in the making, the Vision Pro has delivered a genuine leapfrog in capability and execution of mixed reality.
An advanced wearable system, the Vision Pro has useful features to deliver a more immersive and fundamentally new computing experience and which can change the way we work and play.
The fact that most existing Mac and iPad apps will work on this headset means that when the Vision Pro goes on sale next year, there will be thousands of programs in games, productivity, education and more. That is the reason Apple chose to unveil the headset at its annual developer conference in California. It has also introduced new tools that will help developers create the killer apps for the new computer.
Spatial computing is not a concept invented by Apple. It emerged in the early 2000s as an integration of traditional computing with tools and processes used to capture, process and interact with 3D data.
What Apple has done is to bring spatial computing to life with the Vision Pro.
There have been many reviews of the Vision Pro like here, here and here, all of which are positive but highlight the lack of killer apps, short battery life and high price as the key drawbacks.
The US$3,499 price tag is likely to attract the discriminating customers like the tech savvy singletons who appreciate gadgets and want to be first adopters.
For the industry, the Vision Pro will likely influence the entire tech market’s virtual computing/spatial computing vision.
