Sometime in the coming years you should be able to look down a street and say “zoom in on that” and, as if a pair of binoculars appeared in front of you in mid air, you’ll be able to see much further down that street than you could with the unaided eye.
We’re a long ways off from that and it won’t happen with the first generation Apple Vision Pro. In fact, with Saudi Arabia still pouring $500 million into companies like Magic Leap as Meta continues to invest in optical AR glasses, the question of whether VR-style headsets with passthrough on them are going to be used outdoors in any significant way is not a forgone conclusion.
My guess, though, is that higher resolution cameras producing higher resolution passthrough on a VR headset is the clearer path toward this kind of functionality than widening the field of view of optical AR eyewear. Meanwhile, researchers at Stanford University are currently investigating the very subject of what it means to see the world through degraded eyes for extended periods of time in passthrough on a VR-style headset.
Meta Research Suggests High Brightness HDR Key To VR’s Future
Meta research suggests VR’s most transformative gains in telepresence and visual realism may come from advances in display brightness and dynamic range. Speaking on Meta CTO Andrew Bosworth’s podcast, the company’s head of display systems research talked about the enormous gap in brightness between the 100 nits
The researchers noted in a recent paper largely looking at Quest 3 that, while passthrough will be highly useful, it “will also likely cause visual after effects, lapses in judgments of distance, induce simulator sickness, and interfere with social connection. We recommend caution and restraint for companies lobbying for daily use of these headsets, and urge scholars to rigorously and longitudinally study this phenomenon.”
UploadVR’s early tests with Vision Pro in our hands outside of Apple’s controlled demo rooms suggests that the resolution of passthrough fed from Vision Pro’s camera and sensing systems, then processed on its chips, and reconstructed inside the headset for your sight, allows a person with unaided 20/20 vision to discern details at 6 feet away through passthrough with only roughly the same abilities as someone with 20/30 vision. We noted slightly poorer results with passthrough on a Quest 3, but it’s not a straightforward comparison with issues like field of view, jitter, depth cues, and high dynamic range all playing a part in your perception of passthrough visuals. You’ll have to check in with our in-depth independent review of Vision Pro later this week coming from my colleague David Heaney for a run-down of the trade-offs Apple and Meta have made when it comes to reconstructing human vision in these headsets.
Facebook Researching VR/AR ‘Hear-Through’ Technology For ‘Perceptual Superpowers’
Facebook researchers are investigating AR glasses featuring “hear-through” technology powered by specialized in-ear monitors for “enhanced hearing.” The technology “would be able to recognize different types of events happening around you: people having conversations, the air conditioning noise, dishes and silverware clanking. Then using contextualized AI, your AR glasses would
What You See Is What You Get
For now, keep in mind that the videos and photos you see recorded by influencers and YouTubers using a Meta Quest 3 or Apple Vision Pro do not convey these limitations. What they do convey, however, is the promise of all-day augmentation and the ways it’ll help, like having a to-do list hanging on your fridge, a vacuum cleaner that’s fun to use, or a big TV to watch while doing the dishes.
But let’s consider some of the big ways VR-style headsets could augment vision in passthrough mode over the long term.
Imagine being able to see through walls, or to understand the strength of radio waves emanating from you Wi-Fi router. You’ll be able to see which piece of furniture or wall is blocking the signal from getting a reliable connection to that one device in your home constantly getting disconnected.
Imagine Uber or DoorDash for drone delivery. You’ll be able to watch the drone’s path all the way from its launch to your agreed drop off spot. You won’t be watching a dot move on a flat map, instead, you’ll watch the drone actually soar through the sky to your location following its arrival path through the walls of your own home.
Imagine never losing your keys or wallet ever again. Like a hint in a video game, the headset will simply highlight its exact location in the house for you through the walls.
I can only begin to imagine the possibilities and Apple is currently feeding developers kits at $3,500 each to build what’s in their imaginations. Those are the actual early adopters of Apple Vision Pro — developers who are building the next generation of apps for all-day augmentation or super vision – and not the influencers and YouTubers seeded hardware by Apple or the bots on X harvesting viral videos.
Not many people realize Vision Pro’s visual augmentation in passthrough mode is actually a step backward from human vision in many respects. Perhaps that’s part of the reason so many people greet this technology as dystopian? After all, who wants to volunteer to have degraded eyes on the real world? Even for floating windows or the promise of cool virtual worlds to visit, that’s a tall order.
The flip side to that dystopian argument, however, is that it helps to realize not everyone can see at 20/30 or better. In fact, the vast majority of people, throughout the course of their life, will use positively ancient optical glasses technology to augment their vision. This is usually done to restore their sight to what others have at 20/20. In my opinion, that’s why we need to pay keen attention to a different group of people than the influencers showing Vision Pro being used on a skateboard or in a pool.
I suggest we listen to the people who use assistive devices regularly already, and we can hear about what Vision Pro does for them. They got their headsets starting on Friday as well, and their opinions will be fascinating to read. I also suspect their views will go a long way toward conveying the true promise of this technology over the long term.
If you’re working on an app for Vision Pro or doing something interesting with it, please reach out to tips@uploadvr.com or use our Contact Us page to share your story.
