The Power and Promise of Passthrough
There was a time, not so long ago, that augmented reality and virtual reality were considered very different things. But recently the lines between VR and AR have been blurring. No longer do you have to decide if you want an experience that transports you to another world or one that takes place right in your living room. Thanks to the introduction of passthrough, player’s can switch in and out of reality with the touch of a button.
What is Passthrough?
First made widely available with the Meta Quest 2, passthrough technology allows creators to take a headset that was initially designed exclusively for virtual reality and use it to explore a player’s surroundings in augmented reality. This is accomplished through the use of front-facing cameras — a hardware feature that’s part of the toolset needed to allow for in-headset tracking. Some manufacturers wisely understood that these cameras could pull double-duty, also allowing a sort of limited passthrough to make sure players wouldn’t bump into their surroundings. (On the Quest, the passthrough camera would activate whenever you stepped outside of your guardian boundaries to reveal the world around you.)
In 2021 Meta released its Passthrough API, giving creators the ability to integrate the Quest 2 passthrough camera into their apps and games to experiment with new features. (Resolution Games was actually the first third-party developer to release a passthrough experience on the Quest Store with an update to Blaston that let you turn your home environment into a cyberpunk dueling arena, complete with neon trim!)
This was a first step, but since it wasn’t explicitly designed for new modes of play, the Quest 2 camera captured the world in a low-resolution grayscale: certainly not ideal for pushing the boundaries of what mixed reality can do. But it served as an important proof of concept — and one that’s bearing significant fruit with the latest generation of headsets.
Learn more: Challenges and takeaways of implementing Meta’s Passthrough API in Blaston
The Future: Presented in Living Color!
Starting with the Meta Quest Pro and Pico 4, full-color, high-definition passthrough is quickly becoming the norm for virtual reality headsets, allowing people to realize the full potential of augmented reality gaming — and to experience the wide range of new and exciting genres that will emerge because of this.
We have been experimenting in the area of full-color passthrough this year at Resolution Games, and we’ve cooked up some really exciting projects behind the scenes. Not all of these will see the light of day (as is the case for any studio experimenting with new ideas and technologies), but you may have caught your first glimpse of one of these earlier this year with Fish Under Our Feet, a demo created to showcase an early example of what’s possible on Quest Pro. We also just showed off two additional projects as part of Meta Connect 2022, including the reveal of passthrough mode for our co-op tabletop experience Demeo (coming as a free update on October 25) that will allow players to switch between the virtual environment and the real world, snapping the game board to a physical surface at home — complete with virtual dice hitting a real tabletop.
The other Resolution Games experience teased at Meta Connect 2022 is something truly unique that could only exist thanks to passthrough and wireless headset technology: an experience that sees players moving quickly through their real environment to compete in team-based play, blending virtual objects and real world environments to form the space for play. This early prototype is called Spatial Ops and we’ll be sharing more details about this project in the future.
Better Building and Testing
Passthrough on retail headsets isn’t a boon for the industry just because it puts mixed reality games in reach of players — it also creates a more accessible development environment that allows dev teams to dream, build, test and iterate at incredible speeds. This hasn’t always been the case, and it opens up the door for so many more to invent and deliver great ideas for mixed and augmented reality devices.
The implications for standalone augmented reality, too, are something to consider. While the market for AR glasses has yet to see its major consumer moment, it almost seems to be an inevitability at this point. And when that moment does arrive, it will succeed or fail the same way that all new hardware does: on the strength of the software that proves its use case.
Thanks to passthrough, XR developers around the world are able to get a head start on creating that software — so don’t be surprised if you see more than a few “system-seller” apps for the next generation of AR devices available on day one.
