Challenges and takeaways of implementing Meta’s Passthrough API in Blaston

Luis “Luigi” is a Game Programmer on Blaston. He has a background working as a game developer generalist since 2014, developing applications mostly targeted for VR platforms. In October 2021 he moved from Brazil to Sweden and joined Resolution Games. Here he shares his experience of implementing the brand new Passthrough API feature in Blaston for Meta Quest 2.

During my second week working here, I was assigned the task of implementing Meta’s Passthrough API in Blaston. I wasn't familiar with the code base but my team members helped me understand some of the systems and implement passthrough on the game. I was also helped a lot by a colleague who was working with passthrough on a different project.

Challenging problems with Passthrough for Blaston 

We encountered some big challenges on the technical side. The first one is to implement a completely new feature to an existing game that wasn't designed for that. At the same time we knew that passthrough was unconsciously almost a perfect fit for Blaston.

At the time passthrough wasn't ready for production yet - it was still an experimental feature. This meant that the Passthrough API (application programming interface) from Oculus/Meta might be unstable or contain bugs. And in fact, we did encounter quite a few of them, almost to the point of deciding to not implement it into the game.

The main motivation driving us forward to implement passthrough is that there were no games with this feature and Blaston would be one of the first to feature it. With that in mind we knew that the only way to bypass those issues was to create some workaround on the code side. That's exactly what we ended up doing while also isolating the “ugly code” as best as we could.

Altering the approach to a problem 

Initially we tried to make passthrough work directly on the Blaston project. For some reason it wasn't working even after a few days of trying - all we could see was a black background screen instead of the passthrough compositor.

Then I decided to take one step back and tried to implement passthrough on a new project from Unity (Blaston Engine). This allowed me to remove all the variables that came with the Blaston particularities and I was able to better pinpoint the problems we were experiencing. This approach is rather slow but I think it was a necessary step that we had to take to implement passthrough in Blaston.

After that we slowly merged the changes into the Blaston project, the results started to show up and eventually we managed to see a passthrough view in the game. It was a big relief, and made us believe that we actually might ship this feature on the next update.

Takeaways from this project

Passthrough is still new and only a handful of games feature it. Finding any sort of useful information about implementing passthrough was a challenge. 

However, I have to say, big cheers to the VR game development community. I saw a few game developers on the forums also trying to implement passthrough on their games, some of them had issues that I already knew a solution to. They also fed me with valuable passthrough intel and we ended up helping each other.

I believe that common information should always be public. We sometimes fail to acknowledge the value of sharing our knowledge across the internet. But if you publish or just answer a forum question, chances are that you are going to be helping quite a lot of people. It is certainly something I'll be doing more often.

Are you interested in working on challenging projects in VR and AR? We’re currently hiring XR game programmers - check out our open positions!