Going into the latest hackathon, I knew there were several options for building a virtual reality (VR) app for Android devices. Google currently has two platforms for VR available: Cardboard and Daydream, not to be confused with the screensaver mode on Android phones. Cardboard is Google’s original foray into the VR headset realm, released June 25, 2014. With it, you could put your phone into a cardboard headset to view 360 degree video and some VR content. Interactivity outside of head tracking was done with a singular button that you could press.
Daydream, released November 10, 2016, has three versions: two headsets that you insert your phone into, like the Cardboard, as well as a standalone headset that does not require a phone. Unlike Cardboard, Daydream has a controller that you can use to provide additional interactions with the virtual environments.
I formed a team with Ron Shafii and Daniel Mesquita for the hackathon. Our objective was to create a carnival setting with a VR Whack-A-Toad game, mimicking the classic Whack-A-Mole game. We planned to use the Daydream View remote as the mallet and have the game just sit in front of the user. Additionally, as a stretch goal, we were going to try making it so you had to pick up and could put down the mallet utilizing the Daydream View’s remote. Armed with this plan and the sample apps from Google, we began our trek toward our destination.
We had three paths ahead of us to choose from, the wild and unknown paths of Unreal or Unity, or the familiar platform, tools, and languages of utilizing the Android SDK. Even though I was familiar with Android, it was new to my other team members. At the time, we had made the decision to go with the Android SDK, written in Kotlin, because we wanted to minimize how many things we would have to learn all at once in the two-day timespan of the hackathon. Little did we know at the time, but this decision was going to cause many more problems than if we would have gone with Unity or Unreal.
Even with what we considered to be a reasonable goal for the hackathon project, our aspirations would not be realized in full because of obstacles that appeared on the path we traversed. The biggest of those obstacles ended up being the choice to use the Android SDK for as our platform for Daydream instead of Unity. We did not realize it upon our first look over the documentation, but some of the Android SDK documentation for Daydream links to the Unity section for the same topic. In some areas, this would make sense, since some of the ideas and concepts didn’t change, but there were also aspects of the documentation that felt like they should have been split up to give insight into the specifics of the Android SDK platform. In addition to the lack of documentation, Daydream does not have an integrated physics engine or OpenGL wrapper, and thus relies upon the platform, or developer, to handle that. Android also does not have a first-party OpenGL wrapper or physics engine, which left us having to work around and remove features from the app to make it feasible. Unity, on the other hand, comes with both of those built in and fully supported.
The other category of hurdles came from either not having ever done graphics and OpenGL development or not having used it for many years and needing to brush off the dust. Even though we had the model placed for a toad, we originally started off in a black environment, unsure of where we were in the virtual space Then we overheard another team say, “We’re inside the toad”—we thought about it for a moment and decided to scale our toad down drastically, and the result indicated that we too had been “inside the toad.” If the other team had not mentioned that, we would have likely spent more time troubleshooting this and think we had done something else seriously wrong in our importing of the model and using the OBJ formatted model. The other OBJ file issue we ran into was trying to texture the toad with the provided mtl file. We found a process to attempt to load it in Blender, get the texture mapping, and then apply that mapping to the model in the app, but in the end we decided to do something simpler and color the toads orange. This was easily accomplished using a small orange file and applying that to the entire toad. Our final issue was that the Samsung Galaxy S8 we were using just kept crashing, whereas the Pixel was very solid throughout most of our development.
In the end, we didn’t quite reach our goal of utilizing a mallet controlled by the Daydream remote, but we came up with something close. We have a box, which acts as the playing area, and three toads that are either in the box or outside the box. To “hit” the toad, you look directly at it and then hit the button on the remote, which will cause it to go into the box and another random (potentially the same) toad to appear. Untested, but a side effect of the lowered scope is that the app should be capable of running using regular Cardboard and thus support a wider variety of Android devices.
This project was a great way to get my feet wet with creating VR experiences for Android. Every member of our team learned a lot. Going back, I would likely choose Unity, as it feels like a richer environment with better documentation and a much larger community behind it compared to Daydream with the Android SDK. Over the course of this two-day hackathon, my other team members were exposed to Android development and tools. The three of us got refreshers in OpenGL programming and matrix operations in relation to those OpenGL operations, as well as some much-needed exposure to developing with a VR platform that none of us had utilized before. I will be taking some time in the future to explore Unity, in addition to more OpenGL on Android. Many of Metal Toad’s clients are interested in how this technology can be used in their businesses, and the next time I work on a project like this, I will be more ready to tackle the challenge in front of myself.