Having tried both the Oculus Rift and HTC Vive, I’d like to compare my experiences with each and discuss the state of consumer VR, speculating on where second generation devices might be headed.
The Rift DK2 was the first headset I used, playing only a few rudimentary demos. One thing stuck out to me: resolution. I’d heard about the screen door effect, but never expected it to be that jarring. Not only was the image very soft (it was a struggle to read text on virtual signs and posters). Not only could I see the individual pixels of the display. Not only could I discern each subpixel from its neighbours, but I could also clearly see the black space between them. It wasn’t a screen door effect so much as it was a “sheet of metal with a grid of pinholes in it” effect. Shortly after my time with the DK2, I got to play with a CV1. I did a fair bit more with the CV1, including playing Eve Valkyrie. Resolution was better, but still quite poor. Overall though, the image was much more pleasant. This may be due a higher pixel-to-black-space ratio. Images are still soft, but good enough to not be distracting until you try and read far away text. Developers will need to be mindful of this: Everyone has impaired vision in VR.
As an experience, the Rift underwhelmed me. I saw it as a more immersive way to consume existing media. VR video, while very cool, is just video that you can look around in. It’s a much more immersive way to watch video, but it’s still video. Games were a similar story. With the seated nature of the Rift and the Xbox One controller, pretty much any game you can play on the Rift you can also play just fine on a TV. All the Rift adds is more immersion. When I think of virtual reality, I envisage a virtual world that I can interact with naturally. Navigating a virtual environment with a thumbstick and interacting with it via buttons on a controller just isn’t natural. Using the Rift feels to me like I’m remote controlling a player in a virtual world, rather than actually being inside it.
The Vive HMD has pretty much exactly the same hardware specifications as the Rift, but unlike the Rift, the Vive actually made me feel like I was inside the virtual worlds it presented. The motion controllers and full-room tracking system are the difference between an incremental improvement on immersion and an entirely new experience. The Vive controllers, while not as intuitive as hand tracking, feel natural enough as a way to interact with the virtual environment to give a real sense of presence. Being able to pick things up and throw them around in a virtual world generates more immersion than any increase in HMD resolution ever could.
The full-room tracking system is equally important to the illusion of presence. It’s incredibly immersive to walk around a virtual environment as you would walk around the real world (until you hit your chaperone barriers, that is). It’s clear that navigating through and interacting with the virtual environment in a familiar way is the key to feeling like you’re “inside” virtual reality. The Vive is something you can get lost in. You lose track of the fact that you’re actually stepping about in an office, getting a nice red ski mask imprint pressed into your face. In the Rift, any movement and interaction happens through your controller, and this constantly reminds you that you’re sitting in a chair, with a mobile phone screen and some lenses strapped to your head.
The Rift is still an immersive experience, and $200US cheaper than the Vive. Motion controllers should be coming soon as Oculus Touch (no word on pricing yet though), and John Carmack of Oculus has said he’s got full-room tracking to work with two Oculus cameras, although there are no plans as yet to offer it to consumers. The Rift is also lighter, making it more comfortable for long play sessions, and feels better made. But right now, if you solely care about having the most immersive VR experience, I would have to recommend the Vive.
Now that both major companies have their products released, we can build a clearer picture of where VR is, and what is still to come. VR is no longer a thing that you try at a tech show or an arcade and think “hey that was kinda cool” then forget about. This is something that a lot of people will want in their homes. After trying the Vive I spent a very long time debating whether or not I should buy one. VR is now polished enough to be commercially viable. It’s not perfect, though. There’s still plenty of iterative improvement to come. I’ve put together a list of the shortcomings of the current devices. These are things that I hope or expect will be improved upon in second generation hardware:
Resolution is the most obvious area for improvement. Suitable 4K OLED panels already exist, the limiting factor here is rendering power. Using better lens compensation techniques can provide a significant reduction in required pixel fill rate, and foveated rendering (only rendering to full resolution at the centre of the eye’s gaze) could absolutely slash requirements. NVidia have been polishing up foveated rendering techniques, however eye tracking is still an issue. Fove think they can track the incredibly fast “saccadic” movements of our eyes quickly enough to be unnoticeable, but it remains to be seen if they can do it cheaply enough for consumer headsets. Higher resolutions also require more bandwidth from the graphics card to the headset, with the very latest consumer display standards only supporting up to 4K at 120Hz.
Refresh Rate for me is a small complaint. 90Hz was fine for all but the fastest head movements. However, anyone who has developed immersive experiences knows that immersion is very fragile. A single latent frame is plenty to remind you that you do in fact have a phone screen and some lenses strapped to your face. A modest bump to 120Hz should be enough to eliminate such moments. Again though, this increases the required rendering power and display bandwidth.
The horizontal field of view of the human eye can be up to 210 degrees. That’s right, you can see behind you a little bit. Both major headsets have horizontal field of view around 100 degrees. The result is the feeling that you’re viewing virtual worlds through a porthole. Adding more peripheral vision isn’t easy, though. It may be one of the biggest challenges in VR hardware. Optics are the main issue here. It’s incredibly hard to increase field of view while keeping headsets small and light, and lens distortion low. StarVR claim to be hitting 210 degrees with their prototype headset, but this is geared towards commercial applications like VRcades, and may not be cost effective in consumer headsets.
I got to use the Rift with a Leap Motion hand tracker. It was incredibly cool to hold out my hands in front of me and see them in VR, every knuckle individually tracked. However the Leap still has problems with tracking rate, and doesn’t handle occlusion (one hand obscuring the other from the sensor’s view) very well either. Seamless hand tracking would make VR interaction feel even more natural than it already does with motion controllers, but so far the only reliable solutions involve special gloves. Much like increasing field of view, getting hand tracking right seems like one of the biggest challenges in VR, but also one of the most rewarding, considering the immersion it could add.
Body tracking is a much simpler exercise. Microsoft have this down to a T with the Kinect 2. All that remains is for Kinect support to be integrated into VR APIs, or Kinect-like body tracking sensors being integrated into the next generation of Oculus sensors and Vive Lighthouses.
Audio is not perfect in VR either. In most situations, motion-responsive audio synthesis, combined with neuroplasticity in the parts of our brain responsible for sound localisation, makes generic HRTFs servicable. However I still experienced a number of front-back reversals, where sounds that originated from in front of me in the virtual world seemed to me to sound as though they were behind me, and vice-versa. I’ve written a separate post about how researchers (including myself) are trying to avoid such issues, you can read it here.
The last thing I’d like to see in second generation headsets is a bit of a pipe dream. A lot of high end televisions hitting the market recently have been advertising themselves as having high dynamic range. I think HDR could be really useful for VR. Imagine fumbling around in a horror game, the corridor being just slightly too dark for you to make out what lies at the far end. Imagine being dazed momentarily by the light of day as you burst, trembling, out of the terrible Castle Brennenburg (the setting of Amnesia: The Dark Descent), until your eyes adjust to the flood of light. HDR video is a great spectacle, and HDR audio, which has been used in cinemas for decades, could add to VR immersion also. Players could listen intently for soft footsteps in a stealth game, or be startled by the powerful crack of a sniper shot in the distance. HDR audio is a lot harder to do safely though. With an HDR display built into a headset, the developer can control exactly how bright the display can be, ensuring no damage to your vision. Headphones are largely analog, however, each having their own sensitivity. A signal that is whisper quiet on one pair of headphones may be powerful enough to cause damage to another pair, or even damage to the hearing of the listener.