Head Mounted Displays without positional tracking just aren’t that great. The lack of positional tracking was one of the main pain points of the original Oculus Rift development kit. Omitting positional tracking from an HMD not only dramatically reduces immersion, but can result in increased risk of motion sickness. But positional tracking without sensor drift presently requires external systems such as Valve’s Lighthouses and Oculus’ Constellation sensing IR cameras. This means that self-contained headsets such as Samsung’s GearVR can only offer directional tracking: They know which way the headset is pointing, but not its position in space. Could HMD mounted depth sensing cameras solve this problem with their inside-out approach to positional tracking?

Current HMD positional tracking systems use external sensors or beacons. The location of the headset is measured relative to these external structures. Valve’s Lighthouses send out regular IR laser sweeps along both the vertical and horizontal axes. IR sensors on the Vive HMD and controllers measure the order in which the laser sweeps across each sensor, and the time taken for the laser to sweep from one side of the device to the other. This information can be used to calculate both the orientation and position of the devices relative to each Lighthouse base station. The Oculus Rift uses an IR camera to track a “Constellation” of IR LEDs on the headset. Both of these systems require hardware separate from the headset, mounted in a fixed position.

Inside-out tracking is an emerging technology which uses sensors on the HMD itself to track objects in the real world, and thus the headset’s location relative to the world, without any external hardware. Typically the inbuilt sensors are depth sensing cameras such as Intel RealSense, a stereoscopic camera (two sensors placed about eye distance apart from each other) capable of sensing object depth up to 2m away. Intel demoed an all-in-one HMD codenamed “Project Alloy” which utilised a RealSense camera for inside-out tracking at the 2016 Intel Developer Forum. Stereo Labs, creators of the ZED stereoscopic camera, claim their camera can track the depth of objects up to 20m away, and show their camera (which is small enough to be mounted to an HMD) tracking itself in real time on their website.

Inside-out tracking promises to bring positionally tracked VR to all-in-one HMDs, allowing both a more convenient VR experience, and a larger play area. AIO HMDs with accurate positional tracking will be free from restrictions such as the length of the cable to the PC and the size of the area which external sensors can track within. We have already seen stereoscopic cameras in mobile phones such as the HTC Evo 3D and the LG Optimus 3D. At the time, they were dismissed as gimmicks: 3D photos and videos are only worthwhile if you have a stereoscopic display to view them on, and 3D TVs never really gained a widespread following. As inside-out tracking matures, we may well see a resurgence of stereo cameras in smartphones, bringing positional tracking to mobile phone based VR and AR systems like GearVR.