Microsoft thinks disorientation is the key to touch in VR

To manipulate virtual objects in VR, you have to use an Oculus Touch or other "virtual wand" controller like some kind of hands-off, digital tong. To get an actual "haptic" or touch experience, you need real objects, but it's computationally challenging because the system needs to precisely track each one. Researchers from Microsoft, USC and the University of Waterloo have discovered that by manipulating how you see your body and the world in VR, they can make you think that a single physical object has magically multiplied.

To demonstrate the technique (below), the team set up a hand-tracking Kinect and had subjects don an Oculus Rift. In the first exercise, the user was shown three real cubes on a table, but after they put on the headset, two were removed. The program then asked subjects to grab one of three VR cubes, even though there was only one physical object. By using "body warping," the subjects saw their arm reaching for multiple imaginary objects, and were rewarded haptically by grabbing the one physical cube.

In the second instance, the subjects were asked to look away, and while they did so, the world subtly shifted so that the VR objects corresponded to the real cube's position. A third technique combined the two, warping both the subject's VR body and the virtual world to match the physical object. It works both horizontally and vertically, letting you not only move blocks around but stack them too -- a useful technique for a game like Minecraft.

The team reports that "participants reported higher presence levels from the hybrid approach in comparison to each warp used individually, and overall preferred haptic re-targeting to the virtual wand method." The research is just that right now, and Microsoft hasn't said yet whether it will incorporate the technique into its Hololens or any other AR or VR products. It will reveal a paper on the subject at the CHI 2016 (Human-Computer Interaction) event in San Jose starting May 7th.