VR is all about immersion, and companies like Oculus know that clunky controllers have always stood in the way of that.

At Oculus Connect 6, Facebook unveiled one of the more exciting developments in VR technology we’ve seen lately: Hand tracking. No gloves are required, and neither is a new headset. It’s an update to the Oculus Quest, coming as soon as early 2020.

Sound too good to be true? It’s not.

Unlocking the VR world with your hands

Although Facebook cautioned that hand tracking at this stage is still pre-production beta software, I was able to test how the feature functions on two different demos at Oculus Connect 6 this year. The first was a demo called Elixir, developed by Facebook Reality Labs in partnership with Magnopus VR.

In it, I found myself thrust into the world of witchcraft. Playing the role of a witch’s apprentice as I learn how to cast spells, the tactile use of my hands made the experience feels more immersive as I navigated around the studio.

Poking a dragon’s belly, lighting candles on fire, and dipping my hands into cauldrons and potions felt more natural than dealing with buttons, grips, or triggers on a controller.

Because the Oculus Quest was built with four outward facing cameras, Facebook claims that hand tracking leverages the monochrome cameras to track your hands.

When your hands enter the cameras’ field of view, they are displayed on-screen using 3D rendering. Software maps out your hands and approximates where the joints and knuckles on your fingers are located, which helps to mimic finger motion when you move your finger.

When I poked the dragon’s belly in Elixir, the software accurately identified my hands and its movements, which progressed into a fist with an outward pointing index finger just when the belly was being poked. Tracking was good, but since the cameras are used, whenever my hands were out of the camera’s frame of view off to my sides or too far stretched outwards, they disappeared.

In Elixir, your hands can transform into octopus-like tentacles and Wolverine-like claws, so different overlays other than human hands are possible, adding even more layers of possibilities.

On the enterprise side, Farmers Insurance ran a demo where you’ll star as the insurance claims agent walking through a water-damaged kitchen identifying and tagging leaks.

When you spot water damage — like on the wooden floors or inside a cabinet underneath the sink — you can point to the area with your hand. A virtual menu appears, and you can tag the assessed area as something that can be easily cleaned or as an item that will require repair.

Both simulations performed great. Compared with other demos from the conference, such as Medal of Honor: Above and Beyond, performing actions with my hands definitely felt more natural. In Medal of Honor, the game’s authentic reproduction of Europe during World War II made the game feel very immersive — and real — and once I tried out the hand controls, using controllers felt obtuse.

For example, loading a gun, discharging used ammunition, and firing weapons would feel more natural than using the A button to discharge the ammunition and fumbling with the the grip and trigger on the controllers to reload the guns and rifles.

The limitation, however, is that hand tracking — and more specifically, finger tracking — seems limited to simple in-app interactions, like pointing, grabbing, touching, and holding. More complex interactions will likely require buttons, and a controller would be needed in that case.

Facebook developers working on hand tracking informed me that the company is exploring building in more interactions, and that they are working with game developers to see if specific actions can be controlled without a controller.

Even with hand tracking, you won’t be able to fully get rid of your controllers. On a very basic level, even getting started on Elixir required them. But once I was in, the hands-free VR experience made be never want to go back.

Editors’ Recommendations