The technology, developed by Google Research and Google Daydream Labs, uses a combination of machine learning, 3D computer vision and advanced rendering techniques to "remove" clunky VR headsets in order to allow onlookers to see a person's identity, focus and face in mixed reality.
While the technology does nothing for the real-world headset user, it dramatically improves the experience inside a VR or mixed reality world.
Virtual reality, and really even augmented reality, is a very personal experience in the sense that only the one wearing it can really appreciate what's happening.
By making the headset 'vanish, ' as it were, Google and the tech giant's partners hope to give VR demonstrations a bit more dignity and immerse watchers more fully in the experience, giving them a "complete picture" of the VR world, according to a Google blog post. With some age-old "green screen" Hollywood magic, some 3D modeling, and eye tracking, Google has devised a way to give viewers a better sense of the virtual world and, in a way, makes it actually easier to sell the idea of VR to them. Their face is then calibrated in under a minute and synchronized with a map database of textures.
The way the technology works is by creating a 3D dynamic model of the users face to accurately portray any facial expressions or emotions, and then layer it onto the front of the digital headset.
Michael Flynn unlikely to face charges for lying to Federal Bureau of Investigation , sources say
The president said that while Flynn was "just doing his job", he was "not happy" that the adviser had misled the vice president. Flynn was interviewed by the FBI around the same time , according to a USA official who was briefed on the investigation.
Finally, the composition is rendered and checked to make sure the model face is consistent with the content in the camera stream.
"If you're watching someone else use VR, it's hard to tell what's going on and what they're seeing".
"Headset removal is poised to enhance communication and social interaction in VR itself with diverse applications like VR video conference meetings, multiplayer VR gaming, and exploration with friends and family", the firm says.
Game publishers like Owlchemy Labs and Radial Games have done some of the most extensive work on mixed reality setups and have shared their findings with the greater VR game dev community.
The blank, robotic VR masks seen in mixed reality videos could soon receive a facial fix that's pretty cool but also kind of creepy, just like any good advancement in technology should be.