Fraunhofer-GesellschaftYour favorite sports star comes into your living room to give you private lessons? Mixed Reality makes it possible: The chosen person is inserted into the real environment through mixed-reality glasses. Dr. Oliver Schreer, Wieland Morgenstern, and Dr. Cornelius Hellge from Fraunhofer HHI explain the tricks that make virtual humans look lifelike.
Why don’t you let us in on the secret of making avatars look realistic?
Dr. Oliver Schreer: Until now, avatars have been created using computer animations—but these often seem artificial. So we’ve taken a different approach. We place an actor or an actress in the middle of a rotunda and film him or her with 32 cameras, calculate the 3D information, and use this to create a 3D model. The result is a dynamic mesh sequence, known as a volumetric video.
Wieland Morgenstern: In the next step, we animate this volumetric video with a high level of detail—we use model-based animation to change the actor or actress so that, say, his or her gaze can follow the user. This makes the contact more personal.
Are special end devices needed?
Dr. Cornelius Hellge: If these photorealistic volumetric videos are rendered on the end device, the bit rates and the demands on the graphics cards are too high. Together with Deutsche Telekom, we are therefore moving the rendering to the cloud— so only a normal 2D video stream is transmitted to the end device. This 2D video stream runs on all standard devices and on all browsers.