Streaming high-quality VR360 degree video with resolutions up to 10K x 4K consumes large amounts of bandwidth. Moreover, the encoded video requires decoding capabilities beyond 4K video at the receiving side, e.g. on VR glasses with a mobile phone. Fully standardized tile-based streaming solves these issues by spatially segmenting the 360 degree video into tiles. Each tile is encoded with HEVC at the original high-definition and an additional low-definition resolution. The new MPEG-OMAF standard allows to package the HEVC tile streams in a way that the receiver, e.g. VR glasses or a TV screen, can request the high-definition tiles for the user's viewport and low-definition tiles for the areas out of sight. The tiles are aggregated at the end device into a single HEVC compliant video stream and decoded with a legacy hardware video decoder on the end device.

At IBC 2018, Fraunhofer HHI showcases for the first time in Europe a demonstrator for consistent VR 360 degree live video streaming with a resolution bigger than 4K. This includes high-resolution 360 degree video capturing and live rendering by the Fraunhofer HHI Omnicam-360 with a resolution of 10K x 4K, HEVC tile-based live encoding with the Fraunhofer HHI HEVC encoder, packaging according to the MPEG-OMAF viewport-dependent media profile and high-quality playback on VR glasses and TV screens.

Volumetric Video Production and Workflow

Fraunhofer Heinrich Hertz Institute and VoluCap GmbH present a novel and innovative capture studio as well as a processing workflow for high quality volumetric video productions targeting future VR/AR media productions.

In June 2018, the first volumetric video studio on European mainland was opened at the Filmpark Potsdam-Ba-belsberg, Germany. Real persons are captured with multiple high-resolution cameras in a professional studio environment. A powerful processing suite automatically generates naturally moving dynamic 3D models, which can be integrated in AR/VR applications. The system supports diffuse or synchronized scenic lighting from any direction, automatic keying, and flexible multi-camera arrangement.

Versatile Video Coding (VVC), compression beyond HEVC

Compressed video data are growing at a faster rate than ever before. Already today, video data make up by far the highest percentage of bits on the Internet and in mobile data traffic. This demonstrates the need for even more efficient compression, which goes beyond the current High Efficiency Video Coding Standard (HEVC). In order to master this demanding challenge, the ITU-T Video Coding Expert Group (VCEG) and the ISO/IEC Moving Pictures Expert Group (MPEG) have already started working together in the Joint Video Experts Team (JVET). In April 2018, Fraunhofer HHI and other leading technology companies successfully proposed cutting edge coding technology with compression capability beyond HEVC. This marked the starting point of the Versatile Video Coding (VVC) standardization activity. The VVC standard is expected to provide 50% bit rate reduction over HEVC when finalized by 2020.

At IBC 2018, Fraunhofer HHI showcases the most recent version of the VVC reference codec (VTM-2.0). This early version already demonstrates significant coding efficiency improvements over HEVC for content ranging from standard High Definition (HD) to High Dynamic Range Ultra-HD content.