'Immersiv' teknologiplatform baner vej for nye event-formater
Fraunhofer-forskere er i gang med at udvikle den såkaldte DOMEconnect platform, der gør det muligt at producere high-resolution 'immersive' indhold og livestreame det til f.eks. planetarier (in english).
Standing in front of a huge painting is always something special. But what if the painting suddenly moves, with sound effects that bring it to life? Immersive exhibitions invite audiences to experience art and culture in a multimedia format; interactive museums, concerts, games and other events empower active engagement with the content featured.
Researchers at Fraunhofer HHI in Berlin are currently developing a platform called DOMEconnect, which is creating whole new event formats. The system uses various interactive technologies and real-time rendering engines, making it possible to live-stream high-resolution 360° video to interconnected planetariums.
DOMEconnect can also be used to allow planetariums to participate in events remotely as separate venues. For example, a concert taking place in one location can be broadcast to one or more planetariums elsewhere, allowing for a more impressive and direct event experience than would be possible with a conventional live stream. Novel interfaces and the implementation of synchronization mechanisms give artists space and projection surfaces for their works in forms of presentation that were previously impossible.
But plans call for the connection between the various planetariums and event venues to go beyond mere audio and video playback. Instead, avatars and mobile devices used as controllers will allow the audience to participate in the action live, interact with each other, complete tasks and experience content as a group. To test this function, the researchers are currently developing prototypical multi-user games.
- We’re in uncharted territory with these interactive options in immersive spaces. There’s no playbook for this yet. And that means we’re also opening up new test scenarios for companies and other organizations. For example, we can create training situations in virtual environments for emergency responders, says Christian Weissig, manager of the Capture & Display Systems group at Fraunhofer HHI.
Toolkit: OmniCam-360, RTSE and CineBox
To realize the project, which is receiving funding from the Fraunhofer Science, Art and Design network, Fraunhofer HHI is building on three technologies developed at the institute: OmniCam-360, RTSE and CineBox.
The researchers are expanding on these tools and adapting them to address the needs of planetariums. OmniCam-360 is a scalable mirror-based multi-camera system used to record video in 360° panoramic format. The software-based Real-Time-Stitching Engine (RTSE) processes ultra-high-definition (UHD) panoramic video with a resolution of 10K x 5K in parallel and in real time.
CineBox decodes video streams for multi-projection systems and features intuitive controls to operate projectors, manage various input signals and incorporate diverse audio systems. This toolbox allows the researchers to transmit content to a wide range of installations with different projection surfaces as well as various audio playback systems.
- With the hardware and software components expanded for DOMEconnect, we can output directly from the game engine to the projection and audio playback system in high resolution. We generate and render both the visualizations and the integration of video feeds or spatial audio coherent with the images in real time. Planetariums are ideal venues because they have the technology that makes immersive spaces possible in the first place, Weissig explains.
Experiencing the world of Immanuel Kant in immersive spaces
Weissig and his team are currently working to adapt content from the Immanuel Kant exhibition that ran at the Art and Exhibition Hall of the Federal Republic of Germany (Bundeskunsthalle) in Bonn from November 24, 2023, to March 17, 2024, for immersive spaces.
The goal is to permit the animated content that was originally designed for standard display formats to be shown in other settings such as planetariums. Plans call for bringing the city of Königsberg as Kant knew it to life in the form of virtual worlds in a 360° environment and giving the audience the chance to stroll the streets of the former capital of East Prussia - now the Russian enclave of Kaliningrad - and engage in dialogue with the great philosopher.
The technology is being tested first at the TiME Lab (Tomorrow‘s Immersive Media Experience Laboratory) at Fraunhofer HHI, a research, collaboration and presentation platform for immersive media that is equipped with a high-resolution 180° video projection system and wave field-based 3D sound system.
