Celestial Bodies: Combining MoCap, nomadic HMD and tablets

Can complex matters like the historical explanations of the solar system be more easily understood on an “experiential” immersive base?
Is it i.e. possible to collaboratively walk around in a solar system while embodying planets and making gravitational forces understood in terms of experience?
Celestial Bodies is a concept for using immersion and virtual production tools in education and arts.

To achieve that we aimed at a virtual planetarium, fully developed in 3D, to be viewed and experienced as VR and AR, with a HMD and tablet, while having multiple users “perform” the solar system. The immersants were guided and assisted by a full body motion-captured actress (feedback from previous stage). Adapted software was implemented to achieve a virtual production integration, allowing a live blend of different inputs like CG, MoCap, music, video.

The concept of the event is inspired by “Collaborator”, an earlier concept in Dreamspace: an event with different levels and different content at the same time, thereby using different perspectives and viewpoints and using immersive tools.

In Celestial Bodies we had two “staged sets”: one with 6 scientists giving an illustrated lecture, one with the “performance” involving immersants, tablet users and onlookers. The immersive performance was at several levels:
Level one: Immersant, HMD, headphones. Actor/voice, only to be heard by level one and two. Mocapped actrice, virtually and physically present inside the Virtual Planetarium. A planetarium partly virtual (CG) and augmented. Actor and actress generate effects starting from the real world.
Level two: Spectator with headset and tracked tablet. He can hear level one and point his tablet. He always gets the visual connection between the virtual and the real. Level three: the audience listening to the scientists, and at the same time watching the performance.

The audience was composed of 60 people from the academic world (mainly invited by P.A.R.S.), professionals from French planetaria, and art & media students from the nearby well known art school ‘Le Fresnoye’. Basic research for Celestial Bodies was evaluated and reported during a workshop in February 2015. Evaluation and comments recorded during the workshop with the six academic researchers from P.A.R.S. were highly instrumental in setting up the final Celestial Bodies “Causerie Cosmique” event. The event took place at IKAVS in Tourcoing, France in December 2015.