Binaural Sound for CREW's C.A.P.E experience

Location: CREW & iMinds

The experiment deals with the enhancement of the immersive experience in CREW’s C.a.p.e. (Computer Automatic Personal Environment) configuration (Omnidirectional Video, head tracker, Head Mounted Device) by adding the corresponding auditive sensation.

First attempts with two microphone capsules attached to a person’s ear shells approved binaural recordings can provide an impressively realistic experience of auditory immersion. As it is crucial for the spectator to freely move his head inside the recorded sound field, two solutions shall be tested and compared.

A ‘native BFormat’ (Ambisonics) microphone consisting of one omni- and two figure-eight capsules will be mounted invisibly above the Omnidirectional Video camera in static as well as in mobile positions. Everyday sounds, like footsteps and talking are recorded as well as a symphonic orchestra.

The recorded audio stream will then be decoded into binaural stereo, applying Head Related Transfer Function (HRTF) impulse responses which depend on the listener’s head position. In comparison 'Motion Tracked Binaural’ sound (MTB) offers a rather natural and very realistic, albeit more complex approach. An MTB array (provided by TU Berlin/Audio Kommunikation) consists of a head-sized sphere with 24 microphone capsules uniformly mounted around its equator. By use of a headtracker and interpolation algorithms, the system determines the positions of the listener’s ears directing the corresponding signals to the headphones. In order to evaluate and to compare the two systems regarding their usability for static as well mobile applications several tests will be carried out.