the sound of things interactive sound installation
up previous page
intro realization credits
german

To track the position and rotation of the headphones I use three LEDs attached to the headphones in conjunction with eight infrared cameras and a custom motion-tracking-software. The USB-cameras are not synchronized. Since there are eight cameras, errors average out sufficiently and occlusions are seldom.

Headphones with three LEDs

To make all sound sources appear at the correct positions in real space, I had to build a virtual copy of the table together with the objects standing on it. Reality and virtual elements were then overlaid by a calibration procedure. The result is an augmented reality scenario where the real world is enriched with virtual sounds.

Virtual copy of the real table

To achieve the spatial acoustics via headphones, I am using a combination of delays, filters, acoustic damping, reverb and so-called head-related-transfer-functions (HRTFs). By this means, the sounds seem to originate from the objects themselves. Below is a flow chart depicting the processing chain of a single sound source for a single ear.

flow chart of sound generation for one ear

Tracking and sound processing were both quite easy to parallelize. Both could be run on separate computers if necessary.

I used point sound sources, though always multiple sources per object: Sounds were mostly recorded in stereo on the table itself. Stereophonic sounds allowed me to achieve a richer sound by placing more than one sound source for an object. Recording the sound source on the table made sure that the room acoustics were quite close to reality when an ear moved close to the object.

up previous page
intro realization credits
german