Sample Abstract

Simulating Synesthesia in Spatially-Based Real-time Audio-Visual Performance

Dr. Steve Gibson Reader, Interactive Media Design, Northumbria University. 111 Sample Avenue, Newcastle, UK. Email: [email protected] Web: http://www.telebody.ws/

Abstract

(Abstract should be 300-500 words) In this paper I will describe and present examples of my live audio-visual work for 3D spatial environments. These projects use motion-tracking technology to enable users to interact with sound, light and video using their body movements in 3D space. Specific video examples of one past project (Virtual DJ) and one current project (Virtual VJ) will be shown to ‘illustrate’ how flexible user interaction is enabled through a complex and precise mapping of 3D space to media control. In these projects audience members can interact with sound, light and video in real-time by simply moving around in space with a tracker in hand. Changes in sound can be synchronized with changes in light and/or real-time visual effects (i.e. music volume = light brightness = video opacity). These anges can be dynamically mapped in real-time to allow the user to consolidate the roles of DJ, VJ, and light designer in one interface. This interaction model attempts to reproduce the effect of synesthesia, in which certain people experience light or color in response to music.

Keywords

Synesthesia, motion-tracking, virtual environments, real-time visuals, electronic music, light design, DJing, VJing, immersive art, interactive media