Throughout history technology has offered artists and musicians new modes of expression that continue to transform the way we create and consume music. DELQA is an interactive installation that allows the audience to physically go inside a song created by critically acclaimed music producer and artist, Matthew Dear. A large aluminium frame with mesh walls, housed a 44-channel spatial audio system that allowed people to experience the installation in 3D. As people interacted physically with the elements around them, Microsoft’s Kinect technology detected their movements and used that data to shape the composition in real time, essentially allowing people to create their own version of the song. The vision was to create a dynamic and immersive environment that would push the relationship between music and technology - blurring the lines between creator and audience.

The music was spatialized across a 44 channel speaker array with localized and distributed sound sources to best match the content and interaction of the music. Messages were mapped from the Kinect Cameras to MIDI messages that were received by Max/MSP and then Ableton. This transformed a static song, into a dynamic and interactive one. Max for Live plugins translated the pressure and position data from the cameras and mapped them to timbral, rhythmic and melodic parts of the song. Ableton Live (with Max for Live) created all of the music in real time and routed the audio to Max MSP for spatialization across the speakers. This placed and moved sounds around the listeners in 3D so that as they explore, their perspective on the sonic environment changed.

Thousands of people experienced the installation live at New York’s New Museum driving organic conversation that saw #DELQA generate over 4M impressions over the first few days. Most of the credible major publications spanning technology, music, art, and culture celebrated the project culminating in a reach of over 50 Million. Video content generated over 130K views. In the consumer research 78% of the “creators” surveyed said they were interested in learning more about the Kinect, in addition to an overall lift of 12% points in positivity about Microsoft, with many expressing increased excitement about what the brand is currently doing. Cost per impression has been calculated as $0.0065 per impression. Due to the overwhelming response the concept will now live on. We are currently working with musicians like Grimes, Alt-J and others to bring new versions to some of the most credible festivals, museums and events around the globe.

DELQA is relevant for Creative Data Lions as it adapts data from Microsoft’s Kinect technology in a new and creative way. DELQA created a meaningful story for Microsoft whilst driving a greater personal engagement with their Kinect technology. The consumer narrative implemented for DELQA had data at its core, as the movements made by the audience through the Kinect was transformed into their own interpretation of Matthew Dear’s music. This was also critical in shaping the creative execution.

We used depth cameras to track not humans in this instance, but architecture affected by humans. By using depth cameras we were able to capture and and measure deflections of a non-planar surface in real time. From this data we are able to infer where and how hard visitors are pressing the fabric and pass these values to audio software. The software then alters the composition of the song people are hearing in real time.

Microsoft briefed us to look for opportunities where their technology could empower the music ecosystem by creating meaningful roles for their products in ways that would drive brand favorability. Their ambition was to improve brand perception with the hard-to-reach audience of cultural influencers and creators, specifically in the music and technology community.