Our objective was to bring the “Interactive Cologne” festival, which was thus far only familiar among coders and creative tinkerers, to the attention of a wider audience.

In order to enthuse them for the theme of the festival, we involved the audience in a process in which technology and art merged to form a unique emotional live experience.

In order to do this, we hacked a work of art by the most expensive living artist in the world and played the colours of his glass window in Cologne Cathedral on the visitors’ smartphones via the 10,000 pipes of the organ.

We stimulated people’s curiosity in technology by inspiring them with it—and this still resonates in the minds of those both inside and outside “the scene”.

We managed to rouse people’s interest in technology by involving them in an emotional experience that could only be created through the creative interaction of technology and art.

But the buzz we created will also inspire creative people from within “the scene” to harness the creative power of technology, no matter their field, and to break new ground.

Our goal was to make the visitors as directly involved in the event as possible by using their own smartphones. As it was impossible to predict which device models and operating systems the concertgoers would bring with them, we were faced with the challenge of developing an independent and stable technical solution for all models. We needed to read the IP addresses of the devices and translated them into audio signals for the 10.000 pipes of the organ, while at the same time distributing the hacked colours of the Richter window, based on the MIDI signals, to the smartphones.

Only with uniform visual output and simultaneous triggering of all the devices could the synchronous and thus spectacular experience of colours and music be created.

Since HTML5 and WebSockets in particular are relatively common now, we decided to implement our idea using a combination of HTML5 in the smartphone browser and a Java-based WebSocket server in the back end.

We developed an innovative system integrating various technologies, with which a stable and high-performance real-time communication between the smartphones and the organ was possible.

As the involvement of the guests was of particular importance to us, we went beyond turning their personal smartphones into a digital canvas for the colours of the hacked Richter window. Via their IP addresses, they actually influenced the music and the play of colours themselves.

In order to provide a musical accompaniment to the hacking of the window, we brought together three internationally acclaimed artists. Prof. Dr. Winfried Bönig (cathedral organist), Gregor Schwellenbach (composer) and Damian Dziwis (digital artist) composed an innovative piece of music inspired by the 11,263 squares of colour of the Richter window. Out of this we then created an impressive digital play of colours via a control system developed especially for the interpretation of MIDI signals from the organ and distribution to the smartphones.

In a second part we created music using the IP addresses of the smartphones and came full circle with synchronous colour effects based on the organ’s MIDI signals.