What if a space could be transformed into a sonic platform for the most creative musicians of our generation? This was the premise behind our unprecedented "living" generative sound installation for Sister City Hotel. The project took on the heartbeat of New York City and made it a crucial element of the soundscapes that musicians Björk and Julianna Barwick created. They crafted sounds that corresponded to different environmental triggers that transformed Sister City’s lobby in an ever-evolving soundscape. The music becomes a way to place Sister City in the context of its neighborhood, while bringing that environment into the space in an entirely new way.

In the first phase of the project, Julianna Barwick composed five movements within an overall soundscape that reflect the constantly changing nature of the sky, with bass, synthesizer, and vocal lines that weave in and out. For each “event” identified by Microsoft AI on the rooftop camera feed, she created six synthesizer sounds and six vocal sounds for the generative audio program to choose from, meaning that there were 60 different musical options per day that would be triggered based on the external stimuli and play in the hotel’s lobby. Using Kinect depth cameras, a rooftop webcam, and web-based weather data we captured the events taking place outside. For the project’s next phase, Björk opened up her archives and shared her favorite choral arrangements; a different excerpt would play in the lobby, based on the AI analysis of the rooftop camera feed at any given moment. This time, the AI employed was more advanced, and able to identify a wider range of objects in even greater detail, leading to a nuanced soundscape that will continue to evolve over time, as the AI continues to learn about new events captured by the camera.