You step into the lobby of the popular, recently opened Sister City Hotel, in New York City’s Lower East Side. You're greeted by an atmospheric soundscape. You don’t realize it immediately, but with each passing moment you're hearing the sonic expression of exactly what’s happening outside. Brilliant beams of sunlight bursting through the clouds, the small flock of birds flying across the sky, it’s all being captured by a live camera feed that gets continuously translated into a loop of music using Microsoft AI. That’s how our generative score for the new hotel works. It subtly brings the environment into the property while at the same time connecting the hotel more deeply with its surrounding ecosystem. With help from AI and real-time data capture, the environment is the co-composer of this delightful and unexpected feast for the ears. The other composers: Experimental musicians Julianna Barwick and Björk.

We worked closely with the creative team at Atelier Ace (the parent company of Sister City) and musicians Julianna Barwick and Björk to craft lobby soundtracks that were both welcoming and interesting, sparking curiosity among guests. The lobby score we created for Sister City Hotel in downtown NYC was based entirely upon incoming data from a rooftop webcam atop the hotel, analyzed by Microsoft AI Custom Computer Vision. When Computer Vision recognized a bird, the AI triggered a light synth to play in Barwick’s "Circumstance Synthesis." Rain, sunlight, or even planes triggered different looped sounds. In phase 2 of the project, the AI was even more advanced, not only identifying weather events like clouds, but also able to differentiate types of clouds, as well as the number of birds in the sky at a given time. Depending on the shape, color, and size of clouds, the composition morphed from minimal to more textured and layered. This new specificity unlocked more nuance in the final composition; since the AI will continue to learn, the soundscape itself will also continue to evolve, welcoming guests with soothing (for once!) sounds of New York City.