To promote the Netflix series 'Open Your Eyes' we were briefed to visualize what is going on in the mind of a live performer, with a suggestion to use the EEG technology present in the series (spoiler alert: in the series, the talent of a talented pianist is stolen during a live performance using a strange computer and electrodes connected to the head).

It was to be performed live during event with artists connected to BCI.

The biggest challenges were

- specificity of the EEG test and data (very specific requirements for conducting the experiment in order to be able to read real signals) - we had to use other data, like facial movement events,

- interpreting data and creating visualizations that will emotionally resonate with the audience,

- a live performance and real-time visuals.

What if transferring talent from one person to another could be possible?

This is the central idea behind Open Your Eyes, a Netflix sci-fi series.

To promote it, we decided to run an experiment, and see if we can also transfer talent but in a different way: capturing the emotions of the artistic act and transferring them to visual expression.

Piano playing is the central talent used in the show, so we teamed up with the famous pianist and composer Hania Rani to put our premise to the test.

Using live data from BCI (Brain-Computer Interface - also associated with the series), we created an immersive experience - putting artists and the audience inside the visual interpretation of the performers’ emotions.

To prepare, we collected the signals from BCI (Brain-Computer Interface) during a series of rehearsals with artists. The data was then analyzed side by side with music, video and motion capture to learn as much as possible. Next we used the EEG signals combined with facial muscles’ movement events to generate performance metrics - the set of normalized parameters of the emotional state (excitement, concentration, etc. calibrated for each artist) to drive the simulations (number of particles, intensity, turbulence, diffusion, etc.).

The animation database was an artistic interpretation based on the above-mentioned trials (main parameters, direction of emission, change of effects during climax). The data collected live had a real (not pre-designed) impact of 40-70% on the generated-live image depending on a given moment.

We used the Emotiv Epoc-X headset, and sent the signals wirelessly using the OSC protocol to the application created in Unreal Engine.

Our experiment became an immersive concert experience of the Hania & Dobrawa duo, whose real-time BCI (Brain-Computer Interface) data influenced the surrounding space.

The resulting multimedia installation put musicians and the audience inside the artistic visual interpretation of collected data.

The elusive character of thoughts, the intensity and uncertainty of emotions, inspired us to interpret the signals as fluid simulations combined with flocking movements - capturing the ephemeral performance in a digital, yet very organic, inspired by nature way.

To prepare, we collected the signals from BCI during a series of rehearsals with artists. The data was then analyzed side by side with music, video and motion capture to learn as much as possible. Next we used the EEG signals combined with facial muscles’ movement events to generate performance metrics - the set of normalized parameters of the emotional state (excitement, concentration, etc. calibrated for each artist) to drive the simulations (number of particles, intensity, turbulence, diffusion, etc.).

We used it all to design animations in Unreal Engine that reacted to the emotional state of the performer in real time and showcase them in the form of large-format projections in the Institute of Power Engineering - High Voltage Department - a research center for voltage experiments in Warsaw, Poland, famous for its gigantic space.

The day of the experiment - after the introductory part of the performance - a fragment of Debussy’s ‘Clair de lune’ featured in the series and marketing materials - the duo started their premiere piece ‘Dunkel’. The back and forth between the musicians was vivid on the enormous walls and the floor of the Institute. The culmination point resulted in a set of colorful explosions, slowly calming after it finished.

The performance was beautifully captured by the quantum physicist, artist and a film director Andrzej Dragan.

The experiment and the resulting video helped the series (not based on the known IP) to appear in the minds of the audience. It made a considerable buzz in social media (teaser + additional BTS material), and the video itself received almost 2x more views than the Polish trailer of the series in the first 3 weeks (premiered on September 1).