The joy of playing a musical instrument, and doing so with others, is beyond compare. But it is also true that it takes a huge amount of time and effort to attain that joy, and many people give it up halfway through the process. People with physical disabilities are all the more likely to do so, because however passionate they may be, they face great difficulties with playing a musical instrument on stage together with other musicians due to their physical limitations. That is why Yamaha, a leading company in musical instrument manufacturing, decided to rise to the challenge and make the joy of music equally accessible to anyone.

Introducing THE JOYFUL PIANO:

Yamaha’s AI-assisted Piano is an instrument that learns from a huge amount of performance data and automatically plays the part for unmovable fingers in synchronization with the pianist’s performance. Building on it, Yamaha has now developed and installed a new AI technology that controls the strength of keyboard touch and pedal movement to compensate for the player’s physical limitations. Taking advantage of the technology, Yamaha organized a concert performance of Beethoven’s Ninth Symphony that includes the well-known “Ode to Joy” in the fourth movement, featuring three pianists with different disabilities as soloists. Playing with a leading orchestra and choir in Japan, they delivered to the world an encouraging message that with passion, everyone can become a musician.

Performance data was gathered from the practice sessions that spanned nine months (total 18 hours/month for three pianists) to develop individually aligned AI technology for each pianist. Data gathering continued as the music arrangement was repeatedly revised with consideration to the pianists' skills and balance with orchestra and chorus.

AI is used to assist the pianist born with a three-fingered right hand by playing for the missing two fingers, to help the bed-ridden pianist who strikes the keyboard at the back of a finger by detecting the faintest push on the keyboard and generating other required sounds and pedal movement, and to support the weak-fingered pianist by adjusting the sound volume. This high-quality concert was realized by adopting such AI technologies in order to harmonize the performance of disabled pianists with the orchestra and choir.

Accompaniment data was created based on the data gathered during practice sessions since March 2023 (total 360 hours). It took nine months for Yamaha to develop the best possible AI system for the concert, which reads the physical data entered by the pianist through keyboard on a real-time basis and moves the hammer, while controlling the keyboard and pedal as well.

In the development of performance-assisting AI, Yamaha placed the highest priority on re-creating the pure joy of playing music. To enable three pianists with physical disabilities (two fingers missing on the right hand, being bed-ridden from birth, cerebral palsy) to play the piano by themselves and feel a sense of accomplishment, Yamaha embedded in the piano a sound generating function that enhances their individual performance style.

1. Power-Assisting Technology: It adjusts the strength of hammer in synchronization with the pianist’s performance, when the keyboard is struck too weakly to generate properly audible sound, ensuring that all the sounds reach the audience.

2. Ultra-Low Latency Sound Generation: A technology that prevents accompaniment part from lagging behind the piano, when both should be heard simultaneously. Even the faintest touch on the keyboard is detected and transmitted to the system in order to minimize the delay.

3. Performance Continuation Technology: It adjusts to the pianist's irregular movements, maintaining seamless music flow.

The concert was held in Suntory Hall, one of the most prominent classical concert venues in Japan in December 2023. It was also streamed real-time to the world on YouTube and X, reaching 546 million people in 36 countries including post-event audience.

Yamaha also made this AI system into an app for over 10 million pianos worldwide, creating an enabling environment where anyone, with or without disability, can attempt to play the piano.

The concert was covered by major Japanese TV channels as well as foreign media including AFP, and has reached 546 million people in 36 countries including post-event audience. Media and opinion leaders such as university professors praised Yamaha, saying "Yamaha has transformed AI into a moving entertainment". The pianist who played in the concert emphatically commented: "I'm even more motivated to continue playing the piano".

At the same time, this AI system has been made into an app and released to work on more than ten million pianos worldwide, successfully creating an environment that enables everyone, with or without disability, to attempt to play the piano.