Cannes Lions

KIKI, Virtual Human Interpreter

MEDIA.MONKS, Hilversum / NHK ENTERPRISES & NHK GLOBAL MEDIA SERVICES / 2024

Awards:

1 Silver Cannes Lions
1 Bronze Cannes Lions
1 Shortlisted Cannes Lions
Case Film
Supporting Images
Supporting Images

Overview

Entries

Credits

Overview

Background

For members of the Deaf community, providing only closed captioning only during Emergency Alerts is like translating content into an entirely different language as they read. Transcribing spoken words doesn’t capture the same meaning or nuances for those who communicate primarily through Sign Languages.

Sign Languages have their own word order (or syntax). And what word order is appropriate depends on the context. Proper word order depends on what you’re trying to do: explain, remind, confirm, negate, cause to consider, ask a question, etc. This results in multiple variations of interpretation of a single sentence of spoken word.

The fact that the distinction between closed captioning and the importance of actual Sign language isn’t widely understood is why the world needs to include more sign language interpretation and advocacy of linguistic diversity. Emergency alerts are but one part of answering the broader call. It’s time to embrace true accessibility.

Idea

The creative idea behind Kiki was to develop the world's first text-to-sign translation engine, revolutionizing communication for the Deaf community. Drawing upon advanced technology and collaboration with Sign Language experts, Kiki seamlessly translates plain text into lifelike Japanese Sign Language (JSL) in real-time. The idea was to create a dynamic virtual avatar capable of accurately conveying the nuances of sign language, including facial expressions and emotional cues. This innovative approach ensures rapid and effective communication during emergencies and beyond. By combining cutting-edge technology with linguistic expertise, Kiki represents a breakthrough in accessibility, empowering the Deaf community with vital information and fostering inclusivity on a global scale. The creative idea behind Kiki encapsulates the brand's commitment to leveraging innovation for positive societal impact, driving meaningful change in how we communicate and interact with one another.

Strategy

In crafting our strategy for Kiki, our goal was to create a scalable tool addressing the lack of accessibility for the deaf community during emergency broadcasts in Japan. We collaborated closely with our client and JSL experts, including professors from Tsukuba University of Technology, to ensure complete accuracy of the complexity and nuances in sign language.

Through seamless integration of our first of its kind text-to-sign translation engine and rapid rendering of less than 2 seconds per motion phase, Kiki generates disaster messages within 10 seconds from user text. This breakthrough approach not only aids emergency broadcasts but also benefits the Deaf Community globally. Kiki’s design and API-based approach consider the holistic needs of other government institutions and brands worldwide, making it ideal for future business solutions requiring increased accessibility.

We're in discussions with the government and universities in Europe and beginning the process of capturing audiences for European Sign Language.

Execution

Kiki's execution showcases innovation through her pioneering text-to-sign translation engine, revolutionizing real-time Japanese Sign Language (JSL) interpretation. With a database of over 16,000 individually captured mocaps representing signed words, Kiki achieves unprecedented accuracy and realism, setting a new standard in the field not just in Japan, but globally. The implementation involved a meticulous two-year process, collaborating with Sign Language experts and Tsukuba University of Technology to ensure linguistic authenticity and technical precision. Integration into Japan's emergency broadcast system underscores the project's scalability and immediate impact. Kiki's deployment extends beyond emergencies, enhancing accessibility in various settings such as museums, airports, and aircraft safety videos. This inventive execution not only addresses a crucial gap in emergency communication but also demonstrates adaptability, pushing the boundaries of real-time text-to-sign language translation to exponentially increase opportunities for sign inclusivity for the Deaf community.

Outcome

1. NHK's integration of Kiki into Japan's emergency broadcast system ensures vital communication for over 300,000 Deaf individuals during crises, solidifying its importance in emergency situations.

2. Since it’s launch, Kiki has provided life-saving data at every emergency broadcast most notably during the January 1, 2024 7.6 magnitude earthquake.

3. Created an official sign language repository in conjunction with the Tsukuba University of Technology. Its initial 16,000 forms the basis for all future sign language capture to be easily expanded.

4. Successfully integrated Kiki as the first-ever virtual ambassador at the 2025 Deaflympics in Tokyo, helping support and demonstrate the need and ease of use of our real-time sign service.

5. We’re already working on future developments. Kiki 2.0 will leverage AI and machine learning to capture more efficiently, and cost-effective

6. We are in talks with several European universities to capture new languages to be used for online courses.