Brand Experience and Activation > Brand Experience & Activation: Sectors

KIKI, VIRTUAL HUMAN INTERPRETER

MONKS, Amsterdam / NHK ENTERPRISES & NHK GLOBAL MEDIA SERVICES / 2024

CampaignCampaignLayout(opens in a new tab)
Supporting Images
1 of 0 items

Overview

Credits

OVERVIEW

Why is this work relevant for Brand Experience & Activation?

Kiki's innovative approach not only addresses a critical societal need but also creates a memorable and impactful experience for users. By bridging communication gaps and fostering inclusivity, Kiki effectively communicates the brand's values of accessibility and empowerment. Furthermore, Kiki's deployment in various settings such as museums, airports, and emergency broadcasts, amplifies the brand's visibility and reinforces its commitment to driving positive change. The interactive nature of Kiki's real-time sign language translation adds a layer of engagement that resonates with audiences on a deeper level, making it a compelling example of effective brand experience and activation.

Please provide any cultural context that would help the Jury understand any cultural, national or regional nuances applicable to this work.

Surrounded by rising sea levels, Japan has one of the highest earthquake risks globally. In only the last decade, it has experienced over 30 earthquakes, including a record breaking one in 2011— A 9.0 magnitude quake that resulted in the most destructive tsunami in recorded history. Despite having one of the most advanced early-warning systems, designed to provide life-saving information in multiple languages, it was missing one very important one: Japanese Sign Language.

In order to provide fundamental and equal access to Japan's 300,000+ Deaf Community Members, NHK Enterprises and NHK Global Media (Japan’s Public Broadcaster) created Kiki. The world’s first, text-to-sign translator for Emergency Alerts. Through a first of its kind text to sign translation engine, Kiki, a lifelike virtual avatar, receives text input and generates full dynamic Japanese Sign Language in real-time.

Background

For members of the Deaf community, providing only closed captioning only during Emergency Alerts is like translating content into an entirely different language as they read. Transcribing spoken words doesn’t capture the same meaning or nuances for those who communicate primarily through Sign Languages.

Sign Languages have their own word order (or syntax). And what word order is appropriate depends on the context. Proper word order depends on what you’re trying to do: explain, remind, confirm, negate, cause to consider, ask a question, etc. This results in multiple variations of interpretation of a single sentence of spoken word.

The fact that the distinction between closed captioning and the importance of actual Sign language isn’t widely understood is why the world needs to include more sign language interpretation and advocacy of linguistic diversity. Emergency alerts are but one part of answering the broader call. It’s time to embrace true accessibility.

Describe the creative idea

The creative idea behind Kiki was to develop the world's first text-to-sign translation engine, revolutionizing communication for the Deaf community. Drawing upon advanced technology and collaboration with Sign Language experts, Kiki seamlessly translates plain text into lifelike Japanese Sign Language (JSL) in real-time. The idea was to create a dynamic virtual avatar capable of accurately conveying the nuances of sign language, including facial expressions and emotional cues. This innovative approach ensures rapid and effective communication during emergencies and beyond. By combining cutting-edge technology with linguistic expertise, Kiki represents a breakthrough in accessibility, empowering the Deaf community with vital information and fostering inclusivity on a global scale. The creative idea behind Kiki encapsulates the brand's commitment to leveraging innovation for positive societal impact, driving meaningful change in how we communicate and interact with one another.

Describe the strategy

In crafting our strategy for Kiki, our goal was to create a scalable tool addressing the lack of accessibility for the deaf community during emergency broadcasts in Japan. We collaborated closely with our client and JSL experts, including professors from Tsukuba University of Technology, to ensure complete accuracy of the complexity and nuances in sign language.

Through seamless integration of our first of its kind text-to-sign translation engine and rapid rendering of less than 2 seconds per motion phase, Kiki generates disaster messages within 10 seconds from user text. This breakthrough approach not only aids emergency broadcasts but also benefits the Deaf Community globally. Kiki’s design and API-based approach consider the holistic needs of other government institutions and brands worldwide, making it ideal for future business solutions requiring increased accessibility.

We're in discussions with governments and universities in Europe, and beginning the process of capturing audiences for European Sign Language.

Describe the execution

Kiki's execution showcases innovation through her pioneering text-to-sign translation engine, revolutionizing real-time Japanese Sign Language (JSL) interpretation. With a database of over 16,000 individually captured mocaps representing signed words, Kiki achieves unprecedented accuracy and realism, setting a new standard in the field not just in Japan, but globally.

The implementation involved a meticulous two-year process, collaborating with Sign Language experts and Tsukuba University of Technology to ensure linguistic authenticity and technical precision. Integration into Japan's emergency broadcast system underscores the project's scalability and immediate impact. Kiki's deployment extends beyond emergencies, enhancing accessibility in various settings such as museums, airports, and aircraft safety videos.

This inventive execution not only addresses a crucial gap in emergency communication but also demonstrates adaptability, pushing the boundaries of real-time text-to-sign language translation to exponentially increase opportunities for sign inclusivity for the Deaf community.

List the results

1. NHK Enterprises and NHK Global Media's integration of Kiki into Japan's emergency broadcast system ensures vital communication for over 300,000 Deaf individuals during crises, solidifying its importance in emergency situations.

2. Since its launch, Kiki has provided life-saving data at every emergency broadcast, most notably during the January 1, 2024, 7.6 magnitude earthquake.

3. Created an official sign language repository in conjunction with the Tsukuba University of Technology. Its initial 16,000 forms the basis for all future sign language capture to be easily expanded.

4. Successfully integrated Kiki as the first-ever virtual ambassador at the 2025 Deaflympics in Tokyo, helping support and demonstrate the need and ease of use of our real-time sign service.

5. We’re already working on future developments. Kiki 2.0 will leverage AI and machine learning to capture more efficiently and cost-effective.

6. We are in talks with European universities to capture new languages to be used for online courses.

More Entries from Not-for-Profit / Charity / Government in Brand Experience and Activation

24 items

Grand Prix Cannes Lions
THE FIRST EDIBLE MASCOT

Live Brand Experience or Activation

THE FIRST EDIBLE MASCOT

POP-TARTS, WEBER SHANDWICK

(opens in a new tab)

More Entries from MONKS

17 items

Silver Cannes Lions
KIKI, VIRTUAL HUMAN INTERPRETER

Real-Time Contextual Content

KIKI, VIRTUAL HUMAN INTERPRETER

NHK ENTERPRISES & NHK GLOBAL MEDIA SERVICES, MONKS

(opens in a new tab)