With recent gains in AI technology, innovation for users with disabilities is poised for the beginning of a golden age. Advancements within Google AI and Pixel smartphone technology have opened unique opportunities to better serve people with disabilities – the largest minority group in the world and yet a population that faces ongoing barriers to cultural and technological inclusion. Universal access is core to Google’s mission, and its teams are on a mission to co-design products together with disabled people to create universal access for everyone.

People with disabilities represent one in seven people in the world and one in four Americans, making it the largest minority group in the world. Adequate attention to this important community has been long overdue, and recent mainstream cultural moments – Justina Miles’s viral success as the ASL interpreter at the 2023 Super Bowl and “CODA” earning recognition for Best Picture as the Academy Awards in 2022, to name a few – are signaling that this is the right time to continue to drive the accessibility conversation forward.

Google’s mission is to organize the world’s information and make it universally accessible and useful to all. With a global focus on inclusion, Google Pixel has and will continue to champion innovation, representation, and inclusion with its Google AI technology. Pixel Camera is an industry-leading example of these principles in practice, with an ongoing goal of making photography more widely accessible and representative. Most importantly, Pixel’s accessibility innovations aren’t just created for the disabled community; they’re designed with the disabled community from the start. Creating Google Pixel features together with and for the community is a central part of Google’s approach to innovation development and amplifying the end result is an important aspect of inclusion and representation.

Guided Frame uses cutting-edge Google AI alongside Android TalkBack and the custom-built Google Tensor chip to create a Pixel Camera experience that is more optimized for people with blindness and visual impairment. Even more importantly, it leverages a partnership with blind users and blind community groups to influence the product design and technology – core to how we develop many products and features at Google. It was from this community that we heard how much the blind community loves to take selfies, food photos, and pet photos to share with their friends – and how much frustration and disappointment exists that current phone cameras are not designed for when users have vision impairment. This ongoing conversation inspired the technology that is now found within the latest version of Guided Frame and Android TalkBack on Pixel 8.

Being able to take and share photos allows many people to express themselves and connect with others - this is no different for people who are blind or low-vision. So we built Pixel Camera, which leverages Google AI on Pixel 8’s Tensor G3 chip, to identify when important objects are in frame – like people, food, pets, or even documents – and provide voice guidance and haptic feedback around when to snap a picture. The picture is automatically taken once optimal framing is in place.

The concept for Guided Frame emerged after the Camera team invited Pixel’s Accessibility group to attend an annual hackathon in 2021. Various teams across Google began working together, including the accessibility, engineering, haptics teams and more. It was extremely important to everyone working on this feature that the testing centered on blind and low-vision users throughout the entire process. In addition to internal testers, the team involved Google’s Central Accessibility team, a group of blind and low-vision Googlers, and the Vista Center for the Blind and Visually Impaired. After coming up with a first version, they asked blind and low-vision users to try the not-yet-released feature.

Guided Frame first launched in 2022 on Pixel 7 and Pixel 7 Pro, and we continued to partner with the community to develop more accessible photography. With the most recent update in 2023, Guided Frame can help users take great photos with their rear camera and goes beyond recognizing faces to include pets, food and even documents.

To bring additional awareness to the feature we created a Super Bowl commercial, Javier in Frame, which highlighted the practical application of Guided Frame. Leading up to the Super Bowl, Google released a behind-the-scenes video of how “Javier in Frame” was created on 1/31/24, and debuted the full spot on 2/2/24.

First introduced on the Pixel 7, Guided Frame, a feature in the Pixel Camera, enhances accessibility by guiding people who are blind and low-vision to capture selfies, group photos and pictures. Using Google AI to identify what’s in the scene and communicate that information to the user, Guided Frame uses a combination of audio cues, high-contrast animations and haptic feedback to help people take photos by aligning faces and objects within the frame.

During testing, feedback from blind and low-vision users reiterated how important it is for a camera to automatically take a photo when the person’s face is centered. Users also appreciated not having to find the shutter key and voice instructions. Allowing testers to try Guided Frame in everyday life allowed the team to realize that the product could help with both selfies and group pictures.

What makes Guided Frame not only innovative but also meaningful is that it allows for creative expression, autonomy and joy for the blind and low-vision community. Many accessibility features are designed to help consumers do the bare minimum; with Guided Frame, many people are able to become the photographer and confidently document key moments in their lives and of those around them.

What began as a selfie-assistant for blind and low vision Pixel owners evolved into a photo-coaching tool and visual navigator for all users. The launch of Guided Frame on Pixel saw a +27% increase in blind and low-vision users in Pixel’s camera with Guided Frame and +116% increase in rear camera captures with Guided Frame 2.0 and

Debuting “Javier in Frame” at the Super Bowl to such vast consumer reception elevated not only the Guided Frame feature but also forwarded more representation for the larger number of disabled people and increased visibility of Google’s overall mission around accessibility and building products for all.

Perhaps most importantly, it delivered innovation into the lives of users. Feature usage more than doubled following the campaign.