Our initial intention was to build this application in standard web technologies. In order to achieve the goal of this application we’d need a component that could track a user’s head, determine where his eyes are (even with glasses), and establish whether his eyes where open or closed and whether a given movement could actually be construed as a blink. Even more, we knew people might be in low light scenarios and using laptops and webcams at odd angles.

We began developing prototypes by combining open source javascript libraries, such as head tracking and motion detection, to come up with a solution that worked well under perfect conditions but came up short under other variations. We found this HTML5/JS version accurately detected a blink about 60% of the time.

Not satisfied with this, we rebuilt the application in Flash using another open source library for head tracking and motion detection. This gave us a larger target audience and the blink detection worked over 85% of the time giving better end results.

To tackle how we would swap between the different videos required an incredible amount of trial and error. A key and omnipresent limiter was network bandwidth. In order to be able to swap seamlessly between we needed to be able to balance this along with the optimal number of videos that could be preloaded.

We spent a lot of time getting the video loading and queuing to work properly and buffer as we needed. We came up with methods to jump the playhead up to keep all the videos that were queued synced up by every word and by every second as well as determining the optimal number of video players to layer into the experience. Ultimately we had 4 players loading on top of one another.

At the moment there is no specific decision to further develop this application. However, the execution has seen a lot of interest in other content related applications. The queuing, cutting, and playhead control are custom elements that have many future applications beyond what we’ve achieved here.

Automatic and seamless integration of multiple video sources are typical needs of many campaigns and have a lot to gain by considering a similar approach. Additionally, the integration of such a ubiquitous component as the YouTube player allows for broad adoption and future innovation.

Since launch, the application has captured over 600,000 blinks. 90,000 people have completed the entire video experience and nearly all have come back to repeat the process and see more videos.