We wanted to tell a story in the Sydney botanical gardens, to get kids back into nature while fostering their sense of wonder about the world. We also wanted the ability for hundreds of users to be able to run through the experience.

Being in a park and for children, introduced some interesting limitations: Trees couldn’t be damaged or altered in any way, necessitating small, lightweight installations; the children can be damaging, so everything had to be robust and child-proof; and the experience had to operate over very large distances, so things had to be as autonomous as possible, because technical assistance could be kilometers away. The technology also had to be invisible, to really engross the players.

We were inspired by Augmented Reality (AR) games like Google’s Ingress, and wearables have advanced the cause of small-scale technology, which allowed us to make a smart wand for children.

A user interaction device (the Wand) provides navigation assistance via inbuilt LEDs, compass and GPS. It has an Inertial Measurement Unit (IMU) and proprietary hand-gesture recognition system to detect when varied ‘spells’ are cast. An onboard microprocessor controls the peripheral systems via the UART and SPI protocols on a custom board.

Story locations contain embedded Android phones and effects-technology (lighting, speakers, etc). Wand proximity is detected through BLE communication and GPS positioning for redundancy. Wand information and commands are sent via BLE, and are updated across the whole experience via the Google Cloud Platform, using a NodeJS, MongoDB and Angular technology stack.

Users interact with the phone by speech and gesture -- Text-To-Speech and Google’s Speech-To-Text (OK Google) technologies allow dynamic conversations between them and the virtual characters.

At the end of the conversation, the Wand is told the GPS coordinates of the next node, and starts navigating again.