A former programmer turned designer with a secret life as a novelist came up with the idea after using generative AI to create promotional materials for his own novel. As an agency owner and creative, he was intrigued by the level of production value that AI could provide for little to no budget, as long as there was a strong system in place. This led him on a journey to create a foolproof GenAI platform available to authors, publishers, and booksellers to help promote their own titles. And potentially, in the future, it could become an entirely new digital medium—a new way to consume books.

Also, a big taboo in literature is feeding the audience what they will see in their mind. That’s why book covers are so complex. These animations had to deal with the same issue.

If generative AI can turn words into images, can we use a book as one giant prompt? Not as a poorly directed emulation of a movie but as a new format, created just to enhance the words on the page?

That’s what Hypnovels is. A design idea that solves a problem for authors (producing affordable content for their social channels without having to expose their face all the time, because, you know… they’re writers) and addresses the audience’s need for something visually stimulating yet soothing to force them to stop and pay attention to the words. All in one simple and seamless format.

In order to make Hypnovels possible, we had to first look at books not as stories but as data, and then create a tool that interprets that data and transforms it into images our brains can relate to. So we combined three different forms of AI—GPT4 for interpretation, Stable Diffusion for animation, and Eleven Labs for audio—then fine-tuned it into a format Dr. Paul Li, Cognitive Sciences professor at UC Berkeley, says “sparks the feelings we have in dreams.”

Then we recruited specialists in AI art and literature to craft a list of styles that allowed multiple kinds of stories to be visualized, in a way that protected our “dreamlike” language but made each story unique and original.

In other words: we turned stories into data, and then data back into stories.

A few points we would like to highlight:

The author tool was designed for people with low tech skills and likely some level of tech anxiety. So it was made to be not only super simple but also very familiar. Four easy steps, most of them things authors can describe with their words—no Photoshop skills or prompt engineering knowledge needed.

The biggest challenge: since stories flow between distinct scenes, the animations often lacked unity. So we imported a literary theory idea into the interface: some writers such as Chuck Palahniuk say every chapter has a recurring theme. An underlying thought that repeats itself throughout the story to give it unity. So we included a “recurring image,” and that immediately took care of continuity.

Through the entire setup of a chapter, the only visual input is the visual style, which authors can pick from a range of options the system offers. These are all designed by our own team and based on wide ideas and styles, never the work of a particular artist.

The combination of style, recurring image, the paragraphs themselves and the natural randomness of AI guarantees the uniqueness of each video.

Despite that uniqueness, the fleeting nature of the images, caused by that constant “fly in” motion toward the picture, not only makes a Hypnovel easily recognizable—a dreamlike experience, as Dr. Paul Li, Cognitive Sciences professor from UC Berkeley described it—but also avoids the trap of telling readers what they should see, which is a big taboo in the literary world.

Readers exposed to a Hypnovel are up to three times more likely to continue that book compared to readers exposed to the same chapter in a written format.

The design was so appealing that BookTrib, one of the leading literary websites in the country, immediately reached out and offered an exclusive deal to distribute all the generated videos for free on their own platform, a benefit we are fully transferring to the authors.

The first book to employ the technology, the novel “The Girl from Wudang,” became an instant Amazon bestseller, prompting requests from other authors and readers to see more titles in that format. This early validation led to the development and beta launch of the Hypnovels platform.

Now that the platform is out, lots of authors can turn their simple text files into powerful animations with just a few clicks.