Augmented Reality Brings a Whole New Meaning to the Phrase "Show & Tell"

How to Tell Compelling Stories in AR



BY Daniella

In our "Lessons from Sundance" series, we first looked at how two filmmakers approached augmented reality (AR) as a storytelling medium. We continue that arc with an examination at how experience design shapes AR as a storytelling medium by exploring the design choices the Meta team made as part of Journey to the Center of the Natural Machine at Sundance.


I've been dancing since I was young. To me, dancing is the art of delivering a story while using visuals to evoke emotions. In my routines, I combined different kinds of movements, rhythms, and techniques to tell "stories" and show the emotions underlying the dance as a whole. And, if you add technology to it, dancing is just like augmented reality: they both harness the art of delivering a message while using visual volumetric experience to evoke emotions.


Now how do you take someone through that multi-sensory experience, especially with AR? What do you create to take someone through that kind of experience? How do you get people to feel the experience, ultimately making that human connection?


The obvious answers to these questions are user experience (UX) and user interface (UI) design. But, those are oversimplified answers. When it came to producing Journey, the Sundance Film Festival's first collaborative AR installation, the Meta team carefully weighed and debated several design choices that shaped the final look and feel of the installation. The three design choices below are only a snapshot of how we designed Journey. We hope that they help you when you make your AR experiences and apps.


Design Choice #1: UX and UI Set Aren't Front and Center
Both UX and UI are key elements for artists to express themselves through. After all, they both help artists and creatives figuratively put themselves in their audiences' shoes. Within AR though, UX and UI serve as guides, as opposed to being the defining elements for telling stories through AR. As guides to having users enjoy and appreciate the stories you're immersing them in, UX and UI help people understand "how" and "what" they're supposed to be doing within the story you're showing and telling them. The "why", or significance, of your story is then set up and staged by UX and UI (and more broadly experience design), allowing you to convey whatever message and emotions you want your audience to experience.


We kept this consideration in mind as we designed Journey, making sure that the story of the brain's evolution was not lost among the dazzling holograms, while adhering to our spatial interface design guidelines to ensure that people didn't become overwhelmed. As people take the journey, they'll not only see a holographic interface that prompts them to interact with the brain, but ultimately recall how different parts of brain help us make decisions, build tools, and analyze our surroundings.


Design Choice #2: In AR, Everyone is Now Part of the Story
We all have friends who become really animated when they tell stories – you know, those friends who wildly gesticulate over the course of their stories, even acting out those stories! Those friends have been immersing us deeper into their stories, going beyond the typical way stories engage our senses of sight and hearing. And like those friends, AR brings stories to life through the ability to let us touch the stories we are sharing with friends, while letting us see what we're touching.


As we designed Journey, one thing we wanted the audience to do as part of the experience is to touch the human brain they were seeing in front of them, and making it that much more memorable. What otherwise would have been a semi-active experience instead became a real-time active and immersive experience.


Showing and telling are no longer the same with augmented reality. (Hammonds)


Design Choice #3: Always Keep It Memorable
Chances are the stories you remember most are the ones that evoke specific emotions. The same emotional journey we take people on through stories, whether they be knee-slappers or big fish stories, applies to designing AR experiences. We sought to leave a lasting impression on people, one that not only showed and explained how our brains evolved into the "natural machines", but one that left them wanting to learn more about how our brains work. And while it is easier said than done to make a story as memorable as possible, striving to keep this tenet of storytelling at the forefront of designing AR experiences will go a long way towards making your finished experience that much more compelling.


Through Journey, we aimed to make the Meta 2, and more broadly AR, an educational tool – one that brings history and science (otherwise dry subjects for most people who remember their school days) to life by having people become a part of those lessons. No longer do we have to imagine what classroom lectures look like in the future – we can experience those immersive and highly interactive lessons through AR, and share them with other people by making them the center of a memorable journey.



Subscribe to AR news & best practices