How to Design for Mixed Reality with Kharis O'Connell, a VR / AR Design Expert

A Great Intro to UX and UI Basics for AR

BY Victor Lin

We've been curating augmented reality (AR) and virtual reality (VR) books that we find insightful and helpful for our readers, and ultimately believe will help you along your path to learning AR and VR. In the first installment, we visited VR legend Tony Parisi's top-notch primer for learning how to develop on the Oculus, Gear, and Cardboard VR platforms. In this installment, we take a closer look at user interface (UI) and user experience (UX) design in AR with Kharis O'Connell, a UI design expert with nearly 20 years of experience.

Designing for Mixed Reality: Blending Data, AR, and the Physical World

By Kharis O'Connell

49 pages. O'Reilly Media. List price: Free download

Look anywhere in the news, especially at the "forecasting 2017" articles on the state of AR and VR, and you'll see that the chorus of writers and media publications who say that AR and VR are the new computing platforms has only grown larger (and more unified in thought) since Oculus first launched the Rift DK 1. While these pieces tend to provide delicious food for thought on how AR and VR will be used across different industries, they altogether fail to answer the meatier question of "What's the user experience like in these modified realities?". And in their defense, no one really knows what AR interfaces should look like – not even the awesome artists working in Hollywood who have trained our imaginations to "see" those interfaces in certain ways. What we see in films like Iron Man or Minority Report  offer a tantalizing glimpse of what AR displays could look like (and boy, do they look overwhelming and claustrophobic):


Even a genius like Tony Stark would be hard-pressed to instantly understand what the half-dozen or so displays and gauges in his HUD mean. (Disney / Marvel)

Such a future will be here sooner rather than later, and if this year's eagerly anticipated CES is an indication of that future, it's crucial that we start the conversations on getting AR UI design right and avoid the kind of displays we've seen in the movies. Fortunately, UI and UX design expert Kharis O'Connell points us in the right direction with his brisk, but highly informative, guide on designing for AR. Nomenclature choices aside (he uses the term "mixed reality" throughout the book), O'Connell's guide is the most accessible primer for anyone – regardless of UI design experience – wanting to better understand and think about how people will work with holographic displays.

A few pages into the eBook, and it's immediately apparent that O'Connell put a lot of research and careful consideration into studying VR and AR. The first two chapters are a quick walkthrough the history of "the future of computing", finished with philosophical musings on how AR and VR can be used in the real-world. The third chapter is where things get more exciting as he concisely explains how modern AR headsets work, while pointing out how the complex technologies underlying them (computer vision and display technologies) work together to produce interactive holograms. If you're keen on jumping around the book, or are looking to start at the meatiest part of it, then I highly recommend starting there.

Throughout the book, O'Connell thoroughly shows and explains how designing for AR is an entirely unique and different experience – one that has little to no established best practices, and forces us to think beyond optimizing users' experience for phone and computer screens that range from 4.7 in (11.9 cm) to 27 in (68.6 cm). We all have to think different when it comes to creating optimal AR experiences, and as O'Connell argues, "gesture recognition will play an increasingly important part" in AR. To paraphrase Apple, we all have to start thinking gesturally.

The last two chapters of his book are arguably the most enjoyable not because they are the culmination of the science and technology behind AR, but they are where the road meets the proverbial rubber. O'Connell's experience with different AR headsets really shines here as he draws upon the hundreds of hours of development work to share insightful and practical tips, such as this wireframing spatial template that allows designers to quickly map "the positions of interface elements and objects":


Wireframing spatial template O'Connell created for use and dissemination.

Aside from its short length (Kharis, if you're reading this, please publish a follow-up book!), Designing for Mixed Reality is one of the most practical and authoritative books on UI and UX designs for AR. And with AR being the next computing platform, it really is up to us – both end users and those designing those next generation experiences (yes, even those advertisers who are practically salivating at the fact that holographic ads will become a horrifying reality) – to ensure that AR becomes a comfortable and enjoyable place to stay and work in, not one filled with annoying pop-up ads and terrible websites. The first step to ensuring that AR is nothing but a great experience for us all is to download O'Connell's book (it's free!).

More information

by selecting this option, I affirmatively consent to Meta's use of cookies.