My Eye Opening Experience As a AR/VR Designer — Mistakes, Lessons, & Discoveries
Imagine this. It is the late 2010s and there is one thing everyone is talking about. This brand-new tech called Virtual Reality was making the news with companies like Google, HTC, and Facebook (now Meta) investing millions of dollars into the industry. Remember Google Glasses? This was the first public attempt at “Augmented Reality”, VR’s younger sister.
With VR and AR being the talk of the town, companies designing and developing this tech were coming up. One such company decided to hire a young UX/UI designer who had just started freelancing. This young designer (me) was thrilled to start his journey. Knowing very little about this industry, I decided to learn as much on the job as possible.
Let me take you through all the great things I learned, mistakes I made, and the things that truly surprised me.
A Harmony Of Skills In Tangent
Unlike some other companies, a UX designer working for an AR/VR company needs to know the entire process inside out. Whether it is understanding the core business or getting to know how the developers code the experience, I needed to dive deep into every facet.
I was truly surprised how by how much work goes into effectively designing an AR/VR experience.
One major discovery I made, were the spaces that these experiences were tested in. Most companies will have a Reality Room or space designed to simulate real world AR/VR experiences. The company I worked with was using 3D models to get a feel of the real spaces where users would try on their virtual glasses. In this case, my company 3D printed monuments, since we were designing AR experiences for historical points in the country. I would’ve never known about this without working in the industry.
Now the final results that were shipped in the app, were a culmination of a perfect symphony between the designer (me), the 3D artists, and the programmers. Instead of working with other UX designers, the 3D designer was my partner in crime. This meant that I was always designing for a 3D space, and not for flat screens anymore. Oh boy, was this a challenge!
While UX principles and practices translated well into AR and VR, it was definitely tricky to understand the nuances. Here are some key points that differentiated UX/UI for AR vs other industries.
- UX for AR is closely knit with 3D tools, assets, and environments.
- AR requires extensive knowledge of technology. Designers with little knowledge of AR/VR or tech in general will have a tricky time.
- Training and knowledge sharing can be a long task. Unlike other companies, you will not be starting work on any projects without the proper resources.
- There were few to no resources, UI kits, or design systems for AR/VR a few years ago. Now, everything is ready made, and courses are dime a dozen.
- You rarely use the AR/VR headset while designing. Everything can be tested virtually on your computer using a set of 3D tools like Unity.
- Most AR/VR content is designed to be bare minimum, you will seldom work with information-intensive UIs and screens. Users don’t read long text or like long instructions on screen.
- We work on the smaller elements like buttons and cards individually, instead of building systematic structures and systems.
Interaction Design > UI Design
While UI design for phones or web require a careful process including wireframes, UIs, usability tests, and more; AR/VR designs are more about constantly working in a virtual space. In fact, in some teams they design interfaces inside virtual reality itself. Imagine a person with a VR headset just waving around their hands like a crazy person.
Google actually has an app called Tiltbrush, which is well known to allow designers to sketch and paint in VR. This is just one of the examples of how designers can leave their PCs and build something in a true VR space.
Interaction design was a vital skill I developed while at the company, while my focus shifted from UI design to building interactions. What I noticed was that a majority of the managers and seniors could care less about the UI and visuals at the time. For them, it was more about function and ease of use over visual design or UI. This means that understanding user behaviour and obstacles can build a mature virtual experience.
Spatial design has come quite a distance from where it began, but I seldom see “beautiful” or “mesmerising” UI in any mainstream VR or AR app. A solid reason behind this was the amount of focus that the users give to the UI elements. Their focus lies more on the space in front of them, or the content they are consuming.
A great example of an interface with little focus on the UI elements is the Google Photos app. While using the app, all your attention is on the photos or videos you are browsing through. The UI elements are very minimal or “out of the way” most of the time.
Usability is surely important for any app to be successful, however usability in AR/VR definitely requires constant attention.
Another crucial skill one gets to develop is mapping user tasks and journeys, so as to be aware of the path the user plans to take. Imagine walking through the forest, only to stop to interact with some flowers on the way. In the case of AR/VR, such an example fits right in. While fixed mobile interfaces have a rather straightforward approach, users can often experiment with a spatial experience. Pushing buttons just to see what they do, turning their head all over just to check out whats around. It’s often comparable to a new toy a child gets. They want to play around and experiment with it.
No Fixed Design Process
Ever since being a beginner in the field of UX, I understood that there needs to be a clear set of steps and processes. Often infinite or never ending, these processes have been popularised worldwide. Working with an AR company, I tossed all this knowledge out the window.
You see, once you get onto the job, there will be a lot twists and turns to get to the product ready for production. This means that a lot of what we do at a mobile app company, is often left out in a AR/VR company.
- While a good sketch or rough wireframe is highly appreciated, we often skip Lo-Fi wireframes. Since most UI elements are minimal and bare-bone, you can get by with Hi-Fi wire-framing.
- The visual changes are mostly cosmetic.
- There is heavy focus on prototyping and testing, and not so much on the ideation or wire-framing stages.
Traditional Businesses Getting Into Tech
AR/VR got traditional businesses to finally get into the tech space. While it is extremely easy to get a website developed or a white labeled app into the market, AR/VR requires a completely different level of commitment. We are talking about months of research, design, and development which is often done in house for the most part. Since most businesses have an online presence or platform, introducing a “view in AR” button on their website won’t be difficult to do.
To reduce barriers of entry into spatial design, service companies have rushed to develop solutions for such businesses at all levels. This could range from a basic 3D scan of your physical product using your iPhone camera, to a fully customised experience that the business can demand if the money is right.
While all this sounds super easy to do, the concern for most businesses isn’t the tech, but the user adoption.
You can argue that AR/VR “looks cool”, and everyone would love to use this feature. On the other side of things, getting your current or new users to start using these systems can be a tough cookie to bite into.
- Businesses can often have customers that don’t understand or appreciate new tech as much. This could be due to the age of their customers, the habits of customers, or the unfortunate case of AR/VR seeming like a gimmick.
- AR/VR can be an expensive investment, specially where ROI (return on investment) is considered. Being a new space, not ever business will shell out cash.
- Use cases for such tech can vary for everyone. This tech will not become a priority for business owners until it becomes a major trend everyone starts hopping onto.
- Unlike real world innovations, I’ve noticed business treating AR/VR as a “get it and forget about it” feature in their tech stack.
AR/VR is The Most Misunderstood Innovation
While you can look at virtual headsets and are in awe of their capabilities, this excitement is short lived.
Truth is, AR/VR isn’t very useful for general users. Instead, the real use and importance lies in the hands of industries like medicine, construction, fashion, storage & inventory and the likes. These are the industries that find the most use of such tech.
At it’s developer conference, Apple unveiled it’s first AR/VR headset called “Vision Pro”. While the demonstration showed people using this technology at the comfort of their home and office, I wasn’t fully convinced.
To add fuel to the fire, Apple stated that their focus was businesses who require the tech the most. This hypocrisy seems to flow through the industry. Meta also has normal users having fun with their headsets in ads and launches, while being fully aware of their true audience.
In face, Apple has an entire PDF on their website, convincing businesses of their spatial platform and software.
Now, this seems to be a popular marketing strategy and to drive stocks up. I believe consumer focused companies love the attention they get when they launch something like a VR headset. This is the same reason Google said “AI” a billion times in their new developer event. Whatever gets more eyeballs, isn’t it?
While I think what Apple and Meta are doing is fun, what I’m interested in is what companies like Microsoft and HTC are acheiving in the space. They will rarely advertise their virtual solutions to people like you and I. They know that their market is more niche and focused. Due to this they focus more on features that enterprises will appreciate.
All of what I wrote down is what the present view of the industry really is. The future however is definitely bright and different from what some tech companies want you to believe. As of a 2019 report, almost 23 million jobs can be created in this space by 2030. In fact, the industry is also reported to be a market worth $394 billion by 2023. That my friend, is a lot of money.
I hope I added value to your day, and would appreciate a quick clap and follow.
You can also read my previous article which people seamed to like a lot: https://medium.com/prototypr/googles-new-ai-tools-are-great-for-designers-aeea8263e45
