UX Research in VR Games and Experiences
I’m a freelance games user researcher and UX specialist. I offer various games and non-games research services, outlined at www.garethlloydresearch.com, and have experience in conducting UX research in VR and AR games.
There’s increasing interest and knowledge out there about UX research in VR, but VR games in particular benefit from player-centric design and research. This is because VR games have three sources of UX issue: the first and most generic is the challenge of working with new hardware. The second is the set of particular issues that VR can bring as a medium, from motion sickness to locomotion methods. The third is the whole spectrum of issues encompassed by games user research (I’ve written a separate piece focusing on this topic). In short, a VR game faces all the UX issues that a game on a more traditional device (e.g. a console or mobile phone) would have, plus the challenges of placing a player in an immersive environment with gestural controls, plus the general problems of having lots of players using an unfamiliar device to play your game.
What’s the answer? Thankfully, most games user research methods can be used in VR. An expert UX review is the most straightforward: a researcher can systematically assess the game against best practice principles, placing special emphasis on ensuring that the game not only adheres to best practices with onboarding and reducing barriers for players, but including VR best practices too. The latter is mainly focused on reducing potential motion sickness: it includes movement options (e.g. teleporting style, free movement), motion and vision options (e.g. tunneling), and comfort and safety provisions, to name just a few. Options which may have an impact on player comfort should be explained or demonstrated visually, and players should be pointed towards where they can be changed. Players could also be given a prompt to try out and understand different options, in order to tailor and tweak the game to find the most comfortable way to play.
Playtests can and should also be used. Large-scale playtests — where multiple players experience the game at the same time — can be much more difficult in VR, because of the space and hardware needed, though not of course impossible. This means that for getting the numbers of players required for quantitative studies, a lot of time may be needed to run enough players through the game.
Usability playtests are much more viable in VR. These involve smaller numbers of players per round of testing: it may be sufficient to run 5–10 players in a usability test. However remember that as always, small-scale usability testing is much more powerful when carried out in multiple rounds in accordance with development iterations: 5 players in 5 rounds of testing is infinitely more useful than 25 players in a single round.
Here are some extra considerations when running a test in VR:
Player safety: VR systems have safety features built in, such as guardian systems that show players when they are about to come into contact with walls or objects; ensure that these are set up correctly and conservatively. Remove anything from the environment that could cause harm if the player comes into contact with them, even if they seem in a safe location; players may move in unexpected ways when they are immersed in a game. Pay special attention to trailing leads when using tethered systems; a moderator should monitor for tripping hazards at all times, especially when testing with players who are new to VR. This may mean that you need an additional moderator for a VR test.
Consent and wellbeing: The above notes on player safety mean that you should use extra care and caution when consenting players to take part in a VR study. Your current consent forms may be insufficient or inappropriate for the physical activity and potential discomfort arising from the use of VR. You will need to emphasise that players can stop the session at any point (though this is not sufficient for dealing with motion sickness; see the next point). More so than simply thinking about consent, consider the wellbeing of your players: VR does come with a risk of motion sickness, or other types of negative effects when coming out of sessions (such as the feelings that some have labelled ‘VR hangovers’). Try to ensure that your players recognise these potential effects when they take part: players may not feel comfortable driving home after a playtest for example.
Dealing with motion sickness and discomfort: You should also enforce regular breaks and watch for signs of discomfort. Players may claim to feel fine when you check in with them verbally, but be in the early stages of motion sickness. Sometimes this is because players do not recognise the signs, but other times they just want to play on through the feeling! Often, by the time players are at the point of actually raising verbally that they feel unwell, they are on their way to experiencing medium-to-severe motion sickness. Try to recognise the signs and stop them before they get to that point. And ensure that you have sick bags on hand, as well as cleaning materials for dealing with the aftermath!
Hygiene: VR can be a very physical activity. Headsets and controllers can get sweaty, and there’s nothing worse than handing a player a headset with a damp foam eyeguard from the last session. Various types of eye protector are available for players to wear under the headset, and you might wish to emphasise that these are for the player’s benefit rather than for keeping the headset itself clean. Use antiseptic wipes on controllers and headsets after each and every player.
Tech setup: Every VR system has its own technical challenges to monitor and record what the player is doing. Think aloud techniques can’t be relied upon alone, as the player may not be able to accurately describe what they are doing, and it may distract from the experience. The researcher(s) need to be able to monitor and record what the player sees and potentially hears. While all the current systems have different technical challenges here, most thankfully have a means of monitoring the visual and audio signals, whether on the hosting PC or via a casting solution (as with the Oculus Quest). In addition, use multiple cameras to record the player’s motion in the full physical space from multiple angles; VR allows players to turn around, so it is crucial to be able to see their hand and arm gestures from more than one viewpoint. You may also need multiple room mics (or one more sensitive mic). Use streaming or recording software — such as OBS Studio — to combine these various audio and visual signals, though this may also require you to purchase extra hardware to bring all these signals into the same device for capturing.
Know your hardware: More so than in other forms of usability testing, it is essential that you know the ins and outs of the hardware (and associated software) that you are using. This includes the built-in safety features, how the hardware responds in terms of crashes, how to activate (and cancel) menu screens and how to correctly attach, remove and hold controllers.
Habituation: VR is still new and exciting to many players. Consider having a short game or experience on hand to get naive players used to being in VR for five minutes before loading them into the experience or game you want them to test. Even a short intro experience — such as the Oculus First Contact — will be enough to get players over the initial ‘wow’ factor of being in VR. To standardise the experience across all players, you may wish to follow the same protocol even with players who claim to have more experience: putting all players through the same experience to begin with gives you a good opportunity to gauge VR experience.
Interactions: Conducting a VR usability test can include some hands-on moments — players may need help putting on headsets, or with holding and putting on controllers correctly. You can make players feel more comfortable by letting them know what is involved at the start of the session, and always checking with the player in advance when you need to do something that involves physical contact, such as helping to put on controllers (this is an especially important time, as they may well be already wearing the headset by this point!).
Communication: You will need a way of speaking to and communicating with the player in VR. Some players can find themselves highly immersed when in a VR game or experience, meaning that it can be hard to get their attention (and a tap on the arm can be an unpleasant shock!). Ideally you will find a way to mix in an external microphone signal to what the player hears: this is easily done if using a PC-tethered VR solution, but less so with untethered VR. Until a better hardware solution is reached, consider ‘training’ players to listen for an external sound; you could even ask them to wear a wireless ear bud under the headset earpiece that allows you to speak to them.
What about VR experiences?
Experiences — usually shorter, one-off pieces of VR content, sometimes but not always with interactive elements — are other common uses of VR. They’re not just for home consumers (though there are plenty for them): they’re used in art installations, marketing and promotion, and learning and education. What are the differences in VR UX for games and experiences? There aren’t any really. An experience needs to be immediate and barrier free, and needs UX research as does a VR game.
