Remote usability testing tools
A guide (and infographic) for moderated research.
At People for Research, we often get asked by clients to recommend remote user testing tools and support services. As you might expect, we come across a lot of different tools while working with our wide client base. Because of these requests, we have decided to put together a guide containing the tools we see used most frequently to run moderated user research such as remote usability testing.
Lookback
Lookback claim their remote user testing tools can be used on any device, as well as used for self-moderated testing and for lab-based research. It is built for use in research, unlike many of the other tools which frame themselves as video conferencing or communication tools.
Pros: It seems reliable provided you have a good internet connection, as recordings are sent straight to the cloud and stored in Lookback. This remote user testing tool can be used to test websites, apps, and prototypes.
Cons: Participants have to download an application or piece of software, which be a constraint.
What the experts think about Lookback:
Laurence Tempest, UX Architect
“Lookback is roughly like GotoMeeting, but optimised for user research in the way it guides users and the features it provides for note taking and sharing with stakeholders. You can set up research sessions and send out a link to participants ahead of time. They’ll need to install a browser plug-in, be using Chrome, have a decent internet connection, microphone/speakers and preferably a web cam (works well on mobile too)… If you can guide the participants through all those hoops, you’re golden! To be fair, most of these are hoops for any remote research tool. As with all remote tools, I’ve found that you need to be ready to fall back to plain old telephones pretty quickly. I have started to have a pre-session phone calls with participants to talk them through all of this stuff, which helps improve my success rate! You get a separate link for observers to watch the video in real time. You can also have participants run unmoderated sessions, so assuming you have a decent test plan, this scales really nicely. The app captures user’s screen and webcams and allows you to tag the timeline with your notes, which is a great way to collaborate with stageholders even if they are remote.”
Zoom
Zoom is a video and conferencing service that seems to be increasingly popular with UX teams. It is more user-friendly and nicer to look at than some of the other video conferencing tools out there.
Pros: Easy to interact with the user interface, it is free to sign-up, and the paid for plans start at a reasonable price so there are minimal barriers to entry to begin with.
Cons: Again, it involves downloading an application or piece of software, which can deter some participants from getting involved or make the whole process more complex for the user.
What the experts think about Zoom:
Konrad Black, Principal Experience Consultant at Edo
“We’ve recently switched from GoTo Meeting to Zoom, but are finding issues with using Zoom. Setting up calls can be tricky. We’ve found the meeting ID people get can differ from email to email. This has led to participants being on a call waiting for the other participants to join, even though everyone should be in the same call. Again, this leads to frustration and setup issues and delays. You need an account and to have registered, which makes setting up a spontaneous chat quickly on the fly difficult.”
Adam Babajee-Pycroft, Managing Director (UX) at Natural Interaction
“Anyone who works in an office has probably had an attempted video conference before. The first 15 minutes are spent asking if people can hear you. Individuals disconnect and reconnect, seemingly at random. Zoom is a video conferencing tool which just works and doesn’t do any of that. That’s why we use it to run hundred of remote research sessions every year. We love it because even the least tech savvy users rarely fail to turn up.”
Appear.In
This is a tool to facilitate remote working, but some clients have used it for running remote research because it is easy to use. They claim to be building video conferencing for the future. Again they claim their tool can be used on any device, and they have also recently introduced a Trello integration.
Pros: Nothing to download when being used through a web browser, there is no registration process and they have a free version of the software.
Cons: Because it’s not a remote user testing tool, it doesn’t appear to offer integrated recording, so you will need to find your own solution for this. It uses WebRTC protocol, which doesn’t handle slow connections particularly well. Also, the user has to have specific browsers installed to be able to use it.
What the experts think about Appear.in:
Emma Howell, Senior UX Research Consultant & Research Lead at cxpartners
“Appear.in is good for testing and we are starting to use it more. Big bonus with appear.in is you do not need to sign up or faff around. Literally pick a name for your meeting and you’re off. In fact, I used it recently when a participant was struggling with GoToMeeting. Downside for testing is that (I don’t think) you can give control of your mouse to someone else; unless I haven’t discovered that feature yet!”
Ben Bywater, Independent User Researcher and UX Consultant
“Gloriously simple. It’s just a shame it is not compatible with certain browsers.”
Konrad Black, Principal Experience Consultant at Edo
“Very simple to set up a video chat and invite others to join. Requires no registration; joining is very easy — simply click the link. It’s free and doesn’t require any installation / downloads, though does require access to your camera and microphone. We’ve found Chrome to be the best browser to use with Appear.in, it does struggle with Internet Explorer. We’ve found Appear.in to be the most stable of the lot, and quality of video calls is generally very good.”
Skype & Google Hangouts
We have grouped these together, as often when clients want to use one of these tools we ask them to offer both as options to participants so as not to limit the recruitment to people who already have accounts. Both services seem to be a little unpredictable regarding the reliability of the connection. We usually check that participants have broadband and a good connection speed before recruiting, but invariably there will at least 1 session per day that experiences a poor connection which delays the research.
Pros: Many people already have access to these services and are familiar with how they work, so there is less trepidation from participants when asked to use them as remote user testing tools.
Cons: Skype has to be installed by its users, the connection can be unreliable and the software doesn’t “natively support call recording”, so you will need to make your own arrangements if you want to record calls. It looks like the same is true for a private Google Hangout. Less tech-savvy users tend not to know their Google password can be used to take part in Hangouts. Also, just like Appear.in, Hangouts uses WebRTC.
What the experts think about Skype & Google Hangouts:
Emma Howell, Senior UX Research Consultant & Research Lead at cxpartners
“Skype is faffy! Not everyone has it installed and it’s not reliable enough.”
Konrad Black, Principal Experience Consultant at Edo
“Skype is ok, it does however require the user has a Skype account and the app installed on their device for it to work. Similar with Hangouts, the user needs a Google account and actually joining a Hangout can be tricky. Using Skype and Hangouts does usually result in a delay in getting the session started because of setup teething issues. Generally we’re on the phone talking to the participant and helping them setup to join the video chat. The result is we either overrun to keep a full hour of research or we cut the research short. Skype has been a lot more flakey and unreliable over the last few years. It does also require the Skype details of the participant to be shared in advance and for them to accept you as a contact. This isn’t great for researchers because details will be remembered in future, which you don’t really want.”
GoToMeeting
There are a lot of other video conferencing tools out there, but this seems to be one of the most popular. Some participants have come across this system through their own jobs, which means they already have the plug-in installed on their computers.
Pros: One of the most popular web conferencing tools. They boast end-to-end encryption, which could be an added benefit with the GDPR in mind.
Cons: GoToMeeting was not built solely for user research so shares many cons with the other video conferencing tools already mentioned. It’s more expensive than the other options if you want call recording, and the mobile version is not available on the cheapest plan.
What the experts think about GoToMeeting:
Ben Bywater, Independent User Researcher and UX Consultant
“It’s just so complicated. I’ve never had a nice time with it and avoid it wherever possible.”
Konrad Black, Principal Experience Consultant at Edo
“This is by far the worst tool we’ve used. Very complex to set up and get going. Interface is confusing to use and people often get lost with using the controls. The quality of the screenshare is ok, but not the best. It’s also expensive.”
Emma Howell, Senior UX Research Consultant & Research Lead at cxpartners
“Multiple ways to join, decent screen sharing and (importantly) can hand over control of your screen to the participant. I like that I can record the screen, but it feels like when this fails, it fails hard! Participants struggling to join a meeting and audio not working are the two biggest frustrations for me.”
As we specialise in the recruitment of users for research and UX testing, this guide should be treated as a starting point when running remote user testing. We don’t ever run the research and use these systems ourselves, but the feedback we receive from both clients and participants speaks loud and clear when it comes to selecting the remote user testing tools that provide the smoothest user experience depending on the requirements of each project.
I regularly share this link with clients, which is a much more comprehensive list of remote user testing and UX tools. To find out more, you can also take a look at the co-blog we did with Optimal Workshopabout their range of remote user testing tools to support user experience research.
This article was originally published on the People for Research blog.
We want to keep in touch. Join our mailing list to receive our newsletter and/or our monthly round-up.
