Everything you should know to conduct a kickass user testing
Recently the number of apps in android and iOs market exceeded 8 million, not to forget an unlimited number of website and web-apps popping up from every corner of the world. With the kind of volume, the most scarce element is — attention span of the user.
The most challenging thing in UX design is to capture attention span from users.
This poses a challenge for product owners and UX designers to design an app to capture the attention and engagement from users.
Why is user testing critically important?
One of the very efficient ways to ensure better conversion and engagement is to perform “user testing” (Or more likely, usability testing). It’s a scientific way to find out what users easily figure out and what are they supposed to do. User research has been thoroughly discussed by many design leaders including IDEO, frog design among many others.
We observed some of the leading teams who design experience for global audience catering to millions of users and have drawn few quick learnings that you can use in your next user research:
1. Know what are you expecting
Even before you think about user testing, you should have an objective in place. This way to set up this objective can differ if you are solving a design issue on existing app or validating a new design/flow on the upcoming app.
- For an existing app/website : You get observations on design challenges from Google Analytics, Mixpanel etc. As a solution to this problem, you are user testing on current challenges while exploring other ways to improvement. For example: on an ecommerce app, users are able to add items to cart section, but they are going back to item details instead of proceeding to checkout.
For existing apps, improvements in business results is key objective of user testing.
For example, getting a better conversion at the cart to checkout section on existing app can be the key objective here.
- For an upcoming app/website: In an upcoming app, you make certain design hypothesis including brand guidelines, design pattern etc. For example: in case of an music app, you believe tabs options for different genres of music, instead of genre filter.
In upcoming apps, primary objective can be to validate the design hypothesis.
In our example, we want to validate if users are using tab option for discovering music genres instead of using filter option.
Setting up objectives at the beginning of the test will help you ask right questions.
2. Recruit your test users very carefully
Once you have decided the objective of the user testing, you need to test users to perform user testing. As an established product, you already have drawn the user persona and you have few “real” users who love to see new stuff coming. You can run the test with them.
Though it’s easy to run the test with existing users but carefully remove biases from their existing experience.
To reduce bias from existing experience you may want to create full version of the app to make the entire experience more natural.
You might want to draw a visualization to relate closely with your user (Checkout building user persona in Airbnb usability testing):
However, in case of a new app, you understand the problem you are solving and for “who”. Narrow it down as much as possible. Some of the questions to ask while picking a sample users for user testing:
- Relevance: How critical is your app for test-user?
- Frequency of usage: How frequently this user will use the app in his job to be done?
- Demography: Cultural differences are a major factor in user experience. Focus on one location, age and gender when picking pool of sample users.
- Income range: Are these users in right bracket to pay for your app/service? This includes both, financial capability as well as buying habits.
Anything else comes to your mind? Let us know.
3. When is the right time to perform user testing
“I am waiting for final interface to perform user testing ” — Very often product owners involves User testing in process when they have final version of the interface designs. This is not a very good idea, because later you iterate most time and efforts you invest.
Iterate as early possible in design process to save time and effort
Here’s what to expect at different stage of user testing:
- Idea to hand-drawn sketches: Discuss ideas on pen & paper or whiteboarding. Take snaps of hand drawn sketches put them together in CanvasFlip and invite internal users to test them out.
- Hand-drawn sketch to wireframing: At wireframing stage, you have most formatted version of user interface. Icons and labels are in place. It’s a good idea to show it to external audience at small scale.
- Wireframes to early mockups: Once you have mockups in place, it’s good to start putting this in front of actual audience for first-hand experience. You can iterate and work towards the final User interface based of the user feedback on this stage.
Most popular teams, use a UX analytics tool like CanvasFlip — to create interactive UX-flow ( prototype) and generate UX analytics like User videos, Conversion funnel and interaction heatmap.
4. How much you should be involved in user testing
There is always been a trade-off between creating real-time test scenario and observing user from as close as possible.
When you are observing users closely, he/she becomes conscious and makes more selective decisions on the app compared to his natural decision making process.
Based on level of user engagement there are 3 way a user testing can be done:
- Moderated (aka On-site testing): In a moderated user testing, you are physically present with users. You ask to perform task, ask questions and observe responses. Useful in early stage of design development.
- Remotely moderated (aka remote usability testing): In remote moderated testing, you can connect over Skype, Google Hangout, Zoom or Go-to-Meeting etc. You can ask task to perform and observe response. General practice is to keep interaction as minimal as possible. Useful at wireframing and mockup design iteration.
- Unmoderated: As the name suggests, an unmoderated user testing is done with almost no interruption. Users are given a design prototype and asked to play-around. ( Or sometimes given a task to complete — Example: Buy/Search an item.)
5. How to analyze user testing results for design iteration
Once you have done user testing with your current design iteration, you are done with the easy part. What comes next is even more critical and complicated — analyze the user testing results , figure out takeaways and decide how much these results should impact current designs.
Analyze User Testing results at early stage
At early stage, preferably moderated user testings are conducted. Results of moderated testings are very subjective. These are used to understand user expectation and behaviour, build the design hypothesis based on it. Once you understand features, design patterns, navigation options, iconography and font/label preference, you can draw framework for rest of your design cycle to follow.
Analyze User Testing Results at later stage
When you are conducting user research in the later stage of your design process, it is to validate your design hypothesis that you had establishes earlier. These hypotheses are established in form of “Conversion in UX-flow” or “CTA validation” in screen designs. For example: you want to understand how many users are dropping off between cart to checkout section. Or, How many users are clicking on “Login” CTA compared to rest of the screen.
Since you already have a fair understanding of user behaviour and expectation, you can preferably opt for remotely moderated or unmoderated user testing.
Here’s how CanvasFlip User Testing help you iterate your product designs…
Remote usability testing tool like CanvasFlip can enable lots of actionable insights on user testing (really sorry for the plug):
Know where users are dropping off in your UX-flow:
In above conversion funnel (generated with CanvasFlip.com), we can observe that there is huge drop-off in 1st iteration at “Experience” screen.
Where users are interacting most within the interface:
An Interaction heatmap generated on user testing with CanvasFlip.com helps you understand the frequently interacted areas within your user interface.
Analyze user journeys of individual users
At CanvasFlip.com, you can also analyze navigations followed by an individual users and how much time did they spend on each screen.
Watch full-length user session videos to analyze and improve user behavior:
You can also record navigations followed by your users and analyze them later to figure out where were they stuck and how can experience be improved.
Final Words
In your efforts to achieve a great experience, user testing is critical and for a successful user testing you need to know what are expecting/validating, choose your test users as narrow as possible, conduct user testing as early as possible and use actionable insights to draw conclusion from your user testing outcomes.
If you enjoyed this piece, please click the little green heart. Otherwise, drop a comment on how does your User testing process looks like.
Hope it helps you do an awesome user testing next time.
Happy user testing!!
