Create Better Products by Using Remote Research Sprints
How to remotely validate product designs through user research?
Remote user research
User research is done to help create a good user experience of a product by placing people at the center of the design process. There are many different user research methods that can be used. In this article, we are focusing on remote user research, and more specifically on design validation through prototype testing.
What are the reasons to conduct user research remotely?
- Remote user research can be more affordable as you don’t need a lab, transport tickets, or expensive software to run it. Also, if your team is already spread out working remotely it’s much easier to organize remote research than to organize on-site research at a specific location.
- Reaching out to specific users who may otherwise be neglected is easier than when doing in-person research sessions. This way, your team can be exposed to people in new locations who can challenge the existing assumptions.
- Booking meeting rooms and scheduling meetings with participants is a bigger time commitment compared to running remote research sessions.
- The idea of having an interview from the comfort of your home — without the necessity of going to an office — can be a great motivator for both organizers and participants.
A design sprint
A design sprint is a technique that can be used to figure out if an idea is worth trying out before doing a full-scale launch. It’s a rapid way to test out the market without having to commit directly to product development.
In design sprints, team members usually create a prototype, make customers think it’s the real thing, watch their reaction, get feedback and iterate or pivot. It’s been used by companies like Slack, Nest, and many others. Typically, it’s done over 5 intense days starting from Monday to Friday but can be re-adjusted depending on what is being tested, how much time and which resources are available.
The remote research sprint
The remote research sprint is a combination of these two concepts. The method that I am using consists of these 6 steps:
- Understand the problem
- Generate ideas
- Design the solution
- Recruit participants
- Test the prototype and interview participants
- Summarise
1. Understand the problem
Decide with your team and make it clear exactly which problem is getting solved. Clarify what is the goal of the sprint and sketch the most important user story. Consider choosing and focusing on only a part of the problem, if the initial problem is too big, or even expand a bit if the problem that you are trying to tackle is too small.
Look at existing products and think about how your solution will be different. Think about what is unique to the problem and empathize with the user to help define the problem.
Write down notes and document everything you learned so that later when you are in “creative mode” you can quickly get back to that information without doing research again.
2. Generate ideas
The goal of this step is to ideate. We focus on creating different and diverging ideas, exploring unknown paths and asking questions that “shouldn’t be asked”…
Some of the popular methods are for example 5 Why’s, Crazy 8-s and Storyboards.
- 5 Why’s
5 Why’s is an easy and quick brainstorming technique which, through questioning, aims to ensure the problem is examined properly. It helps to get past the symptoms of a problem and to find its root causes. First, you define the problem and then you ask the question “Why?” 5 times. After 5 questions you will get to the root of the problem and then you should devise an actionable plan.
- Crazy 8-s
In the Crazy 8-s activity, you sketch an idea in 40 seconds then sketch another in 40 seconds until 8 ideas are sketched. A lot of the end results aren’t useful but it helps break through plateaus in creativity. This activity leads to a lot of ideas, so it’s focused on quantity rather than quality. Rapidly iterating triggers the creativity so as an end result there’s usually an innovative result.
- Storyboard
Stories are an effective way to capture and explore experiences in the design process. They are a powerful way to explain and visually predict a user experience idea. You don’t need to know how to draw well to create them. It’s only important that your story has the main character, a scene where the story is happening and that the plot of the story develops in clear and understandable steps. You should also try to capture the emotions of the characters so that viewers can empathize with the story.
3. Design the solution
In the third step, we converge our many ideas from the previous step into one that we want to test with the market. We create the minimum amount of designs, interactions and the user interface that is needed to test the idea. Then we connect individual pieces together to form a prototype that will be tested.
Along with having the prototype, it is important to prepare a good interview plan and to set the expectations and goals.
An interview plan usually consists of these sections:
- Introduction: At the beginning of the interview, the participants can be nervous and unsure about what to expect. It’s best to put them at ease by establishing rapport and giving them an idea of what you’ll be doing and what’s expected of them.
- Context questions: Fight the urge to jump right to questions about the product or prototype you’re testing. Start by asking the participant open-ended questions about their experiences, current behavior and habits, and the problems they are facing. The purpose of this part is to better understand the interview participants.
- Task testing: Make a list of the important features and tasks in your prototype. Which ones are critical to test in this research sprint? You can also think of these as the specific parts of the prototype that you want users to visit during the interview.
- Follow-up questions: After each task, you’ll want to ask follow-up questions. This part of the interview is normally improvised.
- Debrief: Ask the participant to reflect on the prototype and its usefulness. Ask them about what they like and dislike, and how the product might or might not fit into their life.
Take a few minutes to add a time budget to your interview plan as time can fly by during an interview. Then assign time limits for each section and task. In the end, give your interview plan the last check and remove if there are any “leading” questions.
4. Recruit
Usually, it helps to filter the potential interview candidates by using a recruiting screener. To create it, start by listing the characteristics of the people you want to interview (you might look into creating user personas if you don’t already have them). Then figure out the criteria that you can use to identify those people. When using survey screeners, it’s important to write questions that don’t reveal the right answers, as some people will try to game the survey to get the financial reward.
The second step is to post the recruiting screener. If you’re recruiting “normal people” who aren’t familiar with your company, you can find them on Craigslist, or a local equivalent for your target country. Some other places where you can look for participants are LinkedIn groups, Facebook groups, forums, Quora, Twitter, Pinterest, Facebook ads, Google ads.
If you want to interview people who are interested in your company (or at least familiar), you can post a link to the screener on social networks or anywhere else you have an audience.
For example, if you are targeting US participants, you could post an ad in Craigslist with a link to the screener:
Title: $100 usability interviews on 28th of April for Company X
Description: I’m scheduling 60-minute usability interviews on Monday, April 28 for Company X. Selected participants who complete the interviews will receive $100 Amazon gift cards. Please complete this short questionnaire.
Participants will:
– Be willing to sign our standard confidentiality agreement.
– Allow us to record the session.
Figure out how you can incentivize the participants to spend 30 to 60 minutes with you. For acquaintances, you can offer coffee or lunch. For strangers, offer gift cards (for example Amazon, Visa or Airbnb gift cards).
Once you’ve narrowed down the list of potential participants, the final step is to select and schedule participants. Select five or more candidates who fit your criteria the best and call them.
After the phone call, follow up with an email to confirm the time, date, and location of the interview. Remember to include a link to a non-disclosure agreement that people can review and sign ahead of time.
Make sure that each participant confirms the interview and set an event in your calendar a day before to remind each participant personally.
5. Test the prototype
Plan to test the prototype with at least five participants. Interviewing five customers is usually the best return on investment (testing with less doesn’t provide adequate results while testing with more increases the costs while maintaining similar results).
Review the prototype with your team and prepare your team to watch the interviews live or if not possible, at least provide them with a way to access the video recordings.
When you conduct several interviews in a day, your process and surroundings don’t change. This can cause all your sessions to blur together and recall can be difficult. It is best to record the calls so that you can refer to them and distill the findings. Recording user sessions means you don’t have to rely on your own memory to reflect on learning. Don’t forget to get permission at the beginning of the call.
Here are some suggestions for the user tests:
- Make sure to test devices and recording system before the interview.
- Try to build rapport. Make small talk and don’t just jump straight to the prototype.
- Let participants figure things out on their own. Don’t explain the product and better answer questions with more questions.
- Ask who, what, where, when, why, and how questions. Ask follow-up questions.
- Watch the time and check if you are complying with the time budgets that you planned.
Also, don’t forget to:
- Be quieter and let the participants do the talking. Your goal is to mainly stay silent and let the participants speak.
- Watch for non-verbal cues and react accordingly… Are the participants nervous, excited, or annoyed?
- Pregnant pauses are also effective ways to draw people out. This means you can try using a pause after or during your sentences to let the participants reflect on what was said and to provide them with enough time to find the right words.
- Ask for specific, recent, personal examples — instead of theories and projections about other people.
- Go overboard in expressing appreciation for their time and input. “Thank you so much! This has been incredibly valuable for us.”
6. Summarize
It’s time to celebrate because you came to the end of the sprint.
Even if you received negative feedback on some of the designs, you learned from it, which was the goal of the method.
If you didn’t learn anything, then you might reconsider if you followed all of the steps and if there is anything that can be changed in possible future sprints.
Now it’s time to summarize your findings and plan the next steps with your team. Update the prototype to fix problems, create a new higher-fidelity prototype, or decide to focus on a new set of questions and assumptions to tackle.
The tools of the trade
Which tools do I use most often? Here are my favorites:
- For video calls with screen sharing: Google Hangouts
- For recording the interviews: Skype (with recording turned on)
- For scheduling interviews: Calendly
- For sending out surveys: Typeform
- For prototypes: InVision
- For non-disclosure agreements: Sample NDA contract
- For online signatures: HelloSign
Conclusion
So there you have it. A complete easy-to-follow process for running remote research sprints!
Even if you don’t follow this plan completely, just by running research sprints before designing products, you will save a ton of money which would normally be spent developing a suboptimal product. Whatever you decide to do, just don’t forget to start talking with the users from the beginning.
Alright! Let me know how it goes and if you have any suggestions, just shoot and I’ll be glad to update the article. Happy testing!
