Intro to Moderator Guides
An introduction to moderator guides that outlines what they are, what goes into them, and how to make them work well.
Preface
Since I started learning User Experience Design two and half years ago, I’ve seen many different types of moderator guides. I hope what I’ve learned helps you as you start creating your own.
I’ve included a list of resources at the bottom of this post.
Definition
A moderator guide is a document created to conduct a research session.
The content of a guide will vary depending on the type of session it’s made for. Some are scripts with step-by-step instructions, while others consist of topics and questions.
Session Variations
- Usability Test (task-based)
- Interview (question-led)
- Discussion (question-led or topic-led)
- Focus Group (guided or open group discussion)*
*This type of session is usually done for market research, which is different from user research.
Note: We will focus on user research moderator guides in the rest of this post.
What a Guide Does
Before crafting a moderator guide, you should have a research plan. A research plan helps shape the moderator guide. It includes your research question(s), goals, assumptions, and other important information.
A good moderator guide should help the moderator do the following:
- Answer the research question(s)
- Instruct the participant
- Make the participant feel comfortable and safe
- Keep track of the session
- Ask open-ended, non-leading questions
- Go through all tasks needed to get the necessary data
- Acts as a controlled variable to ensure reliable results
A good moderator guide should also help observers follow along.
What to Include in a Guide
Below are different things I’ve found myself including in moderator guides. This is by no means an extensive or prescriptive list.
Ultimately, a guide should help you conduct the session so you can answer your research questions.
- Checklist: Include a checklist in the guide to remind yourself of things to do at the beginning and end of the session.
- Goal of study: It’s good to have a reminder of what you’re trying to learn.
- Research questions: These differ from the actual questions you ask participants. They are the questions you want to ultimately answer with the qualitative data you gather in your interviews.
- Time limits: Keeping track of time is important. You only have a limited amount of time with participants and you want to make every second count. I sometimes add time limit for tasks and sections to stay on track.
- Introduction: Introduce yourself and any note-takers present. Always let participants know if others are watching the session.
- Forms: You may need to ask participants to sign a consent form and/or a non-disclosure agreement depending on the nature of the session.
- Background: Be upfront about what they’ll be doing in the session and give instructions. This will let them know exactly what you expect from them and it’ll make it less nerve-racking. P.S. Let them know if you’re using a script so they don’t feel ignored when you look down.
- Test Subject: Make participants comfortable by letting them know you’re not testing them.
- Participant Questions: Give participants a chance to ask questions. It’s a chance to eliminate confusion. You may even get answers to some of your questions.
- Recording Permission: Always ask for permission to record. Explain what you’re recording and how, along with who you’ll share it with. I always ask participants to state their name and confirm I have their permission to record once the recording starts.
- Questions: Ask warm-up questions to get the participant comfortable. The questions you ask in the beginning can help you later on in the session and give you additional context. Start with an easy question and lead them to tell you a story about their experience.
- Tasks: Give directions for any tasks in your guide. Even if you give instructions at the beginning of a session, remind participants of what you expect them to do (e.g. think aloud, read aloud, explain what they expect to happen, etc). They will forget.
- Tips: If it helps, include reminders throughout your guide to ask follow-up, open-ended, and non-leading questions. Your guide is a tool, use it.
- Check-in: Remind yourself to ask participants how they’re doing, especially if the session is long. They may need a break.
- Wrap-up: Inform participant that the session is about to end. This is a chance to ask questions about the session and get more insights. People usually remember the most frustrating things about their experience.
- Recruit: Invite them to take part in future studies and ask them to tell their friends. This is a chance to grow your participant network. Depending on the audience, you may need to get your employer’s permission to ask this.
- Honorarium: If there is compensation, let them know how and when they will get it. Most people are too bashful to ask.
- Thanks: Always thank participants for their time. Even if you’re compensating them, their time is precious. Tell them how their insights will help — doesn’t have to be specific (e.g. “Your insights will help us improve the design”).
More on Check-ins & Breaks
Being the subject of a research session is exhausting. Be kind to your participants and let them take a breath.
Check in with participants about their comfort level.
- Do they need a bio break? Show them where the bathroom is.
- Are they thirsty? Offer them refreshments.
- Are they feeling overwhelmed? Take a step back and work your way back into the difficult topic again.
- Are they cold? Are they overheated? Adjust your air conditioning system or get a blanket.
- Are they uncomfortable from sitting too long? Let them stand up and stretch or consider doing shorter sessions.
It’s harder to concentrate if you’re uncomfortable.
It’s important to consider your audience and the content of your study guide. Is your study accessible and friendly to people that need accommodations?
I check in with participants throughout a session and include a break in my guide if the session is longer than an hour.
Check in with participants about the time.
Discussion and task time varies from one participant to the next. It’s your responsibility to keep things moving forward and finish the session at the promised time. This sometimes means you have to make a choice.
It’s important to respect others’ time. Your participants have other things to do.
A check-in gives you the chance to ask the participant if they can stay longer than scheduled. Be prepared to compensate them for the extra time. This extra time shouldn’t go over 10 minutes.
Testing Your Guide
Preparing for a study requires many steps. A moderator should test the guide with plenty of time to make changes before the first session.
A participant should never experience the first draft of a guide.
Approaches to testing your guide
There are different ways to improve a moderator guide. A combination of the approaches listed below is best. Number four is always recommended.
- Repurpose an old guide: Using an older guide gives you a head start, allows you to focus on the content of the session, and helps you refine your style.
- Read out loud: Reading the guide out loud allows you to catch odd or confusing phrasing
- Share for feedback: Team feedback offers a different perspective and makes it easier to identify improvement opportunities.
- Dry Run*: A session rehearsal gives you a chance to practice and test the session environment and flow.
*A rehearsal, also known as a “practice run,” of a real session with a fake participant where observers take notes and give feedback on the session plan, guide, and moderation technique.
Other Things to Think About
No two guides look the same. You know best what you need to have on the guide to reach your goals. Make it work for you.
Audience
Consider who’s going to use your guide and how. I’m usually not the only one who uses my guides:
- My observers use the moderator guide to follow along during the session, so they need to be easy to understand — they can’t be a selfish document.
- Sometimes I’m not the only one moderating in a study.
Have a Backup
It’s important to have someone else who’s intimately familiar with the guide you’ve put together. You never know when you’ll need someone else to take over the role of moderator.
Things happen.
Use Your Own Voice
While I mentioned that it’s important to have a guide that anyone could pick up and use, you don’t want to write a guide that doesn’t sound the way anyone speaks. It’ll make you sound like a robot.
Reading your guide out loud helps keep it human.
Don’t Be Too Strict
A guide is there to help you. Don’t let it be what holds you back.
The more comfortable you get with moderating sessions, the easier it is to improvise and go with the flow.
Sometimes a session doesn’t go as planned and following your script won’t get you the information you need. Be prepared to be flexible. Join your participant where they’re at.
Remember, the priority is answering the research question(s) and your plan is only one of many possible ways to do that.
What Do You Do?
You may find yourself doing a combination of the things I’ve mentioned here.
What other things do you include in your guides?
Resources
Looking to learn more about planning and conducting user research? Check out the books and articles listed below.
Articles
First-Time Usability: Moderating
Tips on creating and conducting your first usability test.
medium.com
Books
Thanks to Jessica Ivins and Graham for reviewing this article.
