Effective Solutions Require Digging

Without asking the right questions, you‘re just wasting time

Design Thinking, Lean Startup, Design Sprint, Agile

Debating about the various research methods and activities is a popular past time among designers and product teams. Even among those that are data driven — evidence-based, no one seems to be talking about what makes a good or appropriate guiding research question. A common assumption is that research questions are just what you ask in a survey or user interview, and that you can ask whatever you want to know directly. The truth is no… If you don’t start from the right research question, you’re probably just kicking off dust and obtaining more pebbles in your shoe than before; obscuring more than you’re clarifying.

Questions give data meaning

Without a good question, all the data in the world is as useful as a random page of web search results and no query. Many well-meaning teams make a regular habit of talking to their customers and then don’t know exactly what to do with what they’ve found, except argue about it, or build features based on personal accounts requests rather than facts or research.

Your research question is simply what you want to find out in order to make better evidence-based decisions. However, it takes some care to identify and articulate the most useful question.

A good research question is specific, actionable, and practical. This means that it is possible to answer using the techniques available to you. If your question is too general or beyond your means to answer, it’s not a good question.

Research questions are not interview questions

Your research question is what you need to know to make better product decisions. Your interview questions are what you ask other people in order to learn what you need to know. If you have a strong research question in mind, you might not even need specific interview questions.

For example, if you want to a set up a healthy delivery meal plan in school, you might need to know how to get recent college students to subscribe to your service. But you can’t just go around to recent college graduates and ask “What would get you to subscribe to my food delivery service?” People are good at fabricating answers to give off the best first impression, especially during interviews or dating.

Research question: How do recent college students decide what to have for lunch?

Problem Statements into Research questions

Digital innovation starts with a problem worth solving. It’s amazingly helpful to have a single statement that represents the nature of the problem, who’s experiencing it, and why it’s so important to solve. Here’s one of a way you could refined the problem statement by turning it into a questions, a framework by IDEO — ‘How Might We’ and it’s best conveyed by Andrew Lovett-Barron.

Although being able to execute high quality design is an essential skill to have, what actually needs to come for a good designer is the ability to define the problem we’re trying to solve thoroughly and effectively.

• What problem are we trying to solve?
• How do we know this is a real problem?
• Why is it important to solve?
• Who are our users? What are their goals and motivations?
• How will we know if we’ve solved the problem?

Yes, it takes more effort to identify what you really need to find out than it does to write a set of the wrong interview questions. It takes collaboration and a clear shared goal and a willingness to be proven wrong.

And, by the way, the best interview question is often just “Tell more about that.” and “What do you mean by…”

A peek into the process

The applied research process — using inquiry to solve a specific, practical problem — is not without nuance. These are three essential steps. 1) Form questions 2) Gather data 3) Analyze data to determine what it means.

You cannot skip any of these. Research for product design is too often treated like the inverse of the underpants gnome scenario. Everyone focuses on Step 2, the accumulation of data, without a clear question or insufficient analysis. Given a pile of data without discipline, biased human reasoning sees patterns and draws conclusions that don’t reflect reality.

Goals before questions

Before you can form good questions, you need to have a clear goal. Your goal could be something very general, like “make money by running the dietary food delivery business”. Or, it could be much more specific, like “increase the number of subscribers by 8% in a 2-month period”

Without a goal, it is impossible to identify the highest priority questions, or evaluate whether the research you did was useful and worthwhile. Did what you learn help you make money solving rich-people problems? Then, great. This is why a lot of organizations follow the process of 1) give new types of research a try 2) find it vaguely unsatisfactory, then 3) go back to making stuff without asking as many questions.

Parking Lot — Prioritize the most important question

There are so many things you could ask. How to choose? The best question is the unknown that carries the most risk. So list the questions down onto a post-it note and stick them against a t-diagram that has Risk on the y-axis and Importance on the x-axis.

We seek to identify:

• Items that must be addressed now (AND so we do NOW)
• Items that need to be address but not right now (so they remain in the Parking Lot)
• Items that no longer need to be addressed, for example done (so we remove it…)

Purpose? The Parking Lot helps to track important items, ideas and issues that may not be useful to discuss at a time in the agenda.

Question everything

Don’t rush to test hypotheses before you ask more open-ended questions. Otherwise you could miss the big picture.

It’s tempting to narrow too quickly in the interests of being “lean”. And ending up with the dreaded local maximum. The goal is learning, always learning. Questions before methods. Research methods are simply ways to answer questions. There is no one best method, just the most appropriate for your goal, question, and available resources and expertise.