Reduce your research deficit

Before you try to increase your research budget

UX designers that work closely with developers in the modern software delivery teams are familiar with the concept of technical debt — something that is incurred by writing code or developing software that may solve short term problems but create more problematic issues later down the track.

A recent article on the same issue in mathematics has attempted to describe certain aspects of research debt, and there are parallels in the UX research work that is carried out everyday across the organisations that are adopting a user-centric design process worth exploring.

If you are doing any user research as part of the UX design process, chances are you are creating research debt without even realizing it. The problem with most types of debt is that the longer you allow it to accumulate, the more difficult it becomes to eliminate. And like most types of debts there is also an interest that we have to pay on top of all this. Over time, the cost of the research debt can become such a burden that the value of UX research diminishes very rapidly.

Here are some of the common research debt incurred (in no particular order), and potential solutions:

Complicating the research problem

There are already so many UX related literature, reports, research methodologies and tools available to UX designers than we can possibly process or manage without losing track of the latest changes and developments in this rapidly evolving field. With all the resources at our disposal, or perhaps due to the lack of resources, the temptation to design studies or tests to address complex problems lead to equally, if not more complex results to understand.

The simple truth of the matter is that each factor that you have to consider and each variation you allow for in your research introduces a multiplicative effect in the complexity of the results. When conducting user research, it is more important to delve into the real issues that the research uncovers to gain insight into the underlying cause of the problems, not just aim to provide a statistical significant results, because in the end there is always a degree of unpredictability when it comes to human behaviour (economics is also based on the premise that individuals and societies will act rationally).

Complex problems incur research debt because the effort required for the analysis and the confidence we gain from our findings can outweigh resources spent on carrying out the research. A clear definition of the research problem helps us to reduced a more complex challenge into simpler sub-problems. When translated into research goals that can be measured more objectively, we can be more confident of the results produced or the insights gained rather than worrying about proving the validity of our research design or analysis.

Building on assumptions

Often the first steps involved in any research activity requires making a number of assumptions around the requirements and constraints of the problem. These assumptions are seldom documented, and therefore never validated or possibly neglected because they are assumed to be a valid and previously established ‘fact’. The ‘UX Myths’ that we often hear about and debate with our peers are often seen as general statements made with certain assumptions in mind.

Over a period of time, people accept these assumptions to be true, but when we get to the bottom of these assumptions they are erroneously referred to as myths. At the end of every assumption, there is either a genuine source of truth (because we have understood or recognized the facts that they are built upon) or a trail that leads to a dead end (because of the assumptions that they are built upon).

Building our research efforts on assumptions that are ignored or not validated incurs research debt by introducing uncertainty and risk in our findings. Setting out clear and specific design principles and rationale for your project not only provides a focus for your design efforts, it also reinforces the requirements and constraints you are designing. Furthermore, they serve as a constant reminder to check the assumptions that you are making with each research goal that you set out to achieve.

Insufficient synthesis of research data

Raw data such as photos of sticky notes and sketches on paper are great for documenting the process of deriving at our design decisions and ideas. However, as far as artefacts for sharing and iterating on concepts and ideas go (which is one of the primary outcome for research activities), they are difficult to work with because the information is not distilled.

UX designers communicate constantly with business stakeholders, developers, other designers and end-users. While most of these people do not need to see (and may not care for) the raw data, it is important to communicate your design rationale based on presenting the synthesised data and insights from your research. Basically if your research output can’t be easily shared or understood by the key stakeholders then it needs to be processed and refined further.

Insufficient synthesis of research data incurs research debt by transferring the effort of summarising and translating research output to the users of the information each time they need to access it. Raw and unprocessed data are difficult to manage consistently and efficiently. Summaries, diagrams and storyboards are good examples of synthesising research data into output that translates into clear and actionable design input. The added benefit of clearer and easier communication between team members cannot be emphasized enough.

Referencing existing research

It is easy to fall into the trap of supporting your research with studies or reports that agree with your own findings without a closer examination of the details in other people’s studies. There are many examples of UX design where differences in the context can lead to variations in the behaviour of the users. Just think about how often when we carry out UX research or testing that we end up being surprised by the results because of certain details missed about the user, or assumptions we had to make when coming up with a design.

Referencing existing research incurs research debt by introducing effort required to validate the connection that you are making. Leveraging existing research to support a design decisions can also lead to certain assumptions being overlooked or less rigour in the process applied to the research and testing. When looking for existing research, looking for alternative explanations can be as important as seeking validation of your ideas.

Lack of standards & guidelines

To improve the overall quality of UX research standards and methodology, it is important for the research produced to be communicated clearly and consistently. UX designers have to communicate to themselves, their team members, stakeholders and also their peers. The diversity of the user problems that UX designers have to tackle is reflected in the variation of research artefacts that are produced, yet even for a common asset like personas, it is very difficult to find a common standard across different teams within an organisation, let alone within the UX design community (a quick Google image search should give you some indication of this). Even though many of these artefacts share common properties and content, the differences in format and style introduced templates available online make it difficult for direct comparison.

A lack of standards and guidelines incur research debt on a number of different fronts. It impacts on the ability for research data to be compared and cross-referenced, demands more effort to manage and maintain the research output, takes more time to access and locate the information of interest. Many organisations have already invested in development and design frameworks as part of digital transformation and user-centric design processes. However, many overlook the value of a organisation-wide knowledgebase of their users based on the standardisation of research methodology and output. The more we can engage in conversations around best practices and standards within our team, organisation and the community, the better chance we have of establishing standards that will eliminate the inconsistency of how results are presented and interpreted.

While there is usually a benefit in investing more into research activities, the conversation more often than not is about what type of research activities we should do rather than how we should plan and manage the output. Many of the debts incurred in research often fall under the radar as a result of this, and will continue to undermines the value generated from the useful data and insight of research unless we make the effort to reduce these research debts.