Generalisability in Research – 5 Easy Steps

What is research synthesis?

“Research synthesis is the integration of existing knowledge and research findings pertinent to an issue. The aim of synthesis is to increase the generality and applicability of those findings and to develop new knowledge through the process of integration.”

– ScienceDirect

Research synthesis differs from research analysis. Research analysis considers one text at a time, breaking the bigger concept down to smaller fragments in order to arrive at deductions. However, research synthesis considers multiple, related texts. It is the process of combining the fragmented parts into an aggregated whole.

5 Steps to Excel in Research Synthesis in UX?

1) Establish a baseline

A baseline is necessary to understand what the current state is and what are its associated key metrics. In eCommerce, the key metrics we care about are sales conversion rate and ‘Add to Cart’ rate. Without measuring what these are beforehand, we would not be able to establish a baseline for improvement.

2) Frame your problem statement and success state

“How might we…?”

The cost of not framing the right problem is working on the wrong solutions. I encourage researchers to start with “How might we” statements as it is frame to help you think about solutions and possibilities. It is also worth thinking about what a successful outcome might look like. e.g. How might we help Singaporeans to consistently make healthy financial decisions on a daily basis to reach their individual savings goals?

3) Understand the types of data you may be able to collect

The below is the slide I share with all my students who attended our 2-day UX Intensive Course.

4) Synthesise your data through inference

To infer is to make a logical leap and develop the best possible explanation based on what you know is true.

Affinity mapping may be popular technique to make sense of data in order for you to draw an inference but it is not the only research technique to make sense of data. There are various research techniques for inductive (e.g. affinity mapping and card sorting) and deductive reasoning (e.g. A/B Testing, 5 Whys a.k.a. root cause analysis).

In my design process, I would usually infer from the data and make recommendations for the team. For example, knowing that the majority of car insurance is sold to men in their 30s and 40s with families, I might make a recommendation to the design and marketing team to use this popular customer profile in our hero images.

5) Create experiments to validate your hypothesis

As you make sense of data through synthesis, you may come to a logical conclusion that would require further testing to stand true. A hypothesis stands true only with reproducible results as shared by Sheldon from Big Bang Theory. While working on a global/regional scale, we replicate the experiments in different time periods or locality to see if we get the same results.

Research Synthesis Example

Here is an example to sum it all up in a case below. Inspired by true stories.

You are a social scientist tasked to observe Danny’s morning routine and design solutions around a problem he faces consistently.

Every morning Danny wakes up and goes to work and observes the following: the dishes are left in a mess in the kitchen sink, there are bread crumbs all over the dining table and fruits are missing from the fruit bowl.

Given the problem statement to solve: How might we help Danny wake up to a clean kitchen/dining area and a fully stocked fruit bowl without daily external intervention?

Do you have enough data to brainstorm a few solutions?

• Yes, if you are comfortable with making one key assumption.
• No, if you would like to be precise.

Most of you will take the data and move forward with designing solutions assuming that Danny lives with a partner or flatmate. A crucial missing piece of data that you could have collected by asking Danny (the user).

Now if Danny tells you he lives alone and suffers from a condition known as sleep walking. Wouldn’t that change your design solutions quite drastically?

At this point, do you have sufficient data to design solutions? No, at least I wouldn’t be comfortable to do so if I were you. What else might be missing?

The answer: The root cause of why the state of the kitchen and dining are is what it is. What I might have done in your position is to ask if Danny for permission to place a night vision camera in the bedroom, kitchen and dining area. Given the recordings from the night and an understanding of the root cause of why the kitchen/dining are is in the state of what it is every morning, we can then move forward to design the appropriate solutions.

As you may have noticed, the design process is filled with moments where inductive and/or deductive reasoning are used to help you design the appropriate solutions. Given certain constraints (e.g. Danny might not be comfortable with you recording his apartment in the evening) you may have to make do with the data you have or you may choose to interview and record similar patients to develop a better understanding of sleepwalking; where you can then draw inferences to the root cause.

I hope this helps you with more successful research outcomes that will drive better design solutions in this world. Leave a comment below if this article has been helpful for you.