20 Questions on How To Carry Out UX Research (Part 1)

1. What are the challenges of being a UX Researcher?

• Stakeholder management.
  I think what surprised me when I first started my full time UX Researcher role was how much it wasn’t just about supporting designers or doing the research but also about stakeholder management. This was particularly true for Accenture, a big end to end consultancy with lots of different internal and external stakeholders. I pretty much had to teach them UX at some level as well. But not in a formal classroom training way but more on the go.
• Explaining the importance of my role
  There were a couple of instances where I was put into a project and the client would ask me, “So who are you?” And “Why are you here?” And I have to make it a point to showcase how what I’m doing supports the designers as well as their business outcome. So, that was something that I luckily, quickly picked up on.

2. What is it like working with different organisations?

• Government organisations

This is going to sound super cliche, but I enjoyed working on government projects because I knew that the work I was doing was going to make an impact on the wider population in Singapore. This stemmed from knowing that this design will actually help people get the services they need, and by helping them to get it faster, it will basically reduce some of the red tape.

And knowing that I played a part in that, is something that really kept driving me to want to be involved in a lot of government projects. Even though those are often the most stakeholder heavy.

3. How do you pick an appropriate UX Research method?

The two most common research methods that every UX researcher needs to know are:

1. User Testing – to learn how people actually use your designs.

Scenario:
The company wants to test something that’s already out there and compare it with the competition.

This obviously signals the need for user testing.

4. How do you tell apart an amateur from a professional researcher?

• Their reaction when a user does something wrong

The most common way to tell them apart is the way they jump in during the research. When amateur researchers, especially if they’re the designers too, see a user doing something wrong during the user test, they would feel very uncomfortable. They tend to jump in and try to fix it verbally by saying something like, “No, no, you’re supposed to go here” or nonverbally using their actions.

This results in the user who’s doing the test to realise and correct themselves if they’re paying attention to the researcher. Sadly, the natural reaction of the user required for this research would have been completely lost in such a situation.

Professional researchers are able to balance being a neutral party and building a rapport and connection with the person being tested, especially if the user test is being conducted in a lab or in an unnatural situation.

5. How do you conduct non-biased interviews?

Hence, in order to conduct an unbiased interview, I think something that really helps is to have the person conducting the interview not be someone who is directly involved in the design, if you have that option.

This way, the interviewer does not actually have a stake in how the person responds to the test. It could be someone on the team, but maybe not directly doing the design itself. So a researcher or sometimes in Accenture, it was a business analyst conducting the interviews, rather than a designer or developer.

6. Is it recommended to collect both qualitative and quantitative data?

It really depends on your organisation’s values and what you have access to, because even quantitative tools cost money, right? But if you can have a two-pronged approach, that is absolutely great.

I think the easiest way to distinguish the purposes of quantitative versus qualitative data on a general level is how:

7. How do you deal with respondents who are unenthusiastic and unhelpful?

So there’s two parts to how I approach participants like that.

1. Asking a lot of questions from the very beginning.

When I’m introducing the topic itself I would ask them questions like “What’s going through your mind?” or “How do you feel as we go through this?”, warming them up to the expectation of how they’re going to have to answer a lot of questions.

2. Conditioning them to put their brain on loudspeaker.

I think at the beginning, one way that I like to set up participants in a way that is not necessarily the most natural thing for anyone to simply do without being told to, is to put their brain on loudspeaker.
Here’s what I mean. During the interview, I would give them a moment to think even if they’re completely flat and unresponsive because most of the time, they are thinking at least a little bit about what’s in front of them. Especially if they seem stuck on something, after a little while I’ll say something like “So, I’m just wondering what’s going through your mind at this point of time?”

Or I’ll latch on to a specific behaviour I observe about them and verbalise it in my question. For example, if they’re moving their mouse around a lot but not clicking on something, this usually indicates that they’re searching for something. Hence, I would say “Hey, I noticed you’re looking around a lot on the page using your mouse. Could you let me know what’s going through your mind here?”

When answering such questions repeatedly, they’re obliged to verbalise what is going on in their minds and hence, are putting their brains on loudspeaker.

8. What should you say in your brief before user testing?

Step 1

Invite them in and thank them for their time.

Step 2

Explain a little bit about what we’re going to do today.

“So I’m going to be putting you in front of a prototype for a website that businesses are working on. It’s a prototype, so some parts may not work very well and if you come across such parts, just work with what you can. If there’s something really not working, then I might step in.”

Step 3

Reassure them that there are no implications from their honesty

“One thing I want to emphasise today is that we’re here to collect your honest feedback. I’m not the actually person who put this together as I’m not part of the design team. So, you don’t have to worry about upsetting anybody. We’re really here to hear your thoughts.”

Step 4

Request for their brain to be on loudspeaker

“I’ll give you some tasks, you’ll go through them, and while you’re doing that, it would be great if you could put your brain on loudspeaker. What I mean by that is that if you could let me know what’s going through your mind as you’re going through these tasks, that would be great.”

Step 5

Start the test!

“I might ask you some questions along the way. If you’re ready, let’s get started.”

Bonus – if you have time!

A practice test I like to do before the actual user test is have them go Google the Wikipedia page for Apple, the fruit not the company. They will start clicking and will usually be slightly awkward. Then I’ll ask what’s going on through their minds here.

Their responses are usually “I like the look of the Google logo today,” or “my results show Apple, the company and the fruit. I guess you want the fruit? I’ll click on that.” So, it’s like a warm-up before the test itself. You give them a little practice, you set them up, then you just gently encourage, obviously without leading them into a specific option.

If I don’t have time, I’ll make the first task of the actual user test a relatively simple one instead. One that is often not very consequential to the design but still part of the design. That’s another way to do the practice, to ease them into the user test. The same thing is done during user interviews as well – start with the easy questions. You start broad and then go deeper as the interview progresses.

9. In a survey, if you want to understand the why behind a particular choice, should the ‘why’ question be added into the survey? Or should a question for their contact details be added instead to ask them ‘why’ through a call?

Honestly, there’s no reason why you can’t do both.
When you’re doing a survey, one of the things that you have to balance is the attention span and the cognitive load, that you’re putting on a respondent. So obviously, if you overload with too many open ended questions, they may be less inclined to complete the survey seriously. Personally, I wouldn’t ask a ‘why’ question after every single question, turning a 20-question survey into a 50-question survey.

But there’s no reason why you can’t do a little bit of both. You can have a multiple choice question and follow it with a question like ‘Why did you pick that particular choice?” for particular questions you want to understand the user’s reason for.

If you are going to contact them after, you should put it upfront in the survey that you might contact them afterwards. So that people have that expectation and don’t think that you’re a marketer or a telemarketer.

10. What’s the difference between freelancing and contracting to you?

This definition differs for everyone but to me, freelancing is when I take on projects that may not have full hours. Whereas with contracting, I take on a 9 – 5 role for X number of weeks or months.

Part 1 Summary

1. Challenges of being a UX Researcher

• Stakeholder management. This is especially important in big companies. You need to be able to explain the importance of your own role, and showcase how your work supports designers and the business outcome.

2. What is it like working with different organisations?

• Government organisations: you know that the work you are doing is making a wider impact in Singapore and helps people in society. However, this usually also involves more red tape
• Big tech companies (e.g. Google): you get cool offices and facilities

3. How do you pick an appropriate UX Research method?

• Depends on many factors, including:

1. Time available
2. Resources available
3. Maturity of the team – less mature organisations tend to be less comfortable with quantitative research. One way to persuade stakeholders is to mix in some qualitative research with quantitative research (e.g. user testing).
4. Why you need to do research – for example, if you are testing an existing product with competitors, you should do user testing. If you don’t know much about the users, you could do interviews/surveys/contextual inquiry.

• The most common research methods that every UXer needs to know are:

1. User Testing – to learn how people actually use your designs.
2. Interviews – to learn what people are looking for in your design.

4. How do you tell apart an amateur researcher from a professional researcher?

• You can tell from how neutral they are. When beginner researchers (e.g. designers/developers on the team) see the user doing something wrong, they tend to feel uncomfortable and correct the user. As a result, they can’t observe the natural reaction of the user. When conducting user interviews, beginner researchers tend to show clear differences when reacting to responses they like (“that’s great) compared to responses they don’t like.

5. How do you conduct non-biased interviews?

• If you have the option, have the person conducting the interview not be directly involved in the product. For example, a researcher or business analyst in the team should conduct the research, rather than a designer or developer.

6. Is it recommended to collect both qualitative and quantitative data?

• This depends on what your organisation values and what you have access to, but in general it is good if you can have a two-pronged approach.
• Quantitative data show you what is happening; Qualitative data show you why something is happening.

7. How do you deal with respondents who are unenthusiastic and unhelpful?

• Before starting the user test, remind participants that you are going to ask a lot of questions and remind them to think aloud.
• When they appear stuck on something, give them a moment to think, before probing.
• If you have time, run through a quick practice test at the start to get them used to thinking aloud.

8. What should you say in your brief before user testing?

• Thank them for their time.
• Explain that they are going to be tested on a prototype that might not work perfectly.
• Let them know you are not part of the design, to encourage them to give their honest feedback.
• Remind them to think aloud (“put your brain on loudspeaker”).
• Run through a quick practice test if you have the time. If you don’t have time, make the first task in the user testing a simple one.

9. In a survey, if you want to understand the why behind a particular choice, should the ‘why’ question be added into the survey or done as a follow up through a call, hence ensuring a question for their details to follow up?

• Do both

1. For adding ‘why’ questions: Don’t overload survey with too many such that the number of questions double. Make sure to balance attention span and the cognitive load of survey respondents.
2. For adding a question for their contact details: Put it up front in the survey that you might contact the respondent afterwards. So that people have that expectation and don’t think that you’re a marketer or a telemarketer.

10. What’s the difference between freelancing and contracting to you?

• Freelancing is when I take on projects that may not have full hours.

1. I can take on part-time projects that don’t take up my full day, and I have a huge amount of flexibility in what I do during the day.

• Contracting is when I take on a 9 – 5 role for X number of weeks or months.