How to recruit and screen candidates for user research

Recruitment Criteria

Before recruitment, the researcher needs to consider the appropriate criteria to apply when recruiting in order to find participants that can provide relevant insights.

In some cases, your criteria may be general: age, location, and experience with a particular type of widely used product. In other cases, you might be looking for people who are able to provide more specific insights into the project you are working on – for instance, the user journey of someone who has received a particular type of treatment at a hospital.

Deciding on the right number and type of criteria is a balancing act. Avoid having too few requirements that result in you getting participants that do not represent your end-users. On the other hand, one should avoid adding criteria that might make it difficult to recruit participants, unless it is mission-critical to the research. The more specific the requirements, the more effort is needed for recruitment.

Screening participants

Screening surveys (also known as “screeners”) are questionnaires that gather information about candidate participants’ experiences to quickly identify optimal candidates that are representative of your target audience while excluding any candidates who may not be a “good fit” for your UX interviews or usability testing.

1. Define eligibility criteria

To identify the eligible criteria, consider the demographics of your target audience and its goals as they use your products. These criteria will determine your recruitment strategy and your screener.

Using automated-recruiting platforms may be easier, but be careful about overly restricting your survey with extraneous elimination criteria. For example, an exclusionary question like “When did you last participate in a research study?” may exclude more participants than necessary – like second-time participants who happened to participate in a completely different study.

In the same way, when recruiting marketing professionals, one might choose to accept only individuals who select Advertising and Marketing as their industry. However, marketing professionals exist in different types of organisations and industries. Filtering through job titles or descriptions might be more effective in this case.

2. Design your questions for screening

When designing your questions, use open-ended or multiple-choice questions to avoid revealing the study’s purpose.

For example, a yes/no question like Do you play computer games? might give away the fact that the study might be related to computer games. However, if you phrased the question as Which of these activities have you done in the last month? With options like exercising, reading, shopping, and computer games, then the intent of the study is less obvious. Distractor answers keep respondents honest and prevent them from gaming the questionnaire. Aside from that, it is important to exclude participants who have knowledge of the subject matter or are researchers. Their participation might skew the perspective and result of your research. Like professional testers – people who make part of their income by participating in different kinds of user research – their motivations and expertise may lead to behaviour that skews research results.

You should also consider placing your most important exclusion criteria near the beginning of the survey to quickly eliminate obvious misfits.

3. Strategise your recruitment

There are different ways to recruit participants. Each approach has its strengths and weaknesses. Depending on your study’s research questions, you may want to recruit from multiple sources.

• Professional recruiting agency

There are many professional recruiters dedicated to helping UX or market researchers find qualified research candidates. Most will cover the work of interacting with prospective and final participants, such as scheduling, communication, and payment. Generally, these agencies have a wide reach in their ability to recruit “general audience” participants and are ideal for studies that require specialised user groups (i.e., finding participants with a particular disability, or someone with a specific background). However, they are often more costly than automated tools or leveraging existing pools of users.

• Automated recruiting platform

For recruiting “general audiences”, this method could be beneficial, given its low cost, ability to outsource recruiting work, and the relatively quick turnaround. That said, with an automated recruiting platform, there is a greater risk of recruiting “professional testers”.

• Existing pool of users

This method is ideal for recruiting experienced users of a specific product or for getting insights about employee-facing products. There is a lower immediate recruiting cost and candidates are already semi-qualified. However, you will often need provide participants with compensation – monetary or otherwise – for their time. Time will also be required for scheduling and coordinating participants. The reach of an internal panel is also limited to participants who already are familiar with your brand and offerings. This means that you may not capture new-user perspectives. There might also be some sampling and confirmation bias due to existing brand loyalty.

• Online forums and groups

These groups are great for specialised recruiting needs at a lower cost than with recruiting agencies or platforms. However, the reach will also be narrow and limited to the members of those groups, which are not always representative of the targeted audience. There is also the possibility of sampling bias or groupthink if the group has certain predominant viewpoints or vocal community members.

4. Understanding behaviours of the participants

Time-poor or high-earning professionals may need high incentives to justify the time spent away from work or their personal lives. They are also less likely to spend time filling out a lengthy screener surveys

5. Understanding the mechanics of your own survey

• If you are recruiting two or more types of user segments, you can attempt to use one screening survey for all these sets of criteria. On the plus side, participants don’t have to fill out the survey more than once, which means that there is a higher chance of securing a sufficient sample size. However, you may also end up with a pretty complicated, branching screener or may need to manually filter candidates.
• Review the survey responses of qualified and disqualified candidates before finalising the approved list of participants. Doing so allows you to identify close-fit study applicants that can be considered as backup candidates if they have most, but not all the criteria for your target audience.

Conclusion

Finding the right participants for your study is a crucial step to ensure that your research is based on insights from the users you are designing solutions for. Knowing your research goals will help to determine whether to use your existing customers, recruit a representative audience, or use a mix of both. If you’re updating an existing product, your existing users will be your best audience. If you’re building a brand new feature or product, non-users can provide a fresh perspective. Remember that planning the recruitment is just as important as the actual study!