Design Research Made Easy: How to Use Screener Surveys for Effective Results

Recently I’ve been talking to many Product Design students and noticed a common fear: not having the perfect screener survey jeopardizes their research efforts. So I decided it’s time to demystify this concept, see what a Screener Survey is or isn’t, when to use it, and how to make it appealing to people to participate in your research activities.

Before we dive into specifics and some examples, let’s first clarify what a screener is. Think of it as a filter to optimize your recruiting process and ensure you are talking to the right people. Whether you need someone interested in one particular area or have done a specific task over the past few months, a screener will help you identify those people and focus your research efforts on them. One of the most common (and understandable) mistakes is associating screeners with regular surveys and the tendency to include questions that render insights into the problem. No matter how helpful those insights might seem, they might be statistically insignificant and can also reduce your response rate.

So, now that you know what to focus on when working on a screener; let’s see some of the best practices.

Provide context

As you’ll be sharing your screener survey across various groups, you’ll probably write a message or introduce yourself and the purpose of the survey. Even if you have this accompanying intro, it’s also recommended to have it in the screener in case someone only gets the link or skips the introduction.

Consider including a short intro to the form, where you briefly describe why you are collecting this data and what you will do with it.

Also, don’t be reluctant to include your name and a contact email if people have questions. These details make the whole task more personal or trustworthy, increasing your response rate.

Even though it might seem tedious to spend so much attention on the intro part rather than the questions, having a short but sweet introduction will increase your response rate. Instead of a depersonalized wall of text or paragraph, consider breaking the intro part into sentences and highlighting the key points. This will make the text more readable and friendly. At the same time, make sure to highlight important information (either by bold text or emojis), to make the text easy to scan.

While you are being transparent and making sure people understand why they should take the time to help, it would be good to include other details such as:

• How much time will the survey take (it should not be more than 3 minutes)
• Reassurance that answers are strictly confidential and used only for research purposes;
• The purpose of the survey.

Context doesn’t end with the introductory part, though. As this exercise aims to help you with your recruiting efforts, collecting contact data is one of the crucial parts. Don’t forget to ask people for their email, but don’t do it bluntly. Provide context on why you ask for this information and ensure you give them freedom of choice and transparency. It can be something along the lines of: “Would you be open for an online interview of 30–60 minutes (or the time you estimate the discussion to last) to share your experience with (what you are interested in researching)?”. It can help you get more participants if they know what they are getting into.

Provide clarity

Clarity should be one of your priorities throughout the screener to get honest results. Most of the time, your questions expect an answer that happened in a specific timeframe. Asking for recollection of past events without limiting it to a recent timeframe might cause confusion and lead to biased results. When they cannot recall an event, people tend to choose from available options and continue with the survey. “In the past 2 years” might be a long time for someone to remember things accurately. Also, to receive consistent answers, it’s good to decide on a fixed timeframe on which you want to base all your questions. You can use something like “in the past 6 months” or another specific timeframe depending on the nature of your questions.

As we are deep into our topic and research goals, we sometimes forget that no matter how much we provide context or how trivial the subject we think is, people have different experiences that we may never think of. To make sure you cover all possible scenarios, include options such as “Not applicable,” “None of the above,” and “Other.”

Keep it short

Like regular surveys, screeners are most effective when short, allowing more people to stay focused and not abandon the questions mid-way. There is often a tendency to get away from the survey’s primary purpose and add more questions to uncover insights. The sole purpose of the screener should be to identify the best participants for your research, not to gather insights. A good rule of thumb is to review each question and eliminate those you can incorporate in your further interviews or study. Also, try to reduce the number of open-ended questions. Having to write long answers will drastically decrease the number of responses.

Depending on the number of responses you are expecting to get and the time you have, you can sometimes include an open question at the end of the survey to anticipate the extent to which a participant is interested in the topic and can provide valuable insights. If someone writes a coherent story, chances are you have your ideal user. Provided, of course, that they agree to talk to you.

Apart from including short and sweet questions, there is also a good rule of thumb regarding their order. When thinking about what to ask, depending on your research goals, use an inverted pyramid to structure your survey. That means it’s important to start with the critical questions first, the ones that will “screen out” the majority of participants and are the most important for your research, and only then add the demographic questions at the end.

So, let’s do a quick recap, just in case you need to use this list in crafting your next screener:

1. Always provide some context and tell people what the survey is about and how much time you need from them to complete it. Also, leaving your contact details won’t hurt;
2. When asking people for their e-mail, be as transparent as possible and try not to scare them off. (not many people will be willing to go through an interview anyway);
3. Since you are asking people about past behaviors, include a relevant timeframe. If you are unclear about what you need to know, the answer will be vague as well;
4. To avoid people answering randomly or choosing the closest option from the ones available, make sure to include options such as “None of the above” or “Other” where applicable.