Screener surveys

This article focuses on creating participant recruitment screeners for individual moderated and unmoderated studies, not for phone screening.

To help you create the best possible screener for your research needs, we created a Google Doc checklist – you can copy it to your Google Drive and have it on hand the next time you're designing a screener survey.

What is a screener survey?

The primary purpose of a screener survey is to recruit the right participants for your research study. The goal is to screen the right people “in” and the wrong people “out”. Curating this sample is key to generating quality results for both quantitative and qualitative studies and to make sure that you're gathering input from the people best suited to provide it now or over time via a participant pool.

If you don’t screen well, you’ll gather flawed feedback, and then make poor decisions based on tainted evidence. This is no bueno. So it pays to be clever about how you approach this effort!

Before starting your screener survey, always establish the objectives and information required from the study, then make sure the questions and responses gather actionable feedback and reliable data.

What makes a screener survey effective? 

Successful screener surveys should:

Align with your research objectives:

• Does the screener align with the objectives of your research plan?

• Have the research objectives been clearly defined and validated beforehand?

Gather pertinent information:

• Which information is most valuable and pertinent for the research?

• Are the questions designed to demonstrate that the respondent’s time is valued?

Display neutral and unbiased questioning:

• Are the questions posed in a neutral and unbiased manner?

Minimize bias in the response options:

• Do the response options cover a comprehensive range of possibilities?

• Have measures been taken to minimize bias in the response options?

Provide options for non-applicable or uncertain answers:

• Are there options for respondents to indicate when a question doesn't apply or if they're unsure of the answer?

Offer enough responses for valuable data:

• How many responses are needed to obtain sufficient and valuable information?

• What is the target number of respondents to ensure meaningful data?

Screen the right audience:

• Has the predetermined criteria for participant selection been clearly identified?

• Are efforts made to screen out individuals who don't qualify for further research?

Employ user-centric design:

• Does the screener provide a clear understanding of the study's context upfront?

• Is the survey designed to offer a positive user experience with clear, logical, and concise language?

Set expectations and clarify incentives:

• Do respondents know what to expect after completing the screener?

• Are the incentives offered to selected participants clearly communicated?

Communicate study details and next steps:

• When screening for a future study, are the specific details (dates, format, platform, duration) clearly stated?

• Have you provided information about the next steps and the follow-up timeframe?

Tips for designing screener surveys 

1. Consider how and where you’ll gather screener responses

In some instances, you’ll be screening within the platform where a moderated or unmoderated study will take place. Platforms like Lyssna offer a research panel, where remote participants for unmoderated studies are screened before they’re allowed to begin a test. 

Other times, screening will be a distinct and separate phase of the research process and you’ll review the screener responses individually before inviting people to participate in a study, such as an interview at a future time. 

Both embedded and standalone survey platforms vary in sophistication. Consider your study’s needs and the functionality of the platform you’re using. For example: 

• Do you need to capture geolocation? 

• Will respondents be asked to provide a signature or upload any documents?

• Is piping and/or branching helpful for this recruiting effort? 

• Will you be reviewing the responses before inviting participants to the study? Or will your screener survey immediately redirect them to schedule a session time if they pass the criteria?

2. Set the right tone and choose effective question types

Take first impressions into account. The screener is often the first touchpoint with potential participants and prospective customers, so make sure your solicitation (the survey welcome), and all of the content within the screener, are professional and set the right tone. 

Many people think designing questions and response options are easy tasks. They aren’t. In fact, this is both an art and a science that's difficult to master – 95% of the surveys I see contain fatal flaws! 

Consider using a variety of question types to reduce survey fatigue. Understand the pros and cons of different question types and when to use them. 

Below are examples of question types to consider.  

Single select:

• Radio button (binary): Yes/No

• Dropdown menu (multiple choice): Usually a longer list than radio button options 

• Select image (logos, icons, layouts)

Multi-select:

• A list of responses where respondents may select more than one option

• You can specify 'Select all that apply' or restrict it to a minimum or maximum of questions 

Open text fields, which are often used for:

• Contact forms

• Text boxes – short and long

• Dates (pickers)

• Email addresses

• Percentages 

• Phone numbers (character validation)

• Signatures (sign using a cursor, capture image, via touch)

Grids and rating scales:

• Radio button grids (provide a single select per row) are used to ask multiple but similar questions, and are condensed and consolidated in presentation for respondents and for efficient reporting (each row is essentially one data point). 

• Rating/Likert Scales are horizontal layouts of radio button responses. Make sure you're consistent in regards to left to right anchors, and customize the left and right verbiage.  

• Slider bars allow respondents to drag across a scale to produce a numeric value. Add labels on the left and right anchors and be consistent with the number of points on the scales used in your survey.   

There are several other question types including drag and drop, grouping/card sorting, and more.

3. Consider branching, piping, and skip logic

Evaluate whether branching, piping, or skip logic will be helpful for your screener. These advanced features aren’t available on all platforms, but are incredibly powerful. They allow you to change what questions or pages are displayed to a respondent and tailor questions based on previous answers, which will:  

• Result in a custom pathway through the survey based on rules.

• Reduce the time it takes for respondents to complete the screener. 

• Right-size the data collected.

• Eliminate the need for multiple links or survey variations. 

• Provide a better user experience. 

Here are some considerations to evaluate whether to explore branching, piping, or skip logic. Ask yourself: 

• Is there significant overlap in questions and responses for multiple segments or phases of the study?

• Are you recruiting from the same place and in the same way?

• Will you be able to apply learnings from, or to a sequential survey?

• Will all of the survey content be ready at the same time?

• Do you have access to a more sophisticated survey platform that offers these options?

• Are you prepared for the added complication?

Tips for writing screener surveys

Solicitations are invitations to take a screener survey. When using a standalone platform, they’re typically sent via email and include a link to the screener. On an embedded platform, the solicitation appears first upon login. 

The content of this invitation will vary depending on the survey topic and goals. 

Here are some things to consider when writing solicitations and screeners:

• Set expectations upfront about the topic at a high level (but don’t give it all away!). Include the time to complete and the incentive details.

• Organize your questions into categories (e.g. about you, your car, your insurance, your driving habits). 

• Identify the criteria to screen in: key behaviors, contexts, motivations and/or attitudes representing the type of people you're designing for and seek feedback from. 

• Confirm the criteria to screen out: You may want to remove conflicts of interest, extreme or power users, those who have completed a study for your organization in the past 3–6 months, etc. You’ll need to screen out people who don’t meet the study criteria, don't qualify for your products or service, etc. 

• Discuss the criteria to relax: Before you launch your screener, agree on which criteria you and your stakeholders are willing to release if you run into a difficult recruiting situation. This will save precious time and reduce stress! 

Pay attention to the details:

• Consider the title. Don’t give away the focus of your study in the title as this may sway responses. 

• Revisit the survey questions and response options multiple times during development to make sure they produce the right outcomes.

• Triple check all spelling, grammar, and punctuation are correct. 

• Piloting your screener is key. Pilot to test for question clarity, flow, links, pathways, stimuli, length of time, technology, and outputs. 

• Genuinely thank respondents for their time.

Remember, not everyone you solicit to take your screener survey will qualify, be interested, or be available to participate in your study, so plan accordingly! Think of the solicitation and screening process as a funnel. You may need to keep “feeding your funnel” or relax your recruiting criteria.   

You can access all of the above tips as a handy checklist – save a copy to your Google Drive and have it on hand the next time you're designing a screener survey.

Remember, if you don’t screen properly, you'll be gathering input from people who aren't best suited to providing it. This will result in flawed data, and then you and your stakeholders will be acting on tainted information. Proper screening is key to making sure you're gathering feedback from the right people at the right time. 

Elevate your research practice

Join over 320,000+ marketers, designers, researchers, and product leaders who use Lyssna to make data-driven decisions.

Sign up for free

Sign up for free

Sign up for free

--

This article was authored by Michele Ronsen, Founder and CEO of Curiosity Tank. Michele is a user research executive, coach and educator. She teaches design and user research to people around the world. Her corporate trainings and workshops are inspired by working with Fortune 500s and start-ups for more than twenty years. Fuel Your Curiosity is her award-winning, free, user-research newsletter. In 2020, LinkedIn honored Michele with a TopVoices award in the Technology category. She is the first and only researcher to receive this award.