How to ask the best usability testing questions

As someone with experience in marketing academia, and as a market researcher and participant, I’ve found usability testing to share some similar concepts.

Marketing practitioners like to assume their target audience thinks the same way they do about a product or concept – but the reality is that people respond differently, even within the same target group. And it’s the same with user experience. So to create a great UX experience, researchers and designers need to understand who their users are, as well as what they want or need from a product, and why. But how?

The short answer: by asking potential users lots of questions!

To get more specific, you need to ask “usability testing questions”, which is a period of research UX designers, UX researchers, product managers, and marketers conduct throughout product development and before any official launches (and again when you propose any changes to your design later on, or to check everything is still working as intended). 

In this guide, we’re going to cover the following areas:

• Why you need to ask usability test questions
• Usability testing question categories (with example questions)
• Best practices for asking test questions

Let’s dive right in!

Why do you need to ask usability testing questions?

Asking potential users questions about your product or UX design is important because it helps you validate your hypotheses or any ideas you have, find any issues with your design before launch, and understand how users interact with similar products or services offered by the competition. 

While we’re going to cover best practices further down, one rule of thumb is that you should ask questions at each stage of the user research process – this way you get more in-depth results and gain the maximum benefits from it.

Usability testing question categories (with example questions)

As mentioned above, usability testing questions should be organized by research stage. But to make sure you’re asking users the right questions, you need to define the purpose of your research – what do you want to find out?

By defining the overall research goal, you’ll have a far easier time narrowing down the list of questions you’ll need to ask at each stage of the process.

With that in mind, there are three core stages of UX design research where you can ask users questions: before the test (to screen participants), during the test (to help answer your research questions), and after the test (to gain feedback). 

Another aspect to consider is whether you’ll be using remote usability testing tools, or mobile usability testing tools, as opposed to an in-person test, and whether the test will be moderated or unmoderated. You can find more details about the nuances between these types of testing in our usability testing guide.

For now, let’s take a look at these categories in more detail.

Pre-test questions

As much as we like to think our designs will be useful for everyone, they won’t be. Instead, we need to focus our attention on users that will be the most likely to use the product – our target audience. 

To do that, you need to find the right participants for your test by asking pre-test (otherwise known as “screening”) questions. 

So first, define the type of people you want to involve in your usability tests. Then, to find those people, you can ask “about you” questions to filter your pool of users. These questions are usually about demographics, background, and experience.

Demographic questions

Asking demographic questions can help you find trends or narrow down your participant pool by age, gender, nationality, relationship status, and religion (if relevant). However, in many countries, these points of data are protected characteristics and can be sensitive information for some people. 

Therefore, it’s best to ask these questions in ranges or let the users define themselves (and/or give “prefer not to answer” or “other” options). Here are some examples:

• What age group are you in? (Give ranges as options, e.g. 18–25, 26–34, etc.)
• How would you describe your gender?
• How would you describe your ethnicity?
• What is your household income? (Again, giving ranges as an option)
• How would you describe your relationship status? 

To help you further filter your participants, you can ask additional, more specific background questions.

Background questions

Asking background questions can help you get a better context of the person and how your product or service fits into their life. 

You should also find out if they’re an existing user, a user of competing products, or if they’re completely new to it – each of these offers different insights and you should take them into account during your analysis of the results. 

Some background questions to ask include:

• What does a day in your life look like?
• Do you already use X product, and if so, how often?
• What type of product do you use to do X?
• Which device would you normally use to do X?
• How much time do you typically spend online? (If building a digital product)
• How experienced are you in using X type of product? 

Using these screening questions can help you find the right participants, as well as help explain any abnormalities in your results.  

In-test questions

Now to the questions you probably had in mind when you were thinking about this topic – what to ask during your usability test. Contrary to what you might think, this is the period when you should avoid asking too many questions, as this can distract your users and stop them from taking actions natural to them. 

However, the most important rule to remember when asking usability test questions is to not ask leading questions, meaning a question that influences the user to think a certain way, e.g. “how good was the experience of using X feature?”. 

Leading questions like this will skew the results and won't provide an accurate picture of the experience. Instead, ask neutral, broad questions that encourage users to offer honest answers. 

Examples of non-leading questions can include:

• What do you think about X design?
• Can you name any competitors that you currently or would use X for?
• What features do you find most valuable, and why?
• How would you describe the language used on this page?
• Can you give an example of where you would use X?
• I noticed you [describe something they did]. Why?
• How was your experience completing X task?

Open-ended questions like these help you get an in-depth understanding of the user’s experience while using the product in the test, which is far more useful than simple ‘yes’ or ‘no’ based questions. 

Post-test questions

After they finish a test, users might still have some opinions about the product or testing experience. So the end of the test is a great opportunity to ask some final questions while you still have their attention (at most, ask these up to a few days later so the memory is still relatively fresh). 

These questions can help you improve the testing experience or gather opinions on aspects of the product you hadn’t considered before. Some examples of these questions can be:

• How would you describe the overall experience of the product?
• How would you describe the overall experience of this test?
• Were any of the tasks difficult to complete? (Or) How would you rate the difficulty of the tasks on a scale of 1–10 (10 being extremely difficult)?
• What do you expect X product/feature to be like in the future?
• If you could make any changes to X, what would they be and why?
• Do you have any additional comments or questions?

Another option can be to use a standardized quantitative research feedback survey (such as the System Usability Scale) to help gather any long-term trends or to compare with other similar products using the same survey format. 

Best practices for asking test questions

As you’ve probably picked up, we’ve already introduced two best practices for usability testing:

• Ask questions at each stage of the research process.
• Don’t ask leading questions, ask open-ended questions instead.

However, there are a few other best practices to keep in mind to help you get the most out of your usability test questions.

‍Watch the participant's body language and facial expressions

‍If a participant has permitted you to record them, pay attention to how they physically respond during the test. This can give you a lot of insight into parts of the test when the user subconsciously expresses like/dislike or confusion.‍

Ask participants to think aloud

‍Similar to the above, some people find it difficult to express their thoughts out loud. Asking them to do so gives them “permission” or reminds them you want to understand their state of mind.

Make use of technology

‍While in-person tests are still great, using remote usability testing tools can help automatically process data, such as click maps, path analysis, heatmaps, and time to complete.

Don’t provide too much guidance or ask too many questions during a test

I completed a usability test (as a participant) recently for a banking app. During the test, I often felt the need to ask “how do I find that?”, but the researcher encouraged me to figure it out for myself – for good reason. It helps researchers and designers to see where users might struggle with navigation, and pinpoint potential problems. 

Improve your usability tests and UX design by asking the right questions

Great usability tests help to create better user experiences and can take a lot of time and effort to get right. Being prepared with good questions can help you maximize your return on investment (in time, insights gathered, and costs to conduct tests). 

By planning your questions out in stages, you can identify problems that come up before launch, and find out if your product actually addresses what users want and need. Sometimes, they can also help inspire new ideas based on user feedback from the test. 

Feeling inspired by all the questions you can ask? If you’re ready to start conducting usability tests for your products, sign up for a free Usability Hub account to get started with access to unlimited active tests.

Frequently asked questions about usability testing questions

How do you write usability testing questions?

To write usability testing questions, first define the overall research goal and target audience, then organize questions by research stage: pre-test (screening), during-test, and post-test. Pre-test questions should screen for demographic information and background, while in-test questions should focus on observing and understanding how users interact with the product or service being tested. Avoid asking too many questions during the test as it can distract users from natural actions. Post-test questions can gather feedback on the user experience. Make sure to follow best practices, such as keeping questions concise, avoiding leading questions, and making sure questions are relevant to the research goal.

What are some common usability testing questions?
Some common usability testing questions include:

• What do you think about X design?
• Can you name any competitors that you currently or would use X for?
• What features do you find most valuable, and why?
• How would you describe the language used on this page?
• Can you give an example of where you would use X?
• I noticed you [describe something they did]. Why?
• How was your experience completing X task?
• How would you describe the overall experience of the product?
• How would you describe the overall experience of this test?
• Were any of the tasks difficult to complete? (Or) How would you rate the difficulty of the tasks on a scale of 1–10 (10 being extremely difficult)?
• What do you expect X product/feature to be like in the future?
• If you could make any changes to X, what would they be and why?
• Do you have any additional comments or questions?

Are there any usability testing questions to avoid?

It's important to keep the usability testing questions neutral, focused, and relevant to the specific goals of the test in order to get meaningful results. Here are some questions to avoid:

• Leading questions: Avoid asking questions that suggest a particular answer or guide the participant towards a specific response.
• Double-barreled questions: Avoid asking two questions in one, as this can confuse the participant and lead to inaccurate responses.
• Biased questions: Avoid asking questions that reflect personal opinions or biases, as this can skew the results of the usability test.
• Irrelevant questions: Avoid asking questions that are not related to the specific goal of the usability test, as this can waste the participant's time and compromise the validity of the results.

Alexander Boswell is a freelance writer specialising in B2B SaaS and eCommerce marketing with a particular interest in the world of data, as well as a business Ph.D. candidate. When he’s not writing, he’s nerding out playing D&D and Magic: The Gathering.