Moderated usability testing
A quick guide to understanding moderated usability testing, including the pros and cons, the methods and tools available, and the steps involved.
The role of UX designers and researchers is to make sure the products customers use offer a great experience that's intuitive and easy to navigate. To get the best chances of achieving those goals, you can use moderated usability testing.
Since you’re reading this, I can imagine you’ve come here to learn more about this style of UX research. In which case, buckle in! This guide will cover everything you need to know about moderated usability testing, including:
- What is moderated usability testing?
- Pros and cons of moderated usability testing
- Moderated usability testing methods and tools
- Choosing the right type of testing for your research project
Without further ado, let’s dive right in!
Unlike unmoderated usability testing, moderated usability testing involves an active, live moderator (usually the project researcher).
The moderator’s role is to guide participants through a test by asking usability testing questions, as well as observing participant behavior during the test. That being said, there are two main methods of conducting moderated usability tests: remote and in person.
What’s remote moderated usability testing?
Remote usability testing refers to using online software tools or bespoke testing apps, which takes away location-based limitations (more on this in the pros and cons section below).
For example, I recently completed a remote moderated usability test, which involved me downloading an app specifically designed for testing. The app had a video call-based home screen that allowed the researcher to see and talk to me while we walked through the test stages (which were late-stage Figma prototype designs).
What’s in-person moderated usability testing?
On the other end of the spectrum, in-person moderated usability tests involve a researcher either inviting participants to a testing center, a room at the office, or another mutual meeting space.
While the user is completing the test in person, the moderator is better able to note body language and build rapport with them, which helps encourage open and honest communication.
Now you’re getting a better picture of what moderated testing looks, you’re probably coming to a few conclusions about it. But hold fire! Each method of moderated usability testing has its pros and cons. Let’s take a look at what they are.
Pros and cons of moderated usability testing
No single method of research is perfect for every project, but moderated usability testing could be the best fit for your project, given the right circumstances. So let’s cover the benefits of moderated testing first.
- Less distraction for your users (when conducting in-person tests). When you set up a physical test environment, your participants are less likely to get distracted by their homes or other places they’re comfortable in.
- No need to rely on the internet (when in person). A benefit of in-person testing (but a con for remote testing) is that you don’t necessarily need the internet to test design prototypes, since you can store designs locally.
- Remote moderated tests can be more cost-effective. A benefit specifically of remote testing is that it can be cheaper to do, especially when you consider there’s no travel time and other logistics involved (like providing food/drinks).
- Ability to see participants’ body language. One of the biggest benefits of remote and in-person testing is being able to see how participants physically react to your designs and test tasks.
- Adapt to participants’ questions and struggles on the fly. Another benefit of running moderated usability tests is that you can guide participants through the test – so if they get really stuck or experience errors/glitches, you can help them out and get them back on track.
- Moderated tests aren’t built to scale. Due to the simple fact that moderated tests are limited by the number of moderators available, you can’t use them on a large scale (unless you own a humongous company, of course!).
- It typically costs more compared to unmoderated testing. In a similar vein, moderators cost money, whether you bring in external contractors or use in-house staff. Salaries or individuals paid by the hour can easily rack up research costs, not to mention the incentives you’ll need for your test participants.
- Moderated tests usually take longer. Due to moderators and participants both needing to be either in the same room, or using the same software at the same time, moderated tests often take longer to plan, as well as conduct.
- Being actively involved can influence behavior. People often behave differently when they know they’re being watched (called the Hawthorne Effect). So participants’ behavior might not be an accurate reflection of how they would approach the test tasks in real-life situations. However, this is a common issue in ethical research that you can acknowledge in your results.
As you can see, the benefits of moderated usability testing can often outweigh the drawbacks. Still, it’s ultimately down to each research project. If you think you have a research project that would be best served by moderated testing, then check out some of the tools and steps below.
How to conduct moderated usability testing
In terms of how to run moderated usability testing, most projects will follow a three-phase process: the pre-test phase, the test phase, and the follow-up phase.
1. The pre-test phase
Before you actually start testing, you’ll need to do a lot of planning, as well as sourcing and screening your participants. So when you plan your usability test, consider the following questions to help narrow the scope of your project:
- Who do we want to test?
- To what standard are we testing? (For this, it helps to be as specific as possible to figure out if your design works to address your solution, and to come up with a list of questions to ask.)
- How many people should we test?
- Where should we run the test? (Remote/in-person)
Answering these questions will help you plan the test parameters and find the right participants. Speaking of recruitment, you can do this yourself by sourcing existing customers or you could hire a research recruiter. However, another option several remote tool providers offer is a built-in network of panelists you can recruit from.
Paying to have vetted participants at your fingertips can improve the speed of your research considerably.
2. The test phase
After you’ve determined the specifics of who, what, where, and why you’re testing, you can get around to conducting the tests! One of the great benefits of moderated usability testing is being able to adapt or ask different usability testing questions on the fly and in reaction to how your test participants interact with the product.
However, be careful to ask too many questions during the live test. You can end up distracting your participants or accidentally influencing them and skewing the results.
3. The follow-up phase
Once the test is complete, you’ll be able to ask clarifying questions or answer questions your participants might have about the design or the testing process itself. Once you’ve finished testing, you can move on to analyzing your results.
Many software solutions help you develop reports on quantitative tests you complete using their tools, but another area you’ll need to analyze is the qualitative data you collect from users’ body language and unplanned questions or unforeseen issues they encountered during testing.
You can do this by organizing your data into tables that include the issue, its criticality in terms of whether it needs addressing before release (5=critical, 1=low), and the number of participants that experienced the same problem.
Put these qualitative results together with any quantitative analysis and you’ll have a pretty robust set of data that answers your research question.
Tools of the trade
If you’re looking for a comprehensive overview of usability testing tools, check out our guide. However, it’s worth noting that some of these tools are more suited to particular projects than others. Considering we’re concerned with moderated usability testing, here’s a slightly more refined list of suitable tools:
- UserZoom (custom pricing)
- Optimal Workshop (free plan available, paid plans starting from $208 per month)
- Great Question (paid plans from $10 per month)
- Hotjar (free plan available, paid plans start at $32 per month)
Each of these software tools has various features available to help with moderated usability testing, some more advanced than others. For example, UserZoom can host live interviews as well as automate scheduling and incentive payments, whereas Great Question can only help you analyze uploaded video recordings (from Zoom or elsewhere). We’d suggest assessing each platform and comparing its features to your budget and the needs of your research project.
How to choose the right type of usability testing for your research project
Moderated usability tests are great for conducting in-depth qualitative research and getting rich data from your participants. These kinds of tests are ideal for early-stage solutions, concepts, and designs, where imagination and exploration play a big role.
However, if your designs are further along in the process (such as when you have a functional prototype), your project might be better suited to larger-scale unmoderated usability testing. Here, UsabilityHub has a suite of tools available to help you maximize your research capabilities.
If you’re interested in learning more about usability testing, check out our usability testing guide. Otherwise, if you’d like to dive straight into testing, sign up for a free UsabilityHub account to get access to unlimited active tests!
Frequently asked questions about moderated usability testing
What is moderated usability testing?
Moderated usability testing involves an active, live moderator guiding participants through a test by asking usability testing questions, as well as observing the participants’ behavior during the test. There are two main methods of conducting moderated usability tests: remote and in person. Remote moderated usability testing refers to using online software tools or bespoke testing apps. In-person moderated usability tests involve a researcher either inviting participants to a testing center, a room at the office, or another mutual meeting space.
What is moderated vs unmoderated remote usability testing?
Moderated remote usability testing involves an active, live moderator who guides participants through a usability test by asking questions and observing their behavior, while unmoderated remote usability testing doesn’t involve a moderator, and allows participants to complete the test on their own using online software tools or bespoke testing apps.
How do you conduct moderated usability testing?
The process of conducting moderated usability testing usually involves three key steps. First, define your research goals, recruit participants, and select appropriate tools for recording and analyzing your data. Second, conduct the test by briefing participants, asking them to complete tasks while thinking aloud, and taking notes and recording the session for analysis. Finally, analyze and report the data by reviewing it to identify usability issues and insights, preparing a report that summarizes the findings, prioritizes the issues, and provides recommendations for improvement. Share the report with stakeholders and follow up on any necessary actions.
Alexander Boswell is a freelance writer specialising in B2B SaaS and eCommerce marketing with a particular interest in the world of data, as well as a business Ph.D. candidate. When he’s not writing, he’s nerding out playing D&D and Magic: The Gathering.