10 Oct 2023
|13 min
User testing
We explore the benefits of user testing, and how it can help gather valuable feedback from users to improve product design and user experience.
User testing definition
How do you know when an app or website is ready for the world? Before going live, you need to see how users interact with it and find out what they think about their experience. If they have complaints or aren’t able to accomplish what you’ve set out for them to do, then you’ll have to do a bit more work before you can get your product out there.
Along with evaluating how users react to a design, user testing, or user experience testing, asks bigger questions related to a digital product. Does it fill a consumer need? Do people want the functionality it offers? And does it provide solutions to customers’ problems?
User testing makes it possible to gather a wealth of information from real users, showing you where your product is at, identifying changes that need to be made, and determining if there’s a demand for what it hopes to accomplish.
Why is user testing important?
Internal testing may reveal issues or problem areas, but it can’t catch everything. After working on a project for months, you and your team have become so accustomed to how it works that most cruise through it on autopilot. Stepping outside of this sphere of familiarity, and putting a design in front of people that have zero experience with it, will offer perspectives not possible through internal testing alone.
And let’s not forget the unpredictability of humans. No matter how well-designed a digital product may be, someone will find a way to break it. Though we may envision users taking idealized paths and actions, there’s no way to anticipate the multitude of ways that people can screw things up, resulting in them having a negative experience.
How does user testing work?
User testing takes users through designated actions, observes their natural behavior, and gauges their emotional responses. User testing sessions generally run for 60–90 minutes and focus on a specific area or point of functionality in a digital design.
User testing lets you gather important feedback from real users through questions like:
How did the overall experience make you feel – was it easy, frustrating, or something else?
What were your expectations? Did the product meet your expectations?
Did the architecture and interactive elements help you take the actions you wanted to take?
What features or functionality might be missing that would make it more useful?
What is your overall impression of the product – do you like it, dislike it, or are you indifferent?
User testing vs usability testing: What’s the difference?
User testing and usability testing both fall under the umbrella of user research and have many areas where they overlap. User testing certainly is concerned with the usability of an app or website but is more focused on the overall experience, and whether it provides users with everything they need to successfully interact with the product.
Usability testing hones in on interactions, navigational options, interfaces, and other actions. It asks testers to accomplish specific tasks and gathers data and information about how these tasks are performed.
Elevate your research practice
Join over 320,000+ marketers, designers, researchers, and product leaders who use Lyssna to make data-driven decisions.
How to prepare for user testing
User testing is a crucial aspect of digital product development, as it provides valuable insights into how users interact and engage with your product. By conducting user testing, you can gather feedback on the most important actions and features of your app or website, and make necessary improvements to enhance the user experience.
This section will guide you on how to prepare for user testing, including having a working version of the product and setting clear goals for the testing process.
1. Create a prototype of your app or website
Okay, this may be obvious, but you’ll need to have some sort of functioning version of an app or website to test out. This can be anything from a lo-fi prototype to something more fleshed out.
2. Set goals for user testing
It’s best not to overwhelm your participants with too much to do. Figure out what type of feedback you’re looking for, and what the most important actions are that you want users to take.
For example, let’s say that you’re building an app for a grocery store. You might run a user test to find out how people feel about the categories that customers can shop by. Are they helpful, or do they complicate things? User testing can also reveal features that your grocery app may be missing, like being able to search for items by department, or to choose items from past purchases.
User testing finds out what people prefer in order to optimize how they interact and engage with your product.
3. Decide on a user testing format
User testing can be moderated or unmoderated. Don’t worry about the differences just yet, as we’ll be diving into these methods in just a bit.
4. Recruit test participants
Find people to recruit. Your test participants should represent the customers who will be using your product. You can recruit existing customers while they’re using your product or visiting your website by creating a pop-up request, or sourcing via your customer-facing teams, social media, or online communities. It’s also worth considering offering incentives, like gift cards.
If you’re using a user testing platform like Lyssna, you can use a participant recruitment panel to make recruiting participants easier.
5. Find a place to test
Find an appropriate location, free from distractions, where users can test your product. This can be a physical space, or remotely using an online user testing tool.
User testing best practices
To ensure you conduct effective user testing, it's important to follow best practices. This section offers tips for effective user testing, including starting early, defining clear tasks, and avoiding overtesting.
Begin testing as soon as possible
It’s tempting to roll out user testing when a product is nearing completion. However, identifying problems early in the process makes it easier to address them head on rather than the cascade of side effects that can come from making changes later.
Define the actions and tasks you want users to take
Give users clear directions about what you want them to do. When providing this guidance, leave out the reasons behind why, as this might affect how they run through the test and influence the feedback they provide.
Let’s jump back to our grocery store app example. You might give users a real-life scenario, like asking them to create an account, test out a shopping list feature, and search for groceries they commonly buy. Did they feel good about taking these actions? User testing zeros in on what’s most important and measures how people feel about their interactions.
Remember, don’t test too much! You can’t test every feature or use case, so set realistic goals based on the actions and tasks that the average user would normally want to take.
Moderated vs unmoderated user testing
When running user tests, you can either be directly involved or more hands-off. Moderated testing involves a moderator running a user test and interacting with a participant, while unmoderated testing leaves participants to complete tasks on their own. Let’s look at the difference between the two approaches.
Moderated testing
Moderated testing involves a moderator guiding participants through the testing process. This can either be done in person or remotely, for example via a video call. Moderators take an active role in instructing participants, getting feedback as they move through the test, and providing assistance if they run into trouble. Since moderated testing is structured, it requires preparation and takes more time than unmoderated testing.
Unmoderated testing
Unmoderated testing gives participants autonomy, leaving them on their own to conduct a test. Unmoderated tests are usually conducted remotely. Participants receive written instructions, or a short video introduction telling them what they need to do, but that’s the extent of communication.
Video footage and screen recordings capture what users do, which you can observe and evaluate later. You can also administer follow-up interviews and surveys to find out how users felt about their experiences.
Unmoderated testing simulates how someone would use a product in the real world independent of any sort of guidance or outside help. Since there’s less involvement on your end, you can set up and run unmoderated testing fairly quickly.
Common user testing methods
Now that we’ve covered the basics, let’s look at some popular user testing methods that you can use.
Concept testing
This answers the bigger existential questions around whether people want/need the product you’re planning on building. Concept testing happens in the early stages of the design process and may be as simple as interviewing participants, administering design surveys, or showing participants a very simple representation of what you’re planning on launching and finding out what they think about it.
A/B testing
A/B testing involves testing different prototypes between groups of users and seeing which one performs better. This is used to compare different versions, such as layout or copy. To avoid skewed results, it's crucial to test only one variable at a time.
Group testing
This involves gathering together a focus group to run through your design, and then having conversations about how they felt about its ease of use, features, content, and overall impressions. This can provide a wealth of valuable information about how people feel about your product.
Card sorting
In card sorting, users group topics into categories according to criteria that make the most sense to them. This helps inform the information architecture of a website or app and provides insights into how users categorize content, and can identify trends that align with the mental model of your users. There are three types of card sorting techniques: open, closed, and hybrid. Card sorting activities can be done in person with physical cards or remotely using online tools.
An open card sorting test in Lyssna.
Beta testing
Beta testing takes place when you’re close to launching your product. This is an opportunity to find minor issues that you haven’t addressed previously and taking care of any problems before your product goes live. This is the last stage, where previous user testing has already addressed major issues or problems.
User testing data
User testing generates two types of data: quantitative and qualitative. Quantitative data provides numerical insights into user experience, like time taken for specific actions and results of A/B testing. Qualitative data deals with emotions and the perceptions of users, which can provide valuable feedback you can use to improve your designs.
You can use both types of data to make changes and test designs again for better outcomes.
Quantitative data
Quantitative data involves hard numbers. You can use it to track things like how long it takes users to accomplish actions, comparing the results of A/B testing, and the number of people who met the goals you set out for them. Though you can gather quantitative data from both user testing and usability testing, with user testing, it should provide insights into the overall user experience rather than specific tasks.
Qualitative data
Qualitative data has to do with emotions and how people perceive a design. They’ll have opinions, some good and some not so good. The perspectives and thoughts gathered from user testing make it possible to change a design so that users will have a more positive experience.
How to analyze user testing data
Both quantitative and qualitative data are important in showing what needs improving. From this information, you can make recommendations for adding, removing, or modifying features and design elements. Once you’ve made those changes, you can test your designs again to see if they leave people with a better impression and lead to better outcomes.
Data results gathered from a user test in Lyssna.
Ready to start user testing?
User testing doesn’t have to take up a lot of time and resources. Lyssna is an effective user testing tool where you run prototype tests, design surveys, card sorts, and more, custom tailored to your target audience.
Your go-to user research platform
The best teams use Lyssna so they can deeply understand their audience and move in the right direction — faster.
Frequently asked questions about user testing
--
Jeff Cardello is a freelance writer who loves all things tech and design. Outside of being a word nerd, he enjoys playing bass guitar, riding his bike long distances, and recently started learning about data science and how to code with Python.
You may also like these articles
Sign up to our newsletter
We'll keep you updated with the latest UX insights and more.
