Do you remember learning about categorizing and classifying living things in high school biology? Well, there’s a similar system for categorizing UX research methods. These categories define the underlying concept of each method and include:

• Quantitative and qualitative

• Behavioral and attitudinal

• Moderated and unmoderated

• Generative and evaluative

Broadly speaking, most (if not all) types of UX research will fall into one or two of these categories. For instance, user interviews are a form of qualitative and generative UX research.

It’s important for UX researchers, and anyone else who conducts research, to understand these categories and their purpose. Sometimes, the category can be deduced from the name. For instance, quantitative research concerns numbers, while moderated research means there's a moderator involved.

But generative and evaluative research isn't as self-explanatory, especially for those new to UX research. In this guide, we'll walk you through:

• What generative and evaluative research is 

• Why generative and evaluative research is important 

• How to conduct generative and evaluative research

• The pros and cons of generative and evaluative research 

• Which research method is better

Let's dive in!

What is generative research?

Generative research seeks to generate information about users and their behavior. The goal is to learn critical information about what users do, how they do things, and what situations they take action in.

Generative research aims to find opportunities for innovation and solutions to potential issues. It’s also known as exploratory or discovery research because the insights lead to exploration and discovery.

Generative research is usually conducted at the beginning of the product development cycle. For instance, if you’re at a B2C company that wants to launch a money-saving app for millennials, you might conduct generative research to find out how millennials perceive the app concept and how they think about and approach finances and savings. 

When conducting generative research, you may not be aware of the problem that needs solving yet. But the purpose is to find the problem and work out a way to solve it.

The main types of generative research include: 

• User interviews 

• Surveys 

• Concept testing 

• Diary studies 

• Open card sorting

Why is generative research important?

Generative research helps you to identify a problem, understand its relevance to users, and find a way to solve it. You can use it to detect and understand behavior patterns and how these patterns can be used to solve a problem.

Generative research seeks to reveal user experiences across three main areas:

• User experiences with a product: Understand current or previous user experiences through concept testing and surveys. 

• Knowledge about a product: Gauge how much or little users know about a product through surveys and interviews.

• Future expectations about a product: Based on the research outcomes, you can generate ideas or design concepts that users can expect to see in the future.

By understanding the lives and habits of your users, you can gain insights that go beyond basic information, such as how many times they use a product.

Accessing user information can help you understand the needs and motivations behind specific behavior. Uncovering the reasons behind this behavior can give you the knowledge to positively impact the lives of your users with your product. 

How do you conduct generative research?

Conducting generative research will differ depending on the research method you're using. Let’s explore the main types of generative methods mentioned above.

• User interviews: Interview potential users to learn more about their needs, wants, and pain points. You can do this in person or remotely using video conferencing tools. 

• Surveys: Create surveys that ask open-ended questions to gain a better understanding of user preferences, behaviors, and experiences.

• Concept testing: Present potential users with a concept or idea to gauge their interest, feedback, and overall reaction. Use this to understand whether a new product or concept resonates.

• Diary studies: Ask participants to document their experiences over a period of time, such as a week or a month. Use this to gain insights into how users interact with a product or service in different contexts and over time, as well as identify pain points and opportunities for improvement.

• Open card sorting: Present users with a group of labeled cards and ask them to sort the cards into categories that make the most sense to them. This is helpful for gathering insights into how users naturally categorize information and to help inform the creation of an effective information architecture.

An example of open card sorting in Lyssna

What are the pros and cons of generative research? 

Insights gained from generative research can be incredibly insightful, but there are some downsides too. Let's look at the pros and cons.

Pros

• Develop innovative ideas: By involving users in research, you’re more likely to think of them as active participants in the product design process, rather than passive consumers. This can lead to the development of innovative ideas.  

• Understand how users think and process information: It's easy to do quantitative surveys that explore how often people use a certain product. But you need more than numbers to help you understand why they use a product in a certain way. That's where generative research comes in. It gathers valuable information that helps explain what users think and why they do certain things.

• Build a product roadmap: Generative research is very advantageous at the beginning of the product life cycle. Based on the insights you gather, you can build a product roadmap that aligns with user needs and wants. But it’s also useful throughout the product life cycle, too. Feedback from user interviews or insights from diary studies can generate ideas for new product features. 

Cons

• It’s time-consuming: Generative research takes time to prepare and conduct, even if you have a dedicated ReOps function. You have to consider participant recruitment, interview questions, and other prep work. Conducting the research is also time-consuming, which can limit the number of interviews or activities you can do per week.

• It can be costly: Generative research can also be expensive. After all, time is money. The longer the research session, the more expensive the research incentive is. Not to mention calculating the cost of time and resources required. 

• Extensive synthesis and analysis: Analyzing generative research data can be time-consuming, as it requires watching back interviews, tagging and highlighting critical information, and writing interpretations of the results. For example, going back to our money-saving app idea, if you found users aged 21–30 viewed saving money differently to those over the age of 31, you’d have to explain the reasons. 

What is evaluative research?

Evaluative research is a method used to measure how well a product meets the needs and goals of users. It’s used to explore how well something is working, whether that's a concept, product, or service. Based on the insights generated from evaluative research, UX researchers will work with UX designers and engineers to improve the product. 

Evaluative research is conducted throughout the product life cycle. For example, a UX designer might conduct prototype testing during the early design phase, while first click testing can take place at a later stage. Surveys can also be conducted when there’s a minimum viable product. 

The main types of generative research include: 

• Usability testing

• Closed card sorting

• A/B testing 

• Tree testing

• First click testing 

• Surveys

First click results in Lyssna, displayed as a heatmap

Why is evaluative research important?

Evaluative research helps product teams validate their ideas and prototypes, and ensures if they’re on the right track or if it’s time to return to the drawing board. 

This research includes both qualitative and quantitative methods to assess how successful a product or feature is, and to gather user feedback on the overall experience. 

Through evaluative research, you can:

• Assess whether a product or feature is meeting user needs.

• Check the success of a product or feature against its intended use.

• Collect feedback on the overall product and user experience.

The data gathered from evaluative research helps guide product changes, validates design direction, and allows product teams to execute on user feedback.

How do you conduct evaluative research?

Conducting evaluative research differs depending on the method you’re using. Let's explore the main types outlined above.

• Usability testing: Test the usability of a product by observing users as they perform tasks on it. Usability testing can identify usability problems, validate prototypes, learn about the behaviors and preferences of your users, build empathy, uncover improvement opportunities, and confirm your product meets user expectations.

• Closed card sorting: Present users with a group of labeled cards and ask them to sort the cards into predefined categories. This is a good way to evaluate how intuitive your existing navigation is, or to test the suitability of predefined categories.

• A/B testing: Test two different versions of a product or feature to see which one performs better in terms of user engagement, conversion rates, or other metrics.

• Tree testing: Test the navigational structure of a website or application by asking users to find specific items within a simplified text version of your site map.

• First click testing: Ask users to complete a specific task and note which element on the page they clicked on first. Use this to understand whether the design of the page is intuitive and easy to use.

• Surveys: Survey users to gather feedback about their experience with a product or feature. This can help to identify areas that need improvement and gain insights into user behavior and preferences.

What are the pros and cons of evaluative research?

Evaluative research is a common research method that has its pros and cons. Let's look at them in more detail.

Pros

• Clear understanding of a product's usability: Through prototype testing or first click testing, evaluative research allows you to observe participants using and navigating your product. This gives you a clear understanding of its usability and identifies areas for improvement. 

• Continuous product improvement: Evaluative research enables continuous product discovery and improvement by providing iterative user feedback at any point in the product life cycle. 

• Uncover hidden issues: Evaluative research can reveal hidden issues that you may have overlooked. For example, certain product features or buttons may have been left off prototypes, which users can help pick up on.

Cons

• Difficult to synthesize and analyze results: It can be hard to synthesize and analyze results from evaluative research, especially when it comes to prototype testing. For example, it can be challenging to analyze hours of video footage and store and share multiple recordings, especially for long sessions.

• Lengthy process: Testing existing and new solutions can be a lengthy process, with research projects potentially ongoing for months (or even years in extreme cases) before reaching a conclusion. 

What’s the best research method: Generative or evaluative?

Honestly, there's no correct answer to whether generative or evaluative research is better. It depends on your research goals and the product you’re working on. 

Generative research is more suitable when you want to gain a deeper understanding of user needs and desires. However, evaluative research is best suited for testing a product's usability and user experience against internal expectations.

The best thing to do is to take a hybrid approach and do both types of research. Before developing the product, you can conduct generative research to understand your target audience better. As you design and develop the product, you can carry out evaluative research to assess its usability and gather feedback. By using a hybrid approach, you can get a comprehensive understanding of your users and create a product that meets their needs. Ready to conduct generative and evaluative research for your UX project? 

Elevate your research practice

Join over 320,000+ marketers, designers, researchers, and product leaders who use Lyssna to make data-driven decisions.

Sign up for free

Sign up for free

Sign up for free

Frequently asked questions about generative vs evaluative research

What is generative research?

What is evaluative research?

What is generative vs evaluative research?

--

Megan Johnson is a B2B SaaS content marketing writer. In between writing articles and all things SEO, she’s usually on her next travel adventure.