Generative vs. evaluation research: what’s the difference and why do we need each?

Janelle Estes | February 6, 2020
Generative vs. Evaluative Research: What’s the Difference and Why Do We Need Each?

The goals of generative and evaluation research (sometimes referred to as evaluative research) are very different. Generative research helps you define the problem you’d like to design a solution for. Evaluation research, on the other hand, helps you evaluate an existing design (in prototype, final, or some other form).

In this post, we’ll tease apart these two research design types with detailed differences between the two research methods, why they’re valuable, and the common research approaches associated with each.

Generative research

What is generative research?

Generative research is defined as a method of research that helps researchers develop a deeper understanding of users in order to find opportunities for solutions and innovation. Sometimes referred to as discovery or exploratory research, the goal is always the same.

Generative research definition

The goal of generative research is to look to the world around you to find opportunities for solutions and innovation. These solutions could be new products or experiences or they could be an update or improvement to an existing one.

In order to identify new and innovative solutions, you must define the problem you are trying to solve. This requires you to truly understand how people live, including their environments, behaviors, attitudes/opinions, and perceptions.

When conducting generative research, the most important thing to do is to keep an open mind—you don’t actually know what the problem you’re trying to solve is yet.

Why is it valuable?

Quite frankly, if you don’t conduct generative research, you could create something that no one actually needs or uses. Take a look at this list of failed products. Many failed because they were addressing a (sometimes fictitious) problem the creators didn’t truly understand.

You don’t develop the best solution by doing lots of evaluative research and refining your design—you develop the best solution by properly identifying the problem with generative research.

How UserTesting can help you conduct generative research

At UserTesting, we’re able to conduct generative research with our platform by collecting data and human insights that reveal people’s behaviors, needs, and opinions. Common generative research methods include: ethnography, contextual interviews, focus groups, and data mining.

Our customers often:

1. Ask people to record their everyday activities, behaviors, and thoughts. Previously, this type of research was only possible when a researcher followed a participant around for a given period of time. For example, below is a video of a participant shopping for a television in-store. While shopping, he notes that the price display is not as intuitive as he would have expected. This type of research provides context and insight into customers’ shopping behaviors and habits, and may help teams define a problem they’d like to create a solution for.

2. Explore people’s attitudes, preferences, and opinions. This often helps companies understand their target audience’s point of view so they can create better solutions and experiences. In the video example, below, a woman shopping for shoes describes how it’s extremely beneficial for her to see a picture of someone actually wearing the shoe. This information could be used by the DSW digital team to update their product pages.

3. Understand people’s actions, thoughts, and feelings by conducting focus groups. This allows participants to interact and converse with others while discussing a topic and allows the researcher to gather in-depth details about a given topic. Focus groups are valuable for understanding people’s motivations, but they should never be a replacement for behavioral research, such as usability testing or ethnography.

Evaluation research

What is evaluation research?

Evaluation research, also referred to as evaluative research, can be defined as a research method used for assessing a specific problem to ensure usability and ground it in wants, needs, and desires of real people.

Evaluation research definition

The goal of the evaluative research methodology is to test your existing solution to see if it meets people’s needs, is easy to access and use, and is hopefully even enjoyable. This type of research should be conducted throughout the development lifecycle, from early concept design (think rough sketches or prototypes) to the final site, app, or product.

Why is it valuable?

Evaluative research should always be a part of the iterative design process. Getting designs in the hands of our users as soon—and as often—as possible ensures that the experience will be shaped and refined to truly meet customer needs and expectations.

How UserTesting can help you conduct evaluation research

We can get concepts and designs in front of your target audience quickly and easily. If you have something for your participants to view or even hold, we can get feedback on it. Our customers commonly:

1. Ensure the design meets users’ expectations as early as possible by getting feedback on working designs or prototypes. In the video below, a participant is commenting on a prototype of the UserTesting platform. The screen she is viewing is fully interactive, and she’s able to speak aloud how she interprets what she sees. This provides valuable insight for us as we continue to improve our platform.

2. Optimize existing experiences by asking participants to complete key tasks on live designs. In the example below, see how a user isn’t able to filter her flight options as she would normally expect. When searching for a flight, she would only like to view direct flight options, but is unable to find where to filter. This is a small issue, but removing that obstacle could increase customer satisfaction, decrease site abandonment, and ultimately increase the bottom line.

3. Test “real life” experiences, such as unboxing a product. In the video below, participants unbox and set up a Nest thermostat. This provides insight and context into the experience after customers have completed a purchase, including getting the item, opening the package, installing it, and using it for the first time.

Which is better, generative or evaluation research?

As you can see, there is no “best” approach. The two research methods have different goals and should be done at different times in the development and design process. However both extremely valuable, the bottom line is this: companies that create great experiences are more than likely conducting both generative and evaluative research all the time.

For further reading

The UX research methodology guidebook

Getting started on a new research project—even if you’ve conducted hundreds of studies before—can be complicated.
About this author:

Janelle is UserTesting's Chief Insights Officer and an expert research practitioner fascinated by human behavior and intrigued by data insight. She brings over 15 years of experience conducting large-scale customer research initiatives with both B2C and B2B companies across a variety of industries to help them transform their customer, user, and brand experiences.