Generative vs. Evaluative Research: What’s the Difference and Why Do We Need Each?

| December 17, 2015
Sign up to get weekly resources, and receive your FREE bonus eBook.
Thank you!

Get ready for some great content coming to your inbox from the team at UserTesting!

The goals of generative and evaluative research are very different.

  • Generative research helps you define the problem you’d like to design a solution for.
  • Evaluative research evaluates an existing design (in prototype form or in final form).

In the post below, we tease apart the detailed differences between these two research techniques, why they’re valuable, and the common research approaches associated with each.

Generative research

What is it?

The goal of generative research is to look to the world around you to find opportunities for solutions and innovation. These solutions could be new products or experiences or they could be an update or improvement to an existing one.

In order to identify new and innovative solutions, you must define the problem you are trying to solve. This requires you to truly understand people’s live, environments, behaviors, attitudes/opinions, and perceptions. Indi Young, a user experience consultant and a founding partner at Adaptive Path, believes you must “feather your nest” with generative research. This includes rich data about your target audience and their needs/goals before defining the problem and crafting a solution.

When conducting generative research, the most important thing to do is to keep an open mind: you don’t actually know what problem you are trying to solve yet.

Why is it valuable?

Quite frankly, if you don’t conduct generative research, you could create something that no one actually needs or uses! Take a look at this list of failed products. Many failed because they were addressing a (sometimes fictitious) problem the creators didn’t truly understand.

You don’t develop the best solution by doing lots of evaluative research and refining your design. You develop the best solution by properly identifying the problem with generative research.

How can UserTesting help you conduct generative research?

At UserTesting, we’re able to conduct generative research with our platform by collecting data and insights into people’s behaviors, needs, and opinions. Common generative research methodologies include: ethnography, contextual interviews, focus groups, and data mining. Our clients often:

  • Ask people to record their everyday activities, behaviors, and thoughts with our mobile recorder. Previously, this type of research was only possible when a researcher followed a participant around for a given period of time. For example, below is a video of a participant shopping for a digital camera in a store. While shopping, she looks at price and available features, and she compared the online price to the in-store price while she was in the store. This type of research provides context and insight into customers’ shopping behaviors and habits, and may help teams define a problem they’d like to create a solution for.

  • Explore people’s attitudes, preferences, and opinions. This often helps companies understand their target audience’s point of view so they can create better solutions and experiences. In the video example, below, a woman describes the type of notebook she typically uses while viewing the Moleskine website. She talks about her needs and preferences. This information could be used by the Moleskine product team as they create new solutions or update existing ones.
  • Understand people’s actions, thoughts, and feelings by conducting a focus groups. This allows participants to interact and converse with others while discussing a topic and allows the researcher to gather in-depth details about a given topic. Focus groups are valuable for understanding people’s motivations, but they should never be a replacement for behavioral research, such as usability testing or ethnography.

Evaluative research

What is it?

The goal of evaluative research is to test your existing solution to see if it meets people’s needs, is easy to access and use, and is hopefully even enjoyable. This type of research should be conducted throughout the development lifecycle, from early concept design (think rough sketches or prototypes) to the final site, app, or product.

Why is it valuable?

Evaluative research should always be a part of the iterative design process. Getting designs in the hands our users as soon as–and as often as–possible ensures that the experience will be shaped and refined to truly meet customer needs and expectations.

How can UserTesting help you conduct evaluative research?

We can get concepts and designs in front of your target audience quickly and easily. If you have something for participants to view or even hold, we can get feedback on it. Our clients commonly:

  • Ensure the design meets users’ expectations as early as possible by getting feedback on working design or prototypes. In the video below, a participant is commenting on a prototype of a design or a working version. The screen he is viewing isn’t interactive, but he comments on what he sees and how he expects it to work. This provides valuable insight for the team as they continue to develop the site.
  • Optimize existing experiences by asking participants to complete key tasks on live designs. For example, a key task for site visitors to complete on the EMS website is to make a purchase. In the video, below, a participant tries to add an item to her cart but received an error message because she didn’t select a color. However, the product only came in one color. This is a small issue that the participant was able to correct, but removing that obstacle could increase customer satisfaction, decrease site abandonment, and ultimately increase the bottom line.
  • Test “real life” experiences, such as unboxing a product. In the video below, participants unbox and set up a Nest thermostat. This provides insight and context into the experience after customers have completed a purchase, including getting the item, opening the package, installing it, and using it for the first time.

Which is the best research approach: generative or evaluative research?

There is no “best” approach. The two research techniques have different goals and should be done at different times in the development and design process. However, they are both extremely valuable, and the bottom line is: Companies that create great experiences are conducting both generative and evaluative research all the time.