Moderated vs. unmoderated usability testing: The pros and cons

| September 13, 2018
Sign up to get bi-weekly insights and best practices, and receive your free CX Industry Report.
Thank you!

Get ready for some great content coming to your inbox from the team at UserTesting!

Being customer obsessed isn’t just a competitive advantage these days, it’s a basic necessity. Having a human-centered mindset at an organizational level is important, and one way teams can validate their efforts is through qualitative studies like moderated and unmoderated usability testing. While unmoderated and moderated usability testing can sound intimidating, we’re here to help break it down for you with everything you need to know.

One question that we often hear from customers is which is better: moderated or unmoderated usability testing? The answer, as you might’ve guessed, is that it depends. Like any research strategy, there are differing benefits for each approach making them more or less suitable depending on what you’re studying and your ultimate research goals.

Moderated vs. unmoderated tests, what’s the difference?

We’re about to dig into the pros and cons for each approach, but before we do, it’s important to understand how the two strategies differ.

Moderated usability testing

With moderated usability testing, a real person will be there to help facilitate (i.e., moderate) the test. The moderator will work directly with the test participant, guiding them through the study and will be there to help guide and answer questions if the participant encounters any challenges while completing their tasks.

What’s great about moderated studies is that they can be conducted either remotely or in-person. One of the major challenges with moderated testing, however, is that it can require a bit more planning upfront since you’ll need to coordinate schedules with participants. It usually requires a designated, quiet location, where you can interact with your participant, free of interruptions.

Unmoderated usability testing

Unmoderated usability testing is just like it sounds. It’s not monitored or guided, so there’s no one else present during the study except the participant. The participant completes any tasks and answers questions at their own pace, on their own time, at a time and location of their choosing. Pretty nice, right? Unmoderated testing tends to be faster as participants can complete their tests at any time without any disruption to your daily workflow.

Now that we’ve established the differences between the two let’s dive into how and when to best apply each strategy based on your research needs.

When to use moderated usability testing

It helps to think of moderated user testing as an interview or a real-time conversation that you’re having with a participant or customer. Everything that you’d do to prepare for that type of interaction is what you’ll need to consider for a moderated study.

Pros

Moderated testing works best when you need a high level of interaction between you and your participant. For example, if you want to study a prototype with limited functionality, or a complicated process or concept, moderated testing provides you with the interaction you’d need to guide a participant through the study. It’s also an excellent way to conduct interviews, understand the customer journey, and discover pain points.

Moderated user testing is also a great way to observe body language and pick up on subtle behaviors and responses. It enables you to probe participants for more information if they seem stuck or confused.

This type of test allows you to develop a report and have a natural conversation with your customers which helps establish trust and can lead to candid feedback that might not be possible with other qualitative research methods.

Cons

Because of the additional time and resources, moderated testing does cost more than unmoderated testing. However, you can reduce some of that cost by conducting remote moderated studies, rather than in-person. This will give you the opportunity to connect with participants all around the world and reduce the need to block out a dedicated time and space for onsite interviews. Who doesn’t love getting some hours back in their day?

When to use unmoderated usability testing

This type of testing is best for validating concepts and designs quickly with a diverse group of participants. Unmoderated testing works great if you have specific questions that you need answered, you need a large sample size, you need feedback quickly, or if you want to observe a participant interacting in their natural environment.

Pros

The beauty of remote unmoderated usability testing is that it can be done anytime, anywhere, and you typically have actionable feedback within a day if not sooner. Because a moderator isn’t needed, the cost is typically much lower than moderated tests, which can enable you to run more tests with a wider variety of participants.

Cons

Since unmoderated tests are completely unsupervised, it requires a fixed set of questions and tasks for participants to complete and doesn’t allow for follow up questions or clarification. If participants run into issues, have technical difficulties, or don’t understand the tasks or questions, you won’t have the ability to step in and guide them.

 

At the end of the day, the choice is up to you, but now you can make an informed decision. We’d love to hear what method works best for you, share your experience with us on Twitter using the hashtag #UTConnect.

Want to learn more?

If you’d like to learn more about choosing the best qualitative test for your research, download our eBook, Which qualitative method is right for you? Your guide for evaluating pros, cons, and use cases.

To learn how UserTesting can help you understand your customers through on-demand human insights, contact us here.