Writing your user test plan: open-ended vs. specific tasks and questions

By Spencer Lanoue | May 18, 2015
Image
Writing Your User Test Plan: Open-Ended vs. Specific Tasks and Questions

In any successful user experience study, you’ll need to write a foolproof test plan to guide users through the tasks and questions that will give you the insights you need.

Designing your user test plan isn’t always easy. There are no hard-and-fast rules, and the tasks and questions you should ask are going to depend on what your objective is.

A task is an action or activity that you want your user to accomplish, and questions are used to elicit some form of feedback from the user in his or her own words. Sometimes it’s best to leave tasks and questions open-ended, and in other situations, you need to be more specific.

In this article, I’m going to use examples to explain the difference between open-ended and specific tasks and questions. I’m also going to show you when to use each one, and pitfalls to watch out for. Let’s get started.

Open-ended tasks and questions

Open-ended tasks and questions give your test participant minimal explanation about how to perform the task or answer the question. The key here is to watch users uncover the answer or solution on their own. Keep in mind, responses may vary drastically from one test participant to the next.

Examples

Imagine that you’re testing a fitness app. Here’s an example of what an open-ended task and question might look like:

  • Open-Ended Task: Please spend 5 minutes exploring the app like you normally would.

  • Open-Ended Question: What would you expect to be able to do with a fitness app?

When to Use Open-Ended Tasks and Questions

  • Find Areas of Interest - When you’re not sure where to focus your test, try this: run a test using open-ended tasks and questions. You’ll be sure to find areas of interest to study in a more targeted follow-up test.

  • Exploratory Research - If you’re doing exploratory research, open-ended tasks and questions can help you figure out how people are using your product and the kinds of problems they’re running into.

  • Identify Usability Issues - If you want to find things that are broken or cause friction for your users, letting them explore freely will uncover issues you may not already be aware of.

Pitfalls to Watch Out For

  • Clearly Define Your Test Objective - When you’re asking open-ended tasks and questions, you still need to make sure you have a clear objective in mind. (For example, “Can visitors find the product they’re looking for?”) If you don’t know what you want to learn, your test participants may wander around aimlessly without uncovering anything useful. Make sure that your tasks and questions support the ultimate goal of your research.

  • Keep Users Talking - Make sure you keep the users talking while they’re performing open-ended tasks. You don’t want them to forget to speak their thoughts aloud as they explore freely, so remind them to explain why they’re doing what they’re doing.

Specific Tasks and Questions

Specific tasks and questions will give users clear guidance on what actions to take and what features to speak about. This focuses your research on the exact issues you’re interested in investigating.

Examples

Again, imagine that you’re testing the same fitness app as above. Here’s an example of what a specific task and question might look like:

  • Specific Task: Open up the heart rate tracking feature and try to track your heart rate.

  • Specific Question: Did you find the heart rate feature to be helpful? Why or why not?

When to Use Specific Tasks and Questions

  • Test Specific Features - Give test participants specific instructions if you want to test the usability of a certain feature of your product. For example, “Please use the search bar to find a pair of men’s black dress shoes in size 11.”

  • Complex Products - If you have a complicated, non-traditional, or unusual product that people won’t automatically know how to use, specific tasks will guide them through it and explain the context.

  • Conversion Optimization - If you know there’s a specific point in your conversion funnel where people are bouncing, use specific tasks and questions to watch them go through the funnel. This will give you the context and insights to understand why they’re bouncing.

Pitfalls to Watch Out For

  • Avoid Giving Exact Instructions - Even if you’re giving your test participants specific tasks and questions, you need to keep a balance. You don’t want to tell them every single thing to do, because then you won’t learn anything. Let them do some of the work on their own, and try not to hand-hold them too much.

  • Avoid Leading Questions - Try not to bias your questions by suggesting a specific response. For example, “How easy was it to find the pricing page?” This is such a common trap, and even the most experienced researchers fall into it. Make sure all your questions are as objective as possible. A good way to reframe that questions could be, “How easy or difficult was it to find the pricing page?”

Conclusion

The key to a successful study is to ask users to perform tasks followed up by questions that will give you the type of insights you need. Once you’ve clearly defined your test objectives, you’ll be able to decide whether to use tasks and questions that are either open-ended or specific.

Open-ended tasks and questions help you learn how your users think. They can be useful when considering branding, content, and layouts, or any of the “intangibles” of the user experience. They’re also good for observing natural user behavior.

Specific tasks and questions can help you pinpoint where users get confused or frustrated trying to do something specific on your site or app. They’re great for getting users to focus on a particular feature, tool, or portion of the product they might not otherwise interact with. If you enjoyed this article and you want to learn more about writing great test plans, check out The complete guide to user testing websites, apps, and prototypes.

Insights that drive innovation

Get our best human insight resources delivered right to your inbox every month. As a bonus, we'll send you our latest industry report: When business is human, insights drive innovation.

About the author(s)
Spencer Lanoue

Spencer Lanoue is a marketer who helps UX designers, PMs, and marketers make things people want.