4 Ways to Test Your Product In the Wild

| February 26, 2015
Sign up to get weekly resources, and receive your FREE bonus eBook.
Thank you!

Get ready for some great content coming to your inbox from the team at UserTesting!

These days, you can test more user experiences than ever before.

Here at UserTesting, we recently launched our new mobile recording technology, which makes it possible to test user experiences in real-world locations like retail stores, restaurants, stadiums, airports, and buses.

You can even test non-digital experiences. For example, we’ve run user tests on activities like doing the laundry, using a Nest thermostat, and even mowing the lawn!

Today, we’re going to show you four exciting user experience research techniques. We’ll break them down into categories based on what you’re testing and where you’re testing it. (As you’ll see below, most tests are combinations of two or more of the four techniques.)

  1. Device testing
  2. Beyond-the-device testing
  3. Destination-based testing
  4. Participant-defined location

4 examples of testing techniques

What experience do you want to test?

1. An experience on a device (Device testing)

Device testing refers to a test of a digital experience on a desktop computer, smartphone, or tablet. It’s what most people imagine when they think of user testing.

Here are some of the most common types of device tests:

  • Desktop websites

  • Mobile websites

  • Mobile apps

  • Prototypes

  • Two or more different websites (competitor tests)

  • Longitudinal studies on an app

If you’ve used UserTesting before, you’re probably familiar with something like the video below.

2. A non-digital experience (Beyond-the-device testing)

Here’s where things get really exciting.

With beyond-the-device testing, you can test the customer experience of a non-digital product or location out in the real world.

For example, if you wanted to test the experience of buying toothpaste, you could ask test participants to go to the grocery store, turn on the camera on their smartphone, and speak their thoughts aloud while they find a brand of toothpaste that they like. They wouldn’t need to interact with a website or app at all (although you could certainly ask them to look up toothpaste coupons online while they’re standing in the aisle).

This is a great way to do market research and find out what’s going on on the customer’s head when they make a purchase decision in the real world. It’s also a good way to find out how customers are discovering and using your product (or other products) on a daily basis.

A beyond-the-device test might look something like this example of a user interacting with a front-loading washing machine:

Where do you want to test?

3. In a particular location (Destination-based testing)

Destination-based testing involves any experience that requires the user to be in a designated location (like “the nearest Target store”) or a particular type of setting (like “on a bike ride”).

For example, maybe you want to watch people use a frequent buyer rewards app at Starbucks. In your destination-based test, you can specify that users need to go to their nearest Starbucks before beginning the test.

This type of test is great for:

  • Understanding how users interact with a product in a distracting setting

  • Realistically testing interactions between mobile apps and real-world locations or products

  • Gathering users’ impressions of in-person experiences

  • Mobile experiences and beyond-the-device testing

If you’d like to verify that the user is in the correct location, you can add a task into your test plan that asks the user to turn on the camera on their device and take a picture of their surroundings. Since you’ll see everything that’s happening on their screen, it’s like looking at the world through their eyes.

Here’s a clip from a test we ran on the Yelp app. We asked users to use the app to choose a nearby restaurant for lunch, and then go to that restaurant and open their camera.

Note: Destination-based testing is typically not used to test desktop experiences. (It’s not likely that a user will bring their computer out to a destination test!)

4. Anywhere (Participant-defined location)

Most tests on websites, apps, and physical products don’t require the user to go to a particular place. They can give equally useful feedback whether they’re at home, on the train, at the mall, or anywhere else.

With participant-defined location tests, the participant can take the test wherever they are.

This type of test is great for any user test that doesn’t rely on users being in a particular location. You can also use it to find out where users are naturally using your product (you can ask them to describe or take a picture of their current location, wherever it may be).

Again, if you’re familiar with UserTesting, this is the type of test you’ve probably run before.

Putting it all together

With a little creativity, the possibilities are endless! You can combine different testing methods to gain a holistic view of how a user interacts with a product.

As you might have imagined, most test plans combine at least two of the four techniques. Here’s an example of a study that uses all four techniques.

Let’s say you have a jogging app and you want to find out how people are using it. You could set up a series of tasks that explore the complete user experience.

  • To measure the onboarding experience of the jogging app and find out what users are expecting to be able to do with it, you could ask users to set up the app and start exploring it from wherever they’re located using the mobile recorder. (This would be a device test in a participant-defined location.)

  • Then, in a follow-up test, you could have them open the app the next time they go for a jog. (That would make this a destination-based device test.)

  • In the same test, you could also ask them to take a picture of their location and explain where and how far they’re planning on jogging today. You could even have them take a picture of their jogging shoes and describe why they chose to wear that brand. If you really wanted to get creative, you could ask the user to record themselves jogging. (These tasks would add a bit of beyond-the-device testing into the study.)

Advantages of in-the-wild mobile testing

With the wide variety of testing techniques you can use, you get to go beyond usability testing and really explore how a product fits into a user’s life. You can uncover richer insights about how users naturally interact with products. You can even conduct the kind of market research that used to be very expensive and time-consuming.

It’s just like looking over your customer’s shoulder.

And with UserTesting’s mobile recording technology, it’s faster, easier, and more realistic than ever before.

What are you looking forward to testing next? Tell us on Twitter.