Watch a Demo

Omni-Channel vs. Multi-Channel Testing—What’s the Difference?

| December 15, 2014
Sign up to get bi-weekly insights and best practices, and receive your free CX Industry Report.
Thank you!

Get ready for some great content coming to your inbox from the team at UserTesting!

omnichannel-header

If you’re in the UX industry, you’ve probably heard the words “omni-channel” and “multi-channel” getting thrown around, but when it comes to defining them, things get a little hazy.

The main difference between these two methodologies lies in the process being tested.

If you want to test a single process that spans across multiple devices, you want OMNI-CHANNEL testing.

If you want to test the same process on a range of devices, you want MULTI-CHANNEL testing.

Let’s take a look at a few examples.

Omni-channel testing: Budgeting app example

Imagine you own a budgeting app. The app has both desktop and mobile versions that are used for different purposes. The desktop is where users set up their accounts, incomes, and categories of spending, and the mobile application is for logging and monitoring the users’ expenses as they go.

To test all aspects of the experience, you’d want to have users go through some omni-channel testing:

  • First they would take a test to set up their accounts on the desktops
  • Then they would take a separate test on their mobile devices to log or track their financial activity.

It’s all part of a massive set of tasks, some of which are accomplished on desktop and some of which are suited for mobile.

In this case, the users would need to be the same for each set of devices, so each user would take BOTH a desktop and a smartphone test.

Here’s how to set up omni-channel tests on the UserTesting platform:

Keep in mind that, in this example, the two platforms have to be tested in a particular order: desktops first, and smartphones second.

In the first test (on desktop in this example), you would include a screener question that goes something like this:

“This is part 1 of a two-part study which must be completed in 3 days. The second part of the study will need to be done on your smartphone. Are you willing and able to take part in both studies?”

  • Yes (Accept)

  • No

  • I don’t know

You’d also want to remind test participants in the introduction that this is a two-part test. It’d look something like this:

“You indicated in the screener questions that you are willing to take multiple tests for this study. This is the first part. The next part will appear in your dashboard within [X] hours.”

Once the desktop tests have filled, you can launch the second test to the same group of users.

favorite_users

Select “Reuse favorite users” when you choose your target audience for an omni-channel test.

In the “Choose Your Target Audience” section of your UserTesting dashboard, click the drop-down menu next to the question, “Which panel do you want to use?” and type in the user names of the first test’s participants—the test will be distributed directly to those users instead of the full UserTesting panel.

Note: this option is also available on your dashboard, under each session. Just click “ADD MORE USERS” and type their user names into the “Use your favorite users” field.

favorite_users_2

You can specify users when you add users to an existing study.

Sometimes test participants don’t get a chance to revisit their dashboard for a few days, so it’s best to over-recruit for the first test. If you want to hear from 7 users total, for instance, you might order 10 sessions for the desktop round with the expectation that about 7 will be able to complete both tests.

Multi-channel testing: Cable station website example

Imagine you own a cable television station, and you have set up a site with responsive design. Users are able to look up shows, showtimes, news, and clips on any device.

Of course, that’s something you want to ensure via thorough testing, so you would set up a multi-channel study: have users perform common tasks on desktop, mobile, and tablet devices to determine any issues.

In this case, the users would not need to repeat the study on all devices; each user would complete the test on only one device. In other words, one set of users could test the desktop experience, a second set could take the tablet tests, and a third set could check out the smartphone.

Here’s how to set up multi-channel tests on the UserTesting platform:

Keep in mind that you can release the tests for all devices simultaneously—there’s no need to use the same test participants on both desktop AND smartphone, as you would in the omni-channel example above.

The “Choose Your Target Audience” section of the UserTesting order form lets you set up multiple groups of users, so you can create a group for each device you are looking to test.

multi-channel order

For a multi-channel study, simply designate how many test participants you want on each device type.

Your tasks should be the same across each device type (it’s just the smart thing to do, methodologically), so avoid using language that’s specific to one platform (like “click” or “hover”). Instead, keep your tasks and questions task-oriented:

  • “Determine when Hairspray Heroes will be playing.”
  • “Find the latest news for Love and Other Horrors.”

Combination of both methods: Toy store site example

Imagine your toy store website has a wishlist feature, but your analytics are showing you that conversion is much lower for users who make their lists on their smartphones. You can use BOTH types of testing to figure out why this might be.

Multi-channel

To test the overall usability of the wishlist feature, you could run some multi-channel testing and see if any mobile-specific issues appear.

Omni-channel

You can also set up some omni-channel testing and let users explain when and why they would make the switch to their desktops; the issue might have nothing to do with the wishlist feature at all!

It could be that their payment information isn’t saved on the device, or that people don’t want to pull out their credit cards and make purchases in the middle of a waiting room or supermarket, wherever they might be. It could also be that they need to refer to their personal budgets before committing to a purchase, or that the checkout button isn’t displaying when they reach the cart page. You just don’t know until you test!

Get started with omni-channel and multi-channel testing

As you can see in the last example, many sites and apps benefit from both multi-channel AND omni-channel testing; it depends on what kinds of processes your users are attempting to do.

Having both of these buzzworthy methodologies in your back pocket is just another way to make your UX research—and hopefully, your product—better than ever.

Ready to get started? If you’re a UserTesting Enterprise client, just email your Client Success Manager to get started. If you’re new to UserTesting, click here to start a free trial of a Pro plan. We’d love to help you set up the perfect omni- or multi-channel study!