How to get competitive insights with UserTesting

Identify the competition

Before setting up your competitive user tests, it’s crucial to determine which competitors you’ll want to learn about. There are many ways to define and assess “the competition.” Your competition comes in many shapes and sizes, so don’t limit testing to only your most evident and direct competitors.

Direct: Brands with products very similar to yours, at similar price points, and targeting similar buyers. In your study, participants will share their feelings about these brands and what they like and dislike about their experience.

Comparable: These companies do something similar in your industry but aren’t your direct competitors. 

Aspirational: Best-in-class brands. Even if they aren’t direct competitors or even in your competitive set, you can learn what makes them stand out. 

Unknown: Without pointing them to any brands in particular, explore how people search for a product or solution in your industry or meet a specified need. This allows you to discover unexpected competitors, such as fast-moving newcomers, that you may not have known about. You may even learn that your users solve this problem in other ways, like using pen and paper or a spreadsheet rather than a product.

Non-competitors: Popular and successful businesses—even if they have nothing to do with your company. Some customer experiences, like a seamless checkout process or relevant product recommendations, can apply across many industries and products. You may be surprised to find inspiration from a wide range of brands unrelated to your own!

Decide on your target audience

To recruit participants for your competitive study, you can test with your existing customers, competitors’ customers, target audiences, or even a broader base of test participants to determine if there are additional audiences you should be pursuing.

Known customers: Directly invite existing customers from your email list to participate in your study. Send tests to customers in your database. Note that these customers may already be loyal to your brand, so the feedback you’ll get from them may not be as unbiased as those from users unfamiliar with your brand. That feedback can still be quite insightful, especially if you’re investigating what a competitor might do to entice customers away from you.

Competitors’ customers: Set up screener questions to recruit test participants using your competitors’ products. This can give you a first-person view of why people choose your competitors’ products and what they like and dislike about their experience with those products. 

Target audiences: If you want to recruit users matching your target buyer personas, you’ll determine what demographics, job titles, or other traits you’ll want your test participants to share. In the Human Insight Platform, you can set up screeners to ensure you match with precisely the right users.

Open it up: You may also want feedback from adjacent customer segments to understand similarities and differences. For example, if your product is built for entrepreneurs, you may also want to explore how department leaders feel about your brand and your competitors. Or, you can even conduct interviews to understand better or confirm who you should target—you might be surprised by what you discover!

Get inspired with a real-life use case with ADT

Home security company ADT leveraged UserTesting’s Contributor Network to recruit their competitors’ customers to their competitor analysis study. They observed these users interacting with the competitors’ home alarm system products to identify which features users found helpful or annoying. They even zeroed in on potential pain points in those competitive products, such as whether leaves can trigger the motion detector. These insights were critical to the development of ADT’s mobile app. 

Read the complete case study

When planning out your competitive analysis deciding between moderated and unmoderated conversations are important. Let’s review your options. 

Conduct a live video interview

It can be helpful to have an open, free-form conversation with customers or audiences about their perception of the competition—the competitive landscape of your product category or one specific competitor. With UserTesting, you can do this by using Live Conversation: a type of study where you’ll meet one-on-one with your target users via video call.

In these conversations, start by discussing the test participant’s opinions on the product category or the competitor’s product before bringing up your product. You don’t have to discuss your product if you are focused on learning about the competition. Not disclosing the company you represent can help ensure honest and unbiased customer feedback. You’re not comparing your product against the competition; you’re just trying to learn as much as possible about the competition.

First, segment the audience and identify the one(s) you want to speak with. Maybe they’re familiar with your brand or your competitors or maybe not. Maybe they’re a power user in your product category and are equipped to provide precise and knowledgeable feedback or maybe not. 

Some sample conversation starters for your live conversation could include: 

• How did you hear about the product? 
• What process did you use to pick this product over the competition? 
• What other products were you interested in purchasing? 
• Why did you select this one over others? 
• Is it worth more or less than X product? 
• What do you like about the product/solution? 
• What do you wish that this product/solution was better at? 
• How satisfied are you with customer support and working with the company?

Do a thorough analysis of a competitor using unmoderated testing

Just as you would structure and conduct a test about your brand’s products and experiences, you can do the same for one or more competitors. Setting up unmoderated tests is faster and easier than Live Conversation because your audience can take the test whenever they want. 

Some of the tasks or questions that you’ll want to ask a test participant about a competitor can include: 

• How unfavorable is (1) or favorable (5) is your attitude towards [brand name]? 
• How inconsistent (1) or consistent (5) is [brand name]? 
• How ordinary (1) or unique (5) is [brand name]? 
• How difficult (1) or straightforward (5) is it to recognize [brand name]?
•  If [brand name] were a person, how would you describe their personality? 
• What are your initial impressions of this website? 
• Rating scales on relevant dimensions, including product/website attractiveness, trustworthiness, simplicity in design and usability, and others 
• Tasks that the test participant should complete, with questions on how easy or difficult it was to complete the task

You should include metrics questions when building out your test plans, such as multiple choice or rating ccale, to secure quantitative data alongside the qualitative feedback. 

For example, you could include rating scale questions asking the test participant to rate you and your competitor on specific dimensions.

When the tests are completed, you can use the metrics tab to view the responses. The charts help you see patterns or outliers so that you can decide where to spend your time watching video clips.

Conduct a “bake-off”

The term “bake-off” refers to a competition where cooks compete head-to-head—perhaps in front of an audience. In the business world, the metaphor refers to an effort by a buyer to compare and select from competing vendors.

You can conduct a small-scale bake-off with UserTesting by pitting your brand, designs, products, or messaging against your competitors for a side-by-side comparison. This will help you better understand how your experiences measure up. It also helps ensure you say, share, show, and provide precisely what your customers expect to win their business.

When doing this, you’ll want to set up a study that includes identical tasks and questions across two or more brands to understand test participants’ opinions and experiences with each. You may also want to add tasks and questions at the end that ask the test participant to identify which experience they preferred and to explain why.

Spin up test quickly with templates created by research experts for competitive comparison. 

Get inspired with Spread Group

Made-to-order clothing company Spread Group has made competitor testing a regular part of their user research practice. By testing against the competition, Spread Group discovered they needed more explicit labeling and thorough descriptions of their organic products to avoid being seen as “greenwashing.” With these insights, they increased sales of organic products on their website by 20%. 

Read the complete case study

Finally, it’s worth mentioning that there are many different schools of thought regarding how much brands should invest in monitoring the competition. Some will say that tracking the competition keeps you well-informed and eager to improve to win continually. Others take the opposite view, saying that over-indexing on competitive intelligence can be myopic and deter true innovation and customer-centricity. 

Understanding the competition doesn’t necessarily mean that you should follow it. Jack Ma, the co-founder of tech conglomerate Alibaba Group, has been quoted as saying, “You should learn from your competitor, but never copy. Copy and you die.” Interestingly, he’s also been quoted as saying, “Forget about your competitors. Just focus on your customers.”

The point is: you must have access to data and insights first. Once equipped with relevant and vital customer insights, you can make fast, informed decisions when determining the following steps, allocating resources, and generating your business’ winning strategy. 

Running competitor analysis with UserTesting hits the sweet spot—offering you the competitor insights you need to get ahead while centering the focus on what your customers are doing, saying, and thinking. 

Ultimately, genuinely understanding customer needs and providing an exceptional customer experience is your company's best competitive move.

With an experience research platform, anyone in your organization can run and analyze competitor studies and seasoned UX researchers can setup guardrails like templates and dashboards. 

Use these valuable features to help you conduct competitive analysis. 

Test templates

Suppose you’re short on time or less familiar with conducting competitive analysis. In that case, you can use our ready-to-use test templates when creating a test from scratch. 

Once you select the template, you’ll see tasks and questions automatically inserted into your test plan. From there, customize the test plan as needed and adjust your audience targeting before launching the test. 

Specifically, here are some things you’ll want to keep in mind when using the template: 

• When comparing two website experiences, insert a competitor’s website with either your website or a second competitor’s website. 
• If you don’t want to do a side-by-side comparison, you can customize the test plan by editing or removing questions according to your needs. 

Balanced Comparison

When comparing two things, the order in which they encounter the two items can create a subtle bias within a test. This is known as order bias, and it can skew your research if you aren’t careful. The Balanced Comparison feature was designed to reduce the risk of that bias. 

When you use Balanced Comparison, you create two sections in your study: part A and part B. Your test participants will all interact with both parts, but the order in which participants see the two parts is automatically alternated: half of the participants will see part A first, and the other half will see part B first. This negates the order bias, so your results will be more insightful.

Enhanced Metrics

UserTesting’s Enhanced Metrics show click paths, time spent on each task, and other behavioral data. Set up your test with tasks that ask the test participant to go through a similar flow across different brands’ desktop web experiences. You’ll be able to see how many and which web pages each participant visited in their attempt to complete that task. 

For example, let’s say you want to understand how easy it is for a customer to find and select an item on your site compared to competitors in your industry. You can create this task and replicate it across the competitors you want to evaluate. Once tests are completed, you can view Metrics and see the click paths across the different brands to see which took longer or required more steps. You could then view the video clips associated with areas of interest to understand better each case's behavior, motivations, or challenges.

You can also set a success metric, like a webpage or pages that contain a specific phrase, to immediately understand things like: 

• Did the user visit a page we had in mind? 

• Were they successful in completing the task? 

• At what point in trying to complete the task did they visit this page?

• Once they hit the page in mind, did they realize it and indicate they had completed the task?