Testing with Exact Demographics: When to Use Them and When to Lose Them

Here at UserTesting, we get a lot of questions about what demographics should be used while testing a product.

Being able to pool from a wide variety of demographics is necessary when performing UX research. Users come in all different shapes and sizes. Different companies have different needs, which is why we sometimes want to be able to find the left-handed former basketball players. They can tell you their opinions about your new Lefty Hoops website, for example.

Depending on what type of product you have, your target market can be fairly common or completely obscure. There are some times when it’s important to test your product with your exact market, and other times when users from any demographic can give you the feedback you need.

If you have a small, specific target market, it may take longer to find the right users. This is why we like to encourage our clients to keep an open mind while creating their tests. It will always vary from one scenario to the next, but we do have a couple of general rules that make creating your demographic parameters a no-brainer.

Before you order a test, think about what specifically you’re testing

Rule 1: Testing usability for everyone

Do you want to know if your product simply works?

Want people to make it through the checkout process or find a new feature?

These are examples of when open demographics are completely appropriate. You want to watch someone else with your product and make sure they can navigate easily through it. It’s unnecessary to use specific demographics here because you just need some quick feedback. These types of test fill quickly and can provide feedback in as little as an hour.

Examples of when you can use open demographics:

• Seeing how people interact with your website on a mobile device

• Finding out if people understand how to use your new weather app

• Watching a user complete a transaction on your e-commerce site

Rule 2: Brand-related questions and niche demographics

Want to find out whether your target audience can quickly and easily understand the value of what you’re offering?

Interested in testing brand impressions and recognition, rather than usability?

Or maybe you have a product that is only going to be used by people with a specific set of knowledge and skills, such as software developers or auto mechanics?

This is when you typically want to test with more specific demographics.

In some cases, you might want to use test participants who are already familiar with your product, or might even already use your product. You can have those users compare your product with a competitor. Using specific demographics in that instance can provide you with more specific insight as to why your target market finds Company A more trustworthy than Company B, what type of information they would need before converting, etc.

Examples of when to use specific demographics:

• A high-end department store that wants to test brand impression with higher-income users

• A university that wants to test its website's usability with college students, high school students, and parents of teenagers

• An app for medical professionals that needs to be tested by the people who will be using it

• An up-and-coming company that wants to run a competitor test with people who are current customers of their competitor

Screener questions

When you’ve decided that you do need a more specific demographic group, one way to ensure you find the right test participants is with screener questions.

Screener questions are given to potential test participants before they can accept a test, and depending on the way they answer, they are either qualified or not to complete your test. Good screener questions will be able to filter for qualified respondents. Screener questions are formatted in a multiple choice fashion. Keep in mind that you can have more than one screener question.

For tips and best practices about recruiting the right users, check out our post on writing great screener questions.

Recruiting really hard-to-find participants, and how our team can help

There will be situations when you need a really specific point of view from a user. You might need an opinion from C-level executives, architects who have designed office buildings in California, or nurses over 40 who are familiar with a certain type of medicine. UserTesting's Panel Team can either find these users for you within our existing panel or recruit them to complete the test you need. (Contact us for help with this!)

Ultimately, testing user experience, regardless of your target demographic, will provide insights you didn’t even know you needed. Whether you’re able to ask for opinions from anyone or testing your target market, understanding the user’s experience will be invaluable.