With all the arguments about the ideal number of usability study participants, it’s enough to make your head spin! Because of this, we’re often asked about how many people to test with during studies---but there isn’t an answer that works across the board. Fortunately, there are ways to estimate how many users you should test with.
At UserTesting, we often run qualitative remote usability studies. Qualitative studies help find general usability problems and trends by watching participants use an interface. The goal is to understand WHY the participants are experiencing usability problems so that improvements can be made.
Before you begin running studies, we first suggest grouping your users into your customer segments. Segmenting your customers is important because it helps you organize and group your target audiences. It seems straightforward, but different customers have different needs. One group’s behavior, likes, or needs may be different than another group.
Customer segments (also known as user groups, personas, or user profiles) are groups of people that share certain characteristics. Segmenting your customers means focusing in on the group that buys your goods or services, to understand:
What’s important to them
What they need or are looking for
Their typical web behaviors
How to design for them
What makes them convert
How to customize the experience specifically for them
Customer segments can be broken down in many ways. For example, you could segment your customers by:
There are different ways to approach this, so for a more in-depth explanation, check out this post on segmentation tactics.
You may argue that you just want to test your “target audience.” And while that’s all well and good, you’ll typically have more than one customer segment within that target audience.
For example, let’s say you are a B2B company that sells a SaaS marketing solution. You know that your target audience is marketers, but you’re not sure how to break down that group further. But there are ways! You could segment participants by:
Their knowledge about your product/industry: No knowledge vs. Some knowledge vs. Expert knowledge
Their role: The information-gatherer vs. The decision-maker vs. The buyer
Their buying phase: Research phase vs. Purchase phase
So how many customer segments do you need to test? That’s specific to your company, but it’s worth taking the time to think about and organize your audience into smaller groups. By doing so, you’ll be able to better target and design for the needs of the specific people visiting your site or app.
When running qualitative studies, five participants is the often the accepted number. While there are many schools of thought, Jeff Sauro claims that five users will find approximately 85% of problems in an interface, given that the probability that a user would encounter a problem is about 31%.
However, while five is the generally accepted number, there are some instances in which you could include more (or fewer) users.
There are some situations when you should test with more than the recommended five users. For example, if you’re running a quantitative study and want to:
Compare designs and prove one is better
Gather user opinion and preference versus behavior
Develop statistical proof that X% of users will show the same behavior
Fewer than five users may be acceptable in the case that your participants will run into some of the same usability issues. This could happen when:
You test with users that have overlapping demographics within different segments
Every one of your users is completing the same tasks within all segments
Overall, when setting up your qualitative study, you first need to break down your audience into specific customer segments. This will allow you to target and design for people visiting your site or app.
Once you’ve broken down your audience into customer segments, we suggest including 4-6 users in each group. By testing with multiple participants in each segment, you’ll find out how to give your customers a better experience, which will lead to better results for you!
Get our best human insight resources delivered right to your inbox every month. As a bonus, we'll send you our latest industry report: When business is human, insights drive innovation.