Users are at the heart of what we do at UserTesting and in the UX industry. They tell us what needs they have so we can fill them. They inform our designs. They identify strengths and weaknesses within our products.
In a perfect world, usability testing involves users who wholeheartedly represent your site or app’s target market; they have the easiest time adopting the right mindset, they speak the right lingo, and they have the best insight into an ideal experience.
But whether or not you need to recruit exactly the right target user depends on the nature of your test.
Many UX thought leaders encourage researchers not to be too granular about the users included in their studies. After all, a vast majority of products should be clear and intuitive enough that ANYONE can figure them out: a grandma should be able to buy her grandsons some sports gear online without getting confused, and a 6-year-old kid should be able to check a weather app to make sure his soccer game won’t get rained out.
Of course, there are many circumstances in which researchers need to capture insights from a particular type of user---like if you’re working on a Human Resources tool, you need to test with HR professionals because they're the only ones who would know whether the tool could be helpful to them as they work.
If you’re in one of those circumstances, and you’re testing remotely, then you need to use screener questions---multiple-choice questions that can either eliminate users from taking part in your study or give them access to it.
Here’s an example of a screener question designed to capture plus-sized users:
Only users who select the third or fourth option will be allowed to proceed to the test.
Screeners may seem easy, but it’s often quite challenging to craft screeners properly. If a user misunderstands the questions, you could end up with test participants who don’t fit the needs of your study.
UserTesting’s Research Team knows firsthand how important (and challenging) it is to write solid screener questions, and we have a wealth of experience in capturing the correct users. Below are some guidelines and examples that can help you get just the right user for your next remote, unmoderated user test.
Many of the guidelines for writing good screener questions are the same as the guidelines for writing great Multiple Choice questions:
If you need someone with a particular background (like a medical degree) or someone who is going through a particular experience (like shopping for a new car), we recommend that in addition to screeners, you use the first task of your test to verify this:
“You indicated in the screener questions that you are currently shopping for a new car. Please describe what kind of car you are looking for, where you have looked so far, etc.”
Sometimes, just listening to a user describe their experience can let you know if they’re really the right fit.
A few months ago, I was running a study that required business professionals who are involved with the purchase of IT software for companies with more than 5,000 employees. One of the test participants, when asked to describe their role, turned out to be a cashier at a GameStop who once helped decide what kind of telephone headset to use at his store. While his response to the screener question was honest, he certainly wasn’t someone who knew how to research IT software solutions for a huge company! Having the user explain his background prevented my findings from getting skewed---and taught me to be more careful with how I phrase my screeners.
One of the most common kinds of screener questions that researchers use is capturing users’ level of familiarity with a product or a brand. Sometimes they need fresh users to test out a new tutorial for their app. Other times they are looking for insight from their most frequent users.
Whatever the case, you don’t want to ask point-blank if users fit the mold; people are naturally inclined to say yes, just to please you! Instead, ask users to indicate their familiarity, and then define the different levels of familiarity, like in this example:
This screener makes it easy for users to tell which category they belong in.
As you can see in the example, it’s helpful to define the Novice, Intermediate, and Expert level by a few elements: overall content levels, usage levels, and consideration levels. This keeps a user from overestimating their experience level with your product and accidentally skewing your results.
Similar rules apply for the related---and equally popular---frequency-of-use screener. A researcher may be interested in finding users who regularly check the weather on their phones, or users who used to play a game, but have given up on it over time.
As with experience levels, it’s important to define frequency in solid terms, not just “rarely”, “sometimes”, “often”, etc.
Another common screener related to frequency of use might have to do with how recently a user has participated in a certain activity. For example, many e-commerce product researchers prefer to hear from users who purchase items online fairly often, and many travel product researchers want to hear from those who are planning a trip within the next year.
In those cases, it may be wise to create two screeners: one to confirm that they purchase items online/have an upcoming trip, and then a follow-up screener to determine time frames.
This question will confirm that the user is planning on going on a vacation soon.
This question filters for users who are planning their trip within four months to a year.
Another occasion when multiple screeners might be needed to reveal a single characteristic would be when you need users within a particular occupation.
For example, a massage therapy retailer might want to hear from people in the massage therapy industry.
Obviously, massage therapy is a very specific profession, and it would be hard to come up with an exhaustive list of options inside of one screener question. But you also want to avoid asking a yes/no question, so you might start by listing broader professional categories, including Health (which would encompass massage therapy), and then in a follow-up screener, have users indicate the role they occupy within the Health industry. An example is below:
The first question gives broad categories to keep from overwhelming the user.
The second question narrows it down to specific industries.
The last type of screener that the UserTesting Research Team relies on frequently involves users providing sensitive information, such as their income, their race, their Facebook profile, or their body type.
For example, there are apps that help couples who are attempting to conceive a baby determine peaks in fertility. Obviously, reproductive health is not a topic that everyone is comfortable discussing in a user test, so it’s important to forewarn users of that, and only accept users who are willing to be open about this personal information. The screener would probably look something like this:
This screener gives users a heads-up that the test will deal with information that not everyone is comfortable discussing.
The above examples only scratch the surface of ways that researchers capture their ideal users! Often these are used in some kind of combination, like screening for men who frequently track their fantasy football data, but have never heard of Rotowire. When you use your imagination (or enlist the help of the UserTesting team), the recruiting possibilities are endless.
Shaping screeners can be a challenge, so we encourage you to always perform a dry run, in part to ensure that your screeners are effective!
And if you’re a UserTesting Pro client, you can reach out to your Client Success Manager about creating the ideal screeners for your product.
Get our best human insight resources delivered right to your inbox every month. As a bonus, we'll send you our latest industry report: When business is human, insights drive innovation.