When it comes to unmoderated studies, task-based questions are extremely popular for testing web-based experiences (desktop web, mobile web, and web-based prototypes). However, involved studies may include several task-based questions across multiple participants—making it difficult to evaluate all of the sessions in sum.
Identifying patterns and anomalies within multiple responses can be time-consuming, yet, they’re often the most valuable pieces of analysis. Even worse, finding relationships across various participant sessions can be extremely difficult—causing missed observations that could lead to actionable insights.
That’s why we’re excited to launch instant insight.
Instant insight is one of our newest features that helps you save time during post-test analysis and generate actionable findings by automatically surfacing interesting patterns, anomalies, and correlations for task-based questions.
Instant insight only works for web-based experiences and task-based questions. After a test is completed, the UserTesting platform automatically analyzes the results, identifies layers of data around intent, sentiment, and contributor flows, and generates insights on top of that data.
When analyzing the results, the algorithm explores two things:
Once these findings are identified, you’re able to view them summarized into easy-to-understand sentences as cards above the interactive path flows.
In order to truly speed up your time to insight, the solution displays the insight cards prioritized based on the statistical strength of the findings.
Instant insight is a particularly useful feature when trying to fuel efficient analysis, guide which task data needs to be evaluated further, and quickly understand common themes and anomalies in the flow of a given task.
Specifically, it’s particularly helpful when analyzing tests that have:
Moreover, you can easily share instant insights with colleagues and stakeholders as preliminary findings from research, as well as points of interest to drive product development or UX discussions.
In order to get the most from instant insights, you’ll want to pair this feature with success paths and custom insights through the use of tagging. By tagging via success screens and customer intents you’ll dynamically generate new instant insights.
Let us show you what we mean.
When it comes to machine learning, the more information you provide, the more robust analysis you receive in return. In this case, by tagging via success screens and custom intents, the algorithm will begin to provide you with more meaningful system-generated analysis.
By pairing using instant insight and success paths together, you’ll be able to tell which screens and intents (i.e., actions) are predictive of success or failure.
Here’s how it works:
Note: success screens are persistent and searchable. You can use this functionality to help understand how many contributors were successful, and which interactions or steps were predictors of that success.
Instant insight gets even more powerful when combined with custom intent, another new feature that sits atop interactive path flows.
Let’s say the UserTesting algorithm doesn’t predict an intent that’s specific to your company, or labels it differently from what you’d like to see. By using custom intent relabeling, you can add or adjust the intent labels between the steps on an interactive path flow. Then, we’ll automatically tag all other relevant interactions with the same label.
After you’ve completed your custom intent labeling, if the system detects new findings based on the labeling, corresponding instant insight cards will be generated automatically.
Note: custom intents are persistent and searchable. Now that you’ve tagged it, your teammates can also see the same label and search for that intent when viewing the task.
This feature is now available to accelerate your post-test analysis, and ultimately help you to build better user experiences, by leveraging the power of human insight.
If you’d like to learn more about any of these features, contact your account team or reach out to us here.
Get our best human insight resources delivered right to your inbox every month. As a bonus, we'll send you our latest industry report: When business is human, insights drive innovation.
About the author: Ranjitha Kumar is the Chief Research Scientist at UserTesting, and an Associate Professor of Computer Science at the University of Illinois at Urbana-Champaign. Her research has won best paper awards/nominations at premier conferences in HCI, and is supported by grants from the NSF, Google, Amazon, and Adobe. She received her BS and PhD from the Computer Science Department at Stanford University, and co-founded Apropose, Inc., a data-driven design startup based on her dissertation work that was backed by Andreessen Horowitz and New Enterprise Associates.