AI made teams faster. Human insight makes the work better.
Teresa Torres, author of Continuous Discovery Habits, points out that for most of business history, companies had one signal of success: did people buy the thing.
Usage data changed that. A product can sell just fine and still fail the customer—sitting unused on a shelf or data landfill equivalent.
That gap between what teams expected and what customers needed is where Torres locates the $2 trillion companies lose to rework every year. Rework itself isn't avoidable, she says—but its cost is optional: "we can get rid of the cost of the rework" if teams catch the mismatch from a whiteboard sketch/shipped code to what customers expected, needed or wanted.
That's the tension AI has quietly made sharper.
Artificial intelligence has changed the speed of product development almost overnight. Teams can generate concepts, analyze research, write code and prototype experiences in a fraction of the time it once took.
But faster output has created a new problem: How do you know whether any of it is actually good?
That question runs through a growing collection of UserTesting research, guides, podcasts and webinars about AI. Taken together, they point toward a surprisingly consistent conclusion: the competitive advantage of AI isn't simply automation. It's knowing where to apply human judgment.
Our UserTesting on AI collection brings that thinking together in one place, with resources for researchers, designers, product teams and anyone responsible for building AI-powered customer experiences. You can also download this executive summary of our AI-related resources.
Speed isn't the same as confidence
AI has unquestionably made teams faster. In UserTesting's Defensible design in the age of AI research, 91% of designers said their work moves faster in today's AI-enabled environment. Yet just 15% said they feel much more confident in the quality of their work.
That's the emerging AI paradox: creating something is getting easier; knowing whether you've created the right thing isn't.
Resources including The AI in UX research report, The responsible path to AI-accelerated customer insights and our conversations on the Insights Unlocked podcast explore how teams can close that gap with evidence, validation and human judgment.
Product operations and AI strategy leader Katie Robblee has seen the gap play out inside organizations firsthand. As she put it on the Insights Unlocked podcast, "just because you can build something doesn't mean you should build something."
She points to one striking example: after a company mandated AI usage and tied it to performance reviews, some teams began gaming the system, reportedly burning through tens of trillions of tokens in a single month spinning up agents just to prove they were "using AI." In other words, work with no connection to customer value.
On-Demand Webinar
Evolve your UX research strategy to win in an AI world
Discover how leading UX teams combine AI and human expertise to accelerate research, reduce uncertainty, and make smarter design decisions.
Build AI people actually want
The challenge isn't limited to internal workflows. Customers are developing their own expectations about where AI belongs.
Our State of AI in retail experiences report, for example, found that 49.1% of shoppers want AI-powered search, while 38.4% say the most helpful experience would combine AI and human assistance.
The lesson? Customers aren't necessarily looking for more AI. They're looking for better experiences.
That distinction matters for research methods too, not just product features.
John Whalen, who leads insights and innovation at Brilliant Experience, ran a blind test pitting AI-moderated interviews against two human researchers doing the same study.
The result: AI moderation "hit about 80 or 85% of what we found" as seasoned researchers. Good enough to be useful for a lot of decisions—but Whalen is clear that for a $300 million call, he's not handing it to an AI moderator by default.
The size of the decision still determines how much human judgment belongs in the loop.
Our AI resource center can help teams pinpoint what customers actually want from AI, identify opportunities worth solving and reimagine products to maximize AI's impact. You'll find practical resources covering everything from choosing generative AI features and evaluating AI experiences to understanding how AI is reshaping UX research.
Keep humans in the driver's seat
As AI takes on more work, accountability becomes more important, not less.
Across UserTesting's research and conversations, a common principle emerges: "AI in the loop," rather than simply "human in the loop." AI can accelerate the work, but people remain responsible for deciding what deserves to ship.
Priyanka Kuvalekar, a senior UX researcher at Microsoft leading research for Teams and AI experiences, puts it this way: "trust is built through an AI experience that keeps human in the loop."
In her own workflow, she lets AI take a first pass at clustering themes from interview transcripts—then overrides it with her own read of a participant's hesitation or tone that the AI missed entirely. The acceleration is real; so is the need for someone to catch what the model can't see.
The same principle applies to research. Simulated users and AI moderation can extend researchers' capabilities, but they aren't replacements for real people. Research highlighted in the collection found that 88% of researchers cite quality and accuracy as a concern with synthetic users, while 62.7% say their teams lack governance for using them.
Melissa Garber, a senior UX researcher at Consumer Reports, has watched this tension up close while helping build the organization's own conversational AI agent, trained on nearly 90 years of product-testing data. Her take: AI "can pull patterns, but it can't rationalize for you." That's why Consumer Reports still routes the agent's output through a human-in-the-loop team—not as a formality, but to protect a brand's worth of trust that AI alone can't be responsible for.
EXECUTIVE READOUT
Download the UserTesting on AI resource executive readout
Your guide to building better AI experiences
The collection includes reports, guides, webinars, podcasts, courses and practical resources covering AI research, design, customer expectations, synthetic users, research operations and AI ROI.
Because the question facing teams is quickly changing.
It's no longer simply, "How can we use AI?"
It's "How can we use AI to create something people genuinely want, trust and value?"
Human insight can help answer that.
But wait! There's even more AI resources
- AI-powered product research: Why the fastest builders need to become the fastest learners
- AI won't replace great UX researchers
- AI is making UX research faster. But are we asking the right questions?
- The truth about UX research for AI (it's not just about usability)
- Spotting an AI Cheater in Research: Investigating the Limits of Intuition in Remote Interviews
- What is the New AI in Research Risk Cascade? Experts Explain
On-Demand Webinar
The AI-native product loop: build, test, learn—without slowing down
This session walks through a practical, hands-on workflow for integrating user research into AI-speed development so that validation with real customers becomes a natural step in your process rather than a separate workstream.
