Episode 191 | October 06, 2025
AI in UX research with John Whalen
Discover how AI is transforming UX research. Dr. John Whalen shares insights on simulated users, AI tools, and the future of human-centered design.
How AI in UX research is augmenting (not replacing) human insight
What happens when you put artificial intelligence in the moderator’s seat for a user interview? For Dr. John Whalen, a cognitive scientist and founder of Brilliant Experience, the answer was unexpectedly promising, and a little unsettling.
“We were all set to hate AI moderation,” he admitted on Insights Unlocked. “And we were sort of dumbfounded that it hit about 80% to 85% of what we found as seasoned researchers.”
That experience marked a pivotal moment for John and his team as they began integrating AI in UX research. But make no mistake, John isn’t pushing to automate the human out of the loop. He’s advocating for using AI to augment human insight, not replace it. The result? Smarter, faster, and more inclusive research.
Let’s explore how AI tools, including simulated users and automated analysis, are reshaping UX research and what it means for researchers navigating this evolving landscape.
The real role of AI in UX research
One of the most persistent misconceptions about AI in UX research is that it’s a replacement for real human engagement. John rejects this outright.
“When it’s the big $300 million decision,” he said, “I don’t want you to pull out your favorite AI moderator and just get its default results and do something. Absolutely not.”
Instead, he positions AI as a tool that fills the gaps when speed, language access, or scale become barriers. If a stakeholder needs insights in three days, or if you're working across languages like Finnish and Brazilian Portuguese, AI can be a powerful ally.
AI isn’t about cutting corners, he said. It’s about extending reach.
Why simulated users deserve a second look
If AI-moderated interviews raised eyebrows, simulated users downright shocked many researchers.
“My first reaction was, that’s ridiculous,” John said. “But then I thought—we're researchers. We should test this.”
And test it he did. Over the course of multiple cohorts in his AI for Customer Research course, John and his students ran the same interviews with real people and then recreated them with simulated users using tools available in the marketplace.
The results? Surprisingly consistent.
“When there were maybe seven major findings, synthetic users were getting six or seven of them right,” he explained. “Not with made-up stuff, actual patterns we found in the real data.”
But for John, the value of simulated users isn’t as data points. It’s as an inspiration.
“I think of synthetic user data not as facts, but as a way to broaden my thinking,” he said. “To prepare for being with real humans.”
This reframing shifts simulated users from being seen as a shortcut to being used as a strategic warm-up act—a rehearsal before the real performance.
ON-DEMAND WEBINAR
Effective AI: how to choose the right generative AI features—and build them fast
Use cases across the research workflow
AI in UX research isn’t a one-size-fits-all tool. John outlined several distinct roles that AI—and simulated users—can play throughout the research process.
1. Stakeholder workshops
Spin up a simulated persona to test stakeholder assumptions in real time. This helps clarify audience definitions and refine research questions before the project begins.
2. Screener testing
Use simulated users to test your screeners and catch questions that might inadvertently exclude the very people you’re trying to reach.
3. Interview prep
Run your draft questions with simulated users to see if any answers surprise you—or if your questions fall flat.
4. Post-interview synthesis
Take advantage of available tools to help summarize insights quickly and spot high-level themes. But always verify—“Trust, but click,” John advises.
5. Executive presentations
Create a simulated version of your target stakeholder and run your presentation past them. It’s a great way to anticipate tough questions and tailor your message.
6. Continuous discovery
Instead of relying on sporadic one-off studies, researchers can build ongoing learning loops using simulated users and AI tools to stay close to user needs.
Why human-to-human interviews still matter
Despite the rise of automation and the allure of scale, John firmly believes in the irreplaceable value of live, human conversations.
“It’s the stuff AI misses—the body language, the pauses, the stories between the lines,” he explained. “Those moments are where real insights often live.”
He also emphasized that qualitative research isn’t just about collecting answers. It’s about evolving your mental model of the customer—something that’s only possible through empathetic human interaction.
That said, John sees a future where mixed-methods research becomes even more nuanced. For example:
- Start with a few human interviews
- Scale with AI-moderated interviews
- Analyze using AI tools
- Validate or expand findings with follow-up human sessions
This iterative model embraces both speed and depth, allowing researchers to move fluidly between qualitative and quantitative research modes.
GUIDE
Proving the ROI of UX research
The dangers of overtrusting AI
While John is an advocate of AI-enhanced research, he’s also a skeptic.
“There are weaknesses,” he said. “These tools don’t understand strategic context. They average out perspectives that shouldn’t be averaged.”
He shared examples where AI tools failed to segment respondents by mindset—such as grouping privacy-conscious users and AI enthusiasts together under one generic theme. And others where the most obvious findings were presented as headline insights, overlooking what was actually novel.
So how do you use AI responsibly?
John’s recommendations:
- Always validate AI summaries against source transcripts
- Be aware of your tool’s limitations and blind spots
- Use AI for exploration and acceleration—not final decisions
- Keep humans in the loop for context and critical thinking
The skills researchers need to develop now
As the field shifts, so too must researchers.
John urged practitioners to sharpen skills that help them work effectively with AI:
- Prompt engineering: Knowing how to ask the right questions makes or breaks AI outputs.
- Context engineering: Feeding the right background into a model is just as crucial as asking good questions.
- Synthesis literacy: Understand how to interpret, challenge, and build on what AI tools provide.
- Strategic alignment: Bring your organization’s goals into every stage of the research workflow.
He also pointed out the low-risk nature of experimenting with AI today.
“These are relatively safe ways to get comfortable with the tools,” he said. “You’re not making high-stakes decisions with synthetic users. You’re warming up your thinking.”
The road ahead: from one-off research to continuous insight
Looking toward the next few years, John believes AI will usher in a shift from episodic research to continuous customer understanding.
He envisions a layered approach:
- Test quickly with simulated users
- Interrogate existing data using AI-powered repositories
- Conduct live research only when necessary
As AI tooling becomes more sophisticated, teams can build ongoing streams of customer input—across interviews, social listening, app reviews, and support tickets.
And this shift isn’t just about efficiency. It’s about getting closer to customers more often, without burning out the research team or the people you're interviewing.
Final thoughts: evolve with intention
AI in UX research isn’t a passing trend—it’s a foundational shift. But it doesn’t require abandoning what makes research meaningful.
As John put it, “Be curious, but skeptical. Prepare for the future by experimenting now. Learn the tools, understand their limits, and think about how they’ll evolve. That’s how we stay human in a world that’s changing fast.”
Episode links:
- John on LinkedIn
- John’s podcast: AI for UX
- John’ course: AI for Customer Research
- Brilliant Experience website
- Free simulated user resource
- Book: Design for How People Think
- Lija Hogan on LinkedIn
- AI Experiences: This resource page includes articles, webinars, customer stories and more on how to build great products using AI.
- Beyond the buzz: This on-demand webinar explores how AI can move beyond the hype to become a true strategic partner for product/UX teams, with practical examples.
- Effective AI: This on-demand webinar focuses on best practices for designing and implementing generative AI features.
- Designing AI-powered shopping experiences: This guide outlines how to design AI-enabled shopping features that customers want to use, how to prepare for the rise of agent-to-agent shopping, and how to use real human insight to ensure that AI not only scales efficiency but actually builds trust, drives conversion, and deepens loyalty..
- Using AI to streamline and elevate UX research (Episode 172): In this episode, Google’s Alita Kendrick talks about how generative AI is transforming the UX research workflow, cutting analysis time and enhancing storytelling.
- How AI user research fuels purpose‑built products: This blog post discusses moving beyond just building AI features to solving real user problems using AI‑informed user research.
ON-DEMAND WEBINAR
