The missing link in AI: Human Insight loops for customer-centric conversations
Most AI teams obsess over scale, speed, and efficiency—yet the biggest breakthroughs often come from something far more human: real context from real people.
That was the central message shared by Dr. Duncan Shingleton, VP of Product Strategy at UserTesting, during his recent keynote at the Conversational AI & Customer Experience Summit. Duncan reminded the audience that while AI can deliver astonishing capabilities, it still struggles with something essential: discernment.
As Duncan put it, “AI can definitely scale… but is it lacking a bit of wisdom at times?”
The talk explored why human feedback in AI is not just valuable but necessary to build conversational systems people trust. It also offered a look into how teams can create more meaningful and reliable conversational AI customer experiences by grounding model behavior in lived, human reality.
Watch his presentation:
Why AI needs human context
Generative AI has given product teams the ability to automate conversations, answer questions, and simulate human interactions at unprecedented scale. But as Duncan reminded attendees, scale doesn’t guarantee quality or accuracy.
He highlighted a well-known challenge with large language models: they are intelligent, but not always wise. They can pattern-match without fully understanding context. They can generate convincing but incorrect responses. They can ignore essential details humans would never overlook. They can be tone deaf in their responses.
This gap between capability and expectation is where human-in-the-loop AI becomes indispensable. Without human feedback guiding training, evaluation, and refinement, conversational agents risk becoming unpredictable, unhelpful, or worse—untrustworthy.
“These moments matter when you’re trying to get a piece of information. These moments matter when you’re trying to resolve a pain point, when you’re trying to buy something. They all matter,” he said. “Because when your customer feels misunderstood, the trust that they have in your brand, in your company, in your product erodes faster than the automation can replace it.”
Scaling conversational AI without sacrificing experience quality
AI doesn’t operate in a vacuum. When a customer interacts with a chatbot, assistant, or automated experience, they bring with them a lifetime of expectations from human conversations. Tone, clarity, empathy, and timing all matter. If AI violates these norms, even slightly, trust breaks.
Duncan emphasized that users judge AI interactions more harshly than human ones, and they do it quickly. It only takes a single confusing answer or robotic response for a customer to switch channels or abandon a task.
To bridge this gap, AI teams must validate conversational flows not just with automated metrics, but with qualitative customer insights. This includes observing how real people talk, what they expect, where they get confused, and what they interpret as helpful or trustworthy.
By incorporating these insights into model evaluation, teams can better:
- Prevent hallucinations through contextual grounding
- Improve conversational interface design
- Identify where models misunderstand user intent
- Ensure product experience quality
- Reduce friction in AI-led interactions
- Build customer trust in AI systems
The message is clear: human feedback doesn’t slow you down—it sharpens your aim.
ON-DEMAND WEBINAR
Effective AI: how to choose the right generative AI features—and build them fast
Testing model behavior in real-world scenarios
Duncan shared examples that illustrated how easy it is for AI systems to go off course when judged only against internal benchmarks. When teams use synthetic data alone to evaluate conversational agents, they risk reinforcing the model’s own assumptions rather than correcting them.
Instead, he described a process that mirrors how UserTesting evaluates digital products: by watching real people attempt real tasks using AI tools. This is how researchers uncover misalignments between what the AI thinks is happening and what the user is actually experiencing.
“Users judge AI interactions based on their lived experience, not on the model’s confidence scores,” Duncan said.
Human evaluation reveals issues that automated testing misses—such as tone mismatches, ambiguous phrasing, or emotionally insensitive responses. These aren’t bugs in the technical sense, but they are failures in the customer experience.
Bringing human insight into the AI development lifecycle
One of the most powerful themes from Duncan’s keynote was that AI teams should treat models not as static systems but as evolving products. Just as product teams test prototypes, iterate on design, gather user input, and validate changes, conversational AI needs the same discipline.
He encouraged teams to embed user feedback loops directly into the AI development lifecycle so they can:
- Compare model behavior to real expectations
- Understand how users interpret generated responses
- Evaluate clarity, empathy, and conversational flow
- Align model outputs with brand values
- Test responses across diverse, “edge-case” customer segments
This is especially critical as AI becomes the new front line of customer engagement. A conversational agent is no longer a side feature—it is increasingly the first touchpoint a customer encounters.
Duncan summarized this shift by saying, “AI is becoming the voice of your brand.”
If the AI doesn’t reflect human nuance, cultural context, or emotional intelligence, the brand relationship suffers.
Designing AI experiences that earn trust
Toward the end of his talk, Duncan reminded the audience that the goal isn’t to build AI that sounds human, it’s to build AI that understands humans.
That requires:
- Grounding models in real-world user evaluation
- Using context-driven feedback
- Ensuring responses reflect human communication norms
- Validating conversational flows through usability testing
- Measuring success not just by accuracy, but by clarity and customer satisfaction
When teams bring together AI capabilities with rich human insight, they create systems that feel more intuitive, more reliable, and more aligned with how people naturally communicate.
And that, ultimately, is how AI becomes a tool that enhances the customer experience rather than complicating it.
“Why does AI need human context?” Duncan asked. “So that when our customers engage with it, they don’t just get an answer, they get an experience they can trust.”
