How to build customer trust under the EU AI Act

EU AI Act compliance is becoming an experience problem

Much of the discussion surrounding EU AI Act compliance understandably focuses on lawyers, regulators and risk teams. The legislation establishes a risk-based framework with different obligations depending on how an AI system is used and the potential harm it can cause.

But compliance eventually reaches the interface.

Under the AI Act transparency requirements, chatbots and other interactive systems must make clear when people are interacting with AI rather than a human. Deepfakes must be labeled, while certain AI-generated or altered content must carry machine-readable markings.

That turns a regulatory question into a design question.

Where should the disclosure appear? What should it say? Will people understand it? Does it increase confidence or create anxiety? Could a technically compliant disclosure still leave someone confused about what the AI is actually doing?

"Compliance can tell you what information needs to be disclosed. It can't tell you whether a customer actually understands it,” said Caroline Vize, Director, Solutions Consulting EMEA at UserTesting. “That's where organizations need to go beyond checking the box and understand the experience from the customer's point of view."

AI transparency is really about human understanding

There is a temptation to treat AI transparency as labeling.

Put a notice on the chatbot. Add an icon to the image. Tell people that a recommendation was generated by AI.

Job done.

Except transparency without comprehension is a little like handing someone the terms and conditions for a mortgage and calling it financial education.

The European Commission makes the larger purpose explicit: the measures are intended to reduce deception and manipulation and help people make informed choices.

EMBED EU EXPLAINER VIDEO

That raises the bar considerably.

Companies need to consider not simply whether they've disclosed the presence of AI, but whether customers understand its role. That means watching how real people encounter AI-powered experiences, listening to what they believe is happening and identifying the gaps between the system's intended behavior and the user's interpretation.

Those gaps matter because trust is built in the experience, not in the policy document.

Risk-based AI regulation puts people back into the equation

The EU AI Act organizes AI around risk. The framework distinguishes between unacceptable, high, transparency and minimal-risk uses.

High-risk applications include systems used in areas such as employment, education, essential services, critical infrastructure and law enforcement. These systems face requirements including risk assessment, documentation, human oversight, robustness and cybersecurity.

The details differ, but the philosophy running through the regulation is strikingly human.

What happens to the person on the other side of the algorithm?

A hiring system isn't merely processing data. Someone is waiting to hear whether they got an interview. A credit system isn't simply producing a score. Someone is trying to buy a home. An AI customer-service agent isn't merely completing a workflow. Someone may be frustrated, confused or urgently trying to solve a problem.

"The most useful question organizations can ask about AI risk is often the simplest: What does this feel like for the person experiencing it? Real human insight can reveal risks and misunderstandings that aren't obvious when you're only evaluating the technology," Caroline said.

Generative AI regulation could change how products are designed

The implications extend beyond high-risk AI systems.

Rules covering general-purpose AI (GPAI) include transparency and copyright obligations, while providers of the most advanced models face additional requirements around systemic risk, safety and security.

For product teams, generative AI regulation therefore shouldn't be treated as something applied after an experience has already been built.

Transparency, explainability and human oversight increasingly need to be design inputs.

That means testing more than whether an AI feature works. Teams should explore whether people know when AI is involved, understand what it can and cannot do, recognize when they should question its output and know how to reach a human when necessary.

Those aren't merely compliance questions. They're experience questions.

AI governance has a human side

The EU's experiment with AI governance will continue evolving. Some rules for high-risk systems don't take effect until 2027 or 2028, giving businesses more time to prepare.

But the direction is already clear.

Trustworthy AI won't be defined solely by sophisticated models, technical safeguards or legal compliance. It will also be defined by whether people understand the systems they're being asked to trust.

"The organizations that get this right won't view regulation and customer experience as competing priorities,” Caroline said. “They'll use the same principles — transparency, human oversight and empathy — to create AI experiences that people can understand and confidently use." 

The EU AI Act may establish the rules of the road.

But companies still have to design the journey.

Additional resources

• Design confidence under pressure: Making decisions you can defend in the age of AI. This on-demand webinar connects AI adoption with accountability, evidence, risk and the need for human insight as AI-generated work moves closer to customers.
• State of synthetic users. This report explores what UX researchers and senior practitioners think about synthetic users. 
• Defensible design in the age of AI. Based on research with 183 designers, this report examines speed, risk, validation, accountability and confidence in AI-influenced design decisions.