CarMax + UserTesting

The company looked at how to implement best practices for user testing in the automotive industry. They found that having the product manager, the product designer, and the product engineer all involved in the user testing process was key. Not only did the product team members set up the UserTesting studies themselves, but they also watched and discussed the videos together, allowing them to find new ideas and solutions. Miller said, “This gives the team a sense of shared ownership and shared understanding of how customers use the product.”

The product teams at CarMax now user test everything, including their website, prototypes, landing pages, mobile apps, and even TV commercials. They do research throughout the development cycle of every product, both to generate new ideas to solve problems and to evaluate the solutions they’ve developed. They get feedback through UserTesting within minutes by keeping their demographic requirements fairly open. As Miller shared, “Everybody’s looking to buy a car at some point,” so the products need to be usable for everybody. For special projects, however, the team can also quickly recruit test contributors at a certain stage of the car-buying funnel.

The team members and stakeholders get together to watch the UserTesting video clips and unpack how the customers interact with the product. Any disagreements about the design or product direction can be discussed and resolved then and there, rather than relying on notes and additional meetings. The team uses insights from the videos to make empathy maps that include customers’ feelings, actions, behaviors, and quotes. They apply these pieces of information to prioritize new features and improvements that will have a high impact on the customer experience and the business.

Most importantly, CarMax can quickly evaluate, validate, and iterate on product ideas that will impact revenue and conversions. The product teams often come up with hypotheses during in-person user interviews and then run a series of user tests to refine the idea and use KPIs as a way to measure their impact

CarMax has seen results from their research in multiple products and campaigns. One recent win was the “badge” feature. During discovery, the team noticed customers spending time reviewing photos of the tires. They ran several UserTesting experiments with different solutions for calling out how many new tires a car has and watched and listened as customers noticed (or didn’t notice) that information on the page.

Through rapid iteration, they developed a solution that made customers take note. That has spawned a whole other discovery project: what other valuable and unique content could they badge? In addition to uncovering new ways to increase sales and conversions, the product teams at CarMax use UserTesting results to influence company decisions, effectively giving their customer a seat at the table with executives and other stakeholders.

Trout explained, “I let the users tell their own stories. It’s one thing for a Product Designer to make a recommendation—it’s another, more powerful thing, to combine that recommendation with video from real users. These clips are shown at bi-weekly Open Houses where the company can see firsthand what users think of a new idea or feature. The UserTesting clips have definitely helped us educate stakeholders and influence product decisions.”