Watch a Demo

Translating empathy into actionable results with CarMax’s Archie Miller and Chip Trout

| September 19, 2018
Sign up to get bi-weekly insights and best practices, and receive your free CX Industry Report.
Thank you!

Get ready for some great content coming to your inbox from the team at UserTesting!

Most product teams recognize the importance of customer research and the value of the insights that are gained from those efforts. But teams don’t always know how to make the best use of those insights once they have them. We recently hosted a webinar with Archie Miller, discovery coach, and Chip Trout, product design journey lead at CarMax to learn strategies for effectively translating the empathy gained from user research into actionable results.

There were so many great questions that we didn’t have time to get to them all, so we’re including a few more here that Archie and Chip graciously answered following the webinar. If you weren’t able to attend the webinar live, follow along below for some of the key takeaways or watch the full webinar here.

Where are you finding your participants for the unmoderated and moderated video sessions?

It really depends on your target customer. In our case, almost everyone over a certain age has some experience buying cars. So it’s easier to walk out the front door and find someone to interview. If you’re trying to understand a more specialized customer audience, it’s obviously a different challenge.

When your customer is highly specialized, consider using a customer panel. You can screen for potential users and then offer them incentives to join a panel, or pool, of customers with whom you can routinely engage. We even interview Lyft drivers while traveling on business. We call those in-the-moment interviews. Always look for those opportunities.

We also use “automated” recruitment tools. One of the great things about UserTesting is that the customers are super accessible, and we get unmoderated results back often in an hour or less.

How often should you talk to customers?

If possible, each team should try to talk with at least one person per week. It’s like developing muscle memory in a sport. If you do something repeatedly, it becomes natural. Then it becomes a habit. Teams that fall out of the routine of talking with customers often also lose touch with other important discovery rhythms.

How do you get the larger teams to devote time to listening to research sessions?

We make it one of our team rituals where it’s expected that the core team (three to four primary team roles) will attend research sessions. Other team members are also encouraged to attend as much as they can. Once members of the extended team attend a few, they start to see the direct impact research has on the final product and want to be a part of the process as much as possible.

How do you recommend including remote team members to participate in user research and analysis?

We’ve worked remotely with team members before and leveraged screen sharing and tools like Zoom to allow us to view sessions, live and recorded, as a team.

How long do you take to analyze and present results?

It depends on the research. For something like a few UserTesting videos that show button changes, maybe a few hours. If it’s a large customer journey project, it could take months.

Can you elaborate on quantifying discovery?

It’s not important to measure the amount of Discovery you’re doing, it’s what comes out of the Discovery that is most important. However, if a team is trying to build good habits, then you might want to give the team a measurable Discovery goal. For example, during a  Sprint, we may set a goal to talk to four customers or test two experiments.

How do you align your user research with A/B testing results?

Qualitative interviews are a minefield of bias. What people say is rarely what they actually do. As humans, we all try to portray the best image of ourselves. We all do it. This results in less than honest answers to questions like how often we exercise, how much binge watching we do, how many books we read, or which design do you prefer. They might choose the design that might make them look smarter, more creative, or participants might just pick up on your preference (the new design) and try to make you feel better by choosing it.

Try conducting your A/B test first. Then, before you make assumptions about the results, conduct some interviews around your new design or just the old one, but not both.

User research and A/B testing should go hand in hand. What people say and what they do are often very different. An A/B test can tell you what people are doing, but it can’t tell you the why behind it. Qualitative research enables you to find out why people prefer one thing over another. Then build multiple experiments to validate with quantitative research.

How quickly are you able to implement redesign recommendations into your design cycles from your user research?

It really just depends on the scope. We work in two-week sprints but execute releases daily. When developers are involved in the research, it’s possible to make changes much quicker because they’re part of the solution.

Want to learn more?

If you’d like to learn more about how the team at CarMax uses human insights to get actionable results, you can watch the on-demand webinar here.

To learn how UserTesting can help you understand your customers through on-demand human insights, contact us here.