Most product teams recognize the importance of customer research and the value of the insights that are gained from those efforts. But teams don’t always know how to make the best use of those insights once they have them. We recently hosted a webinar with Archie Miller, discovery coach, and Chip Trout, product design journey lead at CarMax to learn strategies for effectively translating the empathy gained from user research into actionable results. There were so many great questions that we didn’t have time to get to them all, so we’re including a few more here that Archie and Chip graciously answered following the webinar. If you weren’t able to attend the webinar live, follow along below for some of the key takeaways or watch the full webinar here.
It really depends on your target customer. In our case, almost everyone over a certain age has some experience buying cars. So it’s easier to walk out the front door and find someone to interview. If you’re trying to understand a more specialized customer audience, it’s obviously a different challenge. When your customer is highly specialized, consider using a customer panel. You can screen for potential users and then offer them incentives to join a panel, or pool, of customers with whom you can routinely engage. We even interview Lyft drivers while traveling on business. We call those in-the-moment interviews. Always look for those opportunities. We also use “automated” recruitment tools. One of the great things about UserTesting is that the customers are super accessible, and we get unmoderated results back often in an hour or less.
If possible, each team should try to talk with at least one person per week. It’s like developing muscle memory in a sport. If you do something repeatedly, it becomes natural. Then it becomes a habit. Teams that fall out of the routine of talking with customers often also lose touch with other important discovery rhythms.
We make it one of our team rituals where it’s expected that the core team (three to four primary team roles) will attend research sessions. Other team members are also encouraged to attend as much as they can. Once members of the extended team attend a few, they start to see the direct impact research has on the final product and want to be a part of the process as much as possible.
We’ve worked remotely with team members before and leveraged screen sharing and tools like Zoom to allow us to view sessions, live and recorded, as a team.
It depends on the research. For something like a few UserTesting videos that show button changes, maybe a few hours. If it’s a large customer journey project, it could take months.
It’s not important to measure the amount of Discovery you’re doing, it’s what comes out of the Discovery that is most important. However, if a team is trying to build good habits, then you might want to give the team a measurable Discovery goal. For example, during a Sprint, we may set a goal to talk to four customers or test two experiments.
Qualitative interviews are a minefield of bias. What people say is rarely what they actually do. As humans, we all try to portray the best image of ourselves. We all do it. This results in less than honest answers to questions like how often we exercise, how much binge watching we do, how many books we read, or which design do you prefer. They might choose the design that might make them look smarter, more creative, or participants might just pick up on your preference (the new design) and try to make you feel better by choosing it. Try conducting your A/B test first. Then, before you make assumptions about the results, conduct some interviews around your new design or just the old one, but not both. User research and A/B testing should go hand in hand. What people say and what they do are often very different. An A/B test can tell you what people are doing, but it can’t tell you the why behind it. Qualitative research enables you to find out why people prefer one thing over another. Then build multiple experiments to validate with quantitative research.
It really just depends on the scope. We work in two-week sprints but execute releases daily. When developers are involved in the research, it’s possible to make changes much quicker because they’re part of the solution.
If you'd like to learn more about how the team at CarMax uses human insights to get actionable results, you can watch the on-demand webinar here. To learn how UserTesting can help you understand your customers through on-demand human insights, contact us here.