Walmart + UserTesting

Since its beginnings, Walmart has always been focused on the needs of its customers. Founder Sam Walton built and ran his business on this philosophy, insisting, “Customers are not cold statistics. They are flesh and blood human beings with emotions and feelings just like our own.” Yet, in order to improve the customer experience for the more than 600,000 daily visits to Walmart.ca—to increase traffic to the website as well as drive on-site conversions— the Walmart Canada Site Optimization team was largely relying on quantitative data, such as transactional data from analytics and A/B testing.

“For some departments, user research was seen as inferior to ‘real data,’” noted David Raine, Manager, Site Optimization, Walmart.ca. “It was tough to paint a story that, even without numbers behind your insights, user research is still data and valuable towards driving a business decision.” By focusing on quantitative data, the Walmart UX team found that it was running lots of tests, but in a scattershot manner. Lacking an overlying framework or strategy, they just kept moving from test to test according to perceived areas of opportunity and pushing out incremental site enhancements without a sense of where to go to next.

As a result, the team was burning out, perplexed as to why all of their efforts were failing to yield longer term outcomes. And with more tests being run and more results to sift through, there were more opinions and conflicting expectations. There wasn’t a clear answer to the question, “What should we do next?” and—more importantly—no context as to why