Watch a Demo

Laura Klein Interview Part 2: Validating Your Market, Data-Driven UX, and Non-Terrible Qualitative Research

| October 29, 2013
Sign up to get bi-weekly insights and best practices, and receive your free CX Industry Report.
Thank you!

Get ready for some great content coming to your inbox from the team at UserTesting!

SaraWB2Laura has spent more than 15 years as an engineer and designer in Silicon Valley. Her goal is to help startups learn about their customers so that they can build better products faster. Her book, UX for Lean Startups, and her popular design blog, Users Know, teach product owners what they need to know to do research and design. This is part 2 of the interview (read part 1) we did before her joint webinar with Eric Ries, “Lean UX.”

(You can now watch the webinar on demand. Trust us, you’ll love it.)

Is it ever okay to “go with your gut?”

Nothing helps you better realize that you don’t know everything than metrics and testing. Instead of saying “we’re doing this because it’s the best practice” or “because Amazon does it this way,” you’re basing your decisions on real data. This helps you identify patterns better in the long run. Making a habit of testing your assumptions with real data and testing ironically makes you better at guessing the first time. No one’s ever right 100% of the time, but I think if you consistently test, your “gut instinct” will improve and you will more consistently be right.

If you consistently test, your “gut instinct” will improve and you will more consistently be right.

How is “validating your market” part of UX?

I’m going to let you in on a little secret. I believe that product and UX are essentially the same thing. UX designers who don’t think about whether they have the right market are making a big mistake. It’s important to know whether you’re designing for the right market because if you’re not, your design is going to fail and you’re not going to know why. In reality, designing for your market is another way to say you need to understand your users. You need to know whether your design failed because it’s a bad design or whether it’s a bad design for the people that are using it.

In your book you say, “Qualitative research is great except when it’s terrible.” When is it terrible?

Qualitative research is terrible when it’s done incorrectly. One place I often see bad qualitative research is moderated testing. People tend to moderate tests very poorly. They lead, they demo, and they explain way too much to the user. Moderators who do this are setting the user up for confirmation bias. Moderators must go into a test with a plan, and must focus on letting the user truly uncover problems, not confirm what the moderator is saying.

Qualitative research can be terrible when you should be doing quantitative research.

Qualitative research can be terrible when you should be doing quantitative research.

Qualitative research can also be terrible when you should be doing quantitative research. Qualitative testing tells you why something is happening. Most of the time, you need to do quantitative research as well to see what your users are doing in aggregate. You don’t use qualitative research to find out how many people are getting through your registration panel.  You use qualitative research to understand why they’re getting stuck on step 3 after you’ve used quantitative research to determine that everybody’s getting stuck on step 3.

What is data-driven UX?

People often use “data-driven” to mean they’re going to measure everything and look at the data and then try a bunch of stuff to fix that data problem. The common idea is to throw a bunch of stuff against the wall and make a bunch of small optimization changes and hope they fix the problems found within the data. Data-driven design done right is when you recognize a problem in your data and metrics and then you do qualitative research to determine what is causing the problem. If you skip the qualitative step, then you’re just guessing.

Data-driven design done right involves recognizing a problem in your data and then doing the qualitative research to determine what is causing it.

You talk about testing with the right amount of visual design. How rough is too rough? Do you have any guidance on when to test and how much polish you need at each stage?

I’m a big fan of testing with very little visual design—clean, gray scale wireframes—because I find that people will actually focus on the interaction rather than the visual design. If it’s beautiful and looks like it’s been perfectly designed, people will tend to focus on things that you don’t want to hear about, like the colors and the gradients. People are also less likely to criticize things that look too perfect and polished. It feels like a finished product and people don’t want to criticize it because it looks like it’s done.

If your site is beautiful and looks like it’s been perfectly designed, people will tend to focus on things that you don’t want to hear about, like the colors and the gradients.

If it’s beautiful and looks like it’s been perfectly designed, people will tend to focus on things that you don’t want to hear about, like the colors and the gradients.

I’m also a fan of testing once your product is out in the real world. Visual design is incredibly important and can have a big impact on usability. You need to test throughout the process and at each stage you need to test for different things. At the wireframe stage you’re testing for big usability problems, whether people can perform the task at all, how complicated the task is, and whether people understand the messaging.

When you get to visual design, you’re testing to see if the visual design improved or hurt the usability, or whether the button style caused people to be more or less likely to convert. For instance, I often see people trying to use giant buttons to increase their conversion rate. However, these giant buttons sometimes work much worse than normal sized buttons because people don’t think they’re buttons. They look like banner ads or pictures. It’s ironic, but you can actually make text so big and so bold that it loses context and is no longer effective.

If someone wants to learn more about Lean UX, besides the Lean Startup Series, are there any other resources you would recommend?

Luxr.co has put together a great core curriculum of video classes that are taught by Lean experts that I luckily get to be a part of. I teach a class on how to figure out what kind of research you should be doing at what in your product as well as one that helps you understand why users might not be converting on your product.luxrco

The Lean Entrepreneur book is really helpful. Reading any of Eric Ries’ stuff is a good idea as well. If you want to be immersed in the Lean Startup community the Lean Startup Conference is coming up in December and everyone’s going to be there. There will be workshops, talks and mentorship opportunities. There’s also probably a Lean Startup Circle in your area that you can become a part of. There are a lot of people doing this, and we can all learn from each other.

Like what you’ve read? Go ahead and watch the informative on-demand webinar.