How Udemy gets creative with UX research

By Claire Menke | July 10, 2018
How Udemy gets creative with UX research

Today’s article is a guest post from Udemy's Director of UX Research, Claire Menke. Enjoy!

Udemy is an online video-based learning marketplace founded in 2010 where any learner can find the right teacher in an area of interest—from Python programming to painting. Our teachers are everyday experts who want to share their knowledge (and make a little money while doing it) by creating video-based content to host on our platform. Students who are interested in learning from these experts can purchase their courses à la carte—no subscription required. The user experience research team at Udemy is a fully mixed-methods group that approaches research with a question-first approach; with all researchers being versed in a multitude of methodologies it allows us to mix and match tools and approaches to answer research questions in more creative ways. Udemy’s UX research team has been leveraging UserTesting in unique ways and wanted to share some of those use cases with you.

Using unmoderated studies in creative ways

Longitudinal diary study

As we learn more about how users engage with Udemy products on a consistent, daily basis, the biggest challenge is to follow along with them on their journey. To understand the day-to-day user experience we invited a select group of individuals to share how they interact with Udemy on a daily or weekly basis. After doing a moderated interview session to indoctrinate them into the group, we wanted to find a lower-touch way to consistently receive feedback. Enter UserTesting. Leveraging the My Recruit feature we sent links to these individuals directing them through dedicated tasks to accomplish on their own time, in real-time, as they use the product. Giving them access to unmoderated testing links let us be there without having to be on call 24-7. So, whether that was learning while their kids are watching “Octonauts,” or squeezing in a few minutes while lying in bed, we got to be there without having to be there. It’s amazing what insight people will give you into their world!

Competitor evaluation

How your users use your product is largely relative to their other product experiences. Periodically we ask existing and potential users to go through Udemy and voice aloud their thoughts and opinions about the product. In that same unmoderated testing session, we may ask that they visit our competitors’ website and compare our products to theirs. This enriches the commentary by giving them a point of comparison on which to reflect. For example, we’ve been able to learn what encouraged an enterprise customer to choose us over a competitor in that decision moment. Asking testers to review both your own product in conjunction with reviewing your competition elicits more dynamic feedback.

Light Ethnography

Have you ever wanted to see what your users do or where they are as they use your product? The Light Ethnography tool opens that door—we’ve been able to enter our users’ homes around the world without needing to coordinate schedules or buy expensive plane flights. Using UserTesting's light ethnography template, or through your own creative genius, you can develop research plans that ask users to turn their camera around to lead you as the researcher on a tour of various aspects of their lives. We’ve been able to see instructor workspaces in Italy and student study areas in Argentina—how cool is that? I also hear this tool can also be great for product delivery and unboxing experiences.

Beyond unmoderated testing

UserTesting’s Live Conversation tool has been a game-changer for Udemy. It allows us to easily source potential users - people who would consider learning or teaching online - and meet them on a video chat without having to pay expensive panel recruitment fees. So how can you utilize Live Conversation?

Moderated interviews

The basic moderated interview can be easily facilitated through the Live Conversation tool. Whether you use UserTesting’s participant panel or your own users, they streamline the recruitment and scheduling so it can cut down on logistical overhead. For teams without research coordinators, this is clutch! Their partnership with Zoom also means the video quality is high, and you have flexible controls to allow screen-sharing for you or a participant. For Udemy, it's particularly useful to identify and speak with potential customers—carefully screened to be interested in online learning but not having used our products before. When they’re on the video chat, we can do everything from learning more about their first impressions of the product to getting feedback on new YouTube ads, all live and in real time.

Concept validation

One of our favorite things to do with Live Conversation is to do exploratory tests of new concepts—be they new advertising strategies, terminology, or features. Within 48 hours, we can jump on a screen share and pressure test new ideas. That real-time feedback from potential users can be critical to deciding how and where you are going to invest time and money.

Want to learn more?

You can learn more about Udemy here, and if you’d like to learn more about how UserTesting can help you understand your customers through on-demand human insights, contact us here.

Get real-time human insights with Live Conversation

Learn how Live Conversation makes it fast and straightforward to get real-time customer insights.

About the author(s)
Claire Menke

Claire is the Director of UX Research at Udemy, an online learning marketplace that empowers anyone to be able to teach and learn. At her core, Claire believes in empowering communities of people through research—be that indigenous communities in the Peruvian Amazon or learners around the world. In her former life, Claire was a Stanford-trained field anthropologist studying ecotourism and sustainable development. She has since established the UX Research function at two education technology start-ups.

A lifelong learner, Claire is currently teaching herself how to bake sourdough bread, perfect pie crusts, and quilt.