Every Tuesday, UserTesting studies a different topic to share here on the blog. We hope you’ll learn some nifty research techniques and get inspired to run some insightful tests of your own. Enjoy, and check back in next Tuesday!
UX research is something to do early and often. It can start as early as someone having a new idea, thinking, “I have a great idea, but don’t know my users. What do they think about this? Where do I even start?” All the way to the established corporate group that says, “We have a killer product. We know we are good, but we want to be great. I want to learn more.”
From learning about your users’ needs by conducting remote, unmoderated interviews to observing users in their natural environment with destination studies, UserTesting has got your research needs covered from start to finish.
For this study, we looked at the Starbucks Mobile app in two separate groups of user tests that addressed both ends of the development process.
We began with a simple interview-like test, conducted to get the basics on current users, learn the Who, What, When, Where, and Why. In addition to learning current users' general feelings, we were interested in seeing the application in action.
Our follow-up study observed followed the same users through an actual scenario in which they ordered and picked up a drink with the Starbucks app. With both of these studies, we learned a great deal about what the users currently think of the app and how it is used “in the wild.”
Study 1: Remote, unmoderated study
“I have great ideas, but what do my users currently think?”
What are users thinking? Interviews are a great way to explore a user’s general attitude early in the development process. Interviews are a valuable method for exploratory user research and are a great technique to gather qualitative information from existing users.
For this study, we wanted to learn the 411. To do so, we recruited six avid Starbucks mobile app users and asked them a few general questions about their experience with the app.
From these questions, we were able to gain some general insights about the Who? What? When? Where? and Why? of the app.
“Who are my users?”
In this small sample set, participants had used the app for at least a year, either joining in 2013 or early 2014. Participants' ages ranged from 18- 35 and they lived across the United States.
“What do users use the Starbucks mobile app for?”
We heard many reasons as to what makes this app useful. Some of the most common uses for the Starbucks mobile app were to:Pay for drinksAs one participant commented, "[it] simplifies the way in which you pay for the drink.” Another participant claimed it was easier to pay with her phone as it was usually more accessible than their wallet: “I am always on my phone anyway, it’s easy to pull up the app and pay with its card… one less thing I have to do.”Reload their Starbucks cardNearly all of the participants commented the ease of use of re-filling their Starbucks member card. One participant commented that she used the card as a means to keep track of her purchases: “[It's] easy to track how much I spend at Starbucks… it's like a separate account.”Benefits and rewardsAgain, almost all the participants said they used the mobile app to earn rewards. One said, “I like that it gives deals and bonuses for rewards. The app keeps me updated… the app gives me push notification... it’s helpful in getting extra points.” Another stated, “I like how many stars I have so I know how many I need till I get a free one."Participants also commented they used the app to find a location, order a drink, download music, and to explore the menu.
“When do users use my app?”
When asked to describe their last experience using the app, nearly all noted that they use the app to pay when going through the drive-through. The last participant said she uses it right before she wants to get her drink, and uses it to place an order right before picking it up.
“Where are users when they use my app?”
Participants are able to use the Starbucks app wherever and whenever. At home, work, on the bus, or whenever a user has access to their phone, one can look up a menu, check rewards, add cash to balance, receive rewards, and order a drink. Of course, one can also use the app at a Starbucks location to quickly order or pay for a drink.
“Why do users use my app?”
Perhaps one of the most important questions of the 5 W’s is the why. We learned that participants use the app for a variety of reasons. However, there were a few overlapping themes that keep bringing participants back.StatusParticipants used words like “Status,” “Gold Member,” and “VIP” over and over when describing their attitude towards the app. One participant describes being a Starbucks member as “lifestyle” and the company does a good job about keeping things fun and fresh: “Starbucks has a good way of being on pace and keeping it interactive.”Another participant describes the experience, “The gold status is nice…it’s like the VIP treatment from Starbucks.”ConvenienceAlmost all participants expressed an amount of gratefulness for its ease of use. Specifically, it's convenient in terms of paying and reloading value onto accounts. “[It is] convenient because I don’t always have my wallet near, but I always have my phone.”
“How do people use the Starbucks app?”
In summary, we learned the Starbucks mobile app can be used anywhere and for a variety of reasons. Some of the most common ways to use the app are to pay for drinks and reload cards. Users enjoy the app for the convenience and its ease of use. And all of the participants saw being a gold member or app user as a sense of status. As you can see, with just a few general questions and one hour later, we were able to gain valuable insight to existing Starbucks users.
To recap, why are remote, unmoderated interviews helpful? You gain fast, general feedback to understand who your users are and how your product fits in their lives. Ideally, these types of interviews will give you a good idea of the context in which your system needs to work and what types of users you have to cater to.
Study 2: In-the-wild, destination study
“I know my product is a hit, but can I learn more?”
What is happening now that dedicated users are using the product and it’s become a part of their daily lives? If you want to see how your app really impacts your users, it's important to look at things holistically. That means seeing the whole picture, understanding the entire process of adopting a product, and witnessing how it really impacts your user.It is important to learn more than, “Does the blue button feature make sense to the user?” You should be thinking, “Does the feature make this user’s life easier? Does it really get them from A to B faster?”
This is where destination studies can be especially helpful, which is just what we did for the second portion of our study. Destination studies provide an opportunity to watch what people do, not just what they say. Like field studies, destination testing emphasizes the observation of real user behavior.For this study, we wanted to observe how the Starbucks app was used in the real world. We recruited four participants, all active Starbucks members, and observed them going through the process of searching for a location, placing an order, paying, and ultimately picking up their cup o’ joe.
From this study, we discovered the following awesome insights:
Locate the nearest Starbucks location
This proved to be another easy task for participants, as their phones' GPS accurately showed the participants' location and surrounding Starbucks locations. Participants could instantly pinpoint the nearest Starbucks location on the map, which was also their most commonly visited location.
Place an order with the Starbucks app
We found that participants experienced a variety of obstacles when placing their order through the app.
It wasn’t immediately clear to participants if they needed to select their coffee order first, and then find their desired location or vice versa.
Order can’t be placed
Some participants had issues when it came to placing their order. Two of the four participants would persistently try to place their order, only to repeatedly see the error message “Order cannot be placed.” However, this notification did not include any supportive reasoning.
One participant thought it was the particular drink order she was trying to place.
Another participant faced difficulties when trying to place an order and because of her limited location options, she assumed the issue was that the nearest Starbucks didn’t support the mobile ordering feature. However, when she arrived in the store, she saw a banner that prompted the new online ordering feature: “I did see that they had flyers in there to order online so I am not exactly sure why my app wouldn’t work.”
Pick up order from counter
Did the ordering feature work? Were people's drinks ready? The results were mixed. We found that all of our participants each had unique experiences when arriving at the counter to pick up their drink.Knew where to go and her drink was ready: Upon entering the store one participant saw a banner for the mobile ordering features and said, “I know right where to go.”Didn’t know where to go and her drink was ready: One participant came into the store, took a seat down by the counter and waited for her drink to be called. This participant didn’t know she had to approach the barista to claim her drink: “[there was some] awkwardness of not knowing and waiting for them to call my name like they usually do.”Knew where to go but her drink wasn’t ready: One participant knew where to pick up her drink but didn’t see her cup waiting: “I don’t see my drink anywhere. Maybe I should talk to the cashier? Maybe.”Had to order at the counter and wait for her drink: One participant wasn’t able to complete the online ordering task and had to order in-store. She had concluded this was because her particular Starbucks location didn’t support the online ordering feature. However, when she went in the store, she saw banners advertisings mobile ordering.
Overall customer experience
Overall, participants had positive experiences, although no order and pick-up was seamless. Participants were excited about the idea to use the app to enjoy their favorite drink, even if it took a little longer than expected.
Even though the ordering process wasn’t completely seamless, all the participants got what they wanted in the end—their favorite Starbucks fix! When technology didn’t quite work as planned, great service and loyalty to the brand stepped in to compensate... as we can see from one participant who struggled to place an order but was still very excited about her experience of picking up a drink.