Get ready for some great content coming to your inbox from the team at UserTesting!
UX research is something to do early and often. It can start as early as someone having a new idea, thinking, “I have a great idea, but don’t know my users. What do they think about this? Where do I even start?” All the way to the established corporate group that says, “We have a killer product. We know we are good, but we want to be great. I want to learn more.”
From learning about your users’ needs by conducting remote, unmoderated interviews to observing users in their natural environment with destination studies, UserTesting has got your research needs covered from start to finish.
For this study, we looked at the Starbucks Mobile app in two separate groups of user tests that addressed both ends of the development process.
We began with a simple interview-like test, conducted to get the basics on current users, learn the Who, What, When, Where, and Why. In addition to learning current users’ general feelings, we were interested in seeing the application in action.
Our follow-up study observed followed the same users through an actual scenario in which they ordered and picked up a drink with the Starbucks app. With both of these studies, we learned a great deal about what the users currently think of the app and how it is used “in the wild.”
What are users thinking? Interviews are a great way to explore a user’s general attitude early in the development process. Interviews are a valuable method for exploratory user research and are a great technique to gather qualitative information from existing users.
For this study, we wanted to learn the 411. To do so, we recruited six avid Starbucks mobile app users and asked them a few general questions about their experience with the app.
From these questions, we were able to gain some general insights about the Who? What? When? Where? and Why? of the app.
In this small sample set, participants had used the app for at least a year, either joining in 2013 or early 2014. Participants’ ages ranged from 18- 35 and they lived across the United States.
We heard many reasons as to what makes this app useful. Some of the most common uses for the Starbucks mobile app were to:
Pay for drinks
As one participant commented, “[it] simplifies the way in which you pay for the drink.” Another participant claimed it was easier to pay with her phone as it was usually more accessible than their wallet: “I am always on my phone anyway, it’s easy to pull up the app and pay with its card… one less thing I have to do.”
Reload their Starbucks card
Nearly all of the participants commented the ease of use of re-filling their Starbucks member card. One participant commented that she used the card as a means to keep track of her purchases: “[It’s] easy to track how much I spend at Starbucks… it’s like a separate account.”
Benefits and rewards
Again, almost all the participants said they used the mobile app to earn rewards. One said, “I like that it gives deals and bonuses for rewards. The app keeps me updated… the app gives me push notification… it’s helpful in getting extra points.”
Another stated, “I like how many stars I have so I know how many I need till I get a free one.”
Participants also commented they used the app to find a location, order a drink, download music, and to explore the menu.
When asked to describe their last experience using the app, nearly all noted that they use the app to pay when going through the drive-through. The last participant said she uses it right before she wants to get her drink, and uses it to place an order right before picking it up.
Participants are able to use the Starbucks app wherever and whenever. At home, work, on the bus, or whenever a user has access to their phone, one can look up a menu, check rewards, add cash to balance, receive rewards, and order a drink. Of course, one can also use the app at a Starbucks location to quickly order or pay for a drink.
Perhaps one of the most important questions of the 5 W’s is the why. We learned that participants use the app for a variety of reasons. However, there were a few overlapping themes that keep bringing participants back.
Participants used words like “Status,” “Gold Member,” and “VIP” over and over when describing their attitude towards the app. One participant describes being a Starbucks member as “lifestyle” and the company does a good job about keeping things fun and fresh: “Starbucks has a good way of being on pace and keeping it interactive.”
Another participant describes the experience, “The gold status is nice…it’s like the VIP treatment from Starbucks.”
Almost all participants expressed an amount of gratefulness for its ease of use. Specifically, it’s convenient in terms of paying and reloading value onto accounts. “[It is] convenient because I don’t always have my wallet near, but I always have my phone.”
In summary, we learned the Starbucks mobile app can be used anywhere and for a variety of reasons. Some of the most common ways to use the app are to pay for drinks and reload cards. Users enjoy the app for the convenience and its ease of use. And all of the participants saw being a gold member or app user as a sense of status. As you can see, with just a few general questions and one hour later, we were able to gain valuable insight to existing Starbucks users.
To recap, why are remote, unmoderated interviews helpful? You gain fast, general feedback to understand who your users are and how your product fits in their lives. Ideally, these types of interviews will give you a good idea of the context in which your system needs to work and what types of users you have to cater to.
What is happening now that dedicated users are using the product and it’s become a part of their daily lives? If you want to see how your app really impacts your users, it’s important to look at things holistically. That means seeing the whole picture, understanding the entire process of adopting a product, and witnessing how it really impacts your user.
It is important to learn more than, “Does the blue button feature make sense to the user?” You should be thinking, “Does the feature make this user’s life easier? Does it really get them from A to B faster?”
This is where destination studies can be especially helpful, which is just what we did for the second portion of our study. Destination studies provide an opportunity to watch what people do, not just what they say. Like field studies, destination testing emphasizes the observation of real user behavior.
For this study, we wanted to observe how the Starbucks app was used in the real world. We recruited four participants, all active Starbucks members, and observed them going through the process of searching for a location, placing an order, paying, and ultimately picking up their cup o’ joe.
From this study, we discovered the following awesome insights:
This proved to be another easy task for participants, as their phones’ GPS accurately showed the participants’ location and surrounding Starbucks locations. Participants could instantly pinpoint the nearest Starbucks location on the map, which was also their most commonly visited location.
We found that participants experienced a variety of obstacles when placing their order through the app.
It wasn’t immediately clear to participants if they needed to select their coffee order first, and then find their desired location or vice versa.
Some participants had issues when it came to placing their order. Two of the four participants would persistently try to place their order, only to repeatedly see the error message “Order cannot be placed.” However, this notification did not include any supportive reasoning.
One participant thought it was the particular drink order she was trying to place.
Another participant faced difficulties when trying to place an order and because of her limited location options, she assumed the issue was that the nearest Starbucks didn’t support the mobile ordering feature. However, when she arrived in the store, she saw a banner that prompted the new online ordering feature: “I did see that they had flyers in there to order online so I am not exactly sure why my app wouldn’t work.”
Did the ordering feature work? Were people’s drinks ready? The results were mixed. We found that all of our participants each had unique experiences when arriving at the counter to pick up their drink.
Knew where to go and her drink was ready: Upon entering the store one participant saw a banner for the mobile ordering features and said, “I know right where to go.”
Didn’t know where to go and her drink was ready: One participant came into the store, took a seat down by the counter and waited for her drink to be called. This participant didn’t know she had to approach the barista to claim her drink: “[there was some] awkwardness of not knowing and waiting for them to call my name like they usually do.”
Knew where to go but her drink wasn’t ready: One participant knew where to pick up her drink but didn’t see her cup waiting: “I don’t see my drink anywhere. Maybe I should talk to the cashier? Maybe.”
Had to order at the counter and wait for her drink: One participant wasn’t able to complete the online ordering task and had to order in-store. She had concluded this was because her particular Starbucks location didn’t support the online ordering feature. However, when she went in the store, she saw banners advertisings mobile ordering.
Overall, participants had positive experiences, although no order and pick-up was seamless. Participants were excited about the idea to use the app to enjoy their favorite drink, even if it took a little longer than expected.
Even though the ordering process wasn’t completely seamless, all the participants got what they wanted in the end—their favorite Starbucks fix! When technology didn’t quite work as planned, great service and loyalty to the brand stepped in to compensate… as we can see from one participant who struggled to place an order but was still very excited about her experience of picking up a drink.
The most valuable asset of a successful UX team is the information they have about their users. Interviews can be a strong first step in the research process as it provides important general insight every research must know before diving deeper into the product. When teams have all the information about the user, their job of designing a powerful, easy-to-use interface becomes significantly easier.
As an outsider to the Starbucks mobile app, we were able to get great insight on existing users. All data was insightful and essential for us to better understand the current user. This information played a big role in designing the final destination study.
Although interviews can be a valuable tool to answer certain questions, different questions require different research methods to answer. If you are further along in the development process, one of the most valuable things you can do is observe the user in action.
Destination testing can provide some of the most important data you can collect as a researcher: customer behavior. What people do may be different than what they say. Destination tests allow teams to immerse themselves in the context of their users, which allows them to observe critical details for which there is no other way of discovering. Not only were we able to hear what participants thought of the app, we were able to witness their full customer experience, and discover pain points that might not have been apparent through interviews alone.
Want to learn more about destination studies? Check out our recent article on UserTesting’s Mobile Recorder!
Join 120,000 subscribers and get articles like this every week.
Get ready for some great content coming to your inbox from the team at UserTesting!