Watch a Demo

Are consumers ready for healthcare chatbots? Q&A with UserTesting’s Sr. UX Researcher Jacob Smith

| February 22, 2019
Sign up to get bi-weekly insights and best practices, and receive your free CX Industry Report.
Thank you!

Get ready for some great content coming to your inbox from the team at UserTesting!

In a recent webinar, Jacob Smith, Sr. UX Researcher for UserTesting, sat down with us to share the findings from our latest competitive benchmarking study, Healthcare Chatbot Diagnosis: Will consumers trust them with their health? in which we explored consumer satisfaction with the experience of getting a medical diagnosis via a healthcare chatbot and assessed consumer sentiments toward using these types of apps in the future.

We had a great Q&A session with Jacob and included some of our favorite questions below, or you can watch the full webinar here. Enjoy!

Did you measure the accuracy of the diagnosis or self-reported confidence from users?

It was actually both. For the first task, the simple diagnosis, we gave or asked them what diagnosis they received and compared that to get a sense of the overall accuracy. I guess there is still an element of self-report in that, but it wasn’t a reflection of them and how successful they thought it was. Then we also followed up with asking them a multitude of questions asking them their confidence and satisfaction with those measures. In hindsight, the overall scope of the test was getting so large, I think there was room to do a similar approach for that complex task where we had them respond and tell us what diagnosis they received.

What do you think are the most important pieces of information to give users at the end of their symptom routine?

One of the apps fell short partially because it didn’t recommend next steps after providing a diagnosis. That highlighted the importance of providing relevant follow-up information for users.

If you imagine being in the mindset of checking your symptoms, getting a diagnosis might be helpful, but what do you do about it? Without any guidance, it leaves you with no real sense of resolution in terms of how to manage or handle what you’re going through.

Participants in our study responded more positively to those apps that included more thorough information in a way that was helpful and not alarming. Including a little bit more detail like the severity or how common the diagnosis is was helpful.

When we look at the apps that performed best, the fact that they all provided this extra level of information indicates that getting a treatment recommendation is a likely desired outcome.

Did you see any specific trends or differences stand out within the study?

We noted that older generations, for example, were less impressed with the credibility and helpfulness of the apps while younger generations tended to be more forgiving.

While you might be tempted to assume this is strictly a generational trend, we think it’s a bit more complex than that. Experience with life—and health—naturally expands with age. It makes sense that an older person would be more critical of a dubious diagnosis because they’re more likely to have lived through it themselves. It’s a lot easier to clearly spot the difference between an accurate or effective recommendation and one that’s not. Younger generations may have been quicker to overlook any shortcomings simply because they lacked the personal experience to give those diagnosis and recommendations context.

In fact, our pre-test survey showed that older generations were the most familiar of all age groups with healthcare chatbot apps, which makes sense given their increased priority on healthcare.

Do you have any tips based on what you’ve learned from this study?

One of the key hesitations participants had with the apps was due to their lack of familiarity with the brands. If a well-known healthcare provider included this type of chatbot within its app, however, that skepticism would likely disappear given people are already familiar with and trust that company.

Companies that aren’t widely recognized need to understand that trust and credibility will go a long way with consumers. Finding ways to establish their brand as a trusted resource will be a key factor before consumers are likely to trust their app.

It’s also important to highlight how important it is to make a great impression for a first time experience. That’s going to affect how users perceive the credibility of the app. Imagine if you come to one of these apps for the first time and you receive a diagnosis like Scarlet Fever when you suspect you have, worst-case scenario, strep throat. That alarming diagnosis would probably dissuade you from ever using the app again. That first time experience, coupled with the app’s credibility, has a lot of impact and influence on the veracity and the efficacy of the person’s view of the app.

Want to learn more?

You can watch the full webinar here. To learn how UserTesting can help you understand your customers through on-demand human insights, contact us here.