Qualitative vs. quantitative research

While the topic of qualitative vs. quantitative research may sound intimidating, they’re easy concepts to understand, and they represent things that you’re probably doing already. No matter your industry, most business professionals want to get customer feedback and know their audience, whether you call it research or something else.

First things first, suggesting that qualitative and quantitative research are at odds with one another is misleading. While quantitative research is the method that most people are familiar with (and the one that gets all the credit), the two complement each other fundamentally. Together, they can give a more holistic view of a problem or situation. Having one without the other means that you’re only getting half the story. Both play a valuable role in measuring your customer experience.

For example, after a sizable European car rental company spent time and money developing a car rental subscription membership on a hunch, they were stumped by its poor performance. First, analytics showed that users saw the ads but didn’t sign up. Then, after getting feedback from a handful of users, it became painfully clear why no one wanted to belong to the exclusive club.

Watch Kate Margolis, UX/UI Design Lead at Thirdfort, tell the story.

Thanks to both qualitative and quantitative research, the team changed the membership value proposition to boost its desirability for potential users—leading to a 65% conversion of membership after only being asked once.

While there are significant differences between qualitative and quantitative research methods, it’s essential to understand the benefits and blind spots. So let’s start with quantitative research.

What is quantitative research?

Quantitative research is the process of collecting and analyzing numerical data. It aims to find patterns and averages, make predictions, test causal relationships, and generalize results to broader populations by representing data expressed as numbers.

Quantitative research is unlike qualitative research in one critical aspect—it’s numerical. This is because the output of quantitative research is numbers and statistics.

How UserTesting can help you conduct quantitative research methods

The following features, which are popular ways of conducting quantitative research, are all available on the UserTesting platform.

1. Surveys (ratings, ranking, scales, and closed-ended questions)

Not all surveys have to be taken silently. Our customers can customize surveys, hosted on third-party tools like Survey Monkey, for example, and ask contributors to speak aloud and offer reasoning as they answer questions. Not sure when to use them? At UserTesting, they’re commonly done before development, during the design process, or after a product launch.

2. A/B testing

Suitable for product managers and marketers alike, this method tests two options to see which comes out on top. To ensure quality results and prevent do-overs, get a feel for customers’ preferences and even get qualitative feedback before deciding on what to test. When creating the test, it’s always best to over-communicate any questions to better understand why contributors prefer one design over another.

A major pediatric health system came to UserTesting to transform its website and apps, aiming for a more customer-focused experience. Using A/B testing, the team showed contributors multiple homepage image options and dove into why some were preferred over others. Contributors’ feedback showed that there was a strong preference for photos with real patients over stock images, and the team’s digital marketers were able to share this confidently to stakeholders.

3. Benchmarking

You may know all too well that a website or app is never fully done. As the market and customers’ needs evolve, so must our work. Whether your product launched for the first time, or for the 20th, benchmark studies are a way to assess how your product’s experience has changed over months or years. The results historically document how redesigns and changes are perceived by contributors and how competitors stack up. Consider launching these tests before committing to any serious redesign efforts, and keep it focused, whether you’re targeting the overall user experience or a specific feature.

4. Web analytics

UserTesting offers a variety of analytics tools that display contributors’ screen interactions, customer behaviors, keyword maps, positive or negative sentiments, and more. These tools will help track where users tend to click on a screen, offer a glimpse of where you should dig deeper, highlight common behaviors, and showcase any pain points among others. This helps cut down on any manual effort you might have spent pinpointing and organizing data.

Advantages and disadvantages of quantitative data

What’s excellent about quantitative data is that you can easily replicate it. Quantitative data collection is relatively easy to do, and so is analysis. Since you’re dealing with numbers, it’s typically easier to interpret quantitative data and present your findings to others in a less subjective way.

It’s human nature to trust numbers. We tend to believe they’re concrete. Quantitative methods get more attention because it’s easier to tie quantitative measurements to performance metrics and ROI. But unfortunately, there are many ways numbers can be unreliable.

While numerical data can tell you that there’s a problem, it seldom tells you why. Plus, by focusing on numbers only, there’s a risk of missing a factor or common denominator.

Here are some examples where quantitative data isn’t enough information to make an informed decision:

• An e-commerce agency notices that her client’s shoppers are dropping off on one of their biggest channels before reaching checkout, but they don’t know why.
• A product manager is getting survey data showing that new customers are not satisfied with the onboarding process. While she has an idea of what it could be, she’s not sure where to start.
• A marketing team spent weeks developing and rolling out a campaign that flopped. While the team believes they’re on the right track, the President of the company, who never liked the idea, tells them to abandon it altogether.

As you can see from the examples, quantitative data only gives you half of what you need before you can move forward. Lastly, a significant drawback to quantitative research is that numbers don’t convey stories well. So while it’s easy to share a table of data points with an audience, it’s harder to get them to absorb the information and remember it later.

What is qualitative research?

Qualitative research is a behavioral research method that relies on non-numerical data derived from observations and recordings that approximate and characterize phenomena. It’s collecting, analyzing, and interpreting non-numerical data, such as language. It sometimes seeks to understand how an individual subjectively perceives and gives meaning to their social reality.

Instead of numbers, qualitative data comes from studying subjects in their natural environment and focusing on understanding the why and how of human behavior in a given situation. It’s especially effective in obtaining information about people's values, opinions, and behaviors. At UserTesting, qualitative data is collected through contributor observation and interviews.

How UserTesting can help you conduct qualitative research

Researchers, marketers, product managers, and more conduct qualitative research daily using moderated or unmoderated testing with UserTesting. With our platform, business professionals can have access to qualitative data at the speed of quantitative analysis. These are three common qualitative research methods:

1. Contributor observation

Observing and listening is the core of UserTesting’s platform. Whether you’re conducting a moderated or unmoderated test, observing our contributors to obtain their feedback is vital to improving or redesigning your product. We recommend keeping your questions neutral to prevent swaying any feedback, over-communicating your ask (without bias), and letting the contributor take the lead as they complete the tasks.

For a use case idea, consider optimizing in-person experiences by watching your customers record themselves in a new store or at home—known as an ethnography study. This type of study can highlight how contributors, using rear-facing cameras, have something set up or stored in their home and how they live day-to-day and make decisions. For example, REI relied on UserTesting to better understand various types of buyers and supporters. Using light ethnography studies, REI asked contributors to show how they stored their REI equipment and discuss how long they’ve had them. As for the result? REI became more efficient in testing the right demographic for future studies.

2. In-depth or unstructured interviews

In-depth interviews are optimal for collecting data on individuals’ personal histories, perspectives, and experiences, mainly when exploring sensitive topics or follow-up questions are likely necessary. When asking open questions, the interviewer can get a real sense of the person’s understanding of a situation. For example, they might say one thing, but their body language says something else. You may decide on this method to create better solutions and experiences by exploring your users' attitudes, preferences, and opinions as they test out designs and prototypes.

3. Focus groups

Focus groups are what you may consider a “group brainstorm”—when a moderator effectively gathers information from multiple subjects at once and generates broad overviews of issues or concerns related to the demographics represented. These offer a lot of flexibility, and can be done in person or online. Here at UserTesting, focus groups can consist of up to three or five contributors and while they may be less thorough compared to our 1-on-1 Live Conversations, you’ll get a real time look about multiple people’s thoughts and actions—and how they differ.

Though these can be conducted at any time during a product’s life cycle, we recommend setting these up before design development, before you have limited time and budgets on the line.

Advantages and disadvantages of qualitative data

The most significant advantage to qualitative data is that it’s easy to present your data as a story to your audience. In this way, qualitative data has both staying power and the ability to persuade others. People remember stories and how they make them feel. While charts and numbers can convince others to change, they won’t always translate into action or follow-through. One of qualitative data’s biggest advantages is that it offers rich, in-depth insights that allow you to explore new contexts and deeper understandings.

The drawbacks of qualitative research are that it’s often not a statistically representative form of data collection, and it can require multiple data sessions, which can lead to varying analyses and a lot of time and resources spent

Examples of qualitative vs. quantitative research questions

When planning research, you want to be strategic with your test questions. Here are some examples of qualitative vs. quantitative questions to give you a better idea of what to ask.

Quantitative research questions

Quantitative research questions are typically set up so that the answer is numerical or statistical or so that the answer is objective. Typically, this process is automated and answers can’t be followed by more questions.

• How long have you been a customer of our organization?
• On a scale of 1-5, how likely are you to purchase our products again?
• How often do you drink coffee at home?
• Do you prefer to watch movies at home or in the theater ?

Qualitative research questions

Qualitative research questions are open-ended. The interviewer can react to answers and probe for more detail.

• What does the app need to do to improve your experience?
• Do you have any comments, questions, or concerns about our website?
• What do you like most about your favorite coffee shop?
• What did you like about this movie?

Why you need both qualitative and quantitative research

Most importantly, the intersection of quantitative and qualitative data methodologies is where human insights come to life. Both methods can be helpful, but combined, they allow you to see things you may have missed.

According to Justin Wei, Former Head of Digital Marketing at Royal Wins, while quantitative data is the black and white picture of a problem or opportunity, qualitative data can color your understanding.

Quantitative data is 'the what' and qualitative data is 'the why'

Commonly, quantitative data will surface trends that you can use as a springboard for qualitative research. However, it’s important to use qualitative research to drive innovation. Organizations that fall into the habit of only using qualitative research to react to quantitative data run the risk of reducing team efficiency and restricting their ability to optimize.

In general, here are some common reasons to use qualitative research or quantitative research:

• Validate hypotheses: quantitative research will get you the key performance indicators (KPIs) you need when you need objective information to confirm or disprove your theory.
• Find answers: It’s typically easier and less expensive to have people fill out a survey than participate in a focus group. In this way, quantitative methods can help answer questions like: were you satisfied with your experience? Would you recommend us to a friend? On the other hand, qualitative research enables you to respond to open-ended questions like: why were you satisfied with your experience? Why would you recommend us to a friend?
• Uncover emotion: qualitative research is especially good at uncovering the emotions behind data. This can be verbal, body language, or facial expressions caught on video. It helps to hear and see your customers describe wants, needs, concerns, frustrations, etc. Qualitative data will get you that.

Watch Jonathan Greenblatt, User Research and Design Leader, explain how WarnerMedia uses quantitative and qualitative research to flesh out its user personas.

Selecting the right methodology:

Choosing the right qualitative method involves several considerations, which you'll need to map against your research goals. For example, you may need to consider:

• Depth of understanding: Methods like in-depth interviews or ethnographic studies are a good choice when the research goal is to deeply understand motives, beliefs, or practices.
• Exploratory nature: If the research aims to explore new ideas or unknown issues, methods allowing open-ended responses and broad questioning, like exploratory focus groups, are ideal.
• Study environment: The environment where the study is conducted also affects the choice. For instance, naturalistic observation fits best in real-world settings where the behavior of study subjects is not influenced by lab conditions or artificial setups.

How to successfully integrate and balance qualitative and quantitative methods

Combining (or if you prefer, "integrating") qualitative and quantitative research methods can enhance the robustness and depth of your insights considerably. If you're new to this type of research, some of the terms can seem a little daunting, but there's no need to panic. Once you get past the jargon, it's fairly straightforward. Here’s how you can effectively integrate and balance the two approaches:  

Integrating data

Integrating qualitative and quantitative findings can address the limitations inherent in each method alone. For instance:

• Explaining anomalies: When quantitative data reveals unexpected patterns or anomalies (such as a sudden drop in product usage or a spike in a particular service's adoption), qualitative insights can help uncover the reasons behind these patterns. Conducting follow-up interviews or focus groups can reveal user perceptions, misconceptions, or external factors influencing these anomalies.
• Enhancing statistical models: Qualitative insights can be used to refine variables and hypotheses in quantitative models. For example, insights gathered from user interviews might indicate additional variables that should be included in a predictive model, such as user attitudes or external influences that quantitative surveys did not initially capture.
• Validating quantitative data: Qualitative data can provide context to the raw numbers obtained from quantitative research, ensuring that the interpretations align with real-world meanings and applications. This is particularly important in market research, where understanding the nuances behind consumer choices can dictate product development directions.

Techniques for integration:

• Data triangulation: Use multiple data sources to confirm the reliability of findings. For example, combining survey data with interview transcripts to validate the consistency of responses across different data collection methods.
• Sequential explanatory strategy: Start by collecting and analyzing quantitative data, which is then followed by qualitative research to explore particular findings in more depth. This strategy is particularly useful in large-scale studies focusing on general trends followed by in-depth analysis.
• Convergent design: Collect both qualitative and quantitative data simultaneously and then merge the data during the interpretation phase to build on the synergies of both data types. This approach is useful for studies where understanding the breadth and depth simultaneously is crucial.

Balancing approaches

When planning mixed-methods research, consider the following points to ensure you balance out both approaches effectively:

• Timing and phasing: Decide whether the methods will be implemented concurrently or sequentially. Concurrent implementation allows for immediate integration of data, enhancing the responsiveness of the study to emerging insights. Sequential studies–where one method follows the other–can allow each phase of research to inform and refine the next phase.
• Resource allocation: Allocate resources based on the study's goals and the relative strengths of each method. Quantitative methods might require more investment in tools and software for data analysis, whereas qualitative methods might demand more time and expertise in interview skills and content analysis.
• Leveraging strengths: Use each method to complement the other’s weaknesses. For example, if quantitative data provides a broad understanding of user behavior, qualitative methods can delve deeper into the motivations behind those behaviors. This approach ensures that each method feeds into a more holistic view of the research subject.
• Integration points: Plan for specific points within the research process where data integration will occur. This might be during data collection, where qualitative insights shape the design of quantitative instruments, or at the data analysis phase, where qualitative themes are used to interpret statistical results.

Complimentary balancing

Ensure that the methodologies do not just coexist but complement each other to enrich the research:

• Methodological transparency: Be clear about why and how each methodology is used and how the data converges to support overarching research questions.
• Flexibility in design: Stay open to adjusting the research design based on preliminary findings. If initial quantitative results suggest unexplored areas, qualitative methods can be adapted to probe these areas more deeply.

By thoughtfully integrating and balancing qualitative and quantitative approaches, researchers can leverage the strengths of both to gain a richer, more nuanced understanding of their research topics. This integrated approach not only broadens the scope but also deepens the insight into complex research questions, making the findings more robust and actionable.
 

The ethical role of the researcher

Once you have a grasp on the fundamentals, you'll quickly realise that the researcher is responsible for a lot more than just implementing studies. You'll also need to understand how to maintain the integrity and impartiality of your research. 

Again, this can seem a little daunting, but as long as you keep a few key ethical consideration in mind, you can be sure you'll end up with actionable, useful data. Let's consider some of the most important points: 

Data collection ethics

Informed consent:

• Clarity and comprehension: Ensure that consent forms and participant information sheets are clear, concise, and accessible. Use plain language to describe what participation involves, what data will be collected, and how it will be used.
• Voluntary participation: Emphasize that participation is voluntary and that participants can withdraw at any time without any consequences. This should be clearly stated at the beginning of the recruitment process and reiterated at the start of data collection.

Privacy protection:

• Data anonymity and confidentiality: Implement measures to anonymize and de-identify data wherever possible. If personal identifiers are necessary for the study, ensure that they are securely stored and only accessible to the research team.
• Secure data handling: Use encrypted storage solutions and secure methods of data transfer when handling sensitive information. Regular audits and updates of security protocols can further protect data integrity.

Transparency with respondents:

• Open communication: Maintain open lines of communication with participants, providing them with contact information for the research team and an avenue to ask questions or express concerns.
• Results sharing: It can also be valuable to offer to share summaries of your findings with participants, allowing them to see the outcomes of their contributions will encourage them to continue participating.

Maintaining Objectivity

Avoiding researcher bias in qualitative research:

• Triangulation: Use multiple data sources, researchers, and methodologies to cross-validate your findings. Everyone has some biases, so this approach can help mitigate anything that might influence the interpretation of data.
• Reflexivity: Encourage researchers to engage in reflexivity by maintaining reflective diaries where they document their personal reflections on the research process. This can help researchers recognize their own biases.
• Peer debriefing: Engage in regular discussions with peers or supervisors who are not involved in the research to challenge and review the study’s approach and findings. This can provide an external perspective and help identify any potential bias.

Ensuring statistical validity in quantitative studies:

• Rigorous methodological design: Ensure that the research design, sampling methods, and statistical analysis are rigorously planned to address the research questions without leading to misleading conclusions.
• Replication: Design studies so that they can be replicated by other researchers. Replication can confirm the reliability of the findings and strengthen the study’s contributions to the field.
• Error checking: Regularly conduct error checks throughout the data collection and analysis phases. Implementing automated data validation methods and double-checking data entry processes can reduce errors.
• Statistical techniques: Use statistical techniques to control for confounding variables that might bias the results. Techniques such as regression analysis can adjust for these variables, providing a clearer picture of the relationships between the primary variables of interest.

Reporting results:

• Honest reporting: Report findings honestly, including the acknowledgment of any limitations or unexpected findings that contradict the hypotheses.
• Responsible interpretation: Avoid overgeneralizing the results. Be precise about the contexts in which the findings are applicable and distinguish between correlation and causation where applicable.

Sticking to these ethical considerations not only enhances the credibility and reliability of the research but also protects participants. By maintaining high ethical standards, researchers ensure that their work contributes positively to the body of knowledge while respecting those who help make the research possible.

Future trends: How research is evolving

Finally, it's always worth remembering that research methods are constantly shifting, with new technologies and changing societal and global dynamics making for an ever-evolving space that means you can always keep adapting your research approaches. Let's take a quick look at how some of these changes are shaping the future of research:

Emerging methodologies

Virtual reality (VR) in qualitative research:

• Immersive experiences: VR technology is transforming qualitative research by enabling immersive environments that replicate real-life scenarios. Researchers use VR to study behavior and reactions in controlled yet realistic settings, which is particularly useful in fields like psychology, market research, and user experience design.
• Enhanced data collection: VR allows for the collection of rich, detailed data on participant behavior and interactions within a virtual environment. This includes tracking eye movement, body language, and verbal responses, providing a comprehensive dataset that is often unattainable in traditional research settings.
• Scenario testing: VR is ideal for testing reactions to hypothetical situations or products not yet available in the market. It allows researchers to meticulously alter elements within the virtual environment and study the impacts of these changes on user behavior.

Big data and predictive analytics in quantitative research:

• Advanced data processing: The integration of big data technologies offers unprecedented opportunities to handle vast amounts of information rapidly and accurately. This capability is crucial for analyzing complex datasets that traditional statistical methods cannot process efficiently.
• Predictive insights: Leveraging machine learning algorithms and predictive analytics, researchers can identify trends, forecast outcomes, and make data-driven decisions with greater accuracy. This approach is increasingly used in fields such as consumer analytics, financial forecasting, and healthcare.
• Real-time data analysis: With the advent of AI, IoT and continuous data streams, quantitative research can now include real-time data analysis. This enables immediate responses and adjustments to marketing strategies, operational changes, and policy decisions based on current data insights.

Market research evolution

Global economic changes:

• Emerging markets: As economic power shifts towards emerging markets, research methodologies are adapting to explore these new consumer bases. This includes localized studies to understand cultural nuances and consumer behavior that differ significantly from established markets.
• Economic volatility: Global economic instability requires dynamic research methods that can quickly adapt to changing conditions and provide insights into consumer confidence and spending behaviors in the face of unexpected events or political upheaval.

Technological evolution:

• Automation in research: Automation tools are streamlining data collection and analysis, reducing the time and cost associated with traditional research methods. This shift allows researchers to focus more on strategy and interpretation rather than on manual data handling.
• Artificial Intelligence (AI): AI is playing a crucial role in pattern recognition and hypothesis generation, providing new ways to understand complex datasets and derive meaningful insights.

Shifting consumer behavior:

• Digital integration: As consumers increasingly interact with digital platforms, research methods are evolving to incorporate digital footprints as a key data source. This includes analyzing social media behavior, online shopping patterns, and mobile app usage.
• Sustainability and ethics: Growing consumer awareness around sustainability issues is prompting companies to conduct research not just on what consumers buy, but also on their values regarding environmental and social responsibility. This trend is pushing market research to include ethical considerations and long-term impacts in their studies.

The future of research methodologies lies in embracing technological advancements and adapting to the global socio-economic landscape. Innovations like VR and big data are not just enhancing traditional research methods, but are also creating entirely new fields and approaches to how research is conducted and applied. As these trends continue to evolve, they will shape the strategic directions of businesses and governments alike, making research an even more integral part of decision-making processes.

Looking for a deeper insight into qaul and quant techniques? Be sure to check out our guide to qualitative research techniques.