Becoming a data-driven organization is an exciting opportunity for your organization to improve its operations and functionality. However, some data myths can skew results, impact conclusions, and interfere with next steps for your organization. Understanding these data myths can help ensure you are making data-informed decisions.
Here is a breakdown of the common data myths:
Myth 1: All Data is Good Data
Not all data is created equal. Some data is less useful, especially if it is not capturing relevant information. Other information may represent bad data, particularly if you doubt your findings. Poor instrument development, data collection, data recording, and data preparation practices all lead to bad data. For example, ill-formed survey or interview questions can generate data that does not capture the target outcome. Being able to tell the difference between good, bad, and less optimal data are important in making data-informed decisions.
Myth 2: Quantitative Data is the Best Data
While quantitative data may be best when you need to run statistical analyses, qualitative data provides insight into specific topics. Ignoring qualitative data based solely on the idea that it is inferior to quantitative data limits your organization’s ability to gain meaningful insight into what your participants think and feel. A comprehensive understanding of participants’ experiences often requires a healthy mix of both qualitative and quantitative data, tailored specifically to the data-informed decisions your organization is striving to determine. Therefore, it is important to know the pros and cons of both methods when collecting data.
Myth 3: Complex Data Analysis Gives the Best Insights
You do not have to be a statistical guru or use expensive statistical software to analyze data in a meaningful way. While complex data analysis using inferential statistics provides meaningful insight, it is not the only way to understand impact. Many research questions are accurately answered using simpler statistics that still provide the types of information necessary to make recommendations. Descriptive statistics, including averages and frequencies, can provide the insight your organization needs to move forward. Each organizations’ data analysis needs are unique, and understanding the most beneficial data analysis for your organization is key to making well-informed decisions.
Myth 4: The More Data, the Better
Ideally, having more and richer data allow organizations to dig deeper into participant experiences and program impacts. However, because not all data is good data, more data can further confuse and complicate evaluation. For example, more data can make analysis overwhelming and difficult to connect the data pieces necessary to draw meaningful, specific conclusions. Focusing on data that is relevant, actionable, and timely ensures your organization is making the most of your resources.
Myth 5: If Data is Presented Well, Everyone Will Come to the Same Conclusions
People interpret data and evaluation findings in different ways, affecting how they draw conclusions and plan next steps. For example, a finding could be that 85% of participants were Satisfied or Very Satisfied with a program. One person might see that result as a success to celebrate while another person might see it as an opportunity for improvement. Both interpretations are valid, but these different conclusions might lead to different next steps. Understanding that each person brings their own backgrounds, perceptions, and expectations to data interpretation is important when sharing findings with others.
So how do you combat these myths?
Being aware of these data myths will help your organization improve how data is collected, analyzed, and communicated. Understanding how these data myths influence evaluation activities should help you make informed choices around data planning, processes, and decision-making. If you still have questions, reach out to REC for support!
Sources
Bhandari, P. (2023). Descriptive Statistics: Definitions, Types, Examples. Scribbr. Link: https://www.scribbr.com/statistics/descriptive-statistics/
Glen, S. (n. d.). Inferential Statistics: Definition, Uses. StatisticsHowTo. Link: https://www.statisticshowto.com/probability-and-statistics/statistics-definitions/inferential-statistics/
Kendall, S. (2021). More Data Doesn’t Always Mean Better Data. Safe Graph. Link: https://www.safegraph.com/blog/more-data-doesnt-always-mean-better-data#:~:text=Adding%20more%20data%20to%20the,over%2Dcomplication%20and%20incorrect%20results.&text=Here’s%20another%20way%20of%20looking,particular%20entity%20at%20your%20fingertips
Shtivelband, A. (2017). Quantitative Versus Qualitative Data. Research Evaluation Consulting LLC. Link: https://researchevaluationconsulting.com/quantitative-versus-qualitative-data/
Shtivelband, A. (2021). How to Become a Data-Driven Organization. Research Evaluation Consulting LLC. Link: https://researchevaluationconsulting.com/how-to-become-a-data-driven-organization/
Shtivelband, A. (2021). Let Data Drive your Decisions. Research Evaluation Consulting LLC. Link: https://researchevaluationconsulting.com/using-data-to-make-decisions/
TechTalks. (2020). What Is Bad Data and How Will It Affect Your Business? Tech Talks. Link: https://bdtechtalks.com/2020/03/21/what-is-bad-data/
Related Posts: