So you are excited to begin collecting quality data for your organization. That’s great! The information you collect has the power to demonstrate your impact, inform decisions, and improve your programs. Yet, what do you need to ask yourself before you get started?
There are a number of methods to collect quality data. Some data collection methods are better than others. It all depends on context, available resources, and the purpose of the data collection effort. This blog post is the first of a four-part series that discusses strategies to improve the data collection process. This blog post focuses on four critical questions that your organization needs to ask before you start collecting quality data:
1) What data will you collect?
2) What is your organization’s capacity to collect data?
3) Who will manage, collect, analyze, and interpret the data?
4) How do you plan to use your findings?
Such questions will help guide your data collection efforts and inform the evaluation process.
1) What data will you collect?
You need a reason, a purpose to collect quality data. Only collect data that you need to collect. This might include data required by the government, funders, or other stakeholder groups. Think about ways to collect data in the most efficient and effective way possible. You may also decide to collect data as a way to demonstrate the impact your organization is making in the community. In this scenario, you need to determine how your organization or program is making a difference in the lives of your stakeholders and measure it. However, at what point does your organization decide not to collect specific information? If you do not know why you are collecting information, then stop collecting it. You will save your organization both time and money.
2) What is your organization’s capacity to collect quality data?
In other words, what resources does your organization have to collect data? Sometimes data collection efforts are limited by your timeframe, budget, or staff skills. For example, there is only so much data you can gather in a couple of days versus a couple of months. Do you have software that can manage the data you collect? Do you have staff with the skills to collect data? If you lack some of these resources, limit the scope of the data you collect. For example, if no one in your organization knows how to design an effective survey, find someone who does (e.g., external evaluator, intern) or provide your staff with professional development.
3) Who will manage, collect, analyze, and interpret the data?
Once you have data, you need to identify at least one person who will manage and oversee the process. This individual needs to be comfortable managing data and have previous experience in this role. Ideally, this person could train others in your organization on reliable and valid data collection methods. The important thing is that this person develops a system in which to collect accurate data and generate appropriate findings. The person managing the data collection process need not be the same person analyzing the results. However, this person needs to have a clear understanding of research design and methodology. To reduce error, limit the number of people involved in the data collection effort.
4) How do you plan to use these findings?
Organizations typically collect data because it has some kind of value or benefit. Therefore, think about how you will use your findings before you begin collecting data. Work backward, think about your end goal/s and the steps you need to take to get there. Ask yourself if you are you willing to share your results with stakeholders even if the focus becomes areas for improvement rather than success. When you understand the value of the data you collect, you develop more effective dissemination strategies.
When you can answer these questions, you can begin to consider which data collection methods you will utilize. Once your organization has answered these questions, you are well on your way to collect quality data. The next blog post will focus on the difference between quantitative and qualitative data, followed by the pros and cons of quantitative and qualitative data, key data collection methods in program evaluation, and concluding with a discussion of the strengths and weaknesses of each approach. By the end of this series, you should have a more solid foundation in which to collect quality data!
Feel free to contact me with any research and evaluation questions or needs at annette@researchevaluationconsulting.com