In our last post we discussed different survey mistakes, now we provide some solutions to these common survey mistakes! Please note that it is always important to consider organizational and evaluation goals when creating surveys. Know that these are simply a few of the ways to avoid common survey mistakes.
Mistake #1: Using All Qualitative or All Quantitative Questions
Solution: Include both qualitative and quantitative questions
Having a healthy mix of qualitative and quantitative questions keeps respondents engaged and provides more comprehensive data. If there are many quantitative questions, consider if respondents have opportunities to provide additional feedback in the survey. Consider turning a couple quantitative questions into qualitative questions that help respondents share more of their perspective. If there are many qualitative questions, consider adding categories or ratings. For example, ask yourself if any of the questions can be turned into a Likert Rating Scale. Also, consider if similar questions can be grouped into a matrix that offers one set of response options. Surveys do not have to have an exact 1:1 ratio of qualitative and quantitative questions. Simply, consider both types of questions as you design your survey.
Mistake #2: Double-Barreled Questions
Solution: Identify where separation is needed
For most double-barreled questions, the best solution for this common survey mistake is to separate these questions into two different concepts. After recognizing the two areas of interest within one question, identify where the separation creates the most sense. For example, a question asking, “How do your parents and siblings help you with your homework?” can be converted into two separate questions. One question would focus on parents helping youth with their homework. While, the other question would focus on the role of siblings with homework. This approach results in better data. Also, similar questions or statements can be grouped into a matrix with a shared Likert Rating. Make sure to group questions together that measure similar concepts to avoid respondent confusion. Implement the suggested changes for both qualitative and quantitative questions.
Mistake #3: Asking Biased Questions
Solution: Refine questions to be neutral and implement skip logic
To avoid asking biased questions, present survey questions as neutrally as possible. Phrase the questions in an objective tone that allows the respondent to provide both negative and positive feedback. Objective questions gather greater information and provide a more accurate picture of respondents’ views. Another tactic to avoid loaded questions is to implement skip logic which guides respondent to the questions that are most relevant to them. Implementing skip logic can save time and ensure that respondents are only asked questions that are relevant to them. For example, if you want to learn what respondents think of a specific service, skip logic can ensure that only respondents who have received the service answers associated questions. By utilizing these tactics, your survey will be more objective and relevant for respondents.
Mistake #4: Writing Complicated Questions
Solution: Rewrite questions to make them easier to understand
Respondents need to understand exactly what the survey asks about their experiences or perspectives. To resolve complicated questions, consider breaking the question into smaller sections and providing clearer instructions. Use specificity and audience-appropriate terminology to reduce the appearance of ambiguous questions within the survey, making it easier for respondents to understand. Keep questions simple, direct, and straightforward. Know your intended audience and design questions that they would understand. Implementing these suggestions will lead to more accurate answers, improving the quality of data for analysis and interpretation.
Mistake #5: Providing Confusing Item Responses
Solution: Provide clear, distinct answer choices that progress consistently
To address this common survey mistake, use the most appropriate scale or item responses for each question. For instance, questions that ask for a specific time frame, like, “How often do you go to the park?” should represent a period of time such as once a week or once a month. After selecting the appropriate item response, ensure that the answer choices cover distinct categories for respondents to distinguish between choices. For example, a scale that ranges from: “Strongly Disagree” to “Strongly Agree” with “Neither Agree nor Disagree” in the middle creates easy distinctions for respondents. Finally, all answers choices should either be presented in ascending order (i.e., from smallest to largest) or descending order (i.e., from largest to smallest).
While research shows mixed results on whether ascending or descending order provides the most unbiased way to collect data, confirm that answer choice order is consistent throughout the entire survey. Being intentional regarding the order of item responses helps respondents know the flow of the survey, easing their abilities to provide accurate information. Following these steps to provide clear, distinct answer choices will provide clearer data for analysis and interpretation.
Mistake #6: Creating a Disorganized Survey Structure
Solution: Organize survey questions and sections in a smooth flow and provide instructions
Before finalizing your survey, make sure the questions provide a clear path for respondents to follow. Grouping similar questions together, such as those about program structure, components, and outcomes, helps respondents arrange their thoughts and experiences. Designing surveys with clear instructions before each section ensures that surveys are well-organized. Further, creating a survey structure that is logical and intuitive ensures respondents provide answers that align with their experiences.
Need help with your survey?
These solutions to these common survey mistakes and tactics will help your organization design an effective survey that captures quality data. If you have additional questions, connect with REC! We provide tailored support to our clients, helping them design effective surveys and other tools to meet their organizations’ needs.
Sources
Birkett, A. (2022). Survey Response Scales: How to Choose the Right One for Your Questionnaire. CXL. Link: https://cxl.com/blog/survey-response-scales/#h-how-to-design-survey-rating-scales
Chyung, Y. (2019). Evidence-Based Survey Design: “Strongly Agree” on the Left or Right Side of the Likert Scale? ATD. Link: https://www.td.org/atd-blog/training-data-is-the-best-kept-secret-in-enterprise
Cox, Jacqueline. (2019). Eliminating Ambiguity in Survey Questions. GutCheck. Link: http://gutcheckit.com/blog/eliminating-ambiguity-in-survey-questions/
Gleason, D. (2022). Qualitative Vs. Quantitative User Research: The Answers You Will (and Won’t) Get from Each. Hotjar. Link: https://www.hotjar.com/blog/qualitative-vs-quantitative-user-research/
McLeod, S. (2019). Likert Scale Definition, Examples and Analysis. Simply Psychology. Link: https://www.simplypsychology.org/likert-scale.html
Qualaroo. (2022). Skip Logic Survey – A Complete Guide on How to Design Branching Surveys. Qualaroo. Link: https://qualaroo.com/blog/skip-logic-survey/
Qualtrics. (n. d.). Survey Question, Sequence, Flow, & Style Tips. QualtricsXM. Link: https://www.qualtrics.com/experience-management/research/question-sequence-flow-style/
Shtivelband, A. (2023). 6 Common Survey Mistakes. Research Evaluation Consulting LLC. Link: https://researchevaluationconsulting.com/common-survey-mistakes-tips-for-effective-survey-design
Related Posts: