Perfecting A Quality Survey

Thursday, 16 November 2023, 9:20 am
Article: Hugh Grant

Quality surveys are one of the best modern tools for assessing public opinion. Through statistical analysis they can offer deep insight into the opinions of communities, companies, and populations. On a surface level they’re also quite simple, all one needs is questions and people willing to answer them. Unfortunately in reality surveys are much more complex, with lots of pitfalls which need to be avoided along the way.

The first of these is poor planning. It’s a simple and intuitive step to take, but many skip it in the process of rushing their project. Planning consists of asking several key questions. Questions like who’s taking the survey? Where is the survey being conducted? What is the central aim of the survey? What questions will get at this central aim? And when is the survey being conducted?

These questions can be figured out as one goes, but this makes any delays or mistakes far more fatal. Planning and giving proper scope to the survey helps keep it on track and clear in what it's trying to do. Assuming this is done, the next step is the creation of the actual survey questions. This is another deceptively simple step with many pitfalls, with the most nuance of any process within survey creation.

Big mistakes in question creation are common and subtle. Leading questions, loaded questions, and multi-answer questions all seem fine at first but are very problematic. For example, the question “How often do you drink ___ brand soda?” seems like a question. It’s clear, it doesn’t suggest a certain answer, but even this question has a key error. It makes an assumption.

Properly phrased, there would be two questions. The first would ask if the participant drinks soda, and the second would ask about specific brands. This is a subtle difference, but it can meaningfully skew answers and put the results in question. Poor question creation compromises the results of the entire survey.

Alright but let’s say everything so far has gone to plan. The project is planned, the questions are solid, and now it’s time to move into execution. Execution is the simplest step, but it relies heavily on the previous parts of survey creation. If there’s no plan, and the questions are confusing, there’s going to be a lot of holdups in execution. If the plan is proper, and the questions are clear, it should go smoothly.

Of course there will always be unexpected issues, and that’s why testing may also be helpful. Especially for surveys which are large in scope, testing can be an important tool. Moving past execution leads directly to analysis. This is likely the most complicated step. It requires statistical knowledge and data analysis and manipulation expertise. For anyone conducting a survey who lacks that sort of expertise, outsource. There’s no shame in it, and a poorly analyzed survey is useless at best but actively harmful at worst.

Finally after analysis there’s reporting. The big pitfall to avoid here is overcomplication. The process of getting to reporting is already challenging enough, requiring all past steps to go smoothly. In a sense it’s the easiest part of the process, but it’s also the step where everything comes together. Even a survey with the best planning, questions, execution, and data will be useless if reported badly.

So again, when reporting, avoid overcomplication. Easy ways to do this is through the use of graphs, minimal jargon, and clear data. Another key tip is making sure to tailor the reporting to whoever will be viewing it. A survey made for general education versus inciting change are very different.

And through following these tips and avoiding these mistakes, an effective survey can be created. It’s a challenging process, more challenging than people give it credit for. Although it’s absolutely manageable with some forethought and effort. If this process does seem too challenging, there are also companies which created quantitative surveys. This removes some creative control, but it’s better to have an effective survey outsourced than a poor survey done in-house.

© Scoop Media