Survey Analysis: Determining the Right Process for Reporting Your Data
Have you ever been involved in a survey research project and, once data starts to come in, thought, “Now that I have this data, what’s the best way to showcase the information?” It’s a common challenge amongst researchers and one we’ve successfully tackled by evaluating the survey’s components through a series of questions outlined below.
We all know that surveys are a versatile and cost effective way to collect data of all different types. They are considered a relatively simple research tool and are used across all industries in various forms. For example, agricultural industry relies heavily on grower/farmer surveys in order to compile data on management practices, as part of stewardship and outreach efforts. However, once collected the question becomes how to efficiently analyze and present survey-collected data. There are multiple clues found in the collection process itself that factor into how the data can be best reported:
Does the data require anonymization? A basic rule in survey data collection is that information related to identity or privacy protection must be altered or removed in such a way that the subject can no longer be linked to the survey data. We have found that developing a strong data anonymization process provides an additional comfort level for survey subjects who provide data in a voluntary survey format. For example, sometimes the survey can be completely anonymized with no identifying information collected. Other times, a secondary survey can be completed if a list of participants is required. If identifying information is collected, however, we’ve found that the assignment of protected and coded identifiers provides a mechanism to anonymize dataset.
What was the sample size and response type? Data analysis from survey results requires an understanding of the sample size and response types, whether in the form of numerical or text data. Multiple choice responses and close- or open-ended text responses must be considered differently for analysis and reportability. Summary statistics, including evaluation for normal distribution, as well as descriptive statistics and significance testing can be applied to survey response data during this step.
How large and complex is the data? If the survey data is particularly large or complex, various database tools are available for data organization and analysis. Beyond the use of simple spreadsheets, we have used tools such as the ArcGIS Survey123 application from the ESRI Geospatial Cloud. If we’re compiling records from various databases for processing, we have applied python scripts to extract data and store them in Microsoft SQL databases. The development of application programming interfaces (APIs) then allow us to bridge the gap between our database and output reports and web tools.
As a part of the data analysis step, it is also critical to understand the uncertainty in the data set. Questions to consider may include:
- Was the sample size sufficient?
- Was random or non-random sampling applied?
- What was the response rate?
- How did the people who agreed to participate differ from those who declined?
- How was the survey administered (paper, electronically, in-person, or telephone interview), and
- How could the administration format impact participation or responses?
How will the final report be presented? Our goal is to always present survey results in a format that clearly and concisely communicates the data and aligns with the aim of the research. How we accomplish this goal boils down to our end audience. Will it be presented within a scientific setting with in-the-know users or will it have a more general audience, such as a company’s C-Suite Executives? Do we need to provide supporting data or can the results stand on their own?
We strive to make the information instantly understandable and, as such, often employ a mix of graphic and pure data presentations. For example, we have found that customized end-user reports with graphical data presentation has been of particular use for growers and farmers participating in agricultural surveys. We have been able to graph metrics over time for a single grower’s performance, as well as comparisons to other growers in a way that maintains the anonymity of all survey contributors. You likely see similar graphical presentations on your monthly electric bill, which serves as a simple mechanism to communicate trends overtime and comparisons.
Answering the above questions will help ease the challenges associated with presented survey data which, in turn, results in more straight-forward communication and greater data usage.