# Questionnaire design matters most

A questionnaire that never asks the question the study set out to answer will fail when the analysis starts, however tidy the data entry was.

Over years of data entry and analysis for students, lecturers and companies, we have concluded that questionnaire design is one of the critical parts of any study that uses a questionnaire. The instrument can make or break the analysis. Give it time at the start.

A usable questionnaire has obvious headings, clear instructions and concise questions. The questions have to cover the hypotheses, or the main problems, the study is meant to examine. We have seen completed questionnaires loaded into statistical software that cannot answer the purpose of the study. The gap shows at the analysis stage, when the researcher finds that the result they hoped for was never asked as a question.

This happens often. Take care when you write the proposal and set the purpose. If you are unsure the questionnaire can produce the answers you need, get help with the design before you field it.

Choose the statistical methods while you are still writing the questions. That limits the recoding, grouping and rescaling you would otherwise do before the analysis can run. We often see data collection finished on the assumption that a parametric test will do the job, with no check on whether the measure can support that test. A t-test or an ANOVA needs a measure and a sample that fit it. Normality of every raw score is too broad a rule to rely on, and a large sample does not repair a question that was never asked. The mismatch is easier to fix on the draft than after the interviews are done.

A short consultation at the design stage saves time and money. It cuts the later manipulation of the file. It can also spare you a second round of fieldwork.

Run a pilot before the main collection. A pilot shows most of the faults in the questionnaire, so you can change them before the official start.

## Updated October 2026

- Grammar and broken sentences in the June 2011 post were repaired. The argument is the same.
- The claim that parametric tests require normally distributed data and a large sample was rewritten. That rule is too broad. Choose the analysis before fieldwork, and match the questions to it.
- No new external dataset. The pilot recommendation is unchanged.
- The 2011 snapshot names no person. The author line is Balcostics Research.


Source: https://balcostics.com/blog/questionnaire-design/