Updated October 2026. October 2026: causation claim corrected from Altman and Krzywinski (2015); unsourced bad-data line removed.
A mistake in analysis changes what you can say about a population, and what you can say about the link between two measures. The same six errors show up often.
Whose answers are in the file
This error is common in large data sets. The tables can look solid while the people in the file are not the people you set out to describe. Check who was eligible, who was reached, and who refused before you generalise.
Measurement
The numbers are only as good as the recording. A wrong code, a skipped item, or a mixed unit will sit in the table until someone compares the file with the questionnaire. Do that check before you adjust anything.
Correlation and cause
Researchers still treat a correlation as a cause. A test can show that two measures move together. A cause claim needs an experiment, or a design built for that purpose, and it stands only while the assumptions of that design hold. Altman and Krzywinski (Nature Methods, 2015) separate the ideas. Correlation is a form of association. Cause is a further claim.
What the column actually records
A large share of prisoners from one high school records where those prisoners went to school. The reason they were imprisoned is a different measure. Treating the school as the cause goes past what the column contains.
Figures that move when conditions move
Daily and weekly movement is easy to spot. A result that describes the current year can miss the next year once prices, policy or behaviour change. A close fit to past results still leaves the following year open.
The choice of model
Random forests and logistic regression are different models. They are not interchangeable tests. Using one in place of the other can change the result. Report which model you used, and which question it was meant to answer.
Updated October 2026
- The causation paragraph no longer says that no test can separate correlation from cause. A correlation still does not establish cause. Source: Naomi Altman and Martin Krzywinski, "Association, correlation and causation", Nature Methods (2015).
- Removed the line that bad data outnumber good data. No source gives that split.
- Random forests and logistic regression are described as models rather than statistical tests. The warning about method choice is unchanged.
- Grammar corrected. No newer Jamaican official statistic applies to this note.