Statistics Literacy: Reading Numbers in the Wild
2026-08-06 · 9 min read
Averages, base rates, sampling, margin of error and misleading charts — the small toolkit that catches most bad numerical claims.
Most bad statistics are not fraud They are ordinary numbers presented without the context that makes them mean anything. You do not need advanced methods to catch them. A handful of habitual questions catches the majority.
Which average, and why Mean, median and mode answer different questions. The mean is sensitive to extremes; the median is not. For anything skewed — income, house prices, wait times, damages — the median describes a typical case and the mean describes the arithmetic, not the experience.
Whenever you see "average," ask which one and how spread out the data are. An average commute of thirty minutes could mean everyone takes thirty minutes or half take ten and half take fifty. Those are different cities.
Percentages need a denominator "Cases rose 200 percent" is meaningless without the starting number: from two to six is a different world from two thousand to six thousand.
Watch the difference between relative and absolute risk. "Doubles your risk" sounds alarming; if the risk goes from one in ten thousand to two in ten thousand, the absolute change is 0.01 percentage points. Both statements are true, and they suggest very different actions. Reporting that gives only the relative figure is withholding the number you need.
Base rates and the screening trap Suppose a test for a condition affecting 1 in 1,000 people is 99 percent accurate. You test positive. The intuitive read is a 99 percent chance you have it. The correct answer is about 9 percent.
Work it through with counts rather than probabilities. In 100,000 people, 100 have the condition and 99 test positive. Of the 99,900 without it, 1 percent — 999 people — test positive falsely. So 1,098 positives, of which 99 are real: roughly 9 percent. Rare conditions produce mostly false positives even with accurate tests, which is why screening programmes are designed carefully and why a single positive usually triggers a second test.
Sampling decides everything A poll of 1,000 well-sampled people can describe a nation. A survey of 100,000 self-selected website visitors describes people who visit that website and chose to answer.
Ask who was asked, who could have been asked, and who answered. Response rates below about 10 percent make representativeness a real question. Nonresponse is rarely random — people with strong views and spare time answer more.
Margin of error covers sampling variation only. It says nothing about badly worded questions, poor sampling frames or dishonest answers, which are often the larger errors.
Charts that mislead without lying Truncated y-axes exaggerate small differences. Dual axes can manufacture apparent correlation by scaling two series until they overlap. Area charts that scale both width and height to a value overstate the change by squaring it. Cherry-picked date ranges hide trends by starting at a convenient peak or trough.
Look at the axes before the shape. If a chart has no axis labels, treat it as an illustration rather than evidence.
Regression to the mean Extreme measurements tend to be followed by less extreme ones, purely by chance. The worst-performing schools improve after intervention; the best decline. Some of that is real and some is arithmetic, and studies without a control group cannot separate the two.
This one quietly explains a large fraction of claims about treatments, coaching methods and business turnarounds.