Through Time

Numbers: reading other people’s data and charts

How to see what an average, a poll, a chart or a study shown to you actually says, and where it is likely to mislead you.

A course for adults who are shown numbers in the news, at work and in advertising and who need not be mathematicians. You will read averages, samples, percentages, charts and scientific papers and ask the right questions of them, with almost no formulas. Making your own decisions under uncertainty belongs to the course “Reasoning when nothing is clear.”

So far there is only this table of contents. Lessons appear one at a time, and we promise no dates. Feed of new lessons (Atom)

18 lessons, 5 parts

Part 1 Simple numbers with a catch

The most common ways to mislead take no mathematics, and spotting them takes none either.

  1. The average salary and yours

    The mean and the median of the same incomes: why one billionaire lifts a room’s average, and which number you will be shown to impress you.

    coming soon

  2. Percent and percentage points

    A rise from 2 to 3 percent is one point and also a 50 percent increase: both answers are right, and what matters is which was chosen for you.

    coming soon

  3. Twice the risk: from what to what

    How relative risk differs from absolute risk, with Britain’s scare over the pill, where “double” turned out to be a tiny difference.

    coming soon

  4. A number with no denominator

    “Thousands of cases” out of how many, over what period and compared with what: that is where any talk of a large number has to start.

    coming soon

  5. How common it is to begin with

    What a base rate is, and why without it you cannot understand a risk or a test result: the courses on reasoning and health apply the idea.

    coming soon

Part 2 Where the data came from

Before you read a result, ask who was included, who was lost and who chose what to count.

  1. Who was asked

    The Literary Digest gathered millions of replies and picked the wrong president, and Gallup’s far smaller sample got it right: size does not fix a skew.

    coming soon

  2. The planes that came back

    Survivorship bias, with Abraham Wald and the advice of successful people, and what the popular legend added to Wald’s real work.

    coming soon

  3. When a measure becomes a target

    Goodhart’s law: how rankings, reports and model benchmarks break once someone gets paid for the number.

    coming soon

Part 3 Correlation and cause

Two lines moving together explain nothing yet, and establishing a cause is far harder than showing a link.

  1. A link is not a cause

    A third factor, cause and effect reversed and plain coincidence: how to spot each when someone shows you a handsome line.

    coming soon

  2. Simpson’s paradox

    Berkeley admissions: overall women got in less often, by department the bias vanished, so the conclusion depends on how the data is added up.

    coming soon

  3. Why experiments exist

    Observational studies suggested hormone therapy protects the heart, and a randomized trial did not bear that out: what random assignment gives you.

    coming soon

Part 4 Charts

A chart persuades faster than text, which is why it needs reading more slowly.

  1. Same numbers, different pictures

    Anscombe’s quartet: four data sets with nearly the same statistics and quite different plots, and why one summary number is not enough.

    coming soon

  2. How a chart misleads

    The cropped axis, the picture in place of a bar and two scales on one plot: how to spot them in ten seconds.

    coming soon

Part 5 How to read a study

“Scientists have proved” often means one paper with one threshold, and this part explains what stands behind it.

  1. What p below 0.05 means

    What a p-value really says: it gives neither the probability of a hypothesis nor the size of an effect.

    coming soon

  2. Significant does not mean important

    A large study will find even a tiny difference, and the question is what it changes for an actual person.

    coming soon

  3. Fishing for a result

    Freedom in collecting and analyzing data sharply raises the rate of false findings: what to look for in a paper’s methods.

    coming soon

  4. The replication crisis

    What happened when a large project repeated a hundred psychology studies, and why that finding is disputed too.

    coming soon

  5. A look ahead: charts on demand

    Models produce analysis and charts in seconds, so honest work and fudging both get cheaper: what to ask when no person stands behind the numbers.

    coming soon