I am getting fed up with science journalism, and so here I am, bound and determined to make you annoyed too. First, a general introduction to things that annoy me:
- reporting the results of a self-selected survey as a population-wide finding (if you survey readers of Australian Top Gear and furthermore describe it as a
manliness survey
— unless that was the journalist — about how much they feel about their female partners’ driving skills, it’s going to be news if it doesn’t come up aswomen drivers suck
, not if it does) - when reporting on medical results, not reporting on exactly which population was studied
There’s more subtle examples that don’t annoy me so much because you have to have a bit of knowledge of the science in question to get underneath them. For example, studies where a bunch of sick people and a bunch of healthy people are studied and asked questions like do you eat eggs?
and the result is reported as either being sick is correlated with eating eggs
or (more usually) study links eating eggs to cancer!
are difficult to interpret because people who know they are sick tend to over-report (or alternatively healthy people under-report, or possibly both) anything that they think of as a risk factor. Because they too are looking for an explanation of why they’re sick.
Anyway, today’s example is from door number two:
In about 70 per cent of the kids, we’re seeing that if they’re not responding to their name at 12 months, they’ve gone on to receive a diagnosis of autism,[Associate Professor Robin Young of Flinders University] said.
A very important question follows. Was that:
- 70% of a sample of children who have been diagnosed as autistic didn’t respond to their name at twelve months; or
- 70% of a random sample of children who didn’t respond to their name at twelve months went to on receive a diagnosis of autism?
The difference is pretty important: the second is much more concerning to parents of children who aren’t responding to their name than the first one is. (To illustrate the difference, consider the statements 99% of women aren’t blonde
compared to 99% of people who aren’t blond(e) are women
in terms of the kinds of predictions you’re making when you’re told someone has dark hair.)
I don’t know if Associate Professor Young gave a sloppy quote here, was misquoted or whether a sloppy quote was selected from a more informative interview/statement/press release.