Do any news organizations have a Numeracy Editor?

For fifteen years, the New York Times had a Public Editor, whose job was to visibly uphold journalistic ethics. The Public Editor would publicly discuss errors, biases, or gaps in the paper’s coverage. (Some other news organizations continue to have a public editor position, though I think it’s not widespread.)

I’d like to propose something narrower: a Numeracy Editor. The Numeracy Editor’s job would be to help reporters and columnists use numerical and statistical reasoning well.

I’ve been pondering this idea for a while, and finally decided to write about it after reading Vatsal G. Thakkar’s excellent NYT Op-Ed Bring Back the Stick Shift a couple of weeks ago. It’s a good piece, but at one point it veers into unexpected non sequitur in an attempt to use statistics to support its argument:

Backup cameras, mandatory on all new cars as of last year, are intended to prevent accidents. Between 2008 and 2011, the percentage of new cars sold with backup cameras doubled, but the backup fatality rate declined by less than a third while backup injuries dropped only 8 percent.

The more you read that, the less it means. For a three-year period, the percentage of new cars sold with backup cameras doubled from whatever it was before — without knowing what it was before, this doesn’t tell us anything: the result of doubling a very miniscule percentage would still be a miniscule percentage, for example. Meanwhile, during that same three-year period, fatalities due to backups declined by some amount (less than a third) from whatever the rate was before — again, we don’t know. So does that decline represent a greater decline in backup fatalities than should be expected from whatever percentage of cars on the road newly have backup cameras? Or a smaller decline? There is no way to say. Also, we don’t know what percentage of cars driving on the road are new cars, which is highly relevant here.

If the author was trying say to that fatalities should have declined more, this paragraph does not support that case, but it doesn’t support any other case either. It throws some statistics into the air, as if to see how the wind catches them, but they don’t connect to each other and they have no bearing on the question at hand. As my friend Tom put it, it’s just a “number casserole”.

I certainly don’t mean to pick on on Thakkar — again, I liked the piece — or on the New York Times. This sort of thing happens in many publications; you can see it all the time, in the regular reporting just as much as in opinion editorials.

But given that this was the New York Times Op-Ed page — a forum that presumably takes quality control and editorial standards seriously — it’s worth asking: how did such a problematic paragraph make it through the filters? I think the answer is that there is no editor whose reputation are self-respect are on the line when numerical clunkers slip through. A few grammatical or spelling errors and someone’s job is in danger, but even glaring errors of statistical reasoning are currently costless.

I get that journalists and their editors tend to have backgrounds in language, political science, history, and other fields that don’t emphasize math. And that’s fine: this isn’t an “everyone should learn more math” argument. There are only a finite number of days in anyone’s life, there isn’t time to learn everything, and people make the choices they make for reasons. That’s exactly why a Numeracy Editor is needed: it would be her job to own this problem, and along the way help journalists learn the math they need. The writers would start to be more careful just knowing that someone is watching. A Numeracy Editor would have caught the problem in that Op-Ed right away, and once spotted, it’s easy to explain; the conversation with the author can take place before publication, as with any other kind of editing. Many errors of numerical or statistical reasoning are easy to understand once they’re pointed out (although there are also subtler cases, such as Simpson’s Paradox, that occur in real-life, policy-relevant situations and need to be watched for).

Unlike Public Editor, Numeracy Editor need not be a public-facing role. The main point is to help writers and other editors use math appropriately and to prevent mistakes. If the editor also wants to conduct a public discussion about using numbers and graphs in journalism, that would be a great public service too, but it’s a bonus. The role could do a lot of good purely behind the scenes.

Numeracy Editor should be an easier position to hire for than the broader role of Public Editor has been, because it doesn’t require nearly as much journalistic experience (the Numeracy Editor isn’t making hard judgement calls about how much anonymous sourcing is acceptable in a story, for example) and because the advice it provides would be less controversial.

Anyway, I don’t run a newspaper; all I have is this blog. I’d love to hear from anyone who works in or near journalism what they think of this idea.

(You can respond in a comment, or in this Twitter thread, or in this Identi.ca thread.)