This week my former colleague Ira Glass joined the chorus http://www.thisamericanlife.org/blog/2015/05/canvassers-study-in-episode-555-has-been-retracted of those backing off of stories on apparently bogus social science research. The original study, in the respected journal Science, purported to show that canvassers could change the minds of survey subjects initially opposed to gay marriage if they spent a mere 20 minutes talking to them. The data behind the study was more than flawed—it was fabricated, and has since led to a rare retraction by the lead author http://tinyurl.com/luqvdg5. Inquiring minds want to know: should journalists have smelled something fishy?
Let me lead the chorus of critics in saying that I, for one, would never have been so gullible. Journalistic hindsight is better than 20/20, it is full of shadenfreude: I’m so glad I didn’t do it! But slow down. This is not the Rolling Stone article, where reporters and editors simply failed to ask hard questions. When it comes to science pieces reporters are at extreme risk. Typically they trust the data is correct. Journalists can’t be expected to climb inside the cyclotron and verify a physics experiment, right? We have been schooled to trust the peer review process, which depends on the scientific community to validate the evidence before it is published.
That trust puts us at an important disadvantage when it comes to science. Would we trust a defense official to interpret data about spending, or would we examine the figures ourselves? Statistically, fraud in science is pretty rare, so it’s hard to identify harm here. But when it happens, as it has here, it exposes our helplessness when it comes to complex material.
The sad thing here is that it took an enormous effort from other scientists, not journalists, to ferret out the fraud. Stanford’s David Brockman et al went to great lengths to validate the study, and found they could not. Brockman’s group applied a level of statistical acumen that few daily journalists can match. This fact underscores that science journalism is in a strange category. It exists in a twilight zone where few journalists dare to question or even examine underlying data. So maybe it’s time to question to absolute trust many journalists have in their science sources.
