17 lines
6.2 KiB
Plaintext
17 lines
6.2 KiB
Plaintext
<s>Seven</s> Eight Warning Signs of Junk Science
|
|
<p>I’ve written before about scientific <a href="http://esr.ibiblio.org/?p=1642">error cascades</a> and the pernicious things that happen when junk science becomes the focus or rationale of a political crusade.</p>
|
|
<p>The worst example of this sort of thing in my lifetime, and arguably in the entire history of science, has been the AGW (anthropogenic global warming) panic. Now that the wheels are falling off that juggernaut, I’m starting to hear ordinary people around me wonder how I knew it was bullshit and hot air so much in advance…</p>
|
|
<p><span id="more-3974"></span></p>
|
|
<p>Some of the answer to that is complicated and not easily replicable. I happened to have the right sort of knowledge base to know that, for example, specific AGW-panicker claims about historical climate were impossible to reconcile with primary evidence – wine grapes grown at 59 degrees north around the year 1000, that sort of thing. This motivated me to dig for other problems with their narrative well before they were really on the public’s radar.</p>
|
|
<p>But a lot of it was more general. I’ve seen a lot of “scientific” panics ginned up from nonexistent or scanty evidence over the last several decades. There’s a pattern to these episodes, a characteristic stench that becomes recognizable after a while. I’ll describe some of the indicia, which I’ve culled from episodes like the Alar scare, the ozone-hole brouhaha, the AIDS panic (are you old enough to remember when it was predicted to become endemic among heterosexuals in the U.S.?), acid rain, and even the great global <em>cooling</em> flap of 1975.</p>
|
|
<p>So. Here is a non-exclusive list of <s>seven</s> eight symptoms to watch out for:</p>
|
|
<p><em>Science by press release.</em> It’s never, ever a good sign when ‘scientists’ announce dramatic results before publishing in a peer-reviewed journal. When this happens, we generally find out later that they were either self-deluded or functioning as political animals rather than scientists. This generalizes a bit; one should also be suspicious of, for example, science first broadcast by congressional testimony or talk-show circuit.</p>
|
|
<p><em>Rhetoric that mixes science with the tropes of eschatological panic.</em> When the argument for theory X slides from “theory X is supported by evidence” to “a terrible catastrophe looms over us if theory X is true, therefore we cannot risk disbelieving it”, you can be pretty sure that X is junk science. Consciously or unconsciously, advocates who say these sorts of things are trying to panic the herd into stampeding rather than focusing on the quality of the evidence for theory X.</p>
|
|
<p><em>Rhetoric that mixes science with the tropes of moral panic.</em> When the argument for theory X slides from “theory X is supported by evidence” to “only bad/sinful/uncaring people disbelieve theory X”, you can be even more sure that theory X is junk science. Consciously or unconsciously, advocates who say these sorts of things are trying to induce a state of preference falsification in which people are peer-pressured to publicly affirm a belief in theory X in spite of private doubts. </p>
|
|
<p><em>Consignment of failed predictions to the memory hole.</em> It’s a sign of sound science when advocates for theory X publicly acknowledge failed predictions and explain why they think they can now make better ones. Conversely, it’s a sign of junk science when they try to bury failed predictions and deny they ever made them.</p>
|
|
<p><em>Over-reliance on computer models replete with bugger factors that aren’t causally justified.</em>. No, this is not unique to climatology; you see it a lot in epidemiology and economics, just to name two fields that start with ‘e’. The key point here is that simply fitting historical data is not causal justification; there are lots of ways to dishonestly make that happen, or honestly fool yourself about it. If you don’t have a generative account of <em>why</em> your formulas and coupling constants look the way they do (a generative account which itself makes falsifiable predictions), you’re not doing science – you’re doing numerology.</p>
|
|
<p><em>If a ‘scientific’ theory seems tailor-made for the needs of politicians or advocacy organizations, it probably has been.</em> Real scientific results have a cross-grained tendency not to fit transient political categories. Accordingly, if you think theory X stinks of political construction, you’re probably right. This is one of the simplest but most difficult lessons in junk-science spotting! The most difficult case is recognizing that this is happening even when you agree with the cause.</p>
|
|
<p><em>Past purveyers of junk science do not change their spots.</em> One of the earliest indicators in many outbreaks of junk science is enthusiastic endorsements by people and advocacy organizations <em>associated with past outbreaks</em>. This one is particularly useful in spotting environmental junk science, because unreliable environmental-advocacy organizations tend to have long public pedigrees including frequent episodes of apocalyptic yelling. It is pardonable to be taken in by this the first time, but foolish by the fourth and fifth.</p>
|
|
<p><em>Refusal to make primary data sets available for inspection.</em> When people doing sound science are challenged to produce the observational and experimental data their theories are supposed to be based on, they do it. (There are a couple of principled exceptions here; particle physicists can’t save the unreduced data from particle collisions, there are too many terabytes per second of it.) It is a strong sign of junk science when a ‘scientist’ claims to have retained raw data sets but refuses to release them to critics.</p>
|
|
<p>It would be way, <em>way</em> too easy to list the ways these symptoms have manifested with respect to the AGW panic. It’s a more useful exercise for the reader to think back and try to recognize them in <em>previous</em> junk-science flaps. Go and learn. And don’t get fooled again.</p>
|