Quackery, Hokum, Baloney: Separating Science from Stupidity

Suppose I told you that somewhere between Earth and Mars there is an invisible teapot that orbits the sun once every 666 days. The teapot is invisible because it is cloaked using technology developed by space aliens, who left it there to monitor our progress. They believe that once we make contact with the teapot, an alarm will sound and they will return to see if we are truly worthy of being galactic citizens.

The Orbiting Teapot

The question you need to ask is; “Is that true?” and if so, “How do I know it’s true?” This is, of course, the fundamental question for Science. What do we know and how do we know it.

The invisible teapot was created by the philosopher Bertrand Russell back in 1952 and went like this:

If I were to suggest that between the Earth and Mars there is a china teapot revolving about the sun in an elliptical orbit, nobody would be able to disprove my assertion provided I were careful to add that the teapot is too small to be revealed even by our most powerful telescopes. But if I were to go on to say that, since my assertion cannot be disproved, it is intolerable presumption on the part of human reason to doubt it, I should rightly be thought to be talking nonsense.

But this is the 21st century, after all, and we are all sophisticated, intelligent people, and we have a wealth of scientific knowledge and instrumentation to help us test for the presence of the teapot. The “powerful telescopes” of 1952 have been replaced by much more sophisticated technology and we can now see much more of our solar system.

Bertrand Russell

In principle, therefore, we could focus all the world’s telescopes along the elliptic plane and search for the pot. We already know what it looks like, we know it is between Earth and Mars, and we know it is cloaked. The cloaking business may make it trickier but we also know from current research that “cloaking” is little more than deflecting light around a mass. We could spot the teapot by looking to see if there is a teapot-sized region of space that makes stars behind it appear to change position; this is because the mass of the pot will cause light to bend ever so slightly (it why in a solar eclipse we can see stars on the edge of the sun that are actually behindit).

The key thing to note here is that we TEST for the presence of the teapot and refuse to accept it on faith. I may be able to spin the most wonderful story about the pot, about how beautiful and splendiferous it can be, and how much it has changed my life, but if all I have is my personal perceptions and ideas, you would be right to treat what I say as bullshit of the highest order.

The only way for me to prove that I am right is to provide evidence of the pot. If the telescopes suddenly reveal a sea-green piece of revolving pottery, orbited by teacups (hey, there may be more to the teapot than I knew!) then you should start taking me more seriously. When several independent observatories have pictures, and all independently identify its location by numerical coordinates, and spectrograph analyses all show its chemical structure, then I’m pretty much vindicated. And if after a few years NASA’s latest “Pot Probe” reaches that location and scoops up the teapot into its gaping maw, then that’s likely to be as much proof as any reasonable person would require to be able to say, “Yes, there IS a teapot in outer space.”

Testability is a cornerstone of Science. And the thing that has to be tested is a HYPOTHESIS[1], which is defined as;

A proposition or principle put forth or stated (without any reference to its correspondence with fact) merely as a basis for reasoning or argument, or as a premiss from which to draw a conclusion.

The aim of Science is to test a hypothesis, that is, to see if it is true or false. Now in reality, you don’t actually prove something to be “true,” you “support” it. Truth and support are two very different things. If my hypothesis is “All swans are white,” I can test it by sitting by a river bank photographing every swan than lands on the water in front of me. If I have several friends across the world do the same thing, we might find that all the pictures we have turn out to be white swans. Does this mean that “All swans are white” is true? Nope, it just means that there is overwhelming support, based on many observations and measurements by many people, that swans are white. However – and here’s the kicker – if we find just ONE black swan, the hypothesis is dead in the water. Gone. No amount of evidence can make a hypothesis true, but just one observation can make it false.

This is the principle of falsification, promoted and discussed at great length by the great philosopher of Science, Karl Popper, whose Logic of Scientific Discovery[2] is  a classic in the field. For a more relaxed read (and by “relaxed” I mean “requires a little concentration” as opposed to “Oh, my frickin’ head’s about to explode!”) you might prefer Objective Knowledge: An Evolutionary Approach[3], published in 1972.

Karl Popper

It’s also explained eloquently by another Carl, Carl Sagan, in his 1996 book The Demon-Haunted World: Science as a Candle in the Dark. I can’t recommend this book enough to students who are eager to learn about the scientific method in an enjoyable and entertaining fashion. It is, perhaps, his best and most lucid book, and it beats me why this isn’t recommended as a high-school text or at least an undergraduate offering to all students. Many people have a woeful understanding of what science and the scientific method are all about and this one book explains it so well.

One particularly practical offering is Chapter 12: The Fine Art of Baloney Detection, where Sagan offers a number of ways to check whether a proposition or hypothesis is valid. Here’s the list for Baloney Detection:

• Wherever possible you need to find independent confirmation of the facts. One person or test does not a hypothesis prove!
• Encourage and engage in debate on the evidence by knowledgeable proponents of all points of view.
• Don’t fall for arguments from authority alone; I may have a PhD in Astrophysics but that doesn’t mean there IS a teapot.
• Be prepared to try multiple hypotheses.
• Try not to get overly attached to a hypothesis just because it’s yours, THis is harder than you might think.
• Measure, measure, measure. Objective numbers always trump personal beliefs, no matter how many folks share that belief.
• If there is a chain of argument every link in the chain must work.
• Sharpen up Occam’s Razor[3] – if there are two hypothesis that could explain the data equally well, choose the simpler.
• Check to see if the hypothesis can, at least in principle, be falsified: Is it testable? If it isn’t testable, it isn’t science![4]
• Can other people replicate the experiment and get the same result?
• Conduct control experiments, especially “double-blind” experiments where the person taking measurements is not aware of the test and control subjects.
• Check for confounding factors; make sure you separate as many of the variables as you can.

This is why evidence-based practice is so important. It separates the speculative from the scientific. The current rush to buy iDevices as a blanket solution for those individuals who need an AAC device is a good example of where hypotheses precede evidence. When someone turns up at the clinic doors with a kid, an iPad, and a recommendation from a video on YouTube that “this is the answer,” what do you say? There are many purported “evidential” video clips on the Internet that are well-meaninged attempts by parents to show how their kids have “improved” by using technology, but with no pre-testing and no measure of what “improvement” is, it’s impossible to call this evidence.

In their desire to help people with communication problems, it’s sometimes easier to believe in orbiting teapots than measure performance.

Notes
[1] The word hypothesis comes directly from the Greek ὑπόθεσις and means “placing under.” ὑπό is “under” and you see this in words such as hypodermic (under the skin), hypothalamus (under the thalamus), and hypochondria (under the breast-bone). The θέσις part orignal referred to the action of placing a foot or hand down to beat time in poetry or music, and it became, by extension, the putting down of a proposition or idea.

[2] Popper, K.R. (1935) Logik der Forschung (The Logic of Research) , Vienna: Springer; trans. The Logic of Scientific Discovery, London: Hutchinson, 1959.

[3] Popper, K.R. (1972) Objective Knowledge, Oxford: Clarendon Press. If you just want to focus on just one chapter, try Chapter 6: Of Clouds and Clocks, which can be read somewhat independently of the book as a whole, and is less dense than some of the earlier chapters. Popper isn’t the easiest of folks to read and in truth, I still have a hard time with much of his stuff on probability because of the math and logic involved, but he’s well worth the effort.

[4] “Pluralitas non est ponenda sine neccesitate” or “plurality should not be posited without necessity.” This is attributed to William of Ockam (1285-1349), an English Franciscan Monk and philosopher, who used this premise in much of his work and thinking, although the notion was actually a common principle in medieval thought. The actual phrase, Occam’s Razor, appeared first in 1852 and was used by the astronomer and physicist, William Rowen Hamilton. No mention of his looking for a teapot…

[5] The difference between Science and Pseudoscience often comes down to this rule of Testability. An idea that is inherently untestable is called metaphysical or speculation. You may well believe passionately that there are fairies at the bottom of your garden but unless you can subject them to testing, they are no more real than my orbiting teapot.