The Art and Science of Crap Detection

One of the ironies of modern culture is that, despite the remarkable advances of science and medicine, many people continue to believe utter nonsense.

The sea of dubious information has clearly been fed by the emergence of social media.  Anyone who uses Twitter or Facebook will be confronted with a continual stream of bold claims, shocking controversies, and supposed secrets for improved health, wealth and happiness.  Some of these claims come from genuine experts, some come from people trying to make a fast buck, and some come from fools.

Telling the difference can be frustratingly difficult.

Two examples.  I've just been reading a report in a newspaper about advances in cancer treatment. The same issue also includes an article about the powers of Chinese "internal healing".  Another source, this time an online magazine, juxtaposes a piece about recent advances in brain science with a quiz inviting readers to find out whether they are "left brain" or "right brain" thinkers.

In case you are not clear yourself (and who could blame you?), there is no compelling evidence in favour of either internal healing or left/right brain thinking.  They might be attractive ideas. They might conceivably turn out to be true at some point in the future. But, in the words of the late, great Carl Sagan, "extraordinary claims require extraordinary evidence".  And, at the moment, the evidence is missing.

The genius of social media is that it has democratised knowledge. Information is no longer the preserve of a select few.  We can all access vast amounts of information.  But democracy cannot work without an educated population.  It assumes that people have sufficient knowledge, skills, and understanding to make informed judgements about the decisions confronting them.

The simple fact is that the vast majority of people have not been educated to make such judgements.

The writer Ernest Hemingway famously said, "The most essential gift for a good writer is a built-in, shockproof, shit detector."  That inspired Neil Postman to claim:

As I see it, the best things schools can do for kids is to help them learn how to distinguish useful talk from bullshit.

Most schools do not do anything that remotely resembles this. Consequently, they send young people out into the world inherently vulnerable to bullshit.  And that seems rather shameful.

Of course, it is not just schools who have failed to prepare their charges.  Many universities force their students to complete 'learning styles' assessments, despite the fact that it is well-known that most experts on learning and the brain deny that such styles exist at all.  Even worse, an informal review of sports coaching education programmes in the UK by my colleagues and me revealed that all of them taught questionable or erroneous theories as scientific fact.

Remember Power Balance?  The bracelets that used "hologram technology" to "resonate with and respond to the natural energy field of the body", resulting in the improvement of sporting performance.  Sounds implausible?  Not so much that it prevented tens of thousands of sports coaches, athletes, and normal people from from around the world buying one, just in case they actually worked.

People bought the devices even after scientists demonstrated that they had no effect at all.  Television news items even demonstrated, with the most basic of tests, that the bracelets were useless.  But people continued to buy them, right up until the company that produced them went out of business, mainly due to a court ruling that the company should stop making unsubstantiated claims and that dissatisfied customers should be given a full refund.

Anyway, the company has relaunched under a new name, and people have started buying the bracelets again.

What is to be done?

Surely, the ultimate solution must be to change education programmes to better prepare people to do with the barrage of bullshit with which they will inevitably be confronted during their lives. This is as much to do with a sceptical mindset as specific techniques.

For a start, though, consider this list of questions from the writer Michael Shermer, which make up his "Baloney Detection Kit".

THE TEN QUESTIONS

1. How reliable is the source of the claim?
2. Does the source make similar claims?
3. Have the claims been verified by somebody else?
4. Does this fit with the way the world works?
5. Has anyone tried to disprove the claim?
6. Where does the preponderance of evidence point?
7. Is the claimant playing by the rules of science?
8. Is the claimant providing positive evidence?
9. Does the new theory account for as many phenomena as the old theory?
10. Are personal beliefs driving the claim?

Crap detection can be framed even easier than this, as one simple but profound question:

How do you know?

If every student was taught to ask this whenever confronted by a new idea - no matter who offered it - they would be making a giant step towards being genuinely educated.