Field of Science

A truly dreadful study into the effects of prayer

In the middle of 2009, a small group of religious scholars and doctors lead by Candice Brown, Associate Professor of Religious Studies at Indiana University, travelled to Mozambique to find out if charismatic healing really works. So they went to some events laid on by Evangelical missions, and ran some tests on the recipients.

Unfortunately, what resulted was one of the shoddiest studies I've ever read!

Basically, the set up was this. The preacher asks if there's anyone who has hearing or sight problems. The researchers measure their hearing and vision, then the patient gets a bit of laying on of hands, then the researchers re-run the hearing and vision test.

Well, the good news is that every single one of the 20-odd recipients was healed. The bad news is that they broke just about every rule in the book for conducting medical trials. Really, it's a case study in how not to do these things.

First off, there was no control group. That means there's nothing to compare it with. So there's no way of knowing whether it was the healing, or something else that did the magic.

They measured hearing using an audiometer, which fires a a 'beep' at different volumes and frequencies into the ear. Sight was measured using a sight chart. Now, these are fine for use in diagnosis, but it's useless for these sorts of studies because it's entirely subjective.

And that, of course, means that it's heavily susceptible to the placebo effect (and remember, there was no placebo control group). We know that the placebo effect is stronger the higher the level of drama and expectation you generate (think: "laying on of hands"!). And we know that remote prayer doesn't work - probably because there's no opportunity to harness the placebo effect. 

It gets worse than that. They were measuring hearing in a noisy environment (a big no-no). They were also measuring hearing on people who have no experience of such a hearing test. It's well known that people get better at these with practice.

Well, all that is bad enough to disregard this study. But if that was all that was wrong with it, I'd just shrug my shoulders and say "meh".

But the most egregious flaw in their study is something rather more interesting, and that's something called the demand effect. That's where subjects consciously or unconsciously respond in a way that they think is going to please the experimenter.

Cast your mind back to the set-up. They asked people to come forward if they were hard-of hearing. Then they tell them that they are going to test them, to see if they really are deaf. They fire a 'beep' into there ear and ask "Did you hear that"?

Now, what is the subject going to do in a situation like this? He's already told the preacher that he's deaf, and he doesn't want to look a fool. He's in a noisy environment, so it's already quite hard to hear the beeps. He hears something faint... was it a beep? Nah, probably not. He's deaf, after all, and so can't hear very well!

Then the preacher comes along and does all the showmanship. They run the test again. Can they hear the beep this time? Well, he doesn't want to let the preacher down, so of course he tries his damnedest to hear the faintest of beeps.

And that simple trick explains why both hearing and sight appears to have dramatically improved among these poor, superstitious villagers. Both they and the researchers have conned themselves.

You might wonder why they didn't try doing the study in controlled conditions back home in the USA. Well, that's because prayer doesn't heal people in such environments. Seriously, that's what they say!

The remarkable thing is that all these flaws are actually acknowledged in the paper (all they don't seem to have got their heads round just what a headache the 'Demand Effect' is for this study).

They know about the flaws, but they choose to pretend they're not important. Just check out the press release, which doesn't mention a single one.

It's a useless study, then. Now, a lot of people set out to do useless studies, but mostly they don't get funded or published. So what happened here?

Well, it won't surprise you to learn that the Templeton Foundation paid for this pitiful charade of science. They got their money's worth, though, in terms of gullible press headlines.

But why on earth did it get published? The Southern Medical Journal is not exactly a world-class publication, but it does normally publish your more mundane mix of research and medical bits and pieces. They know the difference between a good study and a bad one.

Oh, and prayer. They publish a lot of stuff on prayer - 137 articles in the last 5 years alone.

If this study had been on anything other than prayer, it would never have been published. We can only guess at the reasons why this one got through.

Just to finish off, here's a great (and short) video on the placebo effect. Perhaps we should sit Candice down and make her watch it before she tries her hand at any more medical research.



ResearchBlogging.orgBrown CG, Mory SC, Williams R, & McClymond MJ (2010). Study of the therapeutic effects of proximal intercessory prayer (STEPP) on auditory and visual impairments in rural Mozambique. Southern medical journal, 103 (9), 864-9 PMID: 20686441

Creative Commons License This article by Tom Rees was first published on Epiphenom. It is licensed under Creative Commons.

8 comments:

  1. Note to self, never take Southern Medical Journal papers seriously ever again.

    ReplyDelete
  2. I remember hearing about this a while back. That's just terrible! They could have done an equality good quality study just watching a few episodes of Benny Hinn on TBN and coding the "results".

    ReplyDelete
  3. The newsy article appears in the Health News Common, which was distributed free across the general midwest for a couple of years, and almost every issue had a report of at least one dodgy idea or study. However, you cannot get any critical comments or commentary published, and I tried several times, even speaking to one of the junior editors, a very nervous person. "We're just trying to present both sides." "So the scientific side and the pseudoscience side?" I asked. "I don't think you understand, sir." And it was never clear who was paying for the publication, but whether it went all online, or I was just removed from the mailing list, it no longer comes in hard copy.

    ReplyDelete
  4. Horrible study yes - rediculous. But the Placebo Effect is valid and in many cases nearly equal to prescribed medications without the many damaging side effects. What is wrong with the mind healing the body afterall - we all agree the mind can make the body ill.

    ReplyDelete
  5. While this study wasn't conducted in a scientific way does anyone remember the study Art Bell did on his Coast to Coast radio program a couple of times? He would get large groups of people to concentrate on a particular event and then monitor the outcome. One such event was a rain that ended a severe drought in Texas and the south. He also did this with another incident. Art was so convinced of the success that he was afraid and humbled by the power he had at his command with his talk show. After all absolute power corrupts absolutely.

    ReplyDelete
  6. If you can't dazzle with brilliance, baffle them.

    ReplyDelete
  7. I think that the criticism of this research ignores some of the tantalising possibilities implicit in what was shown.

    1. If proximal prayer (PIP to those in the know) works, then what is the threshold distance? Does it follow the inverse square rule?

    2. What shielding placed between the prayer and the prayee would block the effect? 2 inches of lead? kryptonite mesh? Maybe a tank of water (and does holy water work better or worse than tap water)?

    But what I got from this is that there is a real possibility of glimpsing the rosy cheeks of Yahweh's butt - in a quantitative sense anyway. Two prayees reported hearing improvements of 50 decibels. When I plugged this into a decibel to watts calculator I got 100 kW (http://www.ringbell.co.uk/info/dbw.htm). If less than 1 minute of prayer can produce this much power we may be looking at a solution to the global warming problem (although I have no idea how much hot air is generated as a by-product). Using ohms law we could even work out how many volts are involved (assuming amperage is low enough not to kill the prayee). Could a plasma ball or similar be used to pick up the mojo rays in transit? Does excessive prayer create electrical storms and can it be used for weather control as many already claim?

    If we could work out the frequency could distal PIP be done remotely by radio? If prayer was digitised could we serve medical prayers over the internet and dump all those pesky doctors once and for all?

    Clearly we now have a speed for prayer - the speed of light and I suspect this could then be plugged into E=mc2 yielding all sorts of interesting data.

    Although the visual improvement varied amongst prayees we should be able to work out the refractive index of prayer. I don't understand optics well, but I wouldn't be surprised if we could even work out a wavelength. I suspect we may well find god if we check the gaps between the wavelengths. And we could even take a stab at his colour.

    Once we have colour we can perform spectroscopic analysis and determine god's atomic composition.

    So much for those atheist claims that god doesn't interact with the material world.

    I guess we have to thank The Templeton Foundation and the Indiana University Bloomington for making this all possible.

    There is a lot of solid science in this report. Shame on Epiphenom for belittling it!!! Bad Epiphenom.

    ReplyDelete

Markup Key:
- <b>bold</b> = bold
- <i>italic</i> = italic
- <a href="http://www.fieldofscience.com/">FoS</a> = FoS