Bad Science Part 1: Randomisation

I’ve finally finished ‘Bad Science’ by Ben Goldacre and I intend to give this a bit more than just a review. I intend to take certain bits from it and blog them in order to explain why the scientific concepts you hear expounded about on this blog and others are important – or more accurately, why the lack of them is important. But before I do, here’s a (now traditional) short review of Bad Science by Ben Goldacre: Holy shit this book is good, go buy it (Amazon UK, US, unavailable in Canada).

It is _not_ an autism book, although autism is discussed via the chapter on MMR. However, it will give those who wish to discuss science with some degree of confidence a primer in the basics. First thing I want to talk about is the concept of Randomisation:

Here’s Ben discussing the concept in relation to a Homeopathy study:

Randomisation is another basic concept in clinical trials. We randomly assign patients to the placebo sugar pill group or the homeopathy sugar pill group, because otherwise there is a risk that the doctor or homeopath – consciously or unconsciously – will put patients who they think might do well into the homeopathy group and the no-hopers into the placebo group, thus rigging the results.

(page 48)

The issue with randomisation is not that it is never done – Ben mentions that it is rare to find a study so bad that randomisation is not done _at all_ – but that it is done poorly. Poor examples of ‘randomisation’ (e.g. methods that aren’t really random) include: selecting every other patient to a group depending on which order they sign up to the study i.e. Patient 1 goes into the control group, patient 2 into the non-control group, patient 3 into the control group etc etc, last digit of date of birth, date seen in clinic etc. Ben says that some studies claim to randomise patients by tossing a coin!

All these are open to conscious or unconscious manipulation by the researcher signing people up and are thus not trustworthy.

Ben describes the state of the art randomisation method as being:

1) Sign the patient up to the study
2) Make them call a special phone number
3) Someone at the other end answers the call
4) This person uses a computerised randomisation programme.

This is good because the actual research team are pretty much removed from the physical process of randomisation.

Ben goes on to ask: does randomisation matter?

Turns out it really does:

….people have studied the effect of randomisation in large numbers of trial and found that the ones with dodgy methods of randomisation overestimate treatments effects by 41%.

(Page 50)

Worse that that are the (many it seems) studies where the paper doesn’t tell you _what_ methods of randomisation was used, usually a clear red flag that the study was poorly designed and executed.

…trials with unclear methods of randomisation overstate treatment effects by 30%…..

(Page 50)

This is fascinating stuff from Ben and I promise there’s more to come. I hope this mini-series of articles from Ben’s excellent book will give you the impetus to keep reading when discussions evolve into exchanges of a technical nature. Of course, it is no substitute for buying the book itself, something I urge you to do post haste.