Pharmacologic Treatment of Repetitive Behaviors in Autism Spectrum Disorders: Evidence of Publication Bias.

27 Apr

I had hoped to read this article and have a good review in place when the embargo lifted. Time is working against me and I just haven’t had the resources to apply to this. But I shouldn’t let that stop me from highlighting this study: Pharmacologic Treatment of Repetitive Behaviors in Autism Spectrum Disorders: Evidence of Publication Bias. The full paper is available free on the Pediatrics website.

Selective serotonin reuptake inhibitors (SSRIs) are a class of drugs used most often to treat depression. They are also used off-label for treating anxiety and, also, autism symptoms. A study carried out in 2009 showed that one drug frequently prescribed to autistics, Citalopram, was no different than placebo in reducing repetitive behaviors (Lack of efficacy of citalopram in children with autism spectrum disorders and high levels of repetitive behavior: citalopram ineffective in children with autism.) The study results came as a bit of a surprise to many and was a minor news story for that week.

Publication bias “is the tendency of researchers, editors, and pharmaceutical companies to handle the reporting of experimental results that are positive (i.e. showing a significant finding) differently from results that are negative (i.e. supporting the null hypothesis) or inconclusive, leading to bias in the overall published literature.”

A recent study looked at previous publications (and one unpublished study) and claims that there is publication bias involved in reporting on SSRI’s and repetitive behaviors in autism:

The goal of this study was to examine the efficacy of serotonin receptor inhibitors (SRIs) for the treatment of repetitive behaviors in autism spectrum disorders (ASD).
Two reviewers searched PubMed and for randomized, double-blind, placebo-controlled trials evaluating the efficacy of SRIs for repetitive behaviors in ASD. Our primary outcome was mean improvement in ratings scales of repetitive behavior. Publication bias was assessed by using a funnel plot, the Egger’s test, and a meta-regression of sample size and effect size.
Our search identified 5 published and 5 unpublished but completed trials eligible for meta-analysis. Meta-analysis of 5 published and 1 unpublished trial (which provided data) demonstrated a small but significant effect of SRI for the treatment of repetitive behaviors in ASD (standardized mean difference: 0.22 [95% confidence interval: 0.07-0.37], z score = 2.87, P < .005). There was significant evidence of publication bias in all analyses. When Duval and Tweedie's trim and fill method was used to adjust for the effect of publication bias, there was no longer a significant benefit of SRI for the treatment of repetitive behaviors in ASD (standardized mean difference: 0.12 [95% confidence interval: -0.02 to 0.27]). Secondary analyses demonstrated no significant effect of type of medication, patient age, method of analysis, trial design, or trial duration on reported SRI efficacy.
Meta-analysis of the published literature suggests a small but significant effect of SRI in the treatment of repetitive behaviors in ASD. This effect may be attributable to selective publication of trial results. Without timely, transparent, and complete disclosure of trial results, it remains difficult to determine the efficacy of available medications.

10 Responses to “Pharmacologic Treatment of Repetitive Behaviors in Autism Spectrum Disorders: Evidence of Publication Bias.”

  1. passionlessDrone April 27, 2012 at 13:59 #

    Hello friends –

    Totally nasty observation that doesn’t have a lot of good options. I ran into a discussion somewhere recently (can’t remember where, I thought In the Pipeline, but can’t find it now!) about a similar topic, the problem of funding and/or academic position depending on publication in ‘high impact’ journals as measured by number of papers, as opposed to the quality of the paper. One upshot of this, in some cases were instances of looking for stuff, doing enough optimistic filtering, finding a result, and then writing your paper on that.

    Anyway, one of the suggestions I saw offered was to treat research papers more like clinical trials; you have to state up front what you are looking for. It would be somewhat upside down of the existing model, you’d have to tell the journals you’d like to submit to before hand what your expectations of the research are. The idea was that if you found weird stuff that you weren’t expecting, you could discuss that in the paper, but it would be apparent that this wasn’t the focus of the initial research.

    Secondly, I think that this kind of thing speaks towards another potential problem; the natural drive for entities capable of deriving profits from a treatment for autism driving research. Put another way, an argument I used to make was that the barriers of entry of a well designed, gold standard study are so significant there is a tendency to not study a common, cheap, unpatentable supplement. That’s bad for everyone, we don’t learn about things that might help, and we also don’t learn about other stuff said supplementation might be doing; and right or wrong, there is a lot of supplementation going on out there.


    @Sullivan – Thanks for posting this. Very nice.

    – pD

  2. Neuroskeptic April 27, 2012 at 15:08 #

    Hey passionlessDrone: the idea of pre-registering clinical trials is one I’ve been pushing recently. You might have been reading something I wrote but maybe not because I’m not the only one. Anyway here’s my thoughts on it:

    Regarding the paper, it’s a great example of why registration of clinical trials is so important. It’s the only way anyone knows about the 5 unpublished studies. Of the 5, 2 of them are known to have been negative – see Michelle Dawson’s comment on my post

    The other 3 presumably were. They’d almost certainly have been published, were they positive.

    • Sullivan April 27, 2012 at 20:15 #

      From a shear monetary/taxpayer point of view, federally funded research needs to be published as we’ve paid for the information.

      From a human perspective, subjects have given themselves for the research effort. It doesn’t honor their contribution to withhold the results from the public.

  3. passionlessDrone April 27, 2012 at 20:48 #

    @Neuroskeptic: It was your site that I was reading! Nicely done and good ideas thrown around.

    @Anyone: Was this a case of publication bias, or perhaps, ‘lack of submittal for publication’ bias; i.e., the study was done, results determined, and the paper was not subsequently submitted to a journal because the findings were potentially inconvenient?

    – pD

  4. Julie May 11, 2012 at 10:44 #

    I’m real, I’m a mother who is curing her son with homeopathy. Who have been doing CEASE therapy and seeing amazing results.

    I will give you 2 blogs of other mothers who are reporting every detail of their kids’ treatment:

    Real results, not blabla from people who don’t have kids with autism. Not useless dicussions of what researches were done properly or not. Not official palaver supposed to hide the truth because the Big Pharma spent 40 million euros in lobbying in Europe only.

    This is REAL life, sir.

    By the way, Dr. Wakefield gave an interview and explains everything and I’m absolutely sure of his innocence, like many people are.…..words.html

    If you really have kids with autism, you should try homeopathy because you could risk curing them 100% and having to admit that if you think homeopathy is a joke, your opinion is WRONG.
    Another mother who cured her son with classical homeopathy:

    • Sullivan May 11, 2012 at 17:47 #

      “If you really have kids with autism, you should try homeopathy….”

      We all do. As far as I know, pretty much every autistic child drinks water every day.

      “homeopathy is a joke, your opinion is WRONG.”

      I don’t think homeopathy is a joke. Charging people a lot of money for water is no joke.

  5. Sullivan May 14, 2012 at 22:22 #

    By the way, a good example of possible publication bias is in the Wake Forrest “replication” of the Uhlman study (the study by the Wakefield group that supposedly found vaccine strain measles virus in tissues taken from autistic kids). This study was presented at IMFAR to much publicity.

    The Wake Forrest study was presented in 2006 but has never been published. It does pop up periodically online, with much trumpeting of a “new” study replicating the Uhlmann results.

    Why hasn’t it been published? There are certainly journals that would take it. The lead researcher is now an editor for “autism insights”, an online journal which has been very favorable to Mr. Wakefield.

  6. stanley seigler May 15, 2012 at 01:34 #

    @sullivan: “Charging people a lot of money for water is no joke.”

    wish other ‘no jokes’ were as obvious…especially those veiled in ‘scientific proof’…

  7. passionlessDrone May 15, 2012 at 15:35 #

    @Sullivan –

    I engaged in a real live tequila shootout with Walker one night about a year ago. Regarding the measles study, he told me that the IRB was pulled in mid stream when the Wakefield thing really blew up and would not be resurrected. He also claimed that they had a ton of tissue banked and were going to be doing genetic expression analysis on it. I have no idea if he was full of rocks or whatever; I haven’t seen anything on genetic expression with his name on it though. (?)

    – pD

    • Sullivan May 15, 2012 at 19:29 #


      thanks for that. Looking at the 2006 abstract, I see:

      Walker, Karin Hepner, Jeffrey Segal, Arthur Krigsman,
      Wake Forest University School of Medicine

      The samples were almost certainly taken by Dr. Krigsman. His problems with IRB go back well before this 2006 study.

      My concern here is that Dr. Krigsman went ahead with this work in defiance of his IRB.

      I would think that the publicity surrounding Mr. Wakefield wouldn’t be in the correct time frame to scuttle this work.

      Brian Deer disclosed his findings regarding Mr. Wakefield’s efforts at the Royal Free in 2004, 2 years before the Wake Forest study was presented at IMFAR.

      I don’t doubt that they have a lot of tissue banked. The question is whether they can ethically use it. Krigsman recently did an end run around his old hospital by getting a private IRB to approve the work. That struck me as highly unethical.

Leave a Reply to stanley seigler Cancel reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: