A “reanalysis” of the Price study on autism and thimerosal goes very wrong

25 Sep

A few years ago a study came out looking intently at the question of whether thimerosal in vaccines increases autism risk. That study, Prenatal and Infant Exposure to Thimerosal From Vaccines and Immunoglobulins and Risk of Autism, approached the question from many angles, including those suggested by parent advocates who believed (and still believe) that thimerosal was a driving force in the increased prevalence of identified autism in the U.S.. There were so many questions asked and so many results that the paper did not include all of those results. They just wouldn’t fit in a published paper, for one thing. So the authors put out additional reports including those extra tests.

One question they asked was whether the combination of exposure to thimerosal before birth and after birth would increase autism risk. This was an “interaction” model–how do these factors interact to create any possible autism risk. In their report the authors included not only the final result, but the sub-results. The pieces that they combined to calculate the final analysis.

Below is an example of one of those analyses. These are data involved in calculating how pre-natal thimerosal exposure combined with post-natal exposure up to 7 months of age might interact to create autism risk. The authors were kind enough to put boxes around statistically significant results–blue for results which indicate decreased risk and red for increased risk.

Abt-Autism-873

Now, let’s say I took the exposure by 7 months point and said, “Hey, look! Thimerosal exposure in the first seven months of life reduces autism risk by a factor of 4.28!” Well, that would be dishonest wouldn’t it? Taking a single sub result in an interaction model isn’t valid.

If I chose a different model from the same report I could quote a 1/OR (inverse odds ratio) of 113.59. A huge “protective” effect. Very dramatic. 113 times lower risk! But just as with the previous example, if I cherry picked this data point my own community would call me out for being dishonest. And they’d be right to do so.

I’m going to repeat that for emphasis because it isn’t obvious but it is very important: taking a single sub result in an interaction model is not valid. These sub results are only valid when incorporated into the full model and calculation performed.

Here’s something else that would be incorrect, and is more obvious: reporting only the results one wants to report. Ignoring the other results–especially the final result. In this case the final result is that the interaction of prenatal exposure and post natal exposure by 7 months does not increase autism risk. If I pulled out the sub result and said, “hey post natal thimerosal is protective” that would be misleading. If I ignored that red box you see with an odds ratio of 8.73, indicating a higher risk with thimerosal, that would be misleading.

Incorrect. Dishonest. Invalid. Take your pick of terms. It’s wrong.

So, what if I told you someone sifted through the report and cherry picked the results in the red boxes and was going around telling people “prenatal exposure to thimerosal increases autism risk by as much as a factor of 8.73. Would that be honest in your opinion?

Remember Brian Hooker?

If you don’t, he’s the guy who decided to publicize his paper with a race-baiting YouTube video claiming that a scientific disagreement among CDC researchers amounts to a new Tuskegee experiment? That scientific disagreement being whether to discuss one preliminary result or leave it out of a paper?

For some time now Mr. Hooker has been giving talks about how the data from the Price study shows an increased risk for autism and regression with thimerosal exposure. He doesn’t show exactly where he got the numbers he reports. But the number he likes to quote, 8.73, pretty much only appears in the figure above. He’s reporting on a sub result in an interaction model, which is not valid. He’s not reporting on the other sub results, nor the final result of the interaction model.

You can see him do this in his recent talk which has been posted to YouTube:

Let’s go into this in more detail, for those who would like to see that. His talk on this study starts at about 32 min in to the video. Mr. Hooker claims that he got the background reports from a “congressional request”. That’s around minute 37 in the talk. Sounds really impressive. Perhaps Mr. Hooker did get them through a congressional request. Which would beg the question of why he didn’t just download them from the web. The reports have been freely available since the Price Study was published. If one reads the Price study, as one would expect Mr. Hooker has done, references 10 and 11 are the reports, complete with url. These are online (here and here). Minor point I know, but it goes to show the amount of spin coming from Mr. Hooker and his team.

A few other points from his discussion of the Price study:

Mr. Hooker calls the Price study “fatally flawed”. He goes into great detail about why he feels this study is not valid. It’s a lot of hand waiving and his arguments are not valid. But it’s also problematical to spend a great deal of time tearing down a study to then rely on it for your own argument.

You see after he goes on and on about how he feels this study is not statistically valid, Mr. Hooker then appears to forget why the study is not, in his view, statistically valid and goes for the least statistically strong part of the study–that involving the smallest sub group.

He calls his discussion of the Price study a “reanalysis”. That’s not the word I use for skimming through publicly available reports and pulling out results that the authors put red boxes around. Again, it’s just spin.

At 36 minutes in you can see Brian Hooker claiming that the refusal rate was too high for the study to be valid (again, somehow that argument magically goes away for his “reanalysis”) and cheering and telling his audience to refuse to participate in CDC studies.

This is not the act of someone who wants to know the truth, of a true advocate, in my opinion. This looks to me like someone showing a strong bias towards the CDC. This advice goes counter to what I and my family need.

As an aside: if you watch Mr. Hooker’s full talk (don’t worry, it isn’t the full 8 hours of the video, just about an hour) you will see that it’s basically three parts. Part 1 is about his MMR paper with lots of references to the “whistleblower” (what whistle was blown again?). Part 2 is the Price study “reanalysis” (since when is reading someone else’s report a “reanalysis”). Part 3 is a discussion of tics with references to the “whistleblower” again.

Notice the pattern–there’s no discussion of the “whistleblower” in the discussion of the Price study. The area that is the most important to Mr. Hooker, thimerosal and autism, and no input from the “whistleblower”? Take a look at the Price study and answer this question: who is the second author on that study? That’s right, the “whistleblower”.

As I recall, Mr. Hooker claims to have been in contact with his “whistleblower” for about 10 months. It strains credulity to think that in that time Mr. Hooker never asked about the Price study. It further strains credulity to think that if Mr. Hooker had anything from the “whistleblower” to attack the Price study, he would hold it back.

Which is to say: I think he got nothing from his “whistleblower” on the Price study.

There’s a lot more wrong with Mr. Hooker’s “reanalysis”. But I’ll bring this back to this:

Abt-Autism-873

Yes, I admit it’s a bit obscure to know that you don’t pull a sub result out of an interaction model. It’s not so obscure that Mr. Hooker should be excused for doing it. But reporting the one sub result in the red box and ignoring 8 blue boxes? That’s not obscure. That’s just wrong.


By Matt Carey

Advertisements

14 Responses to “A “reanalysis” of the Price study on autism and thimerosal goes very wrong”

  1. reissd September 25, 2014 at 19:35 #

    If the only way anti-vaccine activists can make their case is by misrepresenting the studies, it speaks even further to the weakness of that case.

  2. Science Mom September 25, 2014 at 19:51 #

    Hooker has demonstrated how woefully incompetent he is at statistics and epidemiology in general but his faithful followers don’t care. He has a PhD (even if in an irrelevant field) and is saying what they want to hear and have even less of an understanding. He’s nothing more than a dishonest crank getting his ego off on credulous, desperate people.

    • brian September 25, 2014 at 23:17 #

      But, but, the results are significant!

      http://xkcd.com/882/

      • Sullivan (Matt Carey) September 25, 2014 at 23:24 #

        If you watch the video, you can see Mr. Hooker tell you about how he used a more conservative p value for some of his “analyses”. Yes, by moving to the more conservative p value of 0.1 instead of 0.05, he found more associations.

        For those not familiar with this math–Mr. Hooker loosened the criteria for something to be considered ‘statistically significant’. Instead of a 1 in 20 chance of a false hit, he makes it a 1 in 10. That’s not more conservative at all. More conservative is when you make it harder to have an association.

      • Chris September 26, 2014 at 00:48 #

        “Yes, by moving to the more conservative p value of 0.1 instead of 0.05, he found more associations.”

        Washington State University should rescind his engineering PhD for that horrendous gaff.

  3. Darwy September 25, 2014 at 20:49 #

    But if he didn’t cherry pick, he wouldn’t know what to do with the results.

  4. Chris September 25, 2014 at 22:53 #

    Is that really an almost eight hour long youtube video? Sometimes I wish they had kept to the ten minute maximum for uploaded videos.

    • Sullivan (Matt Carey) September 25, 2014 at 23:16 #

      I can’t say as I’ve watched more than Mr. Hooker’s talk. Sitting through an hour of Patrick “Tim” Bolen is not on my bucket list.

      • Sullivan (Matt Carey) September 25, 2014 at 23:17 #

        Which isn’t to say that listening to Mr. Hooker abuse statistics is on my bucket list.

      • Chris September 26, 2014 at 00:01 #

        It is hard to say which is a worse video. The almost eight hours of the one above or this ten hours worth of a screaming soda rain loop.

      • Darwy September 26, 2014 at 07:18 #

        I’ll watch 10 hours of Trolololo or, “They’re taking the Hobbits to Isengard” before I watch Hooker’s video.

  5. Lawrence October 4, 2014 at 18:50 #

    Hooker’s study is officially “Retracted.”

    http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4128611/

    • Sullivan (Matt Carey) October 5, 2014 at 00:38 #

      Thanks! Will get something up on that soon.

Trackbacks/Pingbacks

  1. Efectos adversos de la vacunas (ej. MMR y papiloma) asociados al timerosal: VERY WRONG (muy equivocados) | Chaos Theory and Human Pharmacology - September 25, 2014

    […] lbrb : A “reanalysis” of the Price study on autism and thimerosal goes very wrong. […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: