A “reanalysis” of the Price study on autism and thimerosal goes very wrong

A few years ago a study came out looking intently at the question of whether thimerosal in vaccines increases autism risk. That study, Prenatal and Infant Exposure to Thimerosal From Vaccines and Immunoglobulins and Risk of Autism, approached the question from many angles, including those suggested by parent advocates who believed (and still believe) that thimerosal was a driving force in the increased prevalence of identified autism in the U.S.. There were so many questions asked and so many results that the paper did not include all of those results. They just wouldn’t fit in a published paper, for one thing. So the authors put out additional reports including those extra tests.

One question they asked was whether the combination of exposure to thimerosal before birth and after birth would increase autism risk. This was an “interaction” model–how do these factors interact to create any possible autism risk. In their report the authors included not only the final result, but the sub-results. The pieces that they combined to calculate the final analysis.

Below is an example of one of those analyses. These are data involved in calculating how pre-natal thimerosal exposure combined with post-natal exposure up to 7 months of age might interact to create autism risk. The authors were kind enough to put boxes around statistically significant results–blue for results which indicate decreased risk and red for increased risk.

Now, let’s say I took the exposure by 7 months point and said, “Hey, look! Thimerosal exposure in the first seven months of life reduces autism risk by a factor of 4.28!” Well, that would be dishonest wouldn’t it? Taking a single sub result in an interaction model isn’t valid.

If I chose a different model from the same report I could quote a 1/OR (inverse odds ratio) of 113.59. A huge “protective” effect. Very dramatic. 113 times lower risk! But just as with the previous example, if I cherry picked this data point my own community would call me out for being dishonest. And they’d be right to do so.

I’m going to repeat that for emphasis because it isn’t obvious but it is very important: taking a single sub result in an interaction model is not valid. These sub results are only valid when incorporated into the full model and calculation performed.

Here’s something else that would be incorrect, and is more obvious: reporting only the results one wants to report. Ignoring the other results–especially the final result. In this case the final result is that the interaction of prenatal exposure and post natal exposure by 7 months does not increase autism risk. If I pulled out the sub result and said, “hey post natal thimerosal is protective” that would be misleading. If I ignored that red box you see with an odds ratio of 8.73, indicating a higher risk with thimerosal, that would be misleading.

Incorrect. Dishonest. Invalid. Take your pick of terms. It’s wrong.

So, what if I told you someone sifted through the report and cherry picked the results in the red boxes and was going around telling people “prenatal exposure to thimerosal increases autism risk by as much as a factor of 8.73. Would that be honest in your opinion?

Remember Brian Hooker?

If you don’t, he’s the guy who decided to publicize his paper with a race-baiting YouTube video claiming that a scientific disagreement among CDC researchers amounts to a new Tuskegee experiment? That scientific disagreement being whether to discuss one preliminary result or leave it out of a paper?

For some time now Mr. Hooker has been giving talks about how the data from the Price study shows an increased risk for autism and regression with thimerosal exposure. He doesn’t show exactly where he got the numbers he reports. But the number he likes to quote, 8.73, pretty much only appears in the figure above. He’s reporting on a sub result in an interaction model, which is not valid. He’s not reporting on the other sub results, nor the final result of the interaction model.

You can see him do this in his recent talk which has been posted to YouTube:

Let’s go into this in more detail, for those who would like to see that. His talk on this study starts at about 32 min in to the video. Mr. Hooker claims that he got the background reports from a “congressional request”. That’s around minute 37 in the talk. Sounds really impressive. Perhaps Mr. Hooker did get them through a congressional request. Which would beg the question of why he didn’t just download them from the web. The reports have been freely available since the Price Study was published. If one reads the Price study, as one would expect Mr. Hooker has done, references 10 and 11 are the reports, complete with url. These are online (here and here). Minor point I know, but it goes to show the amount of spin coming from Mr. Hooker and his team.

A few other points from his discussion of the Price study:

Mr. Hooker calls the Price study “fatally flawed”. He goes into great detail about why he feels this study is not valid. It’s a lot of hand waiving and his arguments are not valid. But it’s also problematical to spend a great deal of time tearing down a study to then rely on it for your own argument.

You see after he goes on and on about how he feels this study is not statistically valid, Mr. Hooker then appears to forget why the study is not, in his view, statistically valid and goes for the least statistically strong part of the study–that involving the smallest sub group.

He calls his discussion of the Price study a “reanalysis”. That’s not the word I use for skimming through publicly available reports and pulling out results that the authors put red boxes around. Again, it’s just spin.

At 36 minutes in you can see Brian Hooker claiming that the refusal rate was too high for the study to be valid (again, somehow that argument magically goes away for his “reanalysis”) and cheering and telling his audience to refuse to participate in CDC studies.

This is not the act of someone who wants to know the truth, of a true advocate, in my opinion. This looks to me like someone showing a strong bias towards the CDC. This advice goes counter to what I and my family need.

As an aside: if you watch Mr. Hooker’s full talk (don’t worry, it isn’t the full 8 hours of the video, just about an hour) you will see that it’s basically three parts. Part 1 is about his MMR paper with lots of references to the “whistleblower” (what whistle was blown again?). Part 2 is the Price study “reanalysis” (since when is reading someone else’s report a “reanalysis”). Part 3 is a discussion of tics with references to the “whistleblower” again.

Notice the pattern–there’s no discussion of the “whistleblower” in the discussion of the Price study. The area that is the most important to Mr. Hooker, thimerosal and autism, and no input from the “whistleblower”? Take a look at the Price study and answer this question: who is the second author on that study? That’s right, the “whistleblower”.

As I recall, Mr. Hooker claims to have been in contact with his “whistleblower” for about 10 months. It strains credulity to think that in that time Mr. Hooker never asked about the Price study. It further strains credulity to think that if Mr. Hooker had anything from the “whistleblower” to attack the Price study, he would hold it back.

Which is to say: I think he got nothing from his “whistleblower” on the Price study.

There’s a lot more wrong with Mr. Hooker’s “reanalysis”. But I’ll bring this back to this:

Yes, I admit it’s a bit obscure to know that you don’t pull a sub result out of an interaction model. It’s not so obscure that Mr. Hooker should be excused for doing it. But reporting the one sub result in the red box and ignoring 8 blue boxes? That’s not obscure. That’s just wrong.

—
By Matt Carey