As a few people have written about recently, a book was recently published in which we get to read transcripts of conversations between Brian Hooker (a vocal advocate of the idea that vaccines cause autism) and William Thompson (a researcher at the CDC).
The basic story we have been told is that supposedly the CDC team (of which Thompson was a member) saw a “statistically significant” association between MMR vaccination and autism in African American boys. Specifically for boys who got the vaccine late, but before age 3. The story then goes on to claim that he CDC team worked furiously to bury this association. William Thompson, wracked with guilt over this, finally reached out to Brian Hooker to guide him towards this result.
This story is full of holes, as has been discussed here and elsewhere. Rather than go through that again, just take this backstory and ask yourself: given this claim that significant results were omitted by the CDC, wouldn’t Brian Hooker make damned sure that nothing important was left out of the “reanalysis” paper? After all, the main criticism Hooker levied against the CDC team was omission.
You’d think so, wouldn’t you. But you’d also guess that I wouldn’t go through this long introduction if that was the case.
Brian Hooker sold his reanalysis as strong because he took a simple approach. I.e. he didn’t control for any confounding factors. Epidemiologists will tell you the opposite–a raw data analysis is suspect.
In the conversations, we hear that Brian Hooker was well aware that he would get criticism for not making corrections:
Dr. Hooker: Right. Right. Well, what would happen…Okay, so let me play this out … My paper gets published; my MMR paper gets published. They…I get heavily criticized, because I haven’t corrected for scioeconomic factors or maternal education. Yeah. I’ll take my hits, and then eventually it gets published. And then there’s a piece of information that I receive from the CDC, but I don’t source. Is that going to be a red flag?
OK, he knows that a “simple” approach is not considered strong. This begs the question: how important are socioeconomic factors and maternal education he’s worried about? I could tell you that they are huge factors for the conclusion he’s trying to make. But don’t take my word for it, because Hooker and Thompson did discuss the importance of maternal education in the recorded calls. And this discussion is very telling. Consider this exchange:
Dr. Hooker: Right. Right. Well, it does…You know, it does kind of play backwards.
Plays backwards–as in this goes against the story they are trying to tell. Going on:
Dr. Thompson: yeah.
Dr. Hooker:… in terms of: Will those kids that got vaccinated later; then you would expect less healthcare-seeking behavior; so they would be less likely to get an autism diagnosis.
Dr. Thompson: yep.
Yep?!? That’s all he can say? This is KEY to the entire argument that there is an increased risk in African American boys.
Let’s step back a moment and look at this issue. Brian Hooker’s reanalysis is based on very few subjects (seven–African American autistic boys who got their MMR vaccine late). Implicit in his “simple” analysis is the assumption that these boys were in every way similar to the other boys in the study, aside from autism and when they got their MMR vaccine. Let’s say that there’s a reason why these autistic kids were vaccinated late–like their parents didn’t have insurance so they didn’t go to well baby checks. This would correlate with the education of the mother and, thus, this variable could help control for this “health-care seeking behavior”.
This leaves us with a HUGE potentially confounding variable in Hooker’s analysis, and in precisely the population they are most interested in–the kids who got vaccinated later. But he don’t address it in the “reanalysis”. He sweeps it under the rug.
Which is to say, this is a very ironic conversation Hooker and Thompson are having. Hooker appears to be doing exactly what he claimes the CDC was doing: crafting a story around the results he wants to report and omitting important information to make that point.
The conversation goes on.
Dr. Hooker: But if you had maternal education, then you could correct for that.
Dr. Thompson: Right. But …
Dr. Hooker: So, I don’t think I have it.
Dr. Thompson: You could argue the most important confounder was not included in that study. So, why would you even consider the results valid?
Dr. Hooker:Valid.Right. Right, you can’t say either way.
Dr. Thompson: Exactly.
Dr. Hooker: Essentially.
Dr. Thompson: Yep.
Dr. Hooker: You can’t…
Let’s repeat one line for emphasis: You could argue the most important confounder was not included in that study. So, why would you even consider the results valid?
The most important confounder being maternal education. They know this is a big deal.
And this wasn’t just a passing thought in that conversation. Later in that same call, Hooker asked again about how strong an effect one could see from maternal education. In this case, from Thompson’s experience from his 2007 thimerosal paper. And again Thompson tells him that maternal education is a strong factor that should be included in an analysis.
Given this repeated discussion of the need to adjust your results for external variables, you’d expect Hooker to explore maternal education in his own paper, wouldn’t you? As in, do the math and see how much this factor affects his calculated relative risk for MMR in African American boys, right? At the very least, you’d expect a discussion similar to the one above–how this is a potential confounding variable and how exactly it could affect his results (it could go “backwards”).
You’d expect that of an ethical researcher. I would.
Here’s the thing: Brian Hooker did not include maternal education in his re-analysis. the closest he gets in in the final paragraph of his reanalysis paper. This is where he focuses on the weaknesses of his study. He goes into detail about age of diagnosis and how that might affect his results, for example. When it comes to maternal education the closest he says:
In addition, socioeconomic factors were not assessed in the current analysis. Thus, any differences in “healthcare seeking behavior” among individuals vaccinated ontime versus late could not be assessed.
Sorry to be so repetitive, but no word about how important maternal education or other socioeconomic factors are to precisely the conclusion he’s trying to draw. Just a statement that amounts to “controlling for socioeconomic factors can’t be done because I didn’t do it.” The data were available to Hooker in the birth certificates. Hooker even used the birth certificate data elsewhere in his paper (from the paper: “Georgia state birth certificate information was used to further obtain each child’s birthweight”). So he can’t say he didn’t have it.
This is ironic in the extreme: after complaining that the CDC left out a “statistically signficant” result, Brian Hooker doesn’t include a factor which he knows is a major without which one can’t consider the results valid.
I’ll go beyond ironic. This is in my opinion scientifically unethical. And, since his study is really more political than scientific, I consider this politically unethical. Frankly, if someone working for me did something like this as part of his work, he’d be fired.
But, wait, the story is even more strange. Recall how Thompson was bashing the original study (DeStefano, et al.) for being invalid due to not controlling for maternal education (“You could argue the most important confounder was not included in that study. So, why would you even consider the results valid?”). DeStefano et. al did control for maternal education. And not only should William Thompson and Brian Hooker have known this if they were reanalyzing the paper, William Thompson was a coauthor on the DeStefano paper!
One does have to wonder if Hooker did the analysis using maternal education and chose to not report it. Given the discussions he had with Thompson, my opinion is that either Hooker did the analysis or he’s incompetent.
Bottom line, though, if you are going to make some public relations/political hay claiming some group omitted a result, you shouldn’t be omitting important analyses yourself.
Were significant analyses omitted from Hooker’s paper? In my opinion, yes. And from his conversations with William Thompson, we see that Hooker was aware of the importance of factoring in maternal education.
—
By Matt Carey
Left Brain Right Brain 
Reblogged this on Nature in the City and commented:
Reblogging from Left Brain /Right Brain
An excellent breakdown of the unethical (and sloppy) “science” behind the “re-analysis” of the CDC study concerning vaccines and autism.
Great article (or blog post… whatever). I do have one small question:
In Hooker’s re-analysis, he do not mention how many cases there were in the sub-group of african-american boys. I’ve seen another blog estimate that there were around 13 cases (sample size is still tiny), but you wrote that there were 7 cases. I was wondering if you had a source for the number or is your number an estimation as well?
There were seven cases in the vaccinated-after-36 months group
My guess is that the 13 is for the number of autism cases in the vaccinated late but before 36 months group. I.e. vaccinated after 18 months but before 36 months.
An additional point–there is no correction for a cohort effect. The older kids–the ones he is using to compare against–will have a lower diagnosis rate. Of course, that means he has to accept awareness and expansion of diagnostic criteria. Not hard when the study data is from 1996 and the DSM IV had just gone into effect in, what 1994?
We know from multiple studies that kids were missed in the 1980’s. This includes kids with significant intellectual disability. Consider the 1980’s Utah/UCLA project and the followup performed recently. So, comparing kids born in 1993-96 to kids born in the 1980’s, like this dataset does, is very problematic.
Thank you for the answer. A quick search shows that the quote you provided was from the book “Vaccine Whistleblower” (source). I know that the exact number of cases in the study is a tiny detail, but I like knowing those details.
> Sullivan wrote:
Was this a problem (or source of error) in the original DeStefano study as well or was the problem introduced when Hooker changed the study method?
In my opinion, yes, this was a problem in the original DeStefano study. It would have reduced apparent MMR/autism associations.
The few statistically significant associations found in DeStefano et al. (the ones that Hooker denies were included) would have likely been eliminated had they been able to control for a cohort effect.
“Let’s step back a moment and look at this issue. Brian Hooker’s reanalysis is based on very few subjects (seven–African American autistic boys who got their MMR vaccine late).”
But 7 was the number of boys vaccinated *after* 36 months correct? The study found the association in black boys vaccinated *before* 36 months. Sorry I’m a bit confused…how is his reanalysis based on just seven boys then?
Seven cases vaccinated after 36 months, per Brian Hooker in conversation with Thompson.
In the paper, Hooker states that he had 624 cases overall, after all exclusions.
He’s comparing relative risk of MMR for kids vaccinated before 18 months, before 24 months and before 36 months. Compared as in compared to those vaccinated after 36 months. So his comparison–his denominator–is based on the seven cases in the over 36 month cohort. Any factor, systematic or random, that could result in a low count in the vaccinated after 36 month group would strongly bias his result.
Keep in mind–Hooker set an arbitrary cutoff of 5 cases in a cohort. Less than that and he ignored that cohort. Why 5? He pulls that number out of thin air. So, less than 5 he ignores but 7 is magically OK.
It is ironic, yet again, that Hooker arbitrarily throws out a cohort while claiming (even though he knew he was wrong) that the CDC arbitrarily threw out kids without birth certificates. He knew it was wrong because the protocol called for a birth certificate analysis and this predated any data crunching.
There’s a lot of dishonesty in this story. From Brian Hooker.
Hi I have a question! Do we know how many total African American boys there were in the vaccinated after 36 months group? The quote you posted says that there were seven African American autistic males but doesn’t mention how many there were total. Thanks!
I can’t help but note that Hooker, an engineer or some kind, I believe, was looking for this kind of detailed epidemiological/statistical guidance from a psychologist.
Wakefield has stated on a few occasions that Thompson isn’t a front line member of the CDC team. That he’s the “accountant” or some such.
My opinion–Hooker was mostly interested in something to substantiate a claim of fraud so he could try to invalidate the Omnibus Autism Proceeding and not only win his own petition, but get others the chance to retry their cases. He failed to do that–Thompson didn’t admit fraud. The “epidemiology” study appears to have been part of the PR campaign Hooker and Thompson discussed. That study gave them the springboard to claim fraud, but they did that by hiding a lot of information and misrepresenting other.
Luckily Hooker and company placed their hopes for the PR campaign on Andrew Wakefield and his “autism media channel”. Wakefeild’s video was good for rallying the troops and recruiting new parents into the fold, but it was so amazingly over-the-top bad that Wakefield scuttled whatever small chances they had for real press on the issue. The little coverage they got was from journalist turned blogger Sharyl Attkisson (who calls herself “fearless” and then deletes all of the comments on her blog and now only allows fawning comments…) My guess is that Wakefield is hoping that when Attkisson returns to journalism she will take this story on. Which begs the question of how Attkisson feels now that more facts are available.