Left Brain Right Brain

This blog entry first appeared at Respectful Insolence. It is reproduced here with permission.

A bad day for antivaccinationists: Yet another study fails to support an association between vaccines and neurodevelopmental disorders
By Orac.

I’m almost beginning to feel sorry for the mercury militia.

Think about it. They’ve been claiming for the past several years that the mercury in the thimerosal used as a preservative in childhood vaccines is a cause of autism. If you believe Generation Rescue, A-CHAMP, SAFEMINDS, and various other activist groups, vaccines are the root of all neurodevelopmental evil, culminating in what to them seems to be the most evil of evil condition, autism. Yet, in study after study in the new millennium, no correlation has been found to implicated their favorite bte noire thimerosal, which serves as a surrogate for their general dislike of vaccines in general.

It comes as no surprise, then, that A-CHAMP would want to launch a pre-emptive strike against a large study of thimerosal-containing vaccines that was published in the New England Journal of Medicine today. Blazing out of their website is the headline New CDC Study Falsely Claims Thimerosal is Safe

On September 27, 2007 the New England Journal of Medicine will publish a study entitled, “Early Thimerosal Exposure and Neuropsychological Outcomes at 7 to 10 Years.” For more than two years we at A-CHAMP have been hearing rumors of a new study that “exonerates” thimerosal, despite the fact that the study results were supposed to be kept strictly confidential.

Now the rumors have been turned into hype – another government funded study that tries to spin data and clear thimerosal of any suspicion of causing neurodevelopmental disorders. The study authors claim in their “Conclusions” that “[o]ur study does not support a causal association between early exposure to mercury from thimerosal-containing vaccines and immune globulins and neuropsychological functioning at the age of 7 to 10 years.”

The statement is plainly false. The study’s conclusions do not reflect the study’s data or the limitations of the study,

You’d think that at least A-CHAMP would correct that hanging comma at the end of the sentence there.

Sarcasm aside, the study’s conclusions do reflect the study’s data, quite well, as we will see, and A=CHAMP’s complaints boil down to the usual crank technique of cherry picking the evidence, combined perhaps with sour grapes. Let’s lay the abstract out for all to see before we look at A-CHAMP’s individual points:

Early Thimerosal Exposure and Neuropsychological Outcomes at 7 to 10 Years
William W. Thompson, Ph.D., Cristofer Price, Sc.M., Barbara Goodson, Ph.D., David K. Shay, M.D., M.P.H., Patti Benson, M.P.H., Virginia L. Hinrichsen, M.S., M.P.H., Edwin Lewis, M.P.H., Eileen Eriksen, M.P.H., Paula Ray, M.P.H., S. Michael Marcy, M.D., John Dunn, M.D., M.P.H., Lisa A. Jackson, M.D., M.P.H., Tracy A. Lieu, M.D., M.P.H., Steve Black, M.D., Gerrie Stewart, M.A., Eric S. Weintraub, M.P.H., Robert L. Davis, M.D., M.P.H., Frank DeStefano, M.D., M.P.H., for the Vaccine Safety Datalink Team

Background It has been hypothesized that early exposure to thimerosal, a mercury-containing preservative used in vaccines and immune globulin preparations, is associated with neuropsychological deficits in children.

Methods We enrolled 1047 children between the ages of 7 and 10 years and administered standardized tests assessing 42 neuropsychological outcomes. (We did not assess autism-spectrum disorders.) Exposure to mercury from thimerosal was determined from computerized immunization records, medical records, personal immunization records, and parent interviews. Information on potential confounding factors was obtained from the interviews and medical charts. We assessed the association between current neuropsychological performance and exposure to mercury during the prenatal period, the neonatal period (birth to 28 days), and the first 7 months of life.

Results Among the 42 neuropsychological outcomes, we detected only a few significant associations with exposure to mercury from thimerosal. The detected associations were small and almost equally divided between positive and negative effects. Higher prenatal mercury exposure was associated with better performance on one measure of language and poorer performance on one measure of attention and executive functioning. Increasing levels of mercury exposure from birth to 7 months were associated with better performance on one measure of fine motor coordination and on one measure of attention and executive functioning. Increasing mercury exposure from birth to 28 days was associated with poorer performance on one measure of speech articulation and better performance on one measure of fine motor coordination.

Conclusions Our study does not support a causal association between early exposure to mercury from thimerosal-containing vaccines and immune globulins and deficits in neuropsychological functioning at the age of 7 to 10 years.

It’s noted that a similar study looking at autism will be published next year. Of course, the fact that this study didn’t look at autism leaves the antivaxers a huge opening to say, “Well, yes, but you didn’t look at autism.” Never mind that there are now several studies that did look at autism and found no association between thimerosal-containing vaccines and autism. So what are the main complaints that A-CHAMP has about this study? Let’s take a look, starting with the first one:

The Study’s Claim of No Causality is Contrary to the Study’s Data. The study authors claim that the data disproves causality when in fact, several findings show a negative effect on neuropsychological functioning warranting more study. At least one such adverse association was also found to be associated with low dose thimerosal exposure in other studies. As with earlier studies hyped by vaccine promoters, the study is unable to prove or disprove causality. The blanket dismissal of the troubling neuropsychological outcomes in this study is disingenuous and misleading.

First off, the study didn’t claim that there was “no causality.” What it stated is exactly what you see in the Conclusions section: That the study does not support a causal relationship. There’s a difference there too subtle for the ideologues who wrote this press release to understand. It’s impossible ever to prove “no causality.” It is, however, possible to conclude from the data that the data does not support a causal relationship. Second, A-CHAMP is cherry picking associations here. It is true that there were some negative correlations found that achieved statistical significance. When running 42 tests, it would be shocking if there were not a few anomalous findings. What makes the study authors fairly confident that the findings are anomalous is that they were divided roughly equally in both directions, good and bad. Consequently, if A-CHAMP is going to insist that the correlation, for example, with increasing mercury exposure and poorer performance on the GFTA-2 measure of speech articulation test, then it must also accept the findings of a beneficial association between mercury and identification of letters and numbers on the WJ-III test, as there is no reason to reject it. Naturally, the mercury militia picks on the associations as being true that they want to be true and ignore the other associations, which, if true, would be arguments for including thimerosal in vaccines. It’s far more likely that all of them are just noise. Again, the reason that investigators can reasonably conclude that the associations found are most likely due to random chance is because of their random distribution between positive and negative.

Another complaint:

Children with autism were excluded from this study. The early media contacts we have received suggest that this study shows no association between thimerosal and autism. In fact, the study specifically did not look at children with autism as the sample size was too small and the testing is impossible to complete for the typical child with autism. The exclusion of children with autism from the study may have undermined the power of the study to draw any conclusions about thimerosal.

This is a really, really dumb statement. I’m sorry, but there’s no other way to put it. It just is. It’s like criticizing an apple for not being an orange. The explanation for this is right in the paper, “Since the CDC is conducting a separate case-control study of autism in relation to mercury exposure, a measure of autism was not included in the test battery.” Get it, idiots? A separate study is being done and the results will be reported in a separate paper! That renders this complaint utterly irrelevant. It’s nothing more than a red herring designed to fire up the faithful.

This complaint is not really stupid, but definitely overblown:

The Study’s Methodology has Serious Limitations Negating Any Conclusions Drawn. Major flaws that that causes a large underestimation of neurological adverse effects burden the study: 70% of the families recruited for the study failed to participate. This kind of bias in epidemiological studies is well known to distort even large studies of health effects…It is well established that people who choose to participate in this kind of study are probably very different than those who refuse to participate (the “healthy person” or “complier” effect); especially when the ones who refused to participate said they were too busy.

Simply put: if you have a kid with ADHD or mild ASD or other neurodevelopmental disorders, you are likely to be busier, more stressed, and less available than the mother of a healthy normal child. This phenomenon serves to amplify the effect of the “complier”, the ;healthy families,” – those who do cooperate with the study – confounding or confusing the study’s results. The cooperative parents included in the study were more likely to be those with relatively trouble-free kids

There’s no evidence that this sort of bias existed, and A-CHAMP conveniently omits other biases that might be result from such a selection bias. For example, it is also quite possible to argue that parents who think vaccines may have caused their child’s condition would be even more likely to want to participate in the study (self-selection bias), than parents whose children are normal or who don’t believe vaccines had anything to do with their child’s problems; they might have thought that it wasn’t worth their bother. It’s good of the antivaxers who have been championing the execrable telephone survey done by Generation Rescue to recognize that this sort of bias exists; too bad they only mention it when it suits their purposes (i.e., to trash a study whose results they don’t like) and ignore it when it doesn’t, for example when it calls into doubt the results of the Generation Rescue telephone poll.

In reality, the fact that only 30% of the families who were approached for the study agreed to participate is probably the most significant weakness of the study. The authors don’t try to hide it. Rather, they are right up front with it in the Discussion section, while at the same time pointing out that this is this possible bias was ameliorated by enrolling children on the basis of having received vaccinations without regard to the seeking of health care or having been given a neurodevelopmental diagnosis, as well as noting that many children weren’t enrolled because they couldn’t be located. Moreover, exposure information was obtained from many different sources. It also looked at prenatal exposure to thimerosal-containing vaccinations from vaccines that the mother might have had while pregnant.

Next complaint:

Vulnerable Children Were Excluded from the Study; Early Intervention Was Ignored. Children with a birth weight under 5 lbs. 8 oz. were excluded from the study further skewing the results, as these children are likely more vulnerable to thimerosal than larger babies. In addition, the fact that early intervention may have reduced deficits such as speech delay detected by neuropsychological testing of children aged 7-10 was not accounted for in the study results. There also was no analysis of combined prenatal and postnatal mercury exposures. Only 103 mothers who were exposed to mercury from prenatal immune globulins participated in the study, far too small a group for researchers to draw conclusions regarding the safety of thimerosal in these products.

This is a grab-bag of the specious and semireasonable. The reason for excluding low birthweight children was obvious: Such children are more likely to have neurodevelopmental problems completely independent from any external cause, such as thimerosal. Including preemies and lower birth weight children would only contribute to the background noise and make finding true associations more difficult. A-CHAMP should be glad that the investigators did that, given that it was almost certainly done to make the study more likely to find an association between vaccines and neurodevelopmental disorders. In addition, there is no evidence that low birth weight children are “more vulnerable to thimerosal.” Of course, once again, A-CHAMP fails to point out that the authors themselves pointed out the shortcoming in regard to not being able to account for interventions such as speech therapy. As for the whining about not analyzing combined prenatal and postnatal mercury exposures, that’s just a red herring; the study did test an a priori assumption of interaction between pre- and postnatal mercury exposure.

The rest of the complaints are the standard ones: Conflict of interest, for example, because several of the study’s authors had received funding from pharmaceutical companies. These potential conflicts of interest are clearly stated in the study. A-CHAMP also brings up a totally specious complaint of the study supposedly not accounting for “efflux” disorder (i.e., children who are allegedly “poor excretors” of mercury). Fortunately, Prometheus has provided a good explanation of what a crock that whole myth is, so that I don’t have to.

Of course, there is another reason, besides this study failing to show what the mercury militia wants it to, that there is such hostility here:

The politicization of the thimerosal issue has led researchers to take unprecedented measures. Unlike previous studies, the current study included more than a dozen outside consultants, including at least one advocate for families of children with autism. “We have really tried to make the entire process–from experiment design to manuscript review–as transparent as possible,” says Thompson. But that effort may not have made a difference in the long run. Sallie Bernard, executive director of Safe Minds, a nonprofit parents’ organization that focuses on the role of mercury in neurodevelopment disorders, consulted on the study but still takes issue with its findings. “All the studies, including this one have certain limitations in their design and their methodology,” she says.

Sour grapes, anyone? Ms. Bernard was a consultant on this study and helped contribute to its design! She apparently didn’t like the results that it was producing and decided to drop out and start criticizing it–even jumping the gun on the 5 PM embargo yesterday to do so! Indeed, she is listed on the study in a way that I’ve never seen before: as a “dissenting member.”

How many studies by mercury militia enablers Mark and David Geier include even a note of dissent?

Although this study is not without flaws, it nonetheless does not support a correlation between thimerosal-containing vaccines and the neurodevelopmental problems studied. Even better, the New England Journal of Medicine included two editorials along with the story. One editorial discussed the explosion of vaccine litigation that the now discredited thimerosal “hypothesis” has unleashed, immune to the findings of science. The second editorial is by Paul Offit (the Antichrist to antivaxers) and makes an excellent point about the unintended consequences of taking precautionary measures on the basis of little or no evidence:

On July 9, 1999, after much wrangling, the CDC and AAP decided to exercise the precautionary principle. They asked pharmaceutical companies to remove thimerosal from vaccines as quickly as possible; in the interim, they asked doctors to delay the birth dose of hepatitis B vaccine in children who weren’t at risk for hepatitis. A press release issued by the AAP revealed the ambivalence among its members: “Parents should not worry about the safety of vaccines,” it read. “The current levels of thimerosal will not hurt children, but reducing those levels will make safe vaccines even safer. While our current immunization strategies are safe, we have an opportunity to increase the margin of safety.” Critics wondered how removing something that hadn’t been found to be unsafe could make vaccines safer. But many parents, frightened by a sudden change in policy, reasoned that thimerosal was targeted because it was harmful — and their faith in the vaccine infrastructure was shaken. Doctors were also confused by the recommendation.

I could see how that would confuse parents and doctors. Offit concludes:

Despite several years of reassuring studies, the thimerosal controversy continues to be emotionally charged. Physicians, scientists, government policy advisors, and child advocates who have publicly stated that vaccines don’t cause neurologic problems or autism have been harassed, threatened, and vilified, receiving hate mail and occasionally death threats. The CDC, in response to planned protests at its gates, recently beefed up security and instructed personnel about how to respond if physically attacked.

During the next few years, thimerosal will probably be removed from influenza vaccines, and the court cases will probably settle down. But the thimerosal controversy should stand as a cautionary tale of how not to communicate theoretical risks to the public; otherwise, the lesson inherent in the collateral damage caused by its precipitous removal will remain unlearned.

Exactly. With the best of intentions, trying to be as “safe” as possible, the AAP and CDC unleashed a tsunami of fear of vaccines and laid the groundwork for pseudoscientists like Mark and David Geier to stoke the fears of mercury further with badly designed studies. This was an example of framing science at its worst.

In the end, it is always frustrating to watch how such studies are spun by antivaccinationists. Epidemiology is like that, though. It’s virtually impossible to conduct a cohort study like this without permitting significant shortcomings in it. The reason is that, unlike a bench experiment, the investigators can never control all the variables. Trade-offs are inevitably made, and rarely are there adequate resources to assure sample sizes large enough to be bullet-proof or to be able to account for all potentially confounding variables. No one study is ever sufficient to rule out correlations between undesirable outcomes and various compounds. However, as the weight of several studies starts to bear down on the problem, the preponderance of evidence must at some point be accepted, because we do not have unlimited resources to keep doing studies to answer the same question over and over and over again and every repeated study uses resources that could be used to study other potential causes and treatments for autism. This study happens to be one large and convincing chunk of that evidence, but it is not the only one. Coming next year, when the CDC releases a similar trial about autism, will be another. As Dr. Offit said:

In a new study, Thompson and his colleagues are taking another look at thimerosal exposure and autism. But for many, the question has been resolved. “This study is the third one of its kind. When the autism one comes out, it will be the sixth of its kind. They’ve all shown the same thing–that there is no significant correlation,” says Offit. “Meanwhile, the thimerosal question has diverted attention and resources away from the search for more promising leads on what causes autism. How many more studies will it take?”

That’s the difference between science and crankery. If this study had shown a clear and convincing correlation between thimerosal in vaccines and neurdevelopmental disorders, I would have accepted the results and perhaps started to change my mind. On the other hand, for cranks, no number of studies is ever enough to dissuade them from their beliefs. If God Himself were to come down from on high and design the absolutely perfect study, which when carried out showed no correlation between thimerosal in vaccines and neurodevelopmental disorders, the antivaccinationists would insist that it was flawed, that the investigators had conflicts of interest, and that it needs to be repeated until (although they would never admit this) it shows the results they want it to show.

In fact, expect just that. The CDC has pledged to make the raw data available to other researchers. I predict that, within a few months, a study by Mark and David Geier will use their trademark bad statistics and bad math to cook the numbers to make it look as though there are associations between thimerosal in vaccines and neurodevelopmental disorders that the CDC “covered up.”

Just wait.

ADDENDUM: Some more commentary on this study:

From Left Brain/Right Brain:

No, everything was fine and dandy as long as she [Bernard] was enjoying being fawned over as a “representative of the autism community” and a fellow-scientist instead of the commercial marketer she actually is. Here’s a clue, Sallie: If you’re going to play scientist, you have to follow the rules of science, and that means you stand by your results. You don’t get to say “heads I win, tails you lose” by waiting to see the outcome before deciding whether the study was any good.

And you really don’t get to have CDC at your beck and call, spend hundreds of thousands of taxpayer dollars to do a study to your specifications, then turn around and call them liars when you don’t like how it comes out.

And you, CDC? You’re not just a victim here. Every time you say “let’s do more research” or “we are examining this issue” in order to appease the mercury moms, you increase the chances that kids will go unvaccinated because you failed to give their parents confidence in the safety of vaccines. When you say a study is reassuring and then highlight what is virtually certain to have been a chance finding (a statistical association between higher thimerosal exposure and transient tics in boys) without making it abundantly clear that some false associations were inevitable given the study design, you defeat the purpose of doing the study.

From AutismVox:

Safe Minds, whose Executive Director, Sallie Bernard, was an external consultant and dissenting member of the study, put out a press release stating that the new study’s “draws a misleading conclusion.” But this statement is as “misleading” as the headlines cited above are about the study’s actual contents This only furthers the belief that–no matter how clearly it is stated and shown that there is no causal association between early exposure to thimerasol and later neuropsychological outcomes—a link between these has been firmly established in the public mind. And disputing that link will take more studies, more evidence, more efforts, and more self-scrutiny of why we believe what we believe.

From Autism Natural Variation:

The CDC study looked at 42 different outcomes, and determined multiple confidence intervals in each case, since different levels of exposure were tested. In total, I understand there were over 300 confidence intervals. Consequently, assuming the null hypothesis is correct, you should expect that an RR of 1.0 will be outside the 95% confidence interval in over 15 measures. What the study found was that in 12 measures there was an apparent protective factor, and in 8 measures there was an apparent risk factor. This is completely consistent with the null hypothesis. Therefore, the conclusion of the study, namely that the results of the same do not support a causal association between thimerosal-containing vaccines and neurological outcomes, is absolutely the correct conclusion.

From Arthur Allen on The Huffington Post:

Despite the wealth of data showing that vaccines do not cause autism, many parents continue to believe the theory. Jenny McCarthy, an ex-Playboy Bunny and TV personage, has been making the rounds of the media this week (Oprah, ABC, People magazine) to promote her book, in which she claims that her child was made autistic by vaccines but was “recovered” through alternative therapies.

A large CDC study comparing thimerosal exposure rates in autistic and non-autistic kids is due out around this time next year, as is an Italian study of the question. The data will probably duplicate the many previous studies that have shown no effect. The dogs may bark but the caravan moves on. Or is it the other way around

(The comments from die-hard antivaxers after Allen’s post are scary indeed.)

From Derek Lowe at In the Pipeline:

The director of SafeMinds, a group of true thimerosal believers if ever there was, actually was on the consulting board of this latest study. But she withdrew her name from the final document. The CDC is conducting a large thimerosal-and-autism study whose results should come out next year. Here’s a prediction for you: if that one fails to show a connection, and I have every expectation that it’ll fail to show one, SafeMinds will not accept the results. Anyone care to bet against that?

As a scientist, I’ve had to take a lot of good, compelling ideas of mine and toss them into the trash when the data failed to support them. Not everything works, and not everything that looks as if it makes sense really does. It’s getting to the point with the autism/thimerosal hypothesis- has, in fact, gotten to the point quite some time ago – that the data have failed to support it. If you disagree, and I know from my e-mail that some readers will, then ask yourself what data would suffice to make you abandon your belief? If you can’t think of any, you have moved beyond medicine and beyond science, and I’ll not follow you.

Commenting on David Kirby’s truly idiotic take on this study in The Huffington Post, Steve Novella at Neurologica:

What Kirby does is not just really dumb, it’s despicable. He cherry picks all the negative (meaning bad) neurological outcomes and pretends that the study shows a correlation (it doesn’t, when you look at ALL the data). He then tries to dismiss the positive (good) outcomes as absurd. He mockingly writes:

If they (the CDC) really mean that thimerosal increases IQ levels in males, then sign me up for a double-dose flu shot this year.

No, David, they don’t mean that. Not by any stretch of the imagination. It takes incompetent statistical analysis or the blindness of ideology to write something so ridiculous. What the CDC means is that the study does NOT show that thimerosal increases IQ, nor that it causes motor tics, or improve motor skills, or decrease language skills, or anything else. The study showed no correlations because it all averaged out as noise.

Steve, normally polite to a fault even with cranks, seems to be laying down a bit of the old Respectful Insolence(tm). I like it.