Left Brain Right Brain

A few weeks ago, BMC Pediatrics published an article that purports to show that Hyperbaric Oxygen Therapy (HBOT) can produce “…significant improvements in overall functioning, receptive language, social interaction, eye contact, and sensory/cognitive awareness..” in autistic children. This study (Rossignol et al, 2009) is billed as a “…multicenter, randomized, double-blind, controlled trial.”

It’s all that and much, much less.

Let’s start by looking at the six “centers” where this research was carried out.

The Centers

The International Child Development Research Center (ICRDC):

This imposing name is attached to a rather less imposing edifice. The ICRDC, brainchild Dr. Jeffrey Bradstreet, is located in a strip mall in Melbourne, Florida, where it not only carries out “cutting-edge research” but also sells a complete line of “supplements” and treats autistic children with a dizzying array of “alternative”, “biomedical” and “integrative” therapies, including HBOT.

Daniel Rossignol MD (Family Practice), Lanier Rossignol (Nurse Practitioner) and Scott Smith (Physician’s Assistant) were the authors from the ICDRC.

The Center for Autism Research and Education (CARE):

This “center” is located in Phoenix, Arizona and has – according to its website – a single practitioner, Cynthia Schneider, MD (OB/Gyn), who is also an author on this paper. One of the “integrative” therapies this “center” offers is HBOT.

One of the other authors, Sally Logerquist, is a PhD psychologist who – according to the paper – is also associated with CARE, but also appears to run social skills therapy groups for autistic children using the “Logerquist Excellent Attitude Program” (LEAP).

True Health Medical Center:

It’s rather difficult to find anything about this “center”, apart from the fact that it is located in Naperville, Illinois – in what appears to be an office complex. Anju Usman, MD (Family Practice) is the author associated with this location.

Neubrander Center:

Although not officially called a “center”, the office of James Neubrander, MD (Pathology) is apparently one of the “centers” of this study. His office is located in the Menlo Park Mall (near Macy’s) and offers – you guessed it! – HBOT as a treatment for autism.

Princess Anne Medical Associates:

A Family Practice medical group in Virginia Beach, Virginia, this “center” is the home of Eric Madren, MD (Family Practice). It’s not clear if this four-physician practice offers HBOT.

The Rimland Center for Integrative Medicine:

A small, one-physician “center” in Lynchburg, Virginia, this is practice location of author Elizabeth Mumper, MD (Pediatrics). Not surprisingly, this “center” sells HBOT services for autistic children.

So, of the six “centers” involved in this study, five are single-physician operations. The remaining “center” has two physicians (three, if you count the naturopath).

I’m underwhelmed.

Well, what about the research itself? Maybe that’s better than the “facilities” might suggest. Let’s take a look.

The Subjects

This study initially enrolled 62 children (33 treatment; 29 control), but only 29 of the treatment group and 26 of the control group finished all 40 sessions. For reasons that pass my understanding, one treatment subject who only finished 9 sessions was included in the analysis. The authors stated that including this subject did not alter results, which begs the question: “Why did they include this subject if it made no difference?”

Outcome measures

The authors used the Aberrant Behavior Checklist (ABC), the Clinical Global Impression (CGI) scale and the Autism Treatment Evaluation Checklist (ATEC) as their outcome measures. All except the ATEC are widely accepted for use in autism treatment trials.

The ABC is a 58-question checklist of – surprise! – aberrant behaviors which are each given a score from “0” (“not at all a problem”) to “3” (“severe problem”). This test has been use – and validated – in a number of disorders, including autism. It gives a global score as well as five subscales: a total of six measures.

The CGI is a generic rating scale used in a variety of clinical trials. For each parameter (e.g. “overall functioning”, “sleep pattern”), the rater gives a score of between “1” (“very much improved”) and “7” (“very much worse”). The authors had both the treating physician and the parents rate the subjects on overall improvement and eighteen discrete parameters: a total of 38 measures in all (19 by the physician and 19 by the parents).

The ATEC was developed by Bernie Rimland and Stephen Edelson and has not been validated. In fact, it has only been used in two published studies – one by Rossignol et al. The ATEC has 25 questions on which the evaluator rates the subject on either a three-point (“not true”, “somewhat true”, “very true”) or four-point (“not a problem”, “minor problem”, “moderate problem”, “serious problem”) scale. It provides a total score and four subscales: a total of five measures.

In all, each subject had a total of 49 evaluation measures (CGI scores and the change in ABC and ATEC scores), of which 47 are independent. The importance of this will become apparent in the section on statistical analysis.

Analysis

As I mentioned above, the decision to include one treatment subject who only completed nine sessions was curious. Why they included this subject and not any of the other three treatment subjects and three control subjects who also failed to complete the entire course of the study is concerning. The smart thing – and the proper response – would have been to drop this subject from analysis.

The authors’ method of analyzing the CGI scales was also curious. Rather than simply using the scores as they were provided, they took the scores and subtracted them from four (the “no change” score). There are a few problems with this.

For starters, the scores are not linear – the difference between “much improved” and “very much improved” is not necessarily the same as between “no change” and “minimally improved”. Nor is the difference between “no change” and “much improved” twice the difference between “much improved” and “very much improved”. For that reason, these types of numerical scores are often referred to as “pseudo-numbers”.

This may seem like nit-picking, but it is a serious concern. Imagine, if you will, that the numbers were replaced by colors. Is the difference between green and orange twice the difference between orange and red? If half of a population of birds are blue and the other half are yellow, is the “average” bird green? The simple fact is that it is not appropriate to treat these “scores” as though they were real numbers, to be added, subtracted and averaged.

Secondly, it appears that the authors used parametric statistics for their analysis of the CGI scores. This is a problem since – as I indicated above – it is nonsensical to do math on pseudo-numbers. I don’t have the raw numbers, so it isn’t possible for me to calculate the absolute impact of this mistake for all of the CGI subclasses, but I can figure out the raw numbers for one group, so let’s look at that one.

It took a little work, but the authors gave enough clues to tease out the raw numbers in the physician “overall functioning” CGI score. The treatment group had an “average” of 2.87 and the control group’s “average” was 3.62; using the unaltered data, a t-test [Note: not an appropriate use of the t-test] gives p-value of 0.0006, not far from what the authors report. When a more appropriate statistical test [Mann-Whitney U-test] is used, the p-value is 0.002, very different from the reported 0.0008. While this is still less than the threshold p-value of 0.05, see below for a discussion of multiple comparisons.

All of these statistical analyses of the CGI scores ignore the fact that these are pseudo-numbers and need to be treated as discrete groups rather than as actual numbers. In truth, even the ABC and ATEC scores should have been treated this way, as well, although it is fairly common practice to treat such multi-factor scores as real numbers. A Chi-square test or Fisher Exact test would be the ideal test, but the problem with that is that the treatment group has one score of “1” (very much improved) and the control group doesn’t. Likewise, the control group has two subjects with a score of “5” (minimally worse) and the treatment group has none. This prevents a Chi-square or Fisher test from comparing each score independently.

One solution is presented by the authors themselves, although they apparently didn’t use it. In their discussion of the CGI, the authors said:

“Children who received a score of ‘very much improved’ or ‘much improved’ on the physician CGI overall functioning score were considered to be ‘good responders’ to treatment.”

If we “bin” the scores into “good responders” and “others”, we find that there were 9 (out of 30 – 30%) “good responders” in the treatment group compared to 2 (out of 26 – 8%) in the control group. Unfortunately, this is not a statistically significant difference (p = 0.08) in the (Yates) Chi-square test and barely reached significance (p = 0.05, but see below) in the Fisher Exact test.

An even bigger problem in the statistical analysis was the failure to correct for multiple comparisons. This problem was brought up by one of the reviewers, and the authors responded by eliminating a table. They did not make the appropriate corrections.

The reason that multiple comparisons are a problem is that the analysis for statistical significance is based on probability. If the probability (the p-value) that the differences between the two groups (treatment and control) is due to random chance is equal to or less than 5%, that difference is considered to be “statistically significant” and accepted as real. That means that there is still a 5% (or less – look to the p-value) chance that the difference is due to chance and not real.

If multiple comparisons are made on the same group of subjects, the probability that one (or more) of them will be “statistically significant” by chance starts to climb. If 14 comparisons are made, the chance of an erroneous “statistical significance” is over 50%. If 47 independent comparisons are made – as in this study – the chance of an erroneous “statistical significance” is over 90%.

For this reason, it is standard procedure to apply a correction for multiple comparisons. The most well-known (and simplest) of these is the Bonferroni Correction, which changes the threshold for statistical significance by dividing it by the number of comparisons. In the case of this study, the threshold (normally p less than or equal to 0.05 or 5%) is reduced to 0.001.

Applying the appropriate correction for multiple comparisons changes the results of this study significantly. Only the physician CGI scores for overall functioning and receptive language reach significance – and these numbers are already suspicious because they were improperly handled to begin with. In fact, as I have shown above, the CGI “overall functioning” p-value wouldn’t reach significance. It is possible that – if the proper statistical tests were used – that the CGI score for “receptive language” would also not reach significance.

Another curious thing. The authors asked the parents after the study whether they thought their child was in the treatment or the control group. Rather than say that the parent’s guesses were no better than random chance (i.e. 50%), the authors stated:

“…there was no significant difference between the two groups in the ability of the parents to correctly guess the group assignment of their child.”

As I said, this was a curious way to put it. As I read this, all it says is that each group of parent were equally able to guess which group their child was assigned to. That could be a 50% accuracy (which would be equal to chance), but a 90% or 99% accuracy – if both groups were that accurate – would also fit that description.

Now, this could simply be an clumsy phrasing by the authors, or it could be a way to make it sound like their blinding was successful when it actually was not.

Summary

This study may have collected some useful data, but its analysis of that data rendered it useless. The CGI scores – where the only statistically significant result was (possibly) seen – were improperly manipulated and the wrong statistical analysis was used.

The other issue is that there is no discussion of why HBOT is thought to be superior to providing the same partial pressure of oxygen at room pressure. This study used 24% oxygen at 1.3 atm, which gives the same partial pressure of oxygen as 31% at sea level. This concentration of oxygen can be easily attained with an oxygen mask or simple oxygen tent – both of which are vastly less expensive than HBOT.

If the authors are arguing that the mild pressure of their inflatable HBOT chambers contributes to the treatment effect, they need to look at the literature on cell membrane compressibility. For those who want to do the calculations at home, the bulk modulus of water (the major component of cells) is 21,700 atm. This means that a 0.3 atm increase in pressure will reduce the cell volume by 0.0014%. The bulk modulus of the lipid bilayer in cell membranes is around 30,000 atm. This means that an increase of 0.3 atm pressure causes a 0.0010% reduction in membrane volume. These are well below the threshold for any clinical effects.

Real pressure effects on the central nervous system are seen at pressures over 19 atm. These effects are:

dizziness
nausea
vomiting
postural and intention tremors
fatigue and somnolence
myoclonic jerking
stomach cramps
decrease intellectual and psychomotor performance
poor sleep with nightmares
increased slow wave and decreased fast wave activity in EEG

None of these effects could be construed as “improvements”, even in autism.

So, this study fails to answer the following questions about HBOT and autism:

[1] Does HBOT improve any feature of autism?
[2] If so, is HBOT any better than supplemental oxygen (which is much cheaper)?

The only real effect of this study was to give a cover of legitimacy to practitioners who are already using HBOT to “treat” autism.

Prometheus

When I recently wrote about the new HBOT-for-autism study (Rossignol et al. 2009)¹, I took issue with unlikely claimed treatment pressures for at least one of the study locations. While a potential methodological weakness, this is probably a fairly small problem in light of potential issues with blinding and interpretation of the results as quantitatively and objectively meaningful with respect to autism. But let’s set those potential issues aside for a moment.

Let’s assume that treatment with slightly enriched air (24% vs. 21% oxygen) in an inflatable hyperbaric chamber pressurized to 4 PSI^2,3 above ambient atmospheric pressure, could confer some sort of benefit to an autistic child.

I’m not suggesting assumption that it does confer benefit. I’m asking readers to set aside any knowledge of hemoglobin’s role in oxygen transport, as well as any knowledge of real hyperbaric oxygen therapy (breathing 100% oxygen at greater than 1 ATA)⁴, and evaluate a simpler proposition. Accept the proposal that some sort of benefit is scientifically possible, but then ask yourself a fairly simple question:

Compared to 24% O2 at 4PSI above ambient atmospheric pressure in an inflatable hyperbaric chamber, equivalent oxygen delivery can be achieved with simple oxygen therapy (an oxygen mask) at a fraction of the cost⁵ – why is a study of the hyperbaric version of this increased oxygen important?

One possibility: studying what’s already for sale

While some might call it being on the “cutting edge”, others may consider it putting the cart before the horse. No matter how you see it, it’s no secret that some Defeat Autism Now practitioners were already selling this type of hyperbaric oxygen therapy well before this study came out. It should be noted that this study’s authors did disclose this conflict of interest with respect to derivation of revenue in their clinical practices from HBOT.

DAR, LWR, SS, CS, AU, JN, EMM, and EAM treat individuals with hyperbaric treatment in their clinical practices and derive revenue from hyperbaric treatment.

Lisa Jo Rudy over at autism.about.com⁶ had additional comments about the subject:

Dr. Rossignol is “the” proponent of HBOT, and has been speaking at conferences all over the world in support of the treatment. Clearly, he has a personal and professional stake in seeing that the outcomes of a research study are positive.

The present study was funded by the International Hyperbarics Association, a trade group of private hyperbaric therapy centers. Clearly, they have a similar stake in seeing positive outcomes.

While there may certainly be an aspect of genuine scientific interest in understanding if this type of hyperbaric oxygen therapy is beneficial for autistic kids, I think there may also be a certain degree of assumption that it is. After all, why would a practioner already be selling something if they didn’t “believe” it worked? Given the stated conflicts of interest, it doesn’t seem implausible that the authors might have an interest in seeing a long-term revenue stream that could come from additional, and deeper pockets than those of parents willing to “believe” and pay – despite the lack of really convincing scientific evidence at this point.

Consider the following portions of an interview with Dr. Dan Rossignol⁷:

We chose 1.3 ATA because a lot of children with autism are currently receiving this dose and we are hoping to prove that it works.

“Hoping to prove that it works.”

Dr. Rossignol’s point does not seem unclear. HBOT is popular, and he is, in his own words, “hoping to prove that it works”. This is a valid reason, I suppose, if he is also open to the possibility that it may not, or that it may be a completely moot point if something on the order of one tenth of the cost can do the same thing. Following Dr. Rossignol’s communication about the hope to “prove that it works”, the interviewer asks:

How is the insurance situation coming along?

Insurance situation? Coming along? Was this situation already a well-known “work in progress” back in 2006 (e.g. had it been decided by some, prior to the science, that “mild” HBOT for autism does work, and that insurance reimbursement is really the goal now? Let’s see if we can get Dr. Rossignol’s take on this.

Well, obviously, HBOT is not approved for autism, but we hope to get there. Interestingly, if you take the ABC scale and look at the lethargy subset score, we saw a 49% improvement in symptoms at 1.5 ATA with a p-value of 0.008. If you look at the New England Journal of Medicine study on risperidone from 2002, there was a 56.9% improvement on the ABC irritability subscale with a p-value < 0.001. So the results we had on these 6 children with 1.5 ATA approached the percentage improvement seen with a drug approved for the use in autism. We just need to be able to reproduce these type of findings in a placebo study.

Hopefully when we finish these studies and show that hyperbaric therapy works, then insurance reimbursement will follow.

I don’t necessarily see a geniune scientific perspective here, but that could just be me. I get more of a vibe (at least from this interview), that the interest may lie more in “finishing” the studies and showing “that hyperbaric therapy works”, rather that actually finding out, with really good quality scientific methodology, whether or not it really does work. I’ll acknowledge that I could be wrong about this. Do you think readers will have noticed that the study result mentioned for comparison, was from 1.5 ATA, and probably totally irrelevant to the 1.3 (or less) studies?

Is it just me, or would it seem naive to wish that a few studies like the recent one, are really going to catalyze insurance reimbursement in the long run? I get the impression that many parents may believe this. Insurance companies work to achieve cost efficiencies. One of the ways they do this is by reimbursing at higher rates for equivalent things at lower costs – hospital stays in contracted facilities, generic drugs as compared to name-brand versions, etc. Why on earth would an insurance company reimburse for a 4-5% increase in blood oxygen content for a couple of hours at a time, in an inflatable hyperbaric chamber (at a few thousand dollars a month), when the identical oxygen increase could be delivered with a simple oxygen mask (for under $200 a month)?

You don’t have to take my word for this comparison of oxygen delivery, you can take Dr. Rossignol’s acknowledgement in that same interview:

Some people have criticized using mild hyperbarics at 1.3 ATA because they state that when compared to this pressure, you can get just as high an oxygen concentration in the blood with oxygen by face mask without a chamber. And this may be true in some cases.

In fact, it’s true in most (if not all) cases. The physics of partial pressures does not discriminate. But there may be more to the story.

Squeeze in some hope

After acknowledging the reality of the partial pressure comparison problem, Dr. Rossignol continues:

However, we must remember we are dealing with 2 separate components with HBOT — the oxygen and the pressure. So it appears that many of the effects of HBOT are from the increased oxygen, but we cannot dismiss the pressure effect. I think we need more studies on this as well.

So “many of the effects” are from the oxygen increase, but we can’t dismiss the pressure effect? What pressure effect? Is there a demonstrated significant clinical effect for autism from a very slight, and very temporary, increase in atmospheric pressure alone?

Although I suppose it is possible, a clinically significant effect for autism at such low pressures doesn’t seem likely at all. If it turns out that I am incorrect, this may be good news for some of the parents of autistic children in several U.S. cities: Albuquerque, NM (5312′ AMSL), Aurora, CO (5471′ AMSL), Colorado Springs, CO (6035′-7200′ AMSL), Denver, CO (5280′ AMSL), Reno, NV (4505′ AMSL), and Salt Lake City, UT (4226′ AMSL), to name a few. Something as simple as a move to a closer to sea-level city might provide increases in atmospheric pressure not a lot unlike those provided by the inflatable hyperbaric chambers. If there were some beneficial effect of slight additional atmospheric pressure for autism, certainly there would have been some observations (anecdotal or media reports) over the years, of families with autistic children who moved from states like Colorado to lower elevation states like California – and noticed. Who knows? Perhaps this is something to yet be uncovered.

So, aside from the fact that an identical oxygen increase can be achieved with simple O2 therapy without a hyperbaric chamber at all (and at a fraction of the cost). And, aside from the point that the minute pressure increase (while certainly possible in a strict scientific sense) isn’t known to be a likely candidate to significantly clinically impact autism, is there anything else about this newest HBOT-for-autism study that may merit some critical thought? Maybe, but it’s really just a side-note (perhaps interesting to some, but not terribly relevant to the science itself).

Who farted in the HBOT chamber? (Shh!…Don’t say it’s expensive)

The original manuscript⁸ for this study contained what I thought was an appropriately realistic comment from the authors in the conclusion. This comment has value in terms of practical knowledge that readers who are not familiar with hyperbaric oxygen therapy would probably find useful. What follows is the first-draft conclusion of this study with that comment emphasized.

Hyperbaric treatment is a relatively time-intensive treatment and can be costly. However, given the positive findings of this study, and the shortage of proven treatments for individuals with autism, parents who pursue hyperbaric treatment as a treatment for their child with autism can be assured that it is a safe treatment modality at the pressure used in this study (1.3 atm), and that it may improve certain autistic behaviors. Further studies are needed by other investigators to confirm these findings; we are aware of several other planned or ongoing studies of hyperbaric treatment in children with autism.

Again, Lisa Jo Rudy over at autism.about.com notes:

No insurance company will cover the very high cost of HBOT for autism, as it is considered an experimental and unproven therapy.

But the above conclusion is not the conclusion that appeared in the peer-reviewed, edited version. Here it is:

Given the positive findings of this study, and the shortage of proven treatments for individuals with autism, parents who pursue hyperbaric treatment for their child with autism can be assured that it is a safe treatment modality at the pressure used in this study (1.3 atm), and that it may improve certain autistic behaviors. Further studies are needed by other investigators to confirm these findings; we are aware of several other planned or ongoing studies of hyperbaric treatment in children with autism.

Why would the authors remove that valuable bit of practical knowledge about time requirements and high cost? Apparently due to a comment from referee #3 for this paper.

Discretionary Revisions

Page 24 In view of the highly positive findings of this study and the fact that no other trial has demonstrated such benefits under strictly controlled conditions to open the conclusions with negative comments demeans the study. Many other inventions used for ASD children are equally time consuming and hyperbaric treatment need not be expensive.

Authors: “The negative comments were removed from the conclusion.”

Opening the conclusion with negative comments demeans the study? Such comments don’t really touch the content of the study itself, and what the now absent comment did do, was provide some practical perspective – quite likely, very accurate practical perspective. Why would it be suggested by referee #3 that the practical comments demean the study? Perhaps it was meant that the comments demean the use of mild hyperbaric oxygen therapy as an autism treatment (therefore actually demeaning a desired interpretation of this study)? That would seem a real possible concern, since the justification offered, has absolutely nothing to do with the study itself, and doesn’t amount to much more than logical fallacy and simple assertion.

“Many other inventions used for ASD children are equally time consuming…”

This is about as basic an example of the “two wrongs make a right” fallacy as can be presented. Two wrongs don’t make right. Just because other interventions are also time consuming, does not mean a researcher is unjustified, or shouldn’t add the point about practicality that HBOT is relatively time consuming. Further, if the authors are aware of such a potential practical issue, it could be argued that ethics would dictate that it is mentioned. Other treatments presenting similar impracticalities do not automatically relieve any potential ethical responsibility in this regard.

“…hyperbaric treatment need not be expensive.”

Compared to what? Hyperbarics in a gold-plated hyperbaric chamber? If there is no significant effect for autism from the brief, and small increase in added pressure in one of these inflatables, the increased oxygen delivered by providing 24% O2 at 4 PSI above ambient atmospheric pressure, is easily matched (or exceded) with simple O2 therapy. In short, this type of hyperbaric treatment would be the hard way, and the expensive way to achieve the results.

Referee #3 also added the following comment:

The reviewer has a preference for the word treatment rather than ‘therapy’. In view of the proven changes that relate to increased inpsired fractions of oxygen it is suggested that treatment would be preferable.

Authors: “The word “therapy” has been replaced with “treatment” throughout the paper.”

The “T” in the acronym “HBOT” does, in fact, represent the word “therapy” in medical usage. I happen to think the terms “treatment” and “therapy” are fairly interchangeable in the context of drug delivery, but I do wonder if there is any significance to such a preference. Is this a semantics issue that has the potential to impact perceptions of those who make decisions about insurance coverage for autism? But I digress. So what’s up with these comments from referee #3, comments with a little fallacious reasoning, that express possible concern about the perception of a high price tag for mild hyperbaric oxygen therapy, and a commment that communicates a preference for the word “treatment” over “therapy”?

I honestly don’t know. What I can tell you is that referee #3 was Philip James, MD. Dr. James is a professor in the field of hyperbaric medicine and hails from the U.K. He appears to have published quite a bit in the field of hyperbaric medicine as well.

According to the International Hyperbarics Association website:

Dr. James is responsible for founding the Hyperbaric Trust in the United Kingdom which promotes the treatment of cerebral palsy and the brain injured child and was responsible for having the National Health Service pay for this therapy.

Dr. James (Referee #3) appears to have been categorized (with a doctor profile) as a medical advisor to International Hyperbarics Association back in February of 2006 (shortly before this study⁹ began). Hey wait a minute, there’s that name again – International Hyperbarics Association. Where have I seen that before? Oh yeah, in the study itself:

We are grateful for the work of Shannon Kenitz of the International Hyperbarics Association (IHA) for an unrestricted grant which funded this study, which included use of hyperbaric chambers and funding for all hyperbaric technician salaries during the study. The IHA had no involvement in the study design, collection, analysis, interpretation of data, writing of the manuscript, or in the decision to submit the manuscript for publication.

I’m not sure how the International Hyperbarics Association defines itself exactly – are its listed medical advisors excluded from that definition? That would seem likely.

As of this writing, Dr. Rossignol is listed as a medical advisor at the IHA website. ¹⁰
As of this writing, Dr. Neubrander is listed as a medical advisor at the IHA website. ¹⁰
As of this writing, Dr. James (referee #3), is categorized as a medical advisor at the IHA website with a physician profile page.^11,12

Side notes aside, where to, from here

So all in all, it seems that “mild” HBOT-for-autism researchers may have their work cut out for them. Although probably not very likely, it is possible that a small temporary change in atmospheric pressure could do something for autism, and that should be studied next, then, better replications should follow.

In the long run, it will be difficult to ignore the scientific fact that simple oxygen therapy alone can easily provide identical increases in blood oxygen content, at a fraction of the cost of mild hyperbaric oxygen therapy (as it currently being studied for autism). If HBOT-for-autism proponents think insurance companies should step up to pay for an expensive treatment that provides a 4-5% increase in blood oxygen (without scientifically establishing benefit of the small and temporary pressure increases), they might do well to consider these famous words (most recently from Barack Obama) – “You can put lipstick on a pig. It’s still a pig.”

Notes:

¹ Hyperbaric treatment for children with autism: a multicenter, randomized, double-blind, controlled trial

Click to access 1471-2431-9-21.pdf

² Medical device pre-market notification (FDA-cleared)

Click to access K001409.pdf

³ Manufacturer product sheet

Click to access vitaeris-lowres2007-8.pdf

⁴ Definition of Hyperbaric Oxygen Therapy
http://www.uhms.org/ResourceLibrary/Indications/tabid/270/Default.aspx

⁵ Hyperbarics and Hypotheses

Schooling and Statement

⁶ Hyperbaric Oxygen as a Treatment for Autism: Let the Buyer Beware
http://autism.about.com/b/2009/03/14/hyperbaric-oxygen-as-a-treatment-for-autism-let-the-buyer-beware.htm

⁷ Interview with Dr. Dan A. Rossignol: Hyperbaric Oxygen Therapy Improves Symptoms in Autistic Children

Click to access Rossignol%20HBOT%20Medical%20Veritas%202.pdf

⁸ Pre-publication history
http://www.biomedcentral.com/1471-2431/9/21/prepub

⁹ ClinicalTrials.gov Identifier: NCT00335790
http://clinicaltrials.gov/ct2/show/NCT00335790

¹⁰ Medical Advisors
http://www.ihausa.org/

¹¹ Index of /docs
http://www.ihausa.org/docs/

¹² International Hyperbarics Association Medical Advisor – Professor Philip B. James, M.D.
http://www.ihausa.org/docs/james.html

I recently read the BMC Pediatrics article, “Hyperbaric treatment for children with autism: a multicenter, randomized, double-blind, controlled trial”¹. I know this paper is attracting a lot of attention in the media, and it is certainly being ballyhooed about the internet. Hell, I’ve even received e-mail spam about this study! But I’m sorry to say, I don’t really share the excitement. In fact, I see what looks like a pretty significant error in the methodology of this study. It’s one of those types of potential errors that stand out like a strobe light or a siren – it’s really tough for me to pretend it’s not there.

Once again, I’m going to ask readers to set aside, for the moment, anything they may know about the role of hemoglobin in oxygen transport and how the minute increases (probably around 3-4%) in total blood oxygen content afforded by this kind of hyperbaric therapy, or simple O2 therapy for that matter, are probably pretty likely to be insignificant.

Both the paper and ClinicalTrials.gov² list the Center for Autism Research and Education, Phoenix, Arizona, as a study location. This is a problem, because the stated treatment pressure in the study (1.3ATM) seems highly unlikely to actually be achievable in Phoenix with the equipment that was apparently used for this study.

As described in the section titled, “Interventions”:

“These procedures included covering control switches, inflating and deflating the chambers to simulate pressure changes, and masking the sounds from the chambers.”

The use of inflatable monoplace hyperbaric chambers, is a clear indication that the actual total pressures (and quite likely results of this study) would have been affected by the ambient air pressures at the times and locations of treatment. In fact, the ambient air pressure is the largest component of the stated treatment pressure in this study (ambient pressure + added treatment pressure = total treatment pressure).

Ambient pressure

Local atmospheric pressure is typically reported as sea-level pressure³ for its utility to aviation, and the meaningful interpretation of weather maps, etc., but the actual station pressure is affected by the elevation. The expected ambient atmospheric pressure, corrected for altitude, (or station pressure) in Phoenix, Arizona⁴ is 28.69 in Hg (where there is a modest elevation of 1161’ AMSL). Wanting to give this paper the benefit of the doubt, and knowing that “high pressure” weather is typical of the Phoenix climate, I looked at 30-day data⁵ for actual station pressure in Phoenix at a station of slightly lower altitude than the Center for Autism Research and Education. The 30-day mean station pressure is 28.81 in Hg, so I’ll use that one for calculations, as it will yield results more likely to be in the study’s favor.

Added treatment pressure

The actual operating pressure of the inflatable chambers, as stated by the manufacturer, is 4 PSI. ^6,7 This pressure is also indicated on the Center for Autism Research and Education’s website:

“The chambers used at care utilize a pressure of 4 psi.”⁸

Total treatment pressure

The total treatment pressure can be easily calculated with the following conversions:
in Hg * 0.491 = PSI
PSI + PSIG = Total PSI
Total PSI * .068 = ATA

For Phoenix, Arizona, this gives a calculated total treatment pressure of 1.23 ATA.

28.81 * 0.491 = 14.15 PSI
14.15 PSI + 4 PSIG = 18.15 PSI
18.15 PSI * .068 = 1.23 ATA

Damn, that’s a pretty big difference from the paper’s stated 1.3 ATM – representing an addition of only .23 ATM (instead of .30 ATM) above mean sea-level pressure of 1 ATM.

I’ve corresponded with the lead author of this study in the past, and he stated that he observes gauge pressure of 4.15 PSI. Despite the manufacturer specs, the FDA-cleared medical device premarket notification, and the Center for Autism Research and Education’s website (which all indicate operating pressure of 4 PSI), and wanting to give the benefit of the doubt, I’ll use 4.15 PSI for the next calculation, as it will be more likely to yield results in the study’s favor.

28.81 * 0.491 = 14.15 PSI
14.15 PSI + 4.15 PSIG = 18.30 PSI
18.30 PSI * .068 = 1.24 ATA

It could be argued that treatment pressure for the other study locations were properly rounded up to 1.3 ATM (even though the actual pressures were quite likely to be considerably lower), however, even with all the calculations purposely leaned in favor of a higher number for Phoenix, Arizona, the study’s stated treatment pressure, there, should have properly rounded to 1.2 ATA! This suggests an overstatement of the added treatment pressure for the Phoenix location of 50% (.3 ATM is 150% of .2 ATM). Even if given the benefit of the doubt yet again, and an exception to proper rounding were made for solely for the Phoenix location in this study, the study’s likely overstatement in added treatment pressure for Phoenix is still a full 25%. (.3 ATM is 125% of .24 ATM – 25% more added pressure above 1 ATM was claimed in this paper, than was probably delivered).

I think this is a big enough boo-boo, that the editors of BMC Pediatrics should call for detailed errata. In the interest of scientific accuracy, it would seem prudent for BMC Pediatrics to:

1. Clarify for its readership and the scientific community, that the stated pressure of 1.3 ATM in this study is rounded up, and includes the ambient air pressure, or alternatively, state the estimated pressure in terms of ATA.

2. Clarify for its readership and the scientific community, that the stated pressure of 1.3 ATM in this study is an estimated pressure, since no actual measurements of ambient station pressure for the locations, and dates/times of treatments were reported.

3. Note for its readership and the scientific community, that the stated pressure of 1.3 ATM was not likely to be uniformly achievable across all study locations due to the use of inflatable hyperbaric chambers and changes in elevation (and atmospheric pressure) across study locations, potentially confounding the results of this study.

4. Note for its readership and the scientific community, that estimated pressures in the placebo control group are affected by these same issues that affect the treatment group, potentially confounding the results of this study further.

What do you think?

¹ BMC Pediatrics 2009, 9:21doi:10.1186/1471-2431-9-21
http://www.biomedcentral.com/1471-2431/9/21/abstract

² http://clinicaltrials.gov/ct2/show/NCT00335790

³Federal Meteorological Handbook No. 1 – Table 11-2
http://www.nws.noaa.gov/oso/oso1/oso12/fmh1/fmh1ch11.htm

⁴ LAT/LON 33.5º N 118.08º W

⁵ http://www.wrh.noaa.gov/mesowest/getobext.php?wfo=psr&sid=KPHX&num=720

⁶ Medical device pre-market notification (FDA-cleared)

Click to access K001409.pdf

⁷ Manufacturer product sheet

Click to access vitaeris-lowres2007-8.pdf

⁸ http://www.center4autism.org/therapyHBOT.asp