I recently read the BMC Pediatrics article, “Hyperbaric treatment for children with autism: a multicenter, randomized, double-blind, controlled trial”1. I know this paper is attracting a lot of attention in the media, and it is certainly being ballyhooed about the internet. Hell, I’ve even received e-mail spam about this study! But I’m sorry to say, I don’t really share the excitement. In fact, I see what looks like a pretty significant error in the methodology of this study. It’s one of those types of potential errors that stand out like a strobe light or a siren – it’s really tough for me to pretend it’s not there.
Once again, I’m going to ask readers to set aside, for the moment, anything they may know about the role of hemoglobin in oxygen transport and how the minute increases (probably around 3-4%) in total blood oxygen content afforded by this kind of hyperbaric therapy, or simple O2 therapy for that matter, are probably pretty likely to be insignificant.
Both the paper and ClinicalTrials.gov2 list the Center for Autism Research and Education, Phoenix, Arizona, as a study location. This is a problem, because the stated treatment pressure in the study (1.3ATM) seems highly unlikely to actually be achievable in Phoenix with the equipment that was apparently used for this study.
As described in the section titled, “Interventions”:
“These procedures included covering control switches, inflating and deflating the chambers to simulate pressure changes, and masking the sounds from the chambers.”
The use of inflatable monoplace hyperbaric chambers, is a clear indication that the actual total pressures (and quite likely results of this study) would have been affected by the ambient air pressures at the times and locations of treatment. In fact, the ambient air pressure is the largest component of the stated treatment pressure in this study (ambient pressure + added treatment pressure = total treatment pressure).
Local atmospheric pressure is typically reported as sea-level pressure3 for its utility to aviation, and the meaningful interpretation of weather maps, etc., but the actual station pressure is affected by the elevation. The expected ambient atmospheric pressure, corrected for altitude, (or station pressure) in Phoenix, Arizona4 is 28.69 in Hg (where there is a modest elevation of 1161’ AMSL). Wanting to give this paper the benefit of the doubt, and knowing that “high pressure” weather is typical of the Phoenix climate, I looked at 30-day data5 for actual station pressure in Phoenix at a station of slightly lower altitude than the Center for Autism Research and Education. The 30-day mean station pressure is 28.81 in Hg, so I’ll use that one for calculations, as it will yield results more likely to be in the study’s favor.
Added treatment pressure
The actual operating pressure of the inflatable chambers, as stated by the manufacturer, is 4 PSI. 6,7 This pressure is also indicated on the Center for Autism Research and Education’s website:
“The chambers used at care utilize a pressure of 4 psi.”8
Total treatment pressure
The total treatment pressure can be easily calculated with the following conversions:
in Hg * 0.491 = PSI
PSI + PSIG = Total PSI
Total PSI * .068 = ATA
For Phoenix, Arizona, this gives a calculated total treatment pressure of 1.23 ATA.
28.81 * 0.491 = 14.15 PSI
14.15 PSI + 4 PSIG = 18.15 PSI
18.15 PSI * .068 = 1.23 ATA
Damn, that’s a pretty big difference from the paper’s stated 1.3 ATM – representing an addition of only .23 ATM (instead of .30 ATM) above mean sea-level pressure of 1 ATM.
I’ve corresponded with the lead author of this study in the past, and he stated that he observes gauge pressure of 4.15 PSI. Despite the manufacturer specs, the FDA-cleared medical device premarket notification, and the Center for Autism Research and Education’s website (which all indicate operating pressure of 4 PSI), and wanting to give the benefit of the doubt, I’ll use 4.15 PSI for the next calculation, as it will be more likely to yield results in the study’s favor.
28.81 * 0.491 = 14.15 PSI
14.15 PSI + 4.15 PSIG = 18.30 PSI
18.30 PSI * .068 = 1.24 ATA
It could be argued that treatment pressure for the other study locations were properly rounded up to 1.3 ATM (even though the actual pressures were quite likely to be considerably lower), however, even with all the calculations purposely leaned in favor of a higher number for Phoenix, Arizona, the study’s stated treatment pressure, there, should have properly rounded to 1.2 ATA! This suggests an overstatement of the added treatment pressure for the Phoenix location of 50% (.3 ATM is 150% of .2 ATM). Even if given the benefit of the doubt yet again, and an exception to proper rounding were made for solely for the Phoenix location in this study, the study’s likely overstatement in added treatment pressure for Phoenix is still a full 25%. (.3 ATM is 125% of .24 ATM – 25% more added pressure above 1 ATM was claimed in this paper, than was probably delivered).
I think this is a big enough boo-boo, that the editors of BMC Pediatrics should call for detailed errata. In the interest of scientific accuracy, it would seem prudent for BMC Pediatrics to:
1. Clarify for its readership and the scientific community, that the stated pressure of 1.3 ATM in this study is rounded up, and includes the ambient air pressure, or alternatively, state the estimated pressure in terms of ATA.
2. Clarify for its readership and the scientific community, that the stated pressure of 1.3 ATM in this study is an estimated pressure, since no actual measurements of ambient station pressure for the locations, and dates/times of treatments were reported.
3. Note for its readership and the scientific community, that the stated pressure of 1.3 ATM was not likely to be uniformly achievable across all study locations due to the use of inflatable hyperbaric chambers and changes in elevation (and atmospheric pressure) across study locations, potentially confounding the results of this study.
4. Note for its readership and the scientific community, that estimated pressures in the placebo control group are affected by these same issues that affect the treatment group, potentially confounding the results of this study further.
What do you think?
1 BMC Pediatrics 2009, 9:21doi:10.1186/1471-2431-9-21
3Federal Meteorological Handbook No. 1 – Table 11-2
4 LAT/LON 33.5º N 118.08º W
6 Medical device pre-market notification (FDA-cleared)
7 Manufacturer product sheet