Archive by Author

It’s different for girls

13 Apr

One of the puzzling things about autism has always been the disparity between the sexes.  Boys have always been more susceptible than girls. This is not in itself unusual. There are gender differences affecting a whole range of conditions and, if this New York Times article is correct, men frequently come off worse.

But if boys are more susceptible you might have assumed that as the severity of the condition increase this disparity would become more marked. In fact you would have assumed wrong. According to this source:

The greater severity and lower frequency of autism in females has been cited as evidence for a multifactorial polygenic mode of inheritance with differential loading by sex, which predicts greater severity in the less frequently affected sex.

Greater severity is usually taken to include severe cognitive impairment as well and the greater the degree of cognitive impairment the closer the ratio between boys and girls.  But there are problems with this model. David Skuse has argued that the association between cognitive impairment and autism is not because they share a common cause but simply because if you have both conditions you are more likely to be seen by a clinician and get a diagnosis. More able people may be just as autistic but have coping strategies that enable them to avoid a diagnosis.  And if girls have better coping strategies than boys they will be disproportionately overrepresented amongst autistic people without cognitive impairment who are missed by the system.

Last week Woman’s Hour broadcast a segment on Asperger syndrome took up this argument and suggested that there may be as many girls as boys on the spectrum. Most of them are not getting a diagnosis because they present in ways that are unfamiliar to clinicians who are used to seeing the condition in boys. The programme is no longer available but Sunday’s Observer carried a two page spread on the same story.

Experts like Judy Gould and Tony Attwood cited by the Observer still believe there is a gender difference but they estimate that it is only 2.5:1. Asperger girls may be more passive than boys. They do not assert their difference or draw attention to themselves. Instead they observe and copy other people’s behaviour. Their special interests may be intense but are also likely to be more socially acceptable; reading fiction, following soaps, celebrity culture – the sort of thing that lots of other girls do – and so they do not stand out.

Conformity comes at a cost. The Observer quotes Tony Attwood’s estimate that 20% of anorexic girls are undiagnosed autistics. Then there is self harm and other evidence of psychological stress. There are important differences between men and women. They need to be understood and respected. But it does not help autistic women if autism is described as an extreme male brain syndrome. The Observer ends by quoting professor David Skuse who believes that:

if we can prove the ratio of boys to girls is as high as many of us suspect, it would be as significant a milestone in this field as the discovery that the condition is on a spectrum.”

Tags: ,

The Somali Minnesotan Autism Epidemic

13 Apr

One of the most intriguing scenarios to pop up from the so-called autism epidemic happened in Minnesota in the US. It was noted that there were what seemed to be disproportionately high numbers of second generation Somali children in Minnesotan schoolrooms. Of course, this was immediately latched on to by the usual suspects, despite the caution issues by health authorities that until proper epidemiological studies had been done it would be impossible to say whether this really was a cluster worth investigation or just a coincidence.

On the 31st March, the Minnesota Dept Health released the study. Its a fairly substantial read. One thing immediately struck me about this:

Administrative prevalence of Somali children, ages 3 and 4, who participated in the MPS ECSE ASD programs was significantly higher than for children of other races or ethnic backgrounds

OK so this is what the families were saying. But there’s an extreme note of caution that should be noted here. This is basically CDDS all over again. Just like some believed that an increasing number of reports to CDDS meant that there was an autism epidemic and just as CDDS said there reports really shouldn’t be used to study these things, MDH are also saying:

Because of the study’s limitations, it is not proof that more Somali children have autism than other children…

This is a vital point. Back in 2005, James Laidler made the clear point that Department of Education data on autism are not reliable for tracking autism prevalence.

Sadly, the new Age of Autism editor, Abdulkadir Khalif, either misunderstood or elected to ignore this issue when he said:

It was obvious from the numbers that the issue of prevalence has finally been settled, and that there definitely is a cluster of autism in Minneapolis

It is clear from the report that Khalif has grossly overstated the case. Firstly, the issue of prevalence is far from settled. MDH seem to be solely using educational data which, as pointed out by Laidler, is not reliable for tracking autism prevalence. Indeed the phrase ‘administrative prevalence’ used by MDH reflects this. ‘Administrative prevalence’ refers solely to numbers of kids in educational programs. This is a clear distinction from ‘prevalence’ which is the proportion of individuals in a population who suffer from a defined disorder. Using only educational data gives a distorted picture.

As has been shown, the USDE data on autism are at odds with studies of autism prevalence, largely because the criteria used by the school districts (the source of the USDE data) to categorize children as autistic are neither rigorous nor consistent. They are inconsistent over time, as are the medical criteria, and are inconsistent from region to region. The USDE data are not reliable for tracking the prevalence of autism, and they in fact never were meant to fill this need.

Secondly Khalif uses the word ‘cluster’ whereas the MDH report does not use it at all. And it is not a word that should be used in such a throwaway fashion – it has a distinct epidemiological meaning.

So clearly, contrary to Khalif’s assertion that the issue of prevalence has been settled, it has not. Contrary to his statement that there is a cluster, there has been no such epidemiological assessment or statement.

Here are some more quotes from the MNH report that Abdulkadir Khalif either chose to ignore or never actually read:

The fact that a child is participating in an ASD early childhood program is an indicator of educational need, but that child may or may not have a medically diagnosed ASD.

Further, Minnesota’s public school open enrollment policy allows children to attend special education programs in school districts where they are not residents. This raised the question of whether participation rates for Somali children might appear higher than the participation rates for non-Somali children because of an influx of Somali children who are not residents of the Minneapolis school district attending MPS ECSE programs for ASD.

Data on variability of ASD prevalence by race, ethnicity, and SES is limited and inconclusive, and apparent differences between racial and ethnic populations may largely be due to differences in case finding and service provision.

Across all assumptions and ASD program types, administrative ASD prevalence estimates for Somali children were uniformly higher than the U.S. parental reported ASD prevalence, but most of the 95% confidence intervals corresponding to the administrative prevalence estimates for Somali children contained the value of the U.S. parental reported ASD prevalence estimate – suggesting that the 2005-2006 administrative ASD prevalence for Somali children might be no different from what would be expected in the U.S. population of children ages 3 and 4 based on parental report.

So what does this mean?

It means that there are no firm answers and that Khalif is simply wrong to assert that there are.

Its always been one of the great puzzles to me that a section of (mostly) parents who demand accurate answers fast cannot seem to understand that there _are_ no accurate answers until the science – proper science – has been done. And that takes time. What legacy do these parents want to leave the autism community? Fast inaccurate mistakes? Or well planned and rigorous science that helps build the growing knowledge we already have?

California tries to win back “stay put”

13 Apr

Parents, of course, have the primary responsibility to provide for the needs of their children. But, children and adults with special needs often require more than their parents can provide, and that’s where government agencies are supposed to help out.

In California, the Regional Centers manage government sponsored services for those with exceptional needs up to age 3. At age 3, the school districts take over as the lead agency, with the RC’s taking a secondary role.

That is, of course, if all goes according to plan.

What if a school district is swamped or for any other reason isn’t prepared to take over? What happens to the child’s services?

Up until a couple of years ago, the child would stay in the same services until the school district and parents could work out the new plan. This was called “stay put”. A family could say, “Hey, look, you aren’t prepared to take on my kid’s services in the school, so we are going to ‘stay put’ with the services we have until this gets worked out”. During ‘stay put’, the district pays for the same services the RC was paying for until age 3.

Then came a change to IDEA, the Individuals with Disabilities in Education Act. Stay put was lost.

Yeah, parents can pay out of pocket while going through a dispute process. That is, parents with the money to do so. The dispute process, at best, take a few months.

This didn’t happen without a lot of activism to try to keep “stay put”. So far, it hasn’t reinstated “stay put”, but there is hope. State Assembly Bill AB1124 is in process.

There is a hearing on this with the Education Committee, this Wednesday (April 15).

This is a proposed California Law, but, let’s face it, setting the precedent that stay put needs to be in place could help people throughout the US. For Californians, now would be a good time to contact your assembly person about this. If your assembly person is on the Education Committee, now is a REALLY good time to contact him/her.

Speaking clearly

12 Apr

The Canadian Journal of Medicine have an interesting article by MacDonald and Picard pleading for clearer language to be used by academics. In
particular they note the potential for different audiences to make
different conclusions from the 2001 Institute of Medicine report
looking at the relationship between MMR vaccine and autism.

The conclusion in the executive summary of the 2001 Institute of Medicine report about the relation between the measles–mumps–rubella vaccine and autism provides an excellent example of potentially confusing academic language. This conclusion was carefully crafted in precise academic language — so precise that, depending on the reader, different conclusions can be drawn. The academic involved in vaccine research, familiar with the scientific principle that the null hypothesis cannot be proven, would conclude that this report does not find evidence that the measles–mumps–rubella vaccine causes autism. A health care worker reading this same statement may not be sure what to make of it: maybe the vaccine causes autism in some children, just not often. The politician may wonder whether supporting public programs for measles–mumps–rubella vaccination at this time is justified. The journalist may reasonably interpret the conclusion as saying that the vaccine is a cause of autism, albeit an infrequent one. The antivaccine lobby, for its part, would be delighted that this respected academic body has given support to its claims that the vaccine can cause autism.

Go and have a read, and compare the actual IOM conclusion with
MacDonald and Picard’s proposed conclusion. Their arguments have
merit, particularly when it comes to the media interpretation of study or report findings. However, I’m not so sure about the anti-vaccine movement, who will deliberately misrepresent studies to prove the complete opposite of what they say, but we can at least stop giving them easy ammunition with which to dupe others.

Omnibus Expert: Patricia Rodier

10 Apr

Autism just plain isn’t mercury poisoning. When can we move on?

Even some of the people who loudly promoted the mistaken idea that “autism is just a misdiagnosis for mercury poisoning” have backed off. But, the groups that promote autism as vaccine injury are packrats: once they’ve collected an idea, bad or not, they won’t ever let it completely go.

Some of you will be thinking, dang, another mercury post. I agree, there are a lot of good arguments against blogging about the mercury-autism connection any more. For one, it gives the idea press that it just doesn’t deserve.

I do think this is worth posting about, though. “This” is the expert report from Dr. Patricia Rodier, submitted to the Autism Omnibus Proceeding. In a single document, we now have an expert on both mercury toxicology and autism. Not faux experts, or worse, businesspeople and public relations people, but an actual, bone fide expert in both fields. I.e. we have a good document to give to people who are being snowed under by the misinformation campaign promoting autism as mercury poisoning.

When Patricia Rodier testified in the Autism Omnibus Proceeding, I was very impressed–and I blogged it right away. I remember at the time telling a friend that it was good to finally see someone officially debunking things like Sally Bernard et al.’s paper, Autism: a novel form of mercury poisoning. My friend pointed out that any college freshman in science (and most not in science) should be able to tear that “paper” apart.

Unfortunately, “should be able to tear the paper apart” isn’t enough. Many people don’t have the time and/or energy. So, many people still think that paper is valid. Let’s face it, that “paper” should have been retracted by the authors long ago, but they still soldier on with the “autism is mercury poisoning” message.

Dr. Rodier’s qualifications are quite good. Her summary is quite good:

As a research scientist who has studied both the toxic effects of methylmercury in animals and autism in children and animal models, I believe I am qualified to evaluate the scientific merit of the allegation.

She may be the only person in the world who has studied both mercury toxicity and autism.

What does she think? In a nutshell:

My conclusion is that the allegation has no scientific support and is highly improbable

Dr. Rodier notes that the comparison that autism and mercury poisoning appear similar isn’t even close.

In othcr words, because the symptoms of methylmercury poisoning
are not similar to those of autism, the authors have tried to construct a new, hypothetical kind of mercury poisoning from symptoms of toxicity of other mercury species and symptoms never reported for any kind of mercury exposure. The hypothesis is not based of facts; instead, the facts are being selected, manipulated, and shaped to fit the hypothesis. The hypothesis is then offered as evidence. But hypotheses are not evidence.

Ouch. Ouch, that is, if you are someone promoting autism-as-mercury-posinong.

Dr. Rodier can back up her words, as we discussed in the previous blog post. But, let’s say that again, Dr. Rodier uses research based facts, not manipulated hypotheses, to come to her conclusion.

I need to get a clean copy of that document, one that looks as good as the information it contains. That document needs to get into the hands of people being lured by the pseudo scientists promoting autism as mercury poisoning.

Thank you, Dr. Rodier for putting yourself on the line to testify. Thanks also to the HHS for allowing these reports to be made public.

All change

9 Apr

A new design (more a realign than a redesign really) for LBRB and the alteration of some functionality.

The biggest change in functionality is that from now on you’ll be offered the chance to sign in to comment. In order to make this as easy as possible I’ve signed up with the RPX service which allows LB/RB to offer you several easy passwordless ways to comment. If you have a Facebook, GMail, Yahoo, Flickr etc etc account then you no longer need to do anything but click. This does not store your passwords anywhere, its simply an easy way for technology to talk to each other to make life easier for you the user. If you’re still not happy with that you can still elect to sign in anonymously. The biggest advantage for LB/RB for doing this is making the site harder to spam.

The biggest change in interface is I guess the ad placeholders. LB/RB has to bow to the inevitable and do all it can to get some money through the door. But please rest assured the ads will be vetted and will not feature anything contrary to the editorial direction of this site. Its the first time in its nigh on 6 years online adverts have been placed. A necessary evil I guess.

If you have a Gravatar your goods will be displayed 🙂 If you don’t – go get one!

Its been made easy for you to add LB/RB to various Social Networking apps and sites. If we’ve missed one out your want to see please contact us.

Efforts have been made to make the site work primarily on Firefox, IE6, 7 and 8, Opera, Chrome and Safari. There are still some very minor niggles in IE6 but mainly we’re good to go. I’m also aware that Archives aren’t available right now.

You might also note that there is no search box. There’s a reason for this 🙂 keep watching the top right column just above the block of four ads.

Go and play with it, you won’t break it and if you do or you find a bug please contact us.

Lies, Damned Lies and Science

7 Apr

Prior to starting a family I was already a skeptic and a subscriber to Skeptical Inquirer. This means I looked twice at the “helpful” information I got from all sides. From whether or not my baby would be a boy or a girl, what might have caused his seizures (like that fact that I drank milk, which was offered as an explanation after the “helpful” person was told he was infant only on breastmilk), the assurances I got that he would talk “when he was ready” or that someone’s cousin four times removed or Einstein did not talk until age three, five, or thirty-five (by the way, the Einstein is a total myth, don’t believe it — and don’t accept your child being compared to Einstein, he really had several flaws! See Private Lives of Albert Einstein, I should mention that sitting in therapy waiting rooms does provide lots of reading opportunities), and most recently I was told to try cranial sacral therapy (like a light head massage is going to “fix” the pathways in the brain that make him different!).

There is a tendency for people to give unsolicited advice to young parents, and that seems to double when it involves a child with a disability. Not only does the amount of advice double, but the relative distance from reality increases exponentially. While a parent of a typically developing child would be told to buy a certain organic baby food or to try some kind of “teach your baby calculus” computer program, the parent of child on a different developmental path would be encouraged to try a myriad of supplements, various odd treatments and to try the “miracle cure” they heard from some famous guy on the news.

How does a parent of a newly diagnosed child wade through the “help”, and determine what is real and what is hype? Well there is help, and it is not a cure all, but a book that shows how to look and science and separate fact from hype: Lies, Damned Lies and Science by Sherry Seethaler, a science writer and education at the University of California in San Diego..

It is not a long book, and is separated into short chunks to help explain the basics of science, why disputes in science is not really a bad thing, how to interpret numbers, and who the stakeholders on an issue are, and why they are important. She includes many real world examples and even comparisons to situations in the Harry Potter book where he excels at potion making by using the notes in the margins of an old book.

In Chapter 7, “Fun Figures”, there is a subsection titled “Ask whether a statistical change reflects reality or the way the data were collected.” Readers of this blog should be very familiar with the example she uses. They will also be familiar with the tactics described in Chapter 9, “All the Tricks in the Trade”, especially the section on pseudo experts.

This is a quick, actually quite a fun book to read. It can be a bit repetitive, but that is in part makes it easier to understand the concepts. I knew much of the information (like statistics), but I still learned a great deal. Check it out of your local library, and even purchase a copy for those friends and relatives who keep giving you all sorts of “advice.”

What is autism?

6 Apr

Consider a child who undergoes a traumatic event. This child develops “behavioral symptoms normally associated with the syndrome of infantile autism”. After treatment and time, the symptoms improve.

Did the child have autism?

It’s a pretty serious question, and not as simple as many think. Many think autism is defined merely as “a collection of symptoms”.

The question “what is autism” has been discussed a lot in the last year, both directly and indirectly. This was prompted in large part by the Hannah Poling concession, and often centered around the phrase “features of autism”.

Lisa Rudy at autism.about.com recently addressed this issue in a blog post: If It Looks Like Autism and It Acts Like Autism…

In her post she states:

But according to the diagnostic manual, unless the symptoms can be better explained by Retts disorder, Childhood Disintegrative Disorder or Schizophrenia – people with autism-like symptoms have an Autism Spectrum Disorder.

Well, yes and no. I’ll go into it because this bugs me somewhat, so indulge me while I explore this. There are three important facts that many leave out from the DSM-IV’s diagnostic criteria for autism.

First–the DSM-IV is a set of “diagnostic criteria”, not a definition of the disorders.

Second, consider this phrase in the diagnostic criteria:

Delays or abnormal functioning in at least one of the following areas, with onset prior to age 3 years: (1) social interaction, (2) language as used in social communication, or (3) symbolic or imaginative play.

Emphasis added.

The criteria of “before age 3” is very important. I’ll discuss this more below.

The third “fact” that people tend to ignore is related to the age 3 criterion:

The closest thing to a definition of Autism in the DSM-IV is as a “Pervasive Developmental Disorder”. Again, emphasis added.

Sorry for the sidetrack. But, there is a difference between the diagnostic criteria and the definition of what autism (or any condition) is.

However, this does not mean, as I have have asserted incorrectly in the past, that the fact that autism is a developmental disorder precludes giving an autism diagnosis to someone suffering from an injury. Developmental disorders include, for example, traumatic brain injury.

OK, now that we have given more depth to the “definition” of autism, what about that example I started this post with?

Consider a child who undergoes a traumatic event. This child develops “behavioral symptoms normally associated with the syndrome of infantile autism”. After treatment and time, the symptoms improve.

And, no, I wasn’t referring to Hannah Poling or to anyone you probably were thinking of. This example is from this paper, Acute onset of autistic features following brain damage in a ten-year-old . Here’s the abstract:

Abstract We report the case of a 10-year-old boy who, following a prolonged period of unconsciousness, displayed severe eye-to-eye gaze avoidance, sensory inattention, and some other behavioral symptoms normally associated with the syndrome of infantile autism. The symptoms lasted only a few months and were associated with the more permanent behavioral changes of post-encephalitic psychosis. Serial computerized tomography scans were taken during his illness and recovery. The relevance of this case to the etiology of infantile autism is discussed.

I’ll admit: I wasn’t able to get the paper from the library and I didn’t pay the $34 to download it. I’d really like to read this in full, because it raises some very interesting questions.

One big question arises from the fact that this child isn’t/wasn’t autistic. He had autistic features, but he wasn’t autistic. Why wasn’t he given an autism diagnosis? Well, for one big reason, the symptoms had an onset at age 10.

Now, here is where it gets interesting. Consider this hypothetical situation: What if a two year old had the exact same causes for his autistic behaviors as the 10 year old in this paper? Would the 2 year old be “autistic” while the 10 year old isn’t?

It’s worth taking a moment to think that one through. This is a big question.

If symptoms define one as autistic or not, then both the 10 year old and the hypothetical 2-year-old are autistic. But, if we include the criteria that onset of symptoms must occur before age 3, then only the 2 year old is autistic–even though the cause is the same.

Obviously this doesn’t make any sense. How do they solve this contradiction? Does the 10 year old get an autism diagnosis? Does the 2 year old not get a diagnosis?

As I noted above, this is a much more difficult question than most people (even self-designated experts) want to make it out to be.

More Hot Air about HBOT

6 Apr

A few weeks ago, BMC Pediatrics published an article that purports to show that Hyperbaric Oxygen Therapy (HBOT) can produce “…significant improvements in overall functioning, receptive language, social interaction, eye contact, and sensory/cognitive awareness..” in autistic children. This study (Rossignol et al, 2009) is billed as a “…multicenter, randomized, double-blind, controlled trial.”

It’s all that and much, much less.

Let’s start by looking at the six “centers” where this research was carried out.

The Centers

The International Child Development Research Center (ICRDC):

This imposing name is attached to a rather less imposing edifice. The ICRDC, brainchild Dr. Jeffrey Bradstreet, is located in a strip mall in Melbourne, Florida, where it not only carries out “cutting-edge research” but also sells a complete line of “supplements” and treats autistic children with a dizzying array of “alternative”, “biomedical” and “integrative” therapies, including HBOT.

Daniel Rossignol MD (Family Practice), Lanier Rossignol (Nurse Practitioner) and Scott Smith (Physician’s Assistant) were the authors from the ICDRC.

The Center for Autism Research and Education (CARE):

This “center” is located in Phoenix, Arizona and has – according to its website – a single practitioner, Cynthia Schneider, MD (OB/Gyn), who is also an author on this paper. One of the “integrative” therapies this “center” offers is HBOT.

One of the other authors, Sally Logerquist, is a PhD psychologist who – according to the paper – is also associated with CARE, but also appears to run social skills therapy groups for autistic children using the “Logerquist Excellent Attitude Program” (LEAP).

True Health Medical Center:

It’s rather difficult to find anything about this “center”, apart from the fact that it is located in Naperville, Illinois – in what appears to be an office complex. Anju Usman, MD (Family Practice) is the author associated with this location.

Neubrander Center:

Although not officially called a “center”, the office of James Neubrander, MD (Pathology) is apparently one of the “centers” of this study. His office is located in the Menlo Park Mall (near Macy’s) and offers – you guessed it! – HBOT as a treatment for autism.

Princess Anne Medical Associates:

A Family Practice medical group in Virginia Beach, Virginia, this “center” is the home of Eric Madren, MD (Family Practice). It’s not clear if this four-physician practice offers HBOT.

The Rimland Center for Integrative Medicine:

A small, one-physician “center” in Lynchburg, Virginia, this is practice location of author Elizabeth Mumper, MD (Pediatrics). Not surprisingly, this “center” sells HBOT services for autistic children.

So, of the six “centers” involved in this study, five are single-physician operations. The remaining “center” has two physicians (three, if you count the naturopath).

I’m underwhelmed.

Well, what about the research itself? Maybe that’s better than the “facilities” might suggest. Let’s take a look.

The Subjects

This study initially enrolled 62 children (33 treatment; 29 control), but only 29 of the treatment group and 26 of the control group finished all 40 sessions. For reasons that pass my understanding, one treatment subject who only finished 9 sessions was included in the analysis. The authors stated that including this subject did not alter results, which begs the question: “Why did they include this subject if it made no difference?”

Outcome measures

The authors used the Aberrant Behavior Checklist (ABC), the Clinical Global Impression (CGI) scale and the Autism Treatment Evaluation Checklist (ATEC) as their outcome measures. All except the ATEC are widely accepted for use in autism treatment trials.

The ABC is a 58-question checklist of – surprise! – aberrant behaviors which are each given a score from “0” (“not at all a problem”) to “3” (“severe problem”). This test has been use – and validated – in a number of disorders, including autism. It gives a global score as well as five subscales: a total of six measures.

The CGI is a generic rating scale used in a variety of clinical trials. For each parameter (e.g. “overall functioning”, “sleep pattern”), the rater gives a score of between “1” (“very much improved”) and “7” (“very much worse”). The authors had both the treating physician and the parents rate the subjects on overall improvement and eighteen discrete parameters: a total of 38 measures in all (19 by the physician and 19 by the parents).

The ATEC was developed by Bernie Rimland and Stephen Edelson and has not been validated. In fact, it has only been used in two published studies – one by Rossignol et al. The ATEC has 25 questions on which the evaluator rates the subject on either a three-point (“not true”, “somewhat true”, “very true”) or four-point (“not a problem”, “minor problem”, “moderate problem”, “serious problem”) scale. It provides a total score and four subscales: a total of five measures.

In all, each subject had a total of 49 evaluation measures (CGI scores and the change in ABC and ATEC scores), of which 47 are independent. The importance of this will become apparent in the section on statistical analysis.

Analysis

As I mentioned above, the decision to include one treatment subject who only completed nine sessions was curious. Why they included this subject and not any of the other three treatment subjects and three control subjects who also failed to complete the entire course of the study is concerning. The smart thing – and the proper response – would have been to drop this subject from analysis.

The authors’ method of analyzing the CGI scales was also curious. Rather than simply using the scores as they were provided, they took the scores and subtracted them from four (the “no change” score). There are a few problems with this.

For starters, the scores are not linear – the difference between “much improved” and “very much improved” is not necessarily the same as between “no change” and “minimally improved”. Nor is the difference between “no change” and “much improved” twice the difference between “much improved” and “very much improved”. For that reason, these types of numerical scores are often referred to as “pseudo-numbers”.

This may seem like nit-picking, but it is a serious concern. Imagine, if you will, that the numbers were replaced by colors. Is the difference between green and orange twice the difference between orange and red? If half of a population of birds are blue and the other half are yellow, is the “average” bird green? The simple fact is that it is not appropriate to treat these “scores” as though they were real numbers, to be added, subtracted and averaged.

Secondly, it appears that the authors used parametric statistics for their analysis of the CGI scores. This is a problem since – as I indicated above – it is nonsensical to do math on pseudo-numbers. I don’t have the raw numbers, so it isn’t possible for me to calculate the absolute impact of this mistake for all of the CGI subclasses, but I can figure out the raw numbers for one group, so let’s look at that one.

It took a little work, but the authors gave enough clues to tease out the raw numbers in the physician “overall functioning” CGI score. The treatment group had an “average” of 2.87 and the control group’s “average” was 3.62; using the unaltered data, a t-test [Note: not an appropriate use of the t-test] gives p-value of 0.0006, not far from what the authors report. When a more appropriate statistical test [Mann-Whitney U-test] is used, the p-value is 0.002, very different from the reported 0.0008. While this is still less than the threshold p-value of 0.05, see below for a discussion of multiple comparisons.

All of these statistical analyses of the CGI scores ignore the fact that these are pseudo-numbers and need to be treated as discrete groups rather than as actual numbers. In truth, even the ABC and ATEC scores should have been treated this way, as well, although it is fairly common practice to treat such multi-factor scores as real numbers. A Chi-square test or Fisher Exact test would be the ideal test, but the problem with that is that the treatment group has one score of “1” (very much improved) and the control group doesn’t. Likewise, the control group has two subjects with a score of “5” (minimally worse) and the treatment group has none. This prevents a Chi-square or Fisher test from comparing each score independently.

One solution is presented by the authors themselves, although they apparently didn’t use it. In their discussion of the CGI, the authors said:

“Children who received a score of ‘very much improved’ or ‘much improved’ on the physician CGI overall functioning score were considered to be ‘good responders’ to treatment.”

If we “bin” the scores into “good responders” and “others”, we find that there were 9 (out of 30 – 30%) “good responders” in the treatment group compared to 2 (out of 26 – 8%) in the control group. Unfortunately, this is not a statistically significant difference (p = 0.08) in the (Yates) Chi-square test and barely reached significance (p = 0.05, but see below) in the Fisher Exact test.

An even bigger problem in the statistical analysis was the failure to correct for multiple comparisons. This problem was brought up by one of the reviewers, and the authors responded by eliminating a table. They did not make the appropriate corrections.

The reason that multiple comparisons are a problem is that the analysis for statistical significance is based on probability. If the probability (the p-value) that the differences between the two groups (treatment and control) is due to random chance is equal to or less than 5%, that difference is considered to be “statistically significant” and accepted as real. That means that there is still a 5% (or less – look to the p-value) chance that the difference is due to chance and not real.

If multiple comparisons are made on the same group of subjects, the probability that one (or more) of them will be “statistically significant” by chance starts to climb. If 14 comparisons are made, the chance of an erroneous “statistical significance” is over 50%. If 47 independent comparisons are made – as in this study – the chance of an erroneous “statistical significance” is over 90%.

For this reason, it is standard procedure to apply a correction for multiple comparisons. The most well-known (and simplest) of these is the Bonferroni Correction, which changes the threshold for statistical significance by dividing it by the number of comparisons. In the case of this study, the threshold (normally p less than or equal to 0.05 or 5%) is reduced to 0.001.

Applying the appropriate correction for multiple comparisons changes the results of this study significantly. Only the physician CGI scores for overall functioning and receptive language reach significance – and these numbers are already suspicious because they were improperly handled to begin with. In fact, as I have shown above, the CGI “overall functioning” p-value wouldn’t reach significance. It is possible that – if the proper statistical tests were used – that the CGI score for “receptive language” would also not reach significance.

Another curious thing. The authors asked the parents after the study whether they thought their child was in the treatment or the control group. Rather than say that the parent’s guesses were no better than random chance (i.e. 50%), the authors stated:

“…there was no significant difference between the two groups in the ability of the parents to correctly guess the group assignment of their child.”

As I said, this was a curious way to put it. As I read this, all it says is that each group of parent were equally able to guess which group their child was assigned to. That could be a 50% accuracy (which would be equal to chance), but a 90% or 99% accuracy – if both groups were that accurate – would also fit that description.

Now, this could simply be an clumsy phrasing by the authors, or it could be a way to make it sound like their blinding was successful when it actually was not.

Summary

This study may have collected some useful data, but its analysis of that data rendered it useless. The CGI scores – where the only statistically significant result was (possibly) seen – were improperly manipulated and the wrong statistical analysis was used.

The other issue is that there is no discussion of why HBOT is thought to be superior to providing the same partial pressure of oxygen at room pressure. This study used 24% oxygen at 1.3 atm, which gives the same partial pressure of oxygen as 31% at sea level. This concentration of oxygen can be easily attained with an oxygen mask or simple oxygen tent – both of which are vastly less expensive than HBOT.

If the authors are arguing that the mild pressure of their inflatable HBOT chambers contributes to the treatment effect, they need to look at the literature on cell membrane compressibility. For those who want to do the calculations at home, the bulk modulus of water (the major component of cells) is 21,700 atm. This means that a 0.3 atm increase in pressure will reduce the cell volume by 0.0014%. The bulk modulus of the lipid bilayer in cell membranes is around 30,000 atm. This means that an increase of 0.3 atm pressure causes a 0.0010% reduction in membrane volume. These are well below the threshold for any clinical effects.

Real pressure effects on the central nervous system are seen at pressures over 19 atm. These effects are:

dizziness
nausea
vomiting
postural and intention tremors
fatigue and somnolence
myoclonic jerking
stomach cramps
decrease intellectual and psychomotor performance
poor sleep with nightmares
increased slow wave and decreased fast wave activity in EEG

None of these effects could be construed as “improvements”, even in autism.

So, this study fails to answer the following questions about HBOT and autism:

[1] Does HBOT improve any feature of autism?
[2] If so, is HBOT any better than supplemental oxygen (which is much cheaper)?

The only real effect of this study was to give a cover of legitimacy to practitioners who are already using HBOT to “treat” autism.

Prometheus

UK Autism Postcode Lottery

5 Apr

Nick Hornby is in the Telegraph today talking about the so-called ‘postcode lottery’. He talks of how it existed when his son was diagnosed, it definitely existed when my child was diagnosed and from my experiences talking to new autism parents it still exists today.

For those unfamiliar with the phrase it literally means that depending on where you live you may get good/bad/awful/no services. This is, as Hornby states, a national scandal. I’m aware of a location a couple of postcodes away from me in Stoke-on-Trent where the local health authority even go so far as to dissuade GP’s from making referrals to local CDC’s (Child Development Centers). Stoke has a reputation within the UK as being one of the worst, most violent and poor areas in the UK and I guess the local authorities see their financial priorities as being elsewhere.

Just to illustrate that, here’s a comparison of the West Midlands region of the UK according to NAS. Stoke – a large city – gets 27 known resources. Birmingham, also a large city less than 50 miles away gets 109.

And when we drill down deeper we find that Stoke has just one resource for assessment/diagnosis, compared to Birmingham that has four. No surprise autistic people and their families in Stoke struggle.

This is a crazy situation – how are we ever supposed to get a firm fix on prevalence and/or incidence of autism aside from anything else? Something needs to change in the UK and soon.

← Older Entries
Newer Entries →