Is the rate of autism recovery going down with time?

2 Sep

There is evidence that autism recovery is real. At least that is what we are being told on blogs based on data from the latest “National Survey of Children’s Health (NSCH)“.

We’ve already discussed some of the misinterpretations of the NSCH dataset on this site. I had a long post about this question prepared, but let’s just cut to the chase. If biomedical interventions are resulting in more kids recovering from autism, the “recovery rate” would go up with time. Kids born in the early 1990’s, before ideas like chelation and special diets were popular, wouldn’t be recovered at the same rate as kids born in, say, the last 10 years.

This just isn’t the case.  I graphed the “recovery rate”.  Take the number of kids whose parents were told that the kid had autism minus the number where the parents are reporting the child does not presently have autism.  Divide by the total number of kids whose parents were told the kid had autism.  Show as a percentage (click on graph below).

"recovery" rate from the NSCH data

"recovery" rate from the NSCH data

The “recovery rate” is going down. The rate was about 40% for kids born in 1990 but has dropped to below 30% for kids born in 2004.

Note that there is a high “recovery” rate datapoint for birth year 2005. Those kids were 2 years old for the survey and there were very few of them (only 15 kids total compared to about 90 for most birth years). I wouldn’t try to draw any conclusions from that point.

But, what’s the bottom line? The NSCH data don’t support the concept that introduction of “biomedical interventions” are “recovering” kids with autism. If you really wanted to take the data at face value, you would have to say exactly the opposite: the recovery rate has gone down with the growth of “biomed”. I don’t buy that. The easiest explanation is this: older kids have had more time for some medical person to say, “he/she might be autistic”.

31 Responses to “Is the rate of autism recovery going down with time?”

  1. jillteller September 2, 2009 at 07:45 #

    Somehow I doubt your analysis would hold up to scrutiny. Why don’t you try publishing your “news-breaking” results in a peer-reviewed journal and see hwo it fairs?

    • Sullivan September 2, 2009 at 16:15 #

      Jilteller, farmwifeto,

      You do realize that the point of this excersize is precisely to show that others have been twisting these data to “prove” alternative medicine is “recovering” autistic children, don’t you?

      You aren’t angry at the people who promoted the falsehood. Instead you show annoyance with the person who showed that the data don’t support the alt-med recovery notion.

      Somehow I doubt your analysis would hold up to scrutiny. Why don’t you try publishing your “news-breaking” results in a peer-reviewed journal and see hwo it fairs?

      I guess if I could find some heavily biased journal with extremely low standards (like JPANDS, except without the anti-vaccine bias) I could submit this and more. If I paid money I could get it into Medical Hypotheses. I guess iI could do that iff my intent was to deceive people that these “results” are meaningful rather than make the simple statement I hope I have made:

      The NSCH data are not able to show “recovery”. They just aren’t.

      The people pushing the idea that the NSCH data show “recovery” are either sloppy, selling something or both.

  2. David N. Brown September 2, 2009 at 09:44 #

    No biomedical intervention has stood up to sustained scientific peer review. So why throw stones?

  3. farmwifetwo September 2, 2009 at 12:29 #

    Wow!! High science!!! Pick and choose information and twist it to fit your hypothesis… Oh wait… the “real” scientists do it all the time.

  4. K September 2, 2009 at 15:21 #

    OT: Sullivan, this video is worth a blog post all its own: http://www.youtube.com/watch?v=3zEXgQIYCWU&feature=related

  5. Dan Hollenbeck September 2, 2009 at 15:22 #

    Hi Sullivan,

    I assume all data points are from a single survey year (i.e. NSCH 2007). This should be clarified on the graph otherwise someone might assume it is a time series graph. I have been criticized for this in the past.

    How do you control for the age of the cohort in your analysis?

    The older cohorts have had a much longer time to lose their diagnosis than say the 2004 birth cohort.

    How do you control for average age of diagnosis?

    Granted over time the average age of an ASD diagnosis has decreased, but there are a number of birth cohorts in the graph that have not reached the average age of diagnosis yet. I estimate this to be true for the 2005 (2 years old at time of survey), 2004 (3 years old), and 2003 (4 years old) cohorts. If these cohorts have not yet reached the average age of diagnosis isn’t premature to analyze if they lost that diagnosis?

    Is the NSCH dataset the right tool for the job?

    Each research instrument (NSCH, IDEA, ADDM) has its strengths and weaknesses and therefore different utility. I would be interested in a detailed analysis of these three instruments, perhaps others, strengths and weaknesses and reasonable utility. For example, what is the expected utility of the NSCH dataset that is a sampled phone survey of parents that does not ask the same questions from survey to survey?

    Thanks, Dan
    Dan Hollenbeck

    • Sullivan September 2, 2009 at 23:41 #

      Dan Hollenbek,

      I’ll start with your last question “Is the NSCH dataset the right tool for the job?”

      If the job is to look for autism recovery, the answer is a clear “no”. I would never have called this a “recovery” rate. That was, as far as I can see, a concept pushed by David Kirby. It was a bad idea to use that term. Before pushing that term on the community, he should have checked if the data were self-consistent. This is one of the most obvious self consistency tests to do–and it clearly shows that these data are not showing “recovery”.

      Frankly, the main self consistency check is the idea that ~40% of kids are “recovering” should have been a warning sign. I have not seen anyone claim such high numbers, even the most enthusiastic biomed based doctor.

      Mr. Kirby made a comment “Another item that will surely spark fiery debate is the reason why so many children previously indentified with ASD are currently not holding that diagnosis.”

      He obviously either didn’t read the questions in the survey or didn’t understand them. The questions were not whether these children ever had actual diagnoses of autism.

      Or, if anyone is still confused–the recovery rate is not going down. There is no recovery rate in the NSCH data. If anyone wants to claim these data as a “recovery” rate, they can try to explain why the rate is going down.

  6. David N. Brown September 2, 2009 at 17:44 #

    The oddity of this graph is that it itshows a peak and a dropoff every 4 years or so. Probably reflects cyclical “overdiagnosis”.

  7. Laurentius Rex September 2, 2009 at 22:55 #

    It is all a joke, a meaningless construct, an artefact of the times, which will in historical terms appear ridiculous.

    Wittgenstein summed it all up because it is ultimately all a language game, a matter of semantics, what you mean by terms which are not universally shared but only negotiated, and badly at that.

    • Sullivan September 2, 2009 at 23:23 #

      Laurentius Rex,

      spot on analysis. I’m glad someone saw it.

  8. MJ September 2, 2009 at 23:14 #

    You know that your graph is actually showing that the recovery rate is going up as the children get older. Which means as time goes by (for them) it is more likely that they will “recover” and lose their label. So in reality your chart is showing the exact opposite thing to what you are saying.

    Or in simple terms, as time goes by for a child they are MORE likely to recover, not less.

    And given the fact that most biomedical treatments can take a long time to work doesn’t that mean that your data is actually showing that it is plausible that these sorts of treatments can make a difference over the long haul?

    • Sullivan September 2, 2009 at 23:22 #

      MJ,

      I refer you to Laurentius Rex’s comment.

      There is no evidence of recovery in the NSCH data. It was all a misinterpretation pushed by people who took an incredibly superficial look at the data.

      This graph shows that their assertion doesn’t work even within the NSCH dataset. The idea that these data show biomed came along and started recovering kids was wishful thinking.

      I appreciate your comment as it demonstrates very clearly how much people will push and push and push to make data fit their hypothesis.

  9. MJ September 3, 2009 at 00:34 #

    Sullivan, you do realize the irony in your comments, don’t you?

    You just get done putting together a chart of the “recovery rate” then say that there is no evidence of recovery in the data. So what exactly did you chart then?

    You arrange your chart from older children to younger and say that this non-existent recovery is clearly decreasing over time so the children must be aging backwards as they get older for their imaginary rate of recovery to be going down over time?

    Or should they be listed as recovered before they get the diagnosis?

    Or should then be diagnosed and then recovered in the same year?

    How exactly do you think that your chart shows anything at all with time going backwards and showing something that doesn’t exist?

    I humbly submit that you are in fact the one who will “push and push and push to make data fit their hypothesis”.

    • Sullivan September 3, 2009 at 01:40 #

      Sullivan, you do realize the irony in your comments, don’t you?
      You just get done putting together a chart of the “recovery rate” then say that there is no evidence of recovery in the data. So what exactly did you chart then?

      Absolutely, I understand the irony. I wrote the post to be ironic. I don’t know if it is irony or something else that you haven’t caught on yet.

      The entire post is a commentary on those who push the NCHS data as proof of “recovery”. Any who wish to assert that the dataset shows “recovery” also have to accept that the “recovery rate” is going down with time.

      I’ve already discussed the fact that the idea of trying to pull a “recovery rate” out of these data is a bad move. Take a look at the post–the supposedly “recovered” kids are kids with other disabilities. They aren’t “recovered”.

      Did you read the last paragraph of this post? Just take this one line:

      But, what’s the bottom line? The NSCH data don’t support the concept that introduction of “biomedical interventions” are “recovering” kids with autism.

      Did you catch my earlier comment

      You do realize that the point of this excersize is precisely to show that others have been twisting these data to “prove” alternative medicine is “recovering” autistic children, don’t you?

      The NSCH data are not able to show “recovery”. They just aren’t

      I hope you catch on soon.

  10. MJ September 3, 2009 at 01:55 #

    “Any who wish to assert that the dataset shows “recovery” also have to accept that the “recovery rate” is going down with time.”

    No, by your own chart it is going up with time – time being defined properly as based on the age of the child getting older, not younger. Or maybe you could explain how time is supposed to move backwards.

    Or better yet how all of the children in your chart were inexplicably “recovered” the year that they were born?

    “I hope you catch on soon.”

    So you purposely did a completely invalid analysis, complete with a chart going backwards in time showing imaginary numbers, to prove the point that other people who did a similar thing were wrong.

    But somehow your own twisting of the data proves that everyone else was in fact twisting the the data because your own twisting has an accurate conclusion?

    Did I miss anything?

    • Sullivan September 3, 2009 at 05:12 #

      Did I miss anything?

      yes.

      Sorry to be terse, but, yes. You not only missed a lot, but you are now compounding it with some very strange mistakes. I recall your data analysis methods from before, so I am not really surprised.

  11. Prometheus September 3, 2009 at 05:21 #

    Sullivan,

    In reference to your statement:

    I have not seen anyone claim such high numbers, even the most enthusiastic biomed based doctor.

    Bernie Rimland, at the peak of the secretin frenzy, claimed that “up to 70%” of autistic children were “recovering” with the use of secretin. Studies later showed that secretin was no better than placebo, although it is still being used by some “biomedical” practitioners. So much for the estimates of “biomedical intervention” enthusiasts.

    Also, I think that you were a bit too subtle in your delivery with this post. There appear to be some people who don’t “get” that the data are meaningless and that any attempt to derive meaning from them is futile. You might as well try to pull messages out of radio static (actually, there are people who claim to do just that).

    This is similar to the appalling Holmes et al (2003) study that showed autistic children had lower hair mercury than the non-autistic controls. Clearly, the best interpretation of this “data” was that mercury had no relation to autism (or that the lab data was rubbish). A distant second place finisher would be the conclusion that mercury protected children from autism.

    However, since the authors had previously concluded (before the study) that mercury caused autism, they were forced to invent their implausible (risible, actually) “poor excretor” hypothesis.
    Likewise, since the “biomedical intervention” promoters had already concluded (in the absence of data) that their “interventions” were “recovering” autistic children, they interpreted the NSCH results in that light – apparently without actually looking at the data.

    In reality, the best interpretation of the NSCH results – as regards “recovery” from autism – is that children with autism continue to develop and some children diagnosed with autism early on (often by less-than-rigorous practitioners) will eventually “lose” that diagnosis through either development or better evaluation.

    This interpretation even explains the “blip” at the 2005 birth cohort – these children were “diagnosed” too early and later evaluations didn’t bear out the earlier diagnosis. Think about it – these children were two years old at the time of the survey and a few of them had already “lost” the autism diagnosis. There is a study (I’ll look for the citation tomorrow, if anyone wants it) that shows younger age at diagnosis is associated with a higher “loss of diagnosis”. Apparentely, if you try to make the diagnosis of autism too soon, you get more “false positives” (make the diagnosis of autism in the absence of autism).

    Prometheus

  12. Prometheus September 3, 2009 at 05:42 #

    MJ comments:

    But somehow your own twisting of the data proves that everyone else was in fact twisting the the data because your own twisting has an accurate conclusion?

    Did I miss anything?

    Maybe I’m the one missing something. Sullivan did a straightforward graph of the NSCH dataset and MJ is accusing him of “twisting” the data? His graph is something the “biomedical intervention” promoters should have done prior to claiming the NSCH data supported their claims of “recovery”.

    Let’s look at the timeline here:

    “Biomedical interventions” in autism really didn’t start getting widespread attention until the middle 1990’s (DAN! was formed in 1995), so the 1991 cohort would have been 4 years old by the time “biomedical interventions” were getting any attention and about 9 or so years old by the time they were widely known – yet they had the highest “recovery” rate.

    In contrast, the 2000 cohort was born in the middle of the “biomedical intervention” craze and would have been exposed to all of the most “intensive” (i.e. dangerous) “interventions”, yet they have one of the lowest “recovery” rates despite being 7 years old at the time of the survey.

    Clearly, this doesn’t look good for “biomedical intervention”. Fortunately for the “biomed” community, the data are not valid for this sort of comparison. They should be happy that Sullivan is exposing the weakness of the NSCH data.

    In reality, the NSCH data show only that autism is a condition of developmental delay, not developmental stasis. Autistic children will continue to develop over time – some slower, some faster – and some (but not all) will eventually “grow out” of their autism (into Asperger’s syndrome, perhaps, or into a mathematics professorship).

    Prometheus

    • Sullivan September 3, 2009 at 19:35 #

      Prometheus:

      “some (but not all) will eventually “grow out” of their autism (into Asperger’s syndrome, perhaps, or into a mathematics professorship).”

      In an earlier post, I showed that many of the kids who lost their autism label are still in special education (97% still have some support in school). Many are still

      A few (of the many) things that we don’t know about these kids at all is

      a) did they ever have a real diagnosis of a PDD? The way the question is worded, a dentist or a chiropracter could have said, “your kid is autistic” and that would have been counted.

      b) whatever way the autism “label” was applied, we don’t know when it was given. A lot of people are assuming a kid was diagnosed PDD and then “lost” the diagnosis. A kid could have been diagnosed with developmental delay and later a doctor could have said, “He/she might be autistic, go see a psychologist”, whereupon the psychologist said, “no, this child doesn’t fit the criteria”.

      Also, I think that you were a bit too subtle in your delivery with this post.

      Oh, I think it is safe to say that this post was not well written at all.

  13. David N. Brown September 3, 2009 at 10:35 #

    Something I would like to hear thoughts on: How do we explain an apparent “four-year cycle” in diagnoses? My idea, posted as a comment on the previous article, is that clusters of new diagnoses (right or long) happen when children go through educational/social transitions, like elementary to missle school. Anyone have other ideas?

    • Sullivan September 3, 2009 at 19:26 #

      Something I would like to hear thoughts on: How do we explain an apparent “four-year cycle” in diagnoses? My idea, posted as a comment on the previous article, is that clusters of new diagnoses (right or long) happen when children go through educational/social transitions, like elementary to missle school. Anyone have other ideas?

      First–you have pretty good pattern recognition!

      If the pattern is real, I wouldn’t think this would be due to transitions from one school to another (say elementary to middle school). For that, I would expect a step change rather than a dip. Take the idea that, say, kids are pushed out of special ed or pushed out of the autism category on transitioning to a new school. One would observe a drop in autism “rate” and it wouldn’t come back.

      I don’t think I’ve ever seen something like that in the special ed data. I wouldn’t put too much emphasis on these NSCH data for that.

  14. Dan Hollenbeck September 3, 2009 at 15:26 #

    David N. Brown,

    clusters of new diagnoses (right or long) happen when children go through educational/social transitions

    Some states most definitely have age related educational policies that affect which IDEA eligibility/disability category children with autism are counted under for specific ages. The most notable state is NJ which according to ADDM and IDEA data has one of the highest autism prevalence and autism administrative prevalence in the country.

    NJ appears to limit, restrict, or does not completely count the eligible special education children in the autism category for children under the age of 6 years old. For ages 3 and 4 they have recorded zero children in the autism category for the last 8 years of data. For the 5 year olds they appear to count them in some years and not in others.

    http://tinyurl.com/mn98tw

    Please note that I am not claiming that NJ does not serve these children in the special education system, but rather they seem to count them differently.

    http://tinyurl.com/krz8sm

    These graphs are time series graphs looking at specific age cohorts. Each line represents how the administrative prevalence has changed over time for a specific age (i.e. time trend).

    Thanks, Dan
    Dan Hollenbeck

  15. Laurentius Rex September 3, 2009 at 18:37 #

    Well unfortunately no-one seems to recover from looking at autism in the wrong way.

    To speak of recovery from autism doesn’t make any sense whatever, it is like speaking of a high school graduate having recovered from there pre school ignorance of all matters they are now qualified in.

    Autism is not a measurable concept, which lies at the root of all the problems of perspective.

    I don’t care about gold standard diagnostic schedules and all that stuff, it might make for some internal reliability in research but it is does not make for accuracy necessarily as at the end all that is being counted is a subjective interpretation of a set of behaviours that have arbitrarily been tagged with a so called “medical” diagnosis, diagnosis not even being a fact but if one studies the notion in itself, diagnosis is an (informed) opinion no more. It is always subject to flux.

    The time has come to get rid forever of any medical conception of autism and to put it where it belongs in the social and educational sphere (or the cultural, for those who see value in that)

    Progress will not be made through medications, progress can only be made through education, it is education that modifies the pathways in the brain more than anything else.

    Now let us scan my brain to discover that difference between before I could ride a bike, and after, that is the degree of subtley that is needed before one can even talk about measuring autism.

  16. dr treg September 3, 2009 at 19:32 #

    re. the three year cycle
    Mycoplasma infections seem to occur in 4-8 year cycles.
    http://www.humanillnesses.com/Infectious-Diseases-He-My/Mycoplasma-Infections.html
    Patients with autism have a higher incidence of history of mycoplasma infection.
    http://autism.healingthresholds.com/research/evidence-for-mycoplasma-ssp-chla
    Perhaps mycoplasma may be the immunogen in some cases of autism.

  17. David N. Brown September 3, 2009 at 21:53 #

    Sullivan,
    Now that you mention it, it’s beyond dispute that schools are under a lot of pressure to reduce the number of students in dedicated special ed programs. Challenging earlier diagnoses is one way this might be achieved. However, local policies might cause some traffic the other direction, as in prometheus’s example. Parents not savvy about school policies might wrongly infer “recovery” simply because their child was moved out of special ed. I suppose another kind of transition that might be in play is parents moving. It would be of interest to see if clusters correspond to economic downturns, which would be a major contributor to mass migration.

  18. Sullivan September 3, 2009 at 23:19 #

    “Parents not savvy about school policies might wrongly infer “recovery” simply because their child was moved out of special ed.”

    Or moved from an expensive Autism program to a cheaper special day class.

  19. MJ September 4, 2009 at 01:06 #

    Sullivan – I am assuming that you are reverting to being “terse” again that you really don’t have an answer for the question that I am asking. You say I am making “strange mistakes” but I think you are really missing the point of what I am saying, so let me spell it out.

    1. You have no data on the age of the child when they were diagnosed with autism.
    2. You have no data on the age on the child when then were “recovered”.

    Therefore your putting the data in a chart that shows the “recovery rate” per year of birth and implying it shows a trend in linear time is completely meaningless and arbitrary. The only time based factor that you have is the age of the child and the year that they were born and nothing else.

    So for you to say that as time moves forward the “recovery” rate goes down is completely without basis in fact. You are misrepresenting what the “time” axis of your chart represents.

    The only element of time in your chart is the age of the child and the only thing that shows is that as the children get older they have a higher “recovery rate”. This implies that as their personal time goes forward (younger to older) they have a higher, not lower, chance of “recovery”.

    But as you like to point out the data is not robust enough to make ANY meaningful conclusions either for or against the theory that biomedical interventions are “recovering” autism.

    So for you to say that the “data don’t support the concept that introduction of “biomedical interventions” are “recovering” kids with autism” is completely wrong – the data says nothing at all about the question therefore the proper response is to say exactly that – no conclusion can be drawn.

    You may say that you did the comparison just to prove that these comparisons are meaningless but that is just a load of crap. If you are attempting to disprove someone else you cannot do a completely invalid analysis and then say it “proves” that others are misusing the data as well.

    That is the same as saying, see I can do a bad job therefore they must have done one too. Logically that does not follow.

    So yes Prometheus, Sullivan is twisting the data and his analysis is anything but straightforward.

    Also Prometheus I think you are mistaking when “biomedical” interventions in autism got their start. I believe that they have been around and in use since at least the early 80s but I would have to go back and dig up some of the earlier research on the subject to see when it first showed up. It has grown over time and is still growing so 2000 was not the middle of any “craze”.

  20. Sullivan September 4, 2009 at 01:34 #

    MJ,

    I am terse because I have answered your questions multiple times already.

  21. MJ September 4, 2009 at 03:09 #

    Sullivan, I must be blind, please show me where you have answered my concerns about your misuse of time.

  22. Mike Stanton September 4, 2009 at 22:58 #

    MJ
    this is a rerun of a longstanding argument about the value of both administrative data and parental surveys in providing reliable datasets for studying trends in autism.

    As I understand it those who believe there has been an autism epidemic frequently cite administrative data, particularly from California, but from other states as well to prove their point, even though we have studies to show that these statistics are an unreliable guide to time trends in autism incidence. In the case of the California data, the authors of the reports I have read go so far as to state that their data should not be used in this way.

    Those who believe there has been an epidemic also rely on parental reports to justify claims of recovery using biomedical metohds. Parent reports can be unreliable as well. Does recovered mean they no longer have a diagnosis? Have they replaced one diagnosis with another? Are all parents using the same objective measures and definition of terms when they answer this question? Is there clinical data to support their answers?

    Like Sullivan I believe that these problems with the quality of data you get from administrative data and parental reports make it difficult to draw any conclusions about the efficacy of biomedical treatments for autism. To prove his point Sullivan has shown that the data that is being used in some quarters to “prove” that biomedical interventions are working lends itself equally to the hypothesis that, despite the increase in biomedical treatments, the rate of recovery is going down.

    He is not arguing that this is indeed the case.

    His whole point is that if the same data can be used to demonstrate two diametrically opposed positions, it is, in the context of our debate, useless and no conclusions can be drawn either way from this data regarding autism and recovery.

  23. MJ September 5, 2009 at 15:38 #

    Mike, As this is the third or fourth post so that Sullivan has done with this survey and used it to “prove” his points I think we are long past the point that is he just using it show that the analysis that others did was invalid – he is using it in an attempt to prove his own points.

    But in this is really besides the point. Even granting that the data shouldn’t be used this way his analysis isn’t even internally consistent. The conclusions that he is stating isn’t even supported by his own data.

Leave a Reply to Laurentius Rex Cancel reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: