For pattern and context, I would like toÂ recap Kirby’s forays into the world of autism epidemiology.Â They seem to haveÂ started in November, 2005, when Citizen Cain explained to him that “new cases” or “intakes” were not a meaningful epidemiological metric, but that instead, he should be looking at the 3-5 caseload to get an indication of a possible drop in autism incidence.
When 2007 came and Kirby’s predictionÂ of a drop in the California DDS 3-5 caseload famously failed,Â he apparently started to furiously scan various databases for any sign of a drop in “the numbers.” First, Â he suggested we might be seeing a drop in speech-language impairment in IDEA. But the drops he was seeing were easily shown to be nothing out of the ordinary. Additionally, IÂ found there were discrepancies between the IDEA data publicly available and the data he presented. Subsequently,Â David KirbyÂ emailed a messageÂ to the EOHarm mailing list announcing he had found 4 states in IDEA withÂ 3-5 caseload drops in autism between 2004 and 2005. Once again it was demonstrated that these drops were not unusual.
David Kirby never addressed these rebuttals. In fact, links to said rebuttals were not evenÂ allowed by whoever moderates messages in his blog at Huffington Post. But that is a different matter.
In his latest attempt at finding drops in the numbers, David Kirby tells us that while the California DDS 3-5 caseload is still rising as of Q2 2007, he obtained birth-year-cohort data that allowed him to look at 3 year olds in isolation.
But among the very youngest kids counted, the story was the opposite. At the end of June 2006, there were 688 children born in 2003 with autism diagnoses. This June, the number of kids born in 2004 with autism was 632, a statistically significant drop of 56 children, or 8.1 percent less than last year at this time.
I had sent a request to California DDS for this data, but I did not have to waitÂ for itÂ since a fellow blogger already had a copy. (Thanks.) I should note that I could not verify the 632 caseload number for June 2006, as the copy of the data I was given only ran to March 2006, but I’ll take David Kirby’s word for it.
IÂ naturally decided to put the data in a graph.Â The following represents the caseload ofÂ autistic children in California DDS whoÂ should have been 3 years of ageÂ by June of each year shown.
You see that drop between 2006 and 2007? That is what David Kirby found which he believes is significant. Let’s be honest here. Does that look like a significant drop between 2006 and 2007, or does the spike between 2005 and 2006 look more significant than that? Caseload growth between 2005 and 2007 is actually pretty steep and in line with the previous trend, isn’t it?
Clearly, that’s a pretty weak finding and it can’t really tell us anything. If caseload drops to 1995 levels or something of the sort,Â then we’d be discussingÂ something ofÂ interest. As things stand, 2007 has seen less autisticÂ 3 year-olds than 2006, but a lot more than 2005. So?Â
David Kirby, however, asserts it’s a “statistically significant” change. But I’m really not sure if this assertion is correct. Readers with a statistical background might be able to help with this one. If we assume a sample size of 540,000 (the population of 3 year-olds in California) and we use a simple confidence interval calculator (e.g. this one)Â the resultsÂ I get are:
- 2006: 688Â (95% CI 638 – 741)
- 2007: 632 (95% CI 584 – 683)
The confidence intervals (or error bars) overlap, so it does not appear that the change is statistically significant.Â But it’s possible David Kirby is using a different method to calculate the confidence intervals.
Data availableÂ is not sufficient to assert thatÂ the caseload of 3 year-olds served by California DDSÂ might have begun a downward trend or even a leveling-off trend. Furthermore, the claim that the caseload change observed between 2006 and 2007 is statistically significant requires further support and clarification.