Rethinking Expertise

16 Sep

I got an email recently with a link to this blog post by another Sullivan, Andrew Sullivan, of the Atlantic. Mr. Sullivan pulled a section out of this interview by New Scientist managing editor Greg Ross of Harry Collins. They are discussing the new book, Rethinking Expertise.

I would say that the danger to democracy that my own discipline—social studies of science—is not doing enough to combat is the collapse of the idea of expertise. Current social studies of science has difficulty with the notion of expertise. The attitude that anyone’s opinion on any topic is equally valuable could spread, and there are some indications, such as widespread vaccine scares, that suggest it is happening. A world in which there is said to be no difference between those who know what they are talking about and those who don’t is not one that anyone who thinks about it wants. Such a society would be like one’s worst nightmare, exhibiting many of the characteristics of the most vile epochs of human history.

On first read I wanted to say thank you to Prof. Collins for leaving the “autism community” out of the picture. But, this statement rings too true to think that we he wasn’t just being polite: “The attitude that anyone’s opinion on any topic is equally valuable could spread, and there are some indications, such as widespread vaccine scares, that suggest it is happening”? It’s just too accurate a description of the vaccine-rejectionists in the greater autism community.

It’s worth looking at the interview itself. One paragraph that caught my eye is quoted below:

Expertise is important not only within science but also for understanding the public’s relationship with science. Nowadays any parent of a young child, or anyone who can access the Internet, thinks their opinions on technical matters are sound. Many of my colleagues in the social sciences seem to think the same thing. The trouble is that the speed of politics is faster than the speed of scientific consensus formation, so politicians are often faced with making decisions without firm scientific answers to lean on, and this makes science look like anyone else’s opinion. I found I wanted to work out how to value expertise without going back to the bad old days where anyone in a white coat was treated as an authority on anything scientific or technological. We have to solve the very hard problem of reconstructing the value of science when we know it can’t deliver the certainty that people want. Studying expertise may do the trick.

“Nowadays any parent of a young child, or anyone who can access the Internet, thinks their opinions on technical matters are sound.”

Ouch. There are just too many examples of that, just in recent days, that picking any one out is difficult.

What Prof. Collins is proposing is interesting: a way to quantify expertise. I doubt it will happen soon, so we can expect more of the same false expertise. Frankly, even if we could quantify expertise, the same psychology at play will keep people from accepting it.

Seriously, if we could put a real number on expertise, would it matter? As a whole, society puts more credence behind the person sitting on the couch on a talk show than the person who spent years at the bench in a laboratory. Do I need a number to quantify the levels of expertise?

Thank god research is decided through peer review and not the talk show circuit. (If you think peer review of research proposals is a good thing, and you want to keep it that way, contact the IACC).

But, the takeaway for me on this is scary: if vaccine rejectionists are being used as examples of false expertise, how far away is the “autism community”? Let’s face it, the outside world doesn’t see the various communities that make up the greater autism community. And, the loudest voices of the “autism community” are the vaccine rejectionists.

About these ads

11 Responses to “Rethinking Expertise”

  1. Kev September 16, 2008 at 14:31 #

    It’s a very difficult thing. I’m signed up as an ‘expert’ on Opposing Views. Does that mean I’m a science expert? Hardly. But what does it mean? An expert in being experienced in the debate?

  2. alyric September 16, 2008 at 16:37 #

    Wow, what a fiction! So interactional ‘expertise’ is a worthy substitute for years of hard work and real scientific expertise? I don’t think so. This is standard social science humbug and darned obvious. Honestly – can’t do the maths, or design the papraphernalia and still wants to be considered an expert?

  3. Joseph September 16, 2008 at 17:21 #

    I try to stay away from those types of from-authority arguments, because they are fallacious for one, and because they do sound elitist.

    Did you know that in my profession (and Kev probably can relate) you can have all sorts of qualifications and experience and still not be considered an expert? I could go to an interview, explain that I have 10 years of experience coding, successful open source projects that I’ve founded, a MS in CS, published papers, and so forth, and they would still give me a number of problems to solve in order to prove that I can write code at a very basic level.

    In other areas, it’s sufficient to note that you’re a doctor or a PhD, and you’re immediately considered an expert.

    I’m not sure I have a point. I guess it’s better to demonstrate that you’re an expert by showing that you have a handle on the subject matter, rather than through qualifications.

  4. Rick Young September 16, 2008 at 17:55 #

    It’s a scary thing when anyone with a computer and the internet can be considered an expert. Check out “stunning new link between vaccines and autism rates” on the AoA website for a prime example. Here is a “stay-at-home” mom who has done her research almost exclusively on the internet, now quoted as an expert? Apparently she has discovered something none of the myriad doctors or scientists working in the field have considered!

  5. Joseph September 16, 2008 at 18:17 #

    Here is a “stay-at-home” mom who has done her research almost exclusively on the internet, now quoted as an expert? Apparently she has discovered something none of the myriad doctors or scientists working in the field have considered!

    But you see, the problem is not that she’s just a mom. Someone could say the same thing about Kathleen Seidel, for example (and they have, sort of).

    The problem is that she doesn’t understand coincidental trends or how to control for them. If she demonstrated statistical expertise, who cares if she’s just a mom?

  6. Sullivan September 16, 2008 at 18:55 #

    The question of how to quantify expertise is an extremely tough one. With few exceptions, the actual experts are not fighting the misinformation spread on the internet.

    Take the most obvious example: Dr. Offit. When it comes to vaccines, there is a world class expert. He’s willing to step forward and point out the failings of the “False Prophets.” This will have an effect, but the wagons were circled well in advance to defend against this. The same failings of the “anyone with an internet connection is an expert” notion was used to get some people to discount his new book (and whatever he says).

    Take the obvious counter-example: Mr. Dan Olmsted. His recent posts show (a) an active participation in the “circle the wagons” campaign and (b) a demonstration of the sort of “expertise” I see to often in the “internet expert” phenomenon. He started with a hypothesis: occupational exposure to toxins leads to autistic offspring. He dug just deep enough to find support for the thesis and hit “publish” on his post. He did this before with another blogger. It is the same method used by David Kirby and others with the CDDS data–dig until you find support for your thesis, then stop.

    Real science challenges the conclusion. That’s a big part of peer-review: put the paper in front of experts who know which questions to ask to challenge the conclusions.

    But, how is a person supposed to separate the wheat from the chaff? How is the average person supposed to know that “Medical Hypotheses”, for example, is not on the same level as the NEJM? How is a person to know that entire advocacy groups ignore piles of good science in shaping their message?

  7. Regan September 16, 2008 at 19:02 #

    Maybe it’s me, but this kind of statement kind of undercuts the expertise in the AoA article,
    “…I don’t really understand what a metalloestrogen is, but the definition on Wikipedia is, a hormonally active agent…”

    Before someone hastens to publication or stating conclusions, a little humility and caution about the limits of one’s knowledge of the details might be a good thing. That is a characteristic that I often find lacking in the graduates of GoogleU and some members of the media.

  8. Sullivan September 16, 2008 at 19:20 #

    It’s a very difficult thing. I’m signed up as an ‘expert’ on Opposing Views. Does that mean I’m a science expert? Hardly. But what does it mean? An expert in being experienced in the debate?

    Kev, given that debate included people from Montreal Children’s Hospital, I put you on a different level of “expertise”. Compared to SafeMinds and NAA, yes, you have the “chops”.

    Consider a recent TV news story done in the United States for “Autism’s False Prophets”. They intereviewed a DAN (or DAN-ish) doctor, for “balance apparantly. This guy didn’t know the details of the chelation death of Abubakar Tariq Nadama. The doctor made some comment about not knowing the details, but tried to waive it off as an error by the doctor’s staff.

    There’s a clear case of the “expert” interviewed for a story having limited expertise in the subject he was discussing. Has he performed more chelations than you have? Pretty likely. Did he demonstrate expertise in the actual subject–what happened to that poor kid? No. Could you have handled it better than he did? Absolutely. Not from your professional background, but from understanding the situation.

    As I said, defining expertise is a tough battle.

  9. Bad mommy September 16, 2008 at 22:15 #

    I’m just a mom. With a degree in economics, and one in law. What that means is I don’t pretend to be an expert, but when I get on the internet and search, I’m pretty darn good at it — AND I’m likely to understand the science of what I’m reading in scientific journals online, and be able to identify weaknesses in the design of the study, etc. I also understand about the basic fallacies, and about the tendency to see causation where merely correlation has been proven.

    What I see as a problem is the way that science is routinely served up in the media. Some new study shows a correlation between x and y, and we hear nothing for a week on the tele but that “x is linked to y, scientists have shown x causes y,” etc. People are no longer truly educated in what science means and how it works, so we’re left with the examples of what purports to be science we hear on television. Having heard that x and y occur together – and that means x causes y – people then go looking for correlation and draw the wrong conclusions. It’s pretty understandable, I suppose. No wonder everybody thinks that they can be an expert – the witless anchor on television seems to be.

    I think that the solution is that we need to require statistics and scientific method in high school – at a minimum at the university level. If we were all properly educated, and were fed less outright manure, it might improve things.

  10. Sullivan September 16, 2008 at 23:18 #

    Bad mommy,

    good comments. I think the media does need to do more thinking and analysis. I think they also need to do more real reporting and less entertaining.

    The ECBT press conference with Amanda Peet was a good example. They seemed to be interested if there was going to be a fight between Ms. Peet and Ms. McCarthy. Jenny

    But, then there’s plain old correcting stories. There is an old story by Sharyl Attkisson on the Hornig mouse study. Nothing on the Berman study that shows a failed attempt to recreate Hornig. CBS still has the original story on their website.

    It is one thing to understand experimental design and statistics. But, I’ve seen people throwing around biochemistry and other subjects, often with clear examples that they don’t understand the fundamentals of the topics they discuss.

  11. isles September 17, 2008 at 04:26 #

    HealthNewsReview.org is a step toward helping people evaluate the way news is served to them. Unfortunately, the people who most desperately need to be informed that many of the stories they’re reading are BS are unlikely to even understand why a service like HealthNewsReview is valuable.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 1,150 other followers

%d bloggers like this: