Expertise and suitability for policy responsibilities

Another great interview on onthemedia.org. This one dealing with the reliability of expert predictions (in the media and in general).

On The Media-- THE GUESSING GAME
PHILIP TETLOCK: When an expert has very, very strong opinions on an issue, when the expert places a high value on simplicity and has little patience with contradictions or ambiguity, and when the expert is making longer-term predictions, that expert's likely to go off the cliff.
...
PHILIP TETLOCK: And the more knowledge that expert has, the worse, interestingly, it becomes, because the expert is using the knowledge very selectively to justify increasingly extreme predictions.

This reminds of a quote that I've been trying to track down for a while now. I think it went something like this: "I'd rather be governed by the first 1000 people in the New York City phone directory than the combined faculties of Harvard and MIT." (no idea who and when said this). The bottom line is that expertise is as much a blinding as an illuminating trait.

Here's a rather excessively long quote:

BROOKE GLADSTONE:: I guess in this case the experts do what the rest of us do when we consume media. They seek out information that confirms what they already believe.

PHILIP TETLOCK: Experts with that particular style of thinking are prone to that tendency, yes.

BOB GARFIELD:: Which brings us to the foxes and the hedgehogs. It's an old Isaiah Berlin metaphor that you use to divide your experts into two camps. What are foxes and hedgehogs and how do they differ in their forecasts?

PHILIP TETLOCK: Well, the fox/hedgehog distinction goes back at least 2,500 years to classical Greece. The basic idea is that the hedgehog personifies someone who knows one big thing, someone who tries to organize everything into a comprehensive overall framework, whereas the fox is someone who knows many little things, is much more intellectually opportunistic, is willing to pick ideas from wherever. It doesn't matter if it's a liberal idea or a conservative idea. They are very flexible. [DL: Reminds me of another quote: 'An educated man knows everything about something and something about everything' But of course, there's the 'a little knowledge is a dangerous thing' adage to put this into perspective.]

[Identified Churchill as an example of a hedgehog. Who was famously right about Nazi Germany and whose massive miscalculation regarding India (he even compared Ghandi to Hitler is largely forgotten).

PHILIP TETLOCK: By and large, that's right. It is true that if you wanted to identify the experts who have made the most spectacularly far-sighted predictions over the last 50 years, the hedgehogs would be disproportionately represented. But if you were computing batting averages, the hedgehogs would be clearly statistically inferior to the foxes.

BROOKE GLADSTONE:: [LAUGHS] And there's sort of a perverse inverse relationship between how spectacularly wrong some hedgehogs are and how much they're held responsible for those wrong predictions. The more wrong you are, the higher profile you are, the less you're called to account.

PHILIP TETLOCK: Well, that's an interesting feature of the political world. Hedgehogs are typically embedded in political movements or theoretical movements and they typically have people who will back them up. They can fall back on a base of supporters who will help them generate various types of excuses or belief system defenses that will neutralize the unexpected evidence. So they'll be able to argue, "Well, what I predicted didn't happen, but it will happen soon," or, "I predicted that country X had weapons of mass destruction, and, well, it appears that it didn't, but it was the right mistake to have made."

BROOKE GLADSTONE:: [LAUGHS]

PHILIP TETLOCK: It's better to have overestimated them than to have underestimated them.

BROOKE GLADSTONE:: You note in your study that hedgehogs are more frequently wrong than foxes, but you also note that when it comes to experts, the media love, not wisely, but too well, the hedgehog - I guess because they make better TV.

PHILIP TETLOCK: The hedgehogs tend to provide better sound bites. I think that's definitely true. They're much more likely to offer unequivocal predictions, whereas the foxes are much more likely to attach lots of linguist qualifiers to their predictions. Their speech is larded with things like, "but," "however," "although" - all signs that you're putting on the brakes. There's also a team sport aspect to punditry. Pundits are often there to represent a certain point of view so they want to make sure they've got an entertaining liberal hedgehog and an entertaining conservative hedgehog and pit them against each other and let the sparks fly, and so it has this public spectacle aspect to it. It's not particularly conducive to accuracy, but it is entertaining, and it really depends on what you want out of life.

Of course, as always, there are two stories to be told about expertise. One in which a far-sighted expert is right in the face of many simple-minded and bigoted oponents is most often applied. However, the other, of the expert blinded by his or her own expertise to the extent of not seeing 'what is right in front of their nose', is also very popular. In fact, Donald A. Schön's book on the 'Reflective Practitioner' was dealing very much with the aftermath of one such application of the latter story.

A further complication, is the fact that, as I've noted before, the amount of information we need to digest is so vast that we need to rely on experts (or mediators of expertise such as pundits; and even experts on experts) simply because we cannot delve into each topic with sufficient depth. And the decisions we make about which expertise to go with are often ones to do with out political (and other group) identities.

A good example of is the film critic Mark Kermode's response to accusations of misguided advocacy of intelligent design based on having seen the March of the Penguins movie. His response was to quote a name and an idea: Theilard de Chardin and puncutated equilibria (he called it something else and used it to support an argument that would make Gould turn in his grave; furthermore is mistakenly claimed that most people accept punctuated equilibria). This hodgepodge of expertise allowed him to make the claim that he has a valid alternative view to evolution. More details in this post.