RACHEL MARTIN, HOST:
In the world of politics, knowing the future would be really valuable. But it turns out that experts and pundits aren't much better than readers of palms or tarot cards when it comes to predicting the future. And the more confident the expert, the less accurate their guesses. Psychologist Philip Tetlock has been studying this for decades, collecting millions of political predictions and then looking at who's right. Tetlock is a professor at the University of Pennsylvania. He joined us from Philadelphia. And I asked him why the so-called experts might be getting it wrong.
PHILIP TETLOCK: Up to a point, knowledge helps. But knowledge often doesn't help with as much as experts think it does. So, one way to measure whether experts are overconfident is to get them to express their judgments on a probability scale and get them to say, are you 90 percent confident it's going to rain today; 70 percent, 60 percent. And if you do that many times, you can build up a database and a neutral judge keeps score, which is not the way it works typically in politics. In politics, each side - the liberals and the conservatives - they have their own scoring card. Imagine a baseball game, right, where each team has its own umpires. Politics the game is somewhat like that. Each side has its own scorekeepers, and not surprisingly we often have very different conclusions about who won the game at the end.
MARTIN: What makes them so inaccurate?
TETLOCK: We had a rare occasion for assessing how inaccurate many pundits could be during the 2012 election when Nate Silver, formerly of the New York Times - I think he's with ESPN now - he conducted some very careful statistical analyses of polls and reached the conclusion that even though the polls are pretty close, Obama has a 75 or 80 percent likelihood of winning the election. And a lot of pundits jumped on him and said, oh, it couldn't possibly be the case. Many people considered that to have been a debacle for the pundits.
MARTIN: So, who's really good at doing this, at predicting the future?
TETLOCK: Well, it helps to be a bit on the modest side. There are just some things nobody can predict. Accidents happen. Things happen that are just beyond the foreseeable. But beyond that, it is possible to cultivate the skill. We can show experimentally and pretty decisively that it is possible to train people to become better users of probability scales. They become more nuanced in their use of probability scales.
MARTIN: As I understand it, Philip, you engaged in this research along with your wife; is that right?
TETLOCK: Yes, Barbara, Barbara Mellers.
MARTIN: How does this play out, I wonder - if it does at all - in your personal relationship? Can you just off the cuff say, yeah, I think it's going to rain tomorrow? The other one calls the other out on that?
TETLOCK: Barbara's a much better intuitive statistician than I am. So, I defer to her on most quantitative issues.
MARTIN: Professor Philip Tetlock. He talked to us about making predictions - how we can do it better. He joined us from Philadelphia. Thanks so much for talking with us.
TETLOCK: My pleasure.
(SOUNDBITE OF MUSIC)
MARTIN: This is NPR News. Transcript provided by NPR, Copyright NPR.