Superforecasting

SUPERFORECASTING: THE ART AND SCIENCE OF PREDICTION
By Philip E. Tetlock and Dan Gardner

2010 saw the publication of Dan Gardner’s Future Babble, a fascinating look at the often dodgy business of expert predictions. Much of the scientific core of that book was provided by the research of Philip Tetlock, himself an expert on the art and science of predicting the future.

Superforecasting is a sort of follow-up to Future Babble, in which Tetlock (the primary author), uses the results of his multi-year study of forecasting (the Good Judgment Project) to explain the thinking behind predictions that work.

There is an initial hurdle to overcome in first being able to measure the accuracy of predictions in order to compare their results. Once he’s established guidelines, however, Tetlock is able to rank his volunteer forecasters, and so discover a class of people with more than just a knack for making the right calls. He calls these people “superforecasters.”

Of course what we all want to know from a book like this is what makes a superforecaster super, and Tetlock helpfully breaks down the various skills and habits of mind involved. In brief, superforecasters are smart, numerate, and good at processing information from different sources. They know their limitations, and that no one can predict the future perfectly (for example, outside of five years you’re basically into the realm of chance). They are aware of the pitfalls of misleading types of thinking that are easy to slide into. They are good at breaking big problems down into their component parts. They are not dogmatic, and are quick to adjust their thinking in light of any new facts coming into play.

Much of this may seem obvious, but just as obvious is the fact that people don’t think this way naturally. There is a certain discipline that goes along with being a seer.

As well as the inherent interest the subject has, Superforecasting will also appeal to fans of the Freakonomics books and the work of writers like Malcolm Gladwell who popularize the latest developments in social psychology and behavioural economics (Daniel Kahneman in particular stands out as the guru of what has become an entire genre, and his name comes up frequently here).

Such books are also part of a larger intellectual movement, toward an understanding of the universe as essentially probabilistic, with Big Data providing the power to crunch all the numbers and give the best odds. Superforecasting is thus just “another manifestation of a broad and deep shift away from decision making based on experience, intuition, and authority . . . toward quantification and analysis.” Though Tetlock tries to put some distance between them, superforecasters are thus much like modern “quants,” with the next step in the evolution of the human mind appearing to be a human-computer synthesis capable of overcoming our mental failings.

Perhaps the merely human is something to be surpassed. In one of the more interesting points raised in the book, Tetlock notes without further comment that people who find meaning in events tend to be happier than those who only judge probabilities that are themselves meaningless (that is, superforecasters). He also concludes by acknowledging that the tools of forecasting can’t be used to determine what sorts of questions might be worth asking, and that there may be a role for the Humanities in all of this yet.

But this is a prediction that may be informed by wishful thinking — a human frailty — and in any event that’s not the book Tetlock and Gardner are writing. What they have given us is instructive enough to help us judge what’s to come.

Notes:
Review first published in Quill & Quire, October 2015.