Superforecasting is the product of Tetlock's research and experiments running a large-scale and long-running forecasting competition, from which he identified a group of individuals who significantly outperformed expert analysts. The book is the story behind this research and a discussion of the characteristics of these "superforecasters".

At first, I felt that Superforecasting was a rehashed Thinking, Fast and Slow **(or a mix of books from that genre), but it's a wonderful book in its own right and a valuable contribution to the genre. It makes interesting philosophical points about the nature of forecasting (to parry those of NNT in The Black Swan); it provides rich historical examples of forecasting errors (e.g. the Bay of Pigs invasion), forecasting non-errors that were treated as errors (controversially, the existence of WMDs in Iraq), and forecasting successes (several examples of superforecaster reasoning). Most importantly, the book provides practical prescriptions for improving forecasting. All of this is wrapped in an introspective and rational package – the authors are self-aware enough to point out contentious points in their own methodology and to consider alternate views.

As much as I would like to paraphrase the main takeaways, I really can't do any better than the author's own summary:

"Unpack the question into components. Distinguish as sharply as you can between the known and unknown and leave no assumptions unscrutinized. Adopt the outside view and put the problem into a comparative perspective that downplays its uniqueness and treats it as a special case of a wider class of phenomena. Then adopt the inside view that plays up the uniqueness of the problem. Also explore the similarities and differences between your views and those of others—and pay special attention to prediction markets and other methods of extracting wisdom from crowds. Synthesize all these different views into a single vision as acute as that of a dragonfly. Finally, express your judgment as precisely as you can, using a finely grained scale of probability."

Important ideas

• Calibration curves and Breier scores
• Extremise averages
• Auftragstaktik - distributed management
• Perpetual beta - always treat your ideas as if they are in the beta stage

The Manifesto of a Superforecaster

• In philosophic outlook, they tend to be:
• In their abilities and thinking styles, they tend to be:
• In their methods of forecasting they tend to be:
• In their work ethic, they tend to have:

10 Commandments for aspiring superforecasters

1. Triage - choose questions that are forecastable
2. Break down problems into key drivers
3. Balance between priors and Bayesian updates
4. Update your beliefs in light of new information (think carefully about calibration)
5. Synthesise opposing views into a probability
6. Be granular about the spectrum of possibilities.
7. Actively seek to reduce both false positives and false neatives
8. Understand your biases and track your performance.
9. Take perspectives from others but don't be completely swayed (without evidence)
10. Incorporate feedback (from outcomes), but don't overreact.
11. These commandments are not binding rules. Be nimble, mindful, and ever-rational.

Highlights

• Predictability and black swans
• "Experts"
• Failures of forecasting
• Characteristics of superforecasters
• Techniques used by superforecasters
• Evaluation and application of superforecasting
• Misc