Calibration and Accuracy

A forecaster who says "70% chance" should be right about 70% of the time they say that. This is calibration — the alignment between stated confidence and actual frequency of outcomes. Calibration, combined with resolution (the ability to discriminate between events that happen and those that do not), determines forecast quality. This skill covers how to measure, track, and improve forecast accuracy using proper scoring rules, calibration curves, and debiasing techniques drawn from the superforecasting literature.

## Key Points

1. TRIAGE: Focus on questions where effort improves accuracy
- Skip questions that are too easy or too hard
- Prioritize questions in the "Goldilocks zone" of difficulty
2. BREAK DOWN PROBLEMS: Fermi estimation approach
- Decompose into sub-questions
- Estimate each component separately
3. STRIKE THE RIGHT BALANCE (inside vs outside view)
- Start with the base rate (outside view)
- Adjust with case-specific factors (inside view)
4. DISTINGUISH AS MANY DEGREES OF UNCERTAINTY AS THE PROBLEM ALLOWS
- Use granular probabilities (not just "likely/unlikely")
- The difference between 60% and 65% matters