math · Updated 21 Jul 2026
The Brier Score: How We Grade Our Own Predictions
The Brier score measures how close a probability was to what actually happened — averaged over many matches. Lower is better: an all-knowing oracle scores 0, blind guessing scores ≈ 0.667.
A prediction like "Arsenal 62%, draw 22%, away 16%" is a set of three numbers, not a single pick. So grading it needs a method that looks at the numbers themselves — not just whether the favourite won. That method, used everywhere from weather forecasting to football analytics, is the Brier score.
How it works
After the match, write reality as 1s and 0s: the outcome that happened gets 1, the others get 0. Then take the squared difference for each outcome and add them up.
Say we predicted 62% / 22% / 16% and the home side won (reality: 1 / 0 / 0):
- Home: (0.62 − 1)² = 0.144
- Draw: (0.22 − 0)² = 0.048
- Away: (0.16 − 0)² = 0.026
Score for this match: 0.218. Now imagine the draw had happened instead — the same prediction would score (0.62)² + (0.22 − 1)² + (0.16)² = 1.019. Same forecast, very different grade: the score rewards being confidently right and punishes being confidently wrong.
Average this over hundreds of matches and you get a number that summarises how well a forecaster prices football.
The scale, in football terms
| Score | What it means |
|---|---|
| 0.000 | A perfect oracle — always 100% on the actual result |
| ≈ 0.58–0.62 | A typical bookmaker's closing prices (the state of the art) |
| ≈ 0.667 | Blind guessing — ⅓ / ⅓ / ⅓ on everything |
| > 0.667 | Actively worse than guessing (systematically overconfident) |
The differences that matter are small: a gap of 0.01–0.02 between two forecasters over hundreds of matches is a real skill difference, not noise.
Why we grade against the market
On our Performance page every league group shows two bars: Model (our ensemble) and Market (the fair closing odds, bookmaker margin removed). Both are scored on exactly the same matches — only genuine forecasts made before kickoff count.
Whoever has the lower score priced the football better. The "ahead" chip states the winner and the size of the gap: market +0.012 means the closing odds beat our model by 0.012 Brier points in that group.
The betting market's closing price is the strongest public football forecast in existence — it aggregates the money and information of everyone betting. No public model we know of beats it consistently on 1X2. We benchmark against it weekly and show the result either way, because a track record you can audit — losses included — is the only kind worth trusting.
Reading the drilldowns
Click any league group on the Performance page:
- Monthly trend — the model and market lines month by month. Parallel lines mean a stable gap; converging lines mean the model is catching up.
- Promised vs delivered — our confidence chips (▲ FAV, ▲ CLR, ~ LEAN, · OPEN) each promise a probability range. This panel checks the promise: "promised" is the model's average probability in that band, "won" is how often those picks actually landed. A well-calibrated model has the two bars matching.
That last check is the honest version of "how good are your predictions": not did you win, but did reality match the numbers you quoted.
Related
Frequently asked questions
What is a good Brier score in football?
For three-way (1X2) football predictions, blind guessing scores about 0.667 and a typical bookmaker's closing prices score around 0.58–0.62. Anything meaningfully below the bookmaker on the same matches would be world-class — no public model we know of manages it consistently.
Is a lower or higher Brier score better?
Lower is better. A perfect oracle that always assigns 100% to the actual result scores 0. The score punishes both overconfidence and underconfidence.
Why not just use accuracy (percent of correct picks)?
Accuracy only checks whose probability was highest, throwing away the number itself. A model that says 51% and one that says 90% get the same accuracy credit when the favourite wins — but they are very different claims. The Brier score grades the whole probability, which is what a prediction actually is.
What does 'market ahead' mean on the Performance page?
On the same matches, the fair closing odds (bookmaker margin removed) produced a lower Brier score than our model — the market priced those games more precisely. That is the normal state of the art; we publish it because a track record you can audit is worth more than one you have to take on faith.