Skip to main content
ChanceWire may earn a commission when you visit partners via our links. DisclosureNot financial adviceLegal age in your jurisdiction (some partners 21+)
ChanceWireChanceWire

Poly Chance: Scan volume first, then open a category. Prices are implied probabilities — not guarantees. Start with the reading guide if this is new.

← Guides

Noise and judgment inconsistency in forecasting

How SoftCta readers reduce noise in their own forecasts — the difference between bias and noise, occasion noise, and practical ways to make probability estimates more consistent. · 3 min read

Updated 22 Oct 2026

Poly ChanceChanceWire guide · fictional AI persona

Disclosure: ChanceWire may earn a commission if you sign up or trade through certain partner links. SoftCta only until partner tokens allow hard CTAs. Read How we make money and our Affiliate disclosure. Not financial advice. Informational only — not legal, tax, or investment advice. Digital assets and event contracts involve substantial risk of loss.

Noise and judgment inconsistency in forecasting

Ask the same forecaster the same question on Monday and Friday, with no new information in between, and you may get two different numbers. Ask two equally skilled forecasters, and the gap may be wider. That variability is noise, and research suggests it is often as large a source of error as bias. This SoftCta guide explains what noise is and how readers can reduce it in their own probability estimates.

Bias and noise

Imagine a group of shooters at a target. If all shots cluster to the left of the bullseye, that is bias. If they scatter widely around it, that is noise. In forecasting, bias might be a habit of overrating the incumbent; noise is the fact that your number for the same race changes with mood, time of day or the last article you read. Mean squared error, which underlies the Brier score, counts both.

Kinds of noise

Kahneman, Sibony and Sunstein distinguish level noise (some judges are generally harsher or more lenient), pattern noise (judges react differently to particular cases), and occasion noise (the same judge varies over time). Readers mostly face occasion noise, and it is the easiest to test on yourself.

Testing your own noise

Pick a handful of questions, write probabilities, and revisit them a week later without checking your notes or the market. If your numbers move a lot without new information, that spread is noise. Our calibration guide explains how to score forecasts once outcomes are known.

Structure reduces scatter

Breaking a question into components and estimating each separately produces more consistent answers. Our Fermi estimation guide shows how. Checklists that force you to consider base rates, current evidence and the resolution criteria in the same order each time also help.

Averaging independent judgments

Averaging several independent estimates cancels much of the noise, which is one reason crowd forecasts and market prices can perform well. The key word is independent: if everyone reads the same thread first, their errors are correlated. Our aggregation guide and wisdom of crowds guide cover the conditions.

Your own crowd

You can average with yourself. Write an estimate, then deliberately consider why it might be wrong and write a second one. Research on the crowd within suggests the average of the two is often more accurate than either alone.

Why it matters for reading prices

A noisy estimate compared with a market price can create the illusion of a gap that is just scatter. Before acting on a difference, ask whether your number would be the same tomorrow.

Primary sources SoftCta readers open

  • Kahneman, D., Sibony, O. and Sunstein, C. R. (2021). Noise: A Flaw in Human Judgment.
  • Research on the wisdom of the inner crowd by Vul and Pashler (2008) and Herzog and Hertwig (2009).

Reading the quote step by step

  • Read the full title and rules, including the named settlement source and the time zone.
  • Convert the Yes and No prices to implied probabilities and note the gap between them.
  • Subtract fees and spread to see the effective price you would actually face.
  • Write your own probability range and the evidence behind it, then compare.
  • Check related and near-duplicate markets on other venues for consistency.

Closing SoftCta checklist

  • Re-read the venue definition in your own words before reacting to a price.
  • Convert prices with fee and spread awareness; verify live cents on the operator.
  • Prefer primary settlement sources over secondary headlines and social summaries.
  • Treat ChanceWire hubs, movers, and tools as orientation — not order entry.
  • Confirm age, location, and product eligibility; SoftCta pages will not suggest location spoofing.
  • If clarity is incomplete, watching is a complete SoftCta outcome — no pick-selling urgency required.

SoftCta habits

  • Write your own probability range before checking the crowd price.
  • Prefer dated primary sources over social summaries.
  • Compare related markets when a single book looks extreme.
  • Size curiosity with fees and depth in mind; substantial risk of loss.

FAQ

What is noise in judgment?
Noise is unwanted variability in judgments that should be identical. If the same forecaster gives different probabilities for the same question with the same information, that spread is noise.
How is noise different from bias?
Bias is a consistent error in one direction, such as always being too optimistic. Noise is scatter around the target. Both reduce accuracy, and they need different fixes.
Where does the idea come from?
The concept was popularised in the 2021 book Noise: A Flaw in Human Judgment by Daniel Kahneman, Olivier Sibony and Cass Sunstein, building on decades of research on judgment variability.
How can I reduce noise?
Break questions into parts, use checklists, write estimates before discussing them, average independent estimates, and revisit the same question later to check consistency.

Related

Noise and judgment inconsistency in forecasting — SoftCta literacy · ChanceWire