FoundationsBoth sides

How to read a confidence band — and distrust a score

A score with no band is a guess in a suit. What the band is actually measuring, how wide is too wide, and the four things a narrow band still can't tell you.

Adam Murray7 August 20268 min read
On this page

Any tool that gives you a score should also tell you how much to trust it. Most don't, because a single confident number demos better than a number with a shrug attached.

The shrug is the useful part.

What a band actually is

A confidence band is a range around an estimate that says: given the evidence available, the true value is plausibly somewhere in here.

"72" is a claim. "72, plausibly 58–84" is a claim plus an admission about the evidence behind it.

The width is doing the work. A narrow band means the underlying data was consistent and there was enough of it. A wide band means one of those things wasn't true — and the estimate should move you less.

The three things that make a band wide

Not much data. Six comparable pieces produce a wider band than sixty. This is the dominant factor most of the time and the least interesting one.

Inconsistent data. Ten pieces that all performed similarly give a tighter band than ten pieces scattered across a huge range. Content is unusually prone to this — the spread within a single channel's output is often enormous, which is exactly why outliers are the unit of analysis.

Weak evidence. Estimating from public data alone is a different exercise from estimating with your first-party analytics. A band should widen as you move down the evidence ladder, and if it doesn't, the tool is flattering you.

How to read the width

Rough, practical, not statistical:

BandWhat it meansWhat to do
Narrow, highConsistent evidence pointing one wayAct on it
Narrow, lowConsistent evidence it won't workAct on it — this is just as useful
Wide, spanning the middleThe evidence genuinely doesn't decideDon't let the midpoint decide either
Wide, but entirely high or entirely lowUncertain magnitude, clear directionAct on the direction, not the number

That last row is the one people misread. A band of 60–95 is wide, but every value in it is good. The magnitude is unknown; the direction isn't. That's an actionable result.

Conversely a band of 40–75 has a midpoint of 57 that looks like a mild positive and is nothing of the sort — it contains "bad idea" and "good idea" with roughly equal claim.

Overlap is the whole game

When two options' bands overlap substantially, the data has not chosen between them. You still have to choose — but choose on grounds the data doesn't cover, and know that's what you're doing.

Good grounds when the numbers tie: which is cheaper to make, which teaches you more if it fails, which you can execute better, which fits the content goal you set.

Bad grounds: the midpoint.

Four things a narrow band still can't tell you

A tight band means the evidence was consistent. It does not mean the estimate is correct.

Whether the comparables were really comparable. A confident estimate built from the wrong reference set is confidently wrong. Ask what it was compared against before you ask how confident it is.

Whether the world changed. Bands are computed from history. A format that reliably worked for two years can stop working in a month when a platform changes what it surfaces. No band captures that.

Whether you can execute it. Nearly every score is about the idea. Execution variance is usually larger than concept variance, and it isn't in the number.

Whether it's the right thing to make. A high score on a piece that serves no goal is a well-evidenced waste of a shoot day.

Why we show them even when it's inconvenient

A band that says "we don't really know" is a worse demo and a better product.

Acumin's Concept Reads carry a band on every dimension, and the band widens when the evidence is thin — few comparables, a category we can only see publicly, a format without much history. On genuinely novel ideas it can be wide enough that the honest read is "this is unprecedented for you, and that's the finding."

That's deliberate, and it follows from the same rule as everything else here: numbers come from data, and where the data is thin, the number has to say so. A tool that returns a crisp 78 for every input has stopped measuring and started decorating.

The same logic runs through the evidence tiers — a read built on public data alone is labelled as such, rather than being blended into a single confident-looking figure.

When a wide band is the answer

Sometimes "we can't tell" is the finding, and it's worth something.

If a concept comes back with a very wide band because nothing comparable exists in your category or your history, that is telling you: this is genuinely new. Not necessarily good, not necessarily bad — unprecedented.

That's an argument for making it cheaply to find out, rather than either committing fully or dropping it. The band has told you the right size of bet.

How to use this today

Find the last content decision your team made on the basis of a number.

Ask: what was the range around it, and did any competing option's range overlap? If nobody can answer, the number wasn't doing the work you thought it was — someone's judgement was, which is fine, but it should be credited honestly.

From here on, ask for two numbers whenever you're given one: the estimate, and how much data it came from. That single habit fixes most of this.


Related: The evidence ladder explains why the source determines the width. What a content signal actually is covers the three tests an observation has to survive before a band is even worth computing.

Written by
Adam Murray
Founder, Acumin

Adam builds Acumin. He spends his days on the same two problems this library is about: working out what a piece of content is actually worth, and getting a brief through production without it turning into something else.

Want this done on your own channel?

Acumin reads your public content and your category and hands back what to make next — every call with its confidence and the evidence it rests on. The first Snapshot is free.