What a confidence score actually tells you

A confidence score is the model’s estimate of how well the input matched what it expected to see. It is not the probability that the value is correct, and treating it as one is how a review queue ends up trusting the wrong entries. Used properly, it is a routing signal: it decides which items a person looks at, and nothing more.

That sounds like a small distinction. It is the difference between a threshold that catches errors and a threshold that catches blurry photos.

What the number is measuring

Two separate things get labelled “confidence” and they behave differently.

Recognition confidence is about characters. The engine looked at a shape and decided it was a 7; the score reflects how cleanly that shape matched a 7 rather than a 1. Low scores here track image quality — crumpling, glare, faded thermal print, motion blur. This is the score that a better photo fixes.

Extraction confidence is about meaning. The characters are legible; the question is whether the number sitting at the bottom right is the total, the amount tendered, or the change. Low scores here track layout novelty rather than image quality: an unfamiliar receipt format from a vendor the system has never seen, printed perfectly.

These are different problems with the same label, which matters because they have different remedies. If your tool reports only one number, you cannot tell which one you are looking at. That is worth knowing about a tool before you tune anything. The distinction is the same one that separates OCR from data extraction.

Why it is not a probability of correctness

A calibrated score would mean that of the values scored 0.9, nine in ten are right. Nothing guarantees that, and the failure is asymmetric in the direction that hurts.

The dangerous errors are the confident ones. A receipt where the subtotal is printed in a larger, cleaner typeface than the total will yield a crisp, unambiguous, high-confidence read of the wrong field. Nothing about that image is degraded. The model is not uncertain. It is simply answering a different question than the one you asked, with every visual cue supporting it.

Meanwhile a low score often attaches to a value that is perfectly correct — a smudged but recoverable digit, a rotated receipt, an unusual currency symbol. Low confidence means “this was hard”, not “this is wrong”.

So the score correlates with difficulty, and only loosely with correctness. It is useful for deciding where to spend attention. It is not evidence.

Per field, or it tells you nothing

A single score for a whole receipt is close to useless. It averages the easy fields — date, which is usually unambiguous — with the hard one, and the result is a number that neither triggers review when it should nor tells a reviewer where to look.

Field-level scores let you set different thresholds for different fields, which is what you actually want. The total deserves a strict threshold, because an error there is silent and propagates into every report. Tax often deserves its own, for reasons covered in extracting the tax amount. Line items are the least reliable extraction there is, so a low score on them frequently means “do not bother with line items for this vendor” rather than “review this”.

Setting a threshold you can defend

Do not pick a number because it looks strict. Derive it from what your review queue can absorb.

Start loose and measure. Set a threshold that sends more items to review than you think you need, then track the correction rate: of the items flagged, how many actually needed changing? If almost none did, the threshold is too aggressive and you are training reviewers to click accept. Tighten it.

Watch the other direction too. Sample some of what passed. If you find errors below the threshold, the number is not the problem — the score is not separating good reads from bad on that field, and you need a different trigger.

Expect to hold different thresholds per field. One global number is a compromise across fields with genuinely different risk.

The checks that beat the score

The most valuable review trigger on a receipt is not a confidence score at all. It is arithmetic: line items summing to the subtotal, subtotal plus tax equalling the total.

Arithmetic catches the confidently wrong reads, which is precisely the class of error a confidence threshold misses. Read the subtotal as the total and the sum no longer works. No score had to be low for that check to fire.

Two others in the same family: a total that is implausible for the vendor, and a date in the future. Both are rules about the world rather than estimates about an image, and both catch errors that arrive with high confidence attached. This is why a well-designed review queue uses reconciliation failures as its primary trigger and confidence as a secondary one.

How to use it, then

As a router. Low score on a field the reviewer can see highlighted, with the image beside it — that is the whole job. The score decides whether a person looks and where they look first.

What it should never do is appear in your books. A stored confidence value creates the impression that an entry carries a known reliability, and it does not: once a value has been reviewed and accepted, the score that got it there is history. And an unreviewed entry is not “90% reliable” — it is unreviewed.

The accuracy you can reasonably expect is high enough that review is faster than typing and never high enough to skip. Confidence scores help you spend the review where it pays. They do not tell you when you can stop.