Duplicate receipts are the most common data problem

The most frequent defect in receipt data is not a misread total. It is the same expense present twice — usually because a receipt was captured on two occasions, or because a paper slip and an emailed copy of the same purchase both got processed. Duplicates inflate expenses, break reconciliation, and are easy to create and tedious to find.

They also get much less attention than accuracy, which is backwards given how much more often they occur.

How duplicates actually appear

Captured twice. You photograph a receipt, aren’t sure it saved, photograph it again. Two records, same purchase, slightly different images and timestamps.

Paper plus email. A restaurant hands you a slip and emails a copy. An online order emails a confirmation and a separate invoice. Both are legitimate documents describing one expense.

Forwarded twice. An email receipt forwarded to your capture address, then forwarded again by someone else in a thread.

Statement plus receipt, unmatched. The card transaction arrives from the bank feed and the receipt arrives from a scan, and if matching doesn’t connect them you have one expense as two lines.

Reissued receipts. A returned and re-purchased item, or a corrected receipt issued after a pricing error. These look like duplicates and aren’t, which is why blanket deduplication is dangerous.

Why they are hard to detect automatically

The obvious rule — same vendor, same date, same amount is a duplicate — is wrong often enough to matter.

Genuine same-day repeats exist. Two coffees from the same shop on the same day, for the same amount, is completely ordinary. Two fuel purchases at the same station, two identical parking charges, two of the same subscription in a month where a plan changed. Delete those and you’ve under-reported expenses, which is a worse error than over-reporting because it’s invisible in reconciliation.

Meanwhile, real duplicates frequently don’t match exactly. The paper slip shows the pre-tip amount and the emailed copy shows the settled total. One record has the vendor as “SQ *NORTHWEST TOOLS” and the other as “Northwest Tools Ltd”. Dates differ by a day when a transaction settled after midnight.

So exact matching misses real duplicates, and fuzzy matching deletes real expenses. Neither is safe on its own.

The approach that works

Treat duplicate detection as a flagging problem, never a deletion problem.

Flag on strong signals. Same amount, same vendor, dates within a couple of days, and — the strongest signal available — the same card transaction. If two receipts match one bank line, exactly one of them is the evidence for it.

Surface, don’t merge. Present the pair side by side with both images. A human takes two seconds to tell “same slip photographed twice” from “two identical coffees”, and no rule reliably does.

Prefer the better document. When you do merge, keep the one with more information — the emailed invoice with line items over the photo of a thermal slip, the settled amount over the authorised amount.

Never silently delete. Whatever you resolve, keep a trace. A record that vanished with no history is worse than a duplicate, because you cannot audit what you can’t see.

Prevention beats detection

Most duplicates are created by uncertainty about whether something was already captured. Remove the uncertainty and most of them stop happening.

One capture path. Decide where receipts go — one app, one email address — and use only that. Duplicates multiply when there are two routes, because neither knows about the other.

Immediate confirmation. Capture should visibly confirm success. Half of all double-captures come from not being sure the first one worked.

Mark the paper. If you keep physical slips at all, a mark or a separate pile for captured ones ends the problem completely for paper.

Decide about email receipts once. Either you forward them all or you never do. The inconsistent middle — forwarding some, photographing the paper copy of others — is what generates paper-plus-email pairs.

Match to card transactions continuously. Not monthly. The bank feed is the arbiter of how many expenses actually occurred, and checking against it weekly turns duplicate detection into a small routine instead of a reconciliation crisis.

The reconciliation view

There’s a simple invariant worth holding onto: each card transaction has exactly one supporting receipt.

Run your data against it and duplicates become obvious. Two receipts pointing at one transaction is a duplicate. A receipt pointing at no transaction is either a cash purchase, a personal card mistake, or a duplicate whose twin already claimed the match. A transaction with no receipt is missing evidence.

That single check finds duplicates, missing documents and misfiled personal spending in one pass, which is why it’s worth more than any dedicated deduplication feature.

Why it matters more than it sounds

A duplicated expense overstates costs, which distorts every report built on them and, at year end, produces a deduction you can’t support with distinct evidence. It also erodes trust in the data — once someone finds one duplicate, they stop believing the totals, and then they start checking everything manually, which defeats the entire point of automating capture.

Getting duplicates right is unglamorous. It’s also the difference between expense data you act on and expense data you re-verify.