When we are wrong

A score is a measurement, and measurements can be overtaken by events. A brand we rated highly can still halt withdrawals, disclose a breach, or be found insolvent. This page sets out — in advance — exactly what we do when that happens, so our response is a procedure we already committed to, not a decision we make under pressure.

Our score history is append-only. When a brand fails, we do not quietly remove its page, and we do not change a past number. The score is shown frozen — struck through, but still visible, with the value it held at the moment we froze it and a timestamp. The strike-through is the record: it shows what we published and when, rather than hiding it.

These are the targets we hold ourselves to. They are targets, not warranties — a genuinely novel situation may take longer, and we would rather be accurate slightly later than fast and wrong.

• within ~2 hours
  Freeze. We freeze the score (struck through, not removed) and post a factual, timestamped notice on the brand’s page stating what happened.
• within ~24 hours
  Suppress and annotate. We remove the brand from our rankings and publish a factual summary of the event with sources — what we observed and when. The brand’s page stays up; the score stays frozen, not deleted.
• within ~72 hours
  Post-mortem. We publish a standardized review: the score and confidence at the time of failure with its input breakdown; what each part of our evidence said; which signals were present but underweighted versus structurally outside what our evidence can observe; and what, if anything, changes.

We do not make an unscheduled change to our methodology inside the review window, or while a related legal threat is live. Overfitting our method to the last failure is its own kind of error. A methodology release whose plan, scope, and date were recorded before the event proceeds on its own schedule, and we disclose it in the post-mortem rather than letting it look like a quiet reaction. “No change” is a legitimate, and often correct, outcome: some failure modes are outside what our published evidence can see, and our methodology says so.

• We will not scrub the score or take the page down to look clean.
• We will not go silent and leave a stale number standing.
• We will not assign blame beyond what sources establish — we state what happened and what we saw, with citations, not a verdict on motive.

A different case is when the fault is ours: a bug in our pipeline, an input we should have caught, or a weight applied incorrectly. We call this a computation error, and it gets a different remedy — the score was never right, even at the moment we published it.

• Brand failure (§1–§2 above). Reality changed after we published a correct score. The response is §1–§2 above: we freeze the number — struck through, not deleted — because it was an accurate measurement at the moment we published it.
• Computation error. Reality did not change; our arithmetic did. Three things happen, all at once:

• The fix runs through the published methodology. We do not hand-patch a single number — we ship the bug fix or the corrected input and recompute using the current methodology version.
• The public number is corrected. The value shown on the brand’s page today is updated to what the corrected computation produces — visitors see the accurate score, not a preserved wrong one.
• The erroneous historical value is affirmatively marked as an error. Unlike a brand-failure freeze, which strikes through a value that was accurate when published, a miscalculation entry is not merely struck through — it is labelled a computed error in the append-only ledger.

Both invariants hold at once. The ledger keeps every value we ever published — a miscalculation is never deleted, only labelled. The number showing today is always the one we currently believe is correct — a miscalculation is never left standing in public. A brand failure preserves an accurate historical value; a computation error disowns an inaccurate one while still keeping the record of it.

For how a score is computed in the first place, see /methodology. Every published score and every change to it is recorded in our audit trail.