Confidence Score

Assumptions

Everything the model believes, stated before the math. If you do not buy one of these, do not buy the output.

1. Confidence is a fixed rubric, deterministic and published - no learned component.
2. Seven factors map to HIGH / MEDIUM / LOW: source quality, recency, corroboration count, contradiction count, specificity, primary vs secondary, document and extraction quality.
3. It runs at answer time as part of the agent run and is recorded with it.
4. No false precision: if confidence is low, the number shows as provisional.

Derivation

The rubric is a published weighted sum over the seven factors, with corroboration entering as a diminishing count and contradiction as a penalty:

The score maps to three grades - HIGH / MEDIUM / LOW - and the grade travels with every probability the system prints: 'Delivery probability 64%, confidence: medium.' Never '64% certain.'

Identification strategy

Why the estimate supports the claimed meaning - and where the model refuses to claim more than the data carries.

• The rubric separates two things that are easy to conflate: how big the number is and how much to trust it. Both are printed because a 64% on thin evidence and a 64% on a full primary record are different objects to a credit buyer.
• The rubric is validated against the calibration record over time: outputs graded HIGH must calibrate better than outputs graded LOW, or the rubric is revised. A confidence grade that does not sort outcomes is decoration.

Inputs and outputs

Calibration and gate evidence

Validation design

• Validated against the calibration ledger: HIGH-graded outputs must out-calibrate LOW-graded outputs over time, or the rubric is revised.

Evidence on record

• The sorting test requires a matured prediction ledger; results are not yet published in the sealed source.

Failure modes

The ways this model declines, degrades or goes UNKNOWN - and what it prints when it does.

• Empty evidence set: factors floor, grade is LOW.
• Contradicted claims: the contradiction factor drags the grade even when the claim itself is well-sourced - disagreement is visible in the grade, not averaged away.

Sensitivity

• Corroboration count is the fastest-moving factor; contradiction penalty is the sharpest drag.
• Factor weights are published policy: changing them re-grades every answer mechanically, which is why the rubric is versioned like a model.