Reliability and Drift Model

Assumptions

Everything the model believes, stated before the math. If you do not buy one of these, do not buy the output.

1. Source outage is source drift, never outcome drift - the channels stay separate.
2. Drift series use event-time fields only; schema and backfill changes cannot create a drift alert.
3. Tests are segment-aware with Bonferroni control; the adjusted alpha prints.
4. Small samples monitor, never suspend globally.
5. Drift is not automatically degradation - actions follow printed PSI thresholds: 0.1 monitor, 0.25 widen, 0.5 suspend.
6. A suspended segment returns UNKNOWN or its documented fallback; fresh confidence is contractually barred until recalibration clears it.
7. Calibration drift without an out-of-time ledger is reported UNKNOWN - unmeasured is not zero.

Derivation

Source health reads the source registry: a disabled source is an outage, logged as source drift - a coverage event with the expected cadence printed, not a statistic about outcomes.

Data drift compares project-type composition between COD windows (2027 and earlier against 2028 and later) within each state segment, using the population stability index between the base shares p and current shares q:

A segment enters testing only with at least 60 records - 30 per window; below the floor it monitors regardless, and a small sample can never trigger a suspension. Across the tested segments the family alpha is Bonferroni-controlled, with S the number of segments tested, and the adjusted alpha prints:

Actions bind to the printed thresholds: PSI at or above 0.1 monitors, 0.25 widens intervals for the models consuming the segment, 0.5 suspends it. Suspension is per-segment, never global from one segment, and a suspended segment cannot emit fresh confidence until recalibration clears it - the output says so in its contract.

Calibration drift needs an out-of-time scored prediction ledger. None exists yet at sufficient depth, so the calibration channel reports UNKNOWN - unmeasured, not zero - until the ledger fills.

Identification strategy

Why the estimate supports the claimed meaning - and where the model refuses to claim more than the data carries.

• Channel separation is the first rule: an outage degrades coverage, and the model is barred from reading it as evidence about outcomes.
• Event-time-only series identify real drift against artifact: schema and backfill changes physically cannot enter the drift windows.
• Bonferroni control prices the multiple-testing cost in the open: with S segments tested, each test answers at 0.05/S, and the adjusted alpha prints rather than being absorbed.
• The sample floor identifies what a PSI number cannot say: below 30 records per window the statistic is UNKNOWN and the segment monitors, so noise can never suspend a model.

Inputs and outputs

Calibration and gate evidence

Validation design

• Gate-verified discipline: source health prints per source, outages are source drift never outcome drift, every alert prints test, window, and sample, calibration drift without a ledger is UNKNOWN not zero, the adjusted alpha prints, and suspended segments are barred from fresh confidence by contract.

Evidence on record

• Smoke gates at seal (8 m35-specific): source health printed per source; source outage is source drift, never outcome drift; every alert prints test, window, and sample; calibration drift without a ledger is UNKNOWN, not zero; multiple-testing control prints the adjusted alpha; suspended segments barred from fresh confidence by contract; determinism (identical input hash on re-run); audit row written for every run.

Gates at seal

• smoke: 8 m35 gates incl. channel separation, per-alert test/window/sample, calibration UNKNOWN without ledger, determinism + audit row (suite counts at seal per the build agent's seal report)
• sealed at 161ea90 on atlas-nl-shock-redesign

Failure modes

The ways this model declines, degrades or goes UNKNOWN - and what it prints when it does.

• Source disabled or unverifiable: logged as source drift with the expected cadence - a coverage event, never evidence about outcomes.
• Segment below the sample floor: monitors regardless, and the statistic is UNKNOWN, not a small number.
• No out-of-time ledger: calibration drift is UNKNOWN - the monitor reports the hole instead of filling it.

Sensitivity

• The PSI thresholds are printed policy: moving the suspend line changes which segments halt, and every alert names its test and window so the effect of a threshold change is traceable.
• The COD window split (2027 and earlier versus 2028 and later) is fixed in the contract; shifting it re-draws the base and current populations, and the window prints on every alert.