Building an Independent Verification Oracle
A checker that shares its subject's assumptions cannot falsify anything. How to build a second implementation that is independent enough to disagree, and how to prove it is.
Learn / complete list
Every tutorial, newest first.
The whole library in one list. Each piece is written from work on this site, carries original code, and states what it does not establish.
01Published
13 tutorials
Calibration Is Not Classification Accuracy
A model can be accurate and badly calibrated, or well calibrated and useless. What each property means, how to measure both, and why per-class thresholds are a decision rather than a result.
Designing Replay as a Normal Path
Offset handling, publish-then-commit ordering, and broker confirmation before HTTP success — the consumer-side decisions that make re-running a stream routine rather than an incident.
Equal Counts Do Not Prove Two Stores Agree
Why a matching row count is the weakest possible integrity check, and what a ladder of comparisons on identity, content and provenance actually rules out.
Evidence Provenance for Knowledge Systems
What a stored generation has to carry so a claim about it can be re-checked later: captured bytes, ruleset digests, and the difference between recorded and installed.
Hybrid Retrieval Without Silent Failure
Dense and sparse retrieval fused server-side, embedding contracts that are checked rather than assumed, and refusing to index rather than storing a truncated vector.
Idempotent Ingestion Across Heterogeneous Stores
Deriving identifiers so that a redelivered event converges three stores on the same state, without a distributed transaction and without a deduplication table.
Promotion Gates for Model Release
Promotion is not the next box along a diagram. It is a conjunction of independent checks, and one short check stops the candidate where it stands.
Production AI Systems Need Refusal Paths
Every stage that can degrade should be able to decline. A synthesis across ingestion, retrieval, extraction and inference of where refusal belongs and what it costs to leave it out.
Re-ingest, Verify, Reconcile, Withdraw
Four operator actions that make a knowledge system maintainable: what each one is allowed to mutate, why the boundaries matter, and how to make every outcome machine-readable.
Template-Driven Document Extraction With LLMs
Schema-validated extraction with a deterministic fallback, corruption detection before the model sees the text, and a contract that survives replacing the model.
What a Saliency Map Does Not Prove
Grad-CAM++ shows where activations were, not why a decision was made. How to use attribution honestly, and the test that tells you whether yours means anything.
Selective Prediction: When Models Should Abstain
A practical guide to risk-coverage trade-offs: use uncertainty to decide which predictions to accept, review, or defer.
Where these come from
Each one is written from work that is published beside it.
The case studies carry the evidence and the limitations; these carry the reasoning and the code. Where a tutorial names a project, that project links back to it.
Selected work