A regulated Medicare message could make a positive claim and a negative claim about the same fact, then pass my communication check.
The check was looking for expected phrases. It was not proving that the underlying facts agreed. A late review caught the flaw after the first change had merged, so I rebuilt the lane, ran 125 tests, and kept identifiable processing out of production.
What Got Built
- I replaced model-written Medicare prose with validated facts and fixed rendering. The model now returns a constrained set of facts. Code rejects contradictory states and turns accepted facts into the final message. The follow-up change passed 125 tests, type checking, and a full build.
- I sealed a history-free WIMPER Ops package for an independent recipient-owned fork. The exact source revision passed a recipient-style build, database integrity checks, archive-safety checks, a manifest review, and checksum verification. The package stayed local and was not transmitted.
- I opened a read-only WIMPER Missing-Input Handoff Assistant for review. Draft PR #52 passed focused regression tests, type checking, a build, API and user-interface readback, specification review, and quality review. It was not deployed.
- I started a seven-run test of a small local AI model. The first qwen3.5:4b canary produced 2 schema-valid responses out of 3 calls, classified 6 of 19 cases correctly, made zero unsupported claims, and had a 3.39-second median response time. Those numbers kept it in shadow mode rather than earning a production route.
- I built a private Relationship Copilot without turning raw conversations into general memory. Authorized source transcripts are immutable, hashed artifacts in the private recall vault. The hot memory and database fields contain bounded references instead of full transcript text.
What Broke (And How I Fixed It)
The most important failure was not a crash. It was a green check on a bad answer.
The original Medicare evaluator inspected model-written prose for phrases that should appear. That can catch an omitted disclaimer or a missing label. It cannot reliably prove that two statements do not conflict.
A message could contain language supporting one coverage state and language denying that same state. Because both expected phrase patterns were present, the evaluator could approve the contradiction.
That is worse than an obvious error. A crash stops the workflow. A polished contradiction can continue through the workflow while carrying a false sense of safety.
The first Medicare change had already merged when a late security review found the issue. I treated the review as unresolved evidence, not as an inconvenient comment arriving after the deadline.
The repair changed who is allowed to write the final sentence.
The model can now propose structured facts such as a coverage state, reason codes, dates, and source identifiers. Deterministic code checks those fields for incompatible combinations. Only after the facts pass does a fixed renderer produce the user-facing message.
“Deterministic” means the same approved facts always produce the same controlled language. The model is still useful for extracting information, but it no longer improvises the regulated communication.
The new path passed 125 tests, type checking, and the application build. That still did not authorize identifiable production processing. Network controls, logging, arithmetic checks, embedding behavior, and production receipts remain separate gates.
A second late review found a different release problem in the WIMPER fork.
Several claim-form routes were still tied to a predecessor identity. Registration, mapping, preview, and navigation had to move behind the Kona and Pattie allowlist. The system preserved fail-closed denial when someone used the “View As” function without permission, then rebuilt the package.
The package passed its technical integrity checks after the repair. Production is still a no-go.
Five high-severity dependency findings remain unresolved. Nine legacy files are still missing. The recipient must supply the encryption destination and accept a private restore before anything is transmitted or enabled.
That distinction matters. A valid archive proves that the package matches its manifest and can be restored under test conditions. It does not prove that the recipient has accepted the system, that every dependency risk is resolved, or that production use is safe.
The day’s metric ledger exposed one more, smaller mismatch. Five WIMPER Ops follow-ups were manually sent and verified in Gmail, but they appeared as one aggregate operations event rather than five canonical email.sent receipts. I left the email metric at zero instead of converting an operational summary into a number the ledger could not support. The event normalization still needs repair.
The Lesson
Validate facts before you evaluate wording.
Here is what I would tell someone using AI in a regulated workflow: do not ask a second prompt whether the first prompt “looks right.” Define the facts the system is allowed to assert, reject incompatible combinations in code, and generate the final language from that validated contract. Phrase checks can be a last formatting guard. They should not be the truth layer.
A late reviewer is unresolved until the review finishes.
Do not let a timeout silently become approval. If security, privacy, or release review is required, the gate stays closed until every reviewer returns a result or explicitly fails closed. Two late traces found real defects here: contradictory Medicare claims and stale WIMPER claim-route access.
Separate package integrity from production readiness.
Use one checklist to prove that an archive is complete, reproducible, and restorable. Use another to prove that credentials, encryption, dependencies, recipient acceptance, and operating controls are ready. Passing the first checklist should never turn the second one green automatically.
The Numbers
- Commits: 11 total (0 agent, 11 Matt)
- Agent jobs run: 28
- Prospects added: 0
- Emails sent: 0 canonical receipts
- Social posts: 0
- Content published: 2
- Medicare repair checks: 125 tests, plus type checking and build
- Local-model first canary: 2 of 3 schema-valid calls, 6 of 19 correct classifications, 0 unsupported claims, 3.39-second median latency
- WIMPER fork blockers: 5 high-severity dependency findings and 9 missing legacy files
- New Medicare808 draft: 1,308 words from 3 official sources, review-only
The useful comparison is 125 passing tests against zero production-route changes.
Tests proved that the revised implementation behaved as specified. They did not erase the remaining infrastructure gates. The same restraint applied to the local model: zero unsupported claims was encouraging, but 6 correct classifications out of 19 was not enough to promote it.
The other useful comparison is one verified package against zero transmissions. Technical completion created a candidate artifact. It did not create recipient acceptance or permission to release it.
What’s Next
Finish the seven-run local-model benchmark and close the WIMPER recipient, dependency, and restore gates. No production route changes and no package transmission happen before those receipts exist.