I moved 5,166 messages out of the memory service behind my AI system and into a local vault I control.
The migration passed 39 of 40 recall checks and all 6 runtime checks. It also uncovered a historical X access token that still matched a current credential, which is exactly why I did not treat “the import completed” as proof that the migration was safe.
What Got Built
- I completed a staged migration from Honcho to built-in Hermes memory. Honcho was the outside service that stored conversation history and helped the system recall earlier context. The replacement keeps that memory on my own machine with source labels and a rollback path.
- The export preserved 5,166 messages before the cutover. The local system indexed 2,267 documents rather than blindly placing every raw record into the live recall path.
- I tested recall before changing production. The replacement passed 39 of 40 recall checks, all 11 safety checks, and 6 of 6 runtime checks after the switch.
- I started a seven-day Business Brain Problem Room pilot. It produces seven editable drafts and learns from the difference between its draft and my final edit. It still cannot publish automatically.
- I rebuilt the weekday owner agenda around one measurable target. The current objective is one partner-attributable employer account in 90 days, with one visible stage bottleneck instead of a long list of loosely related tasks.
- I ranked five proof gaps inside the existing revenue system. The priority is better receipts for replies, applications, commissions, qualified reviews, and social posting—not another venture or another scheduler.
What Broke (And How I Fixed It)
The dangerous part of a memory migration is not copying files. It is deciding what deserves to become active memory.
A raw archive can contain old assumptions, private conversations, duplicate statements, and credentials that should never be available to a recall system. If I had imported all 5,166 messages directly and declared success when the command returned zero errors, I would have confused data movement with safe operation.
The privacy review found a historical X access-token pair inside the Honcho archive. That pair matched the current credential stored in the private credential file.
The migration process isolated the finding instead of allowing it into the active memory index. Credential rotation is still pending, so I am not calling that problem fixed. The honest status is narrower: the local cutover passed, the exposed secret was detected and separated, and the remaining security action is to replace that token pair.
This is a useful failure because it appeared before I trusted the new system. An archive can be technically valid and still contain material that should not survive into production recall.
The fix was a sequence, not a better prompt.
First, I exported the original data so I had a complete recovery source. Then I separated records by provenance, which means the system retained where each memory came from instead of blending everything into one voice. I removed sensitive material from the recall path, indexed only the selected documents, and ran known-answer tests.
The recall benchmark passed 39 of 40 checks. I did not round that up to perfect. One miss means the new memory layer is usable, not infallible.
After the offline checks, I changed the runtime and ran six production-path checks. All six passed. The system also kept a shadow audit and rollback path so a bad cutover would not trap me inside the new provider.
Two other constraints remained visible.
X API credits are still exhausted, so the system confirmed zero social posts. It did not use an unauthorized fallback just to keep the posting metric alive.
The same-day work log was also unavailable when the chronicler ran. The report used 42 event receipts, GitHub commit receipts, the prior work log, current system status, and Atlas ledgers instead of guessing what the missing summary might have said.
Different problem, same rule: missing proof should reduce the claim, not increase the model’s freedom to invent one.
The Lesson
Migrate load-bearing systems in gates.
Here is what I would tell someone replacing an AI memory provider: do not make “old system off, new system on” one step. Use separate export, privacy review, indexing, benchmark, cutover, and shadow-audit stages. Each stage should produce a receipt and preserve a way back.
A migration is complete only when the new path works and the old path can be restored if it does not.
Treat an archive as untrusted input.
Conversation history feels familiar because you created it. That does not make every line safe to index.
Scan for credentials, private identifiers, stale instructions, and conflicting facts before the archive becomes searchable memory. Keep source labels attached so a future answer can distinguish something Matt said from something an agent proposed or a tool reported.
Test memory with questions that have known answers.
A demo where the agent remembers one favorite phrase proves very little. Build a set of expected answers across user preferences, project facts, operational rules, and safety boundaries. Record every pass and miss, then run separate checks through the real production path after cutover.
My score was 39 of 40 before cutover and 6 of 6 after it. Those numbers are more useful than saying the new memory “seems better.”
Do not solve a proof shortage with more activity.
WIMPER has 15 sent partner messages and zero replies, zero tracked clicks, and zero fresh bounces. Every exact pilot authorization has already been consumed.
The next useful artifact is not a larger batch. It is better proof at the missing stages: reply, application, commission, qualified review, and posting receipts. When a funnel stalls, identify the first missing receipt before adding volume.
Let tone improve without handing over publication.
The Business Brain pilot can store its delivered draft, compare it with my edit, and learn repeated changes. That creates a feedback loop without giving the system a publish button.
Learning authority and external-action authority are different permissions. An agent can get better at sounding like me while I retain the final decision about what leaves the system.
The Numbers
- Commits: 2 total (0 agent, 2 Matt)
- Agent jobs run: 20
- Prospects added: 0
- Emails sent: 0
- Social posts: 0
- Content published: 3
- Messages preserved in the memory export: 5,166
- Documents indexed locally: 2,267
- Recall checks passed: 39 of 40
- Safety checks passed: 11 of 11
- Post-cutover runtime checks passed: 6 of 6
- Event receipts used by the chronicler: 42
- Business Brain pilot duration: 7 days
- Editable pilot drafts: 7
The most important number is not 5,166. A large import can still be unsafe.
The important sequence is 39 of 40, 11 of 11, and 6 of 6. It shows that the replacement was tested for recall, safety, and real runtime behavior before I treated it as production.
The zero-send numbers matter too. Fifteen WIMPER partner messages remain under observation with no replies, no tracked clicks, and no fresh bounces. The system did not turn a quiet funnel into permission for another batch.
What’s Next
Rotate the exposed X token pair, keep the memory cutover under shadow audit, and build the missing WIMPER reply and application receipts before authorizing more outreach.