Note · keeping AI memory honest

Catching stale engineering notes before they mislead someone

A small daily check that keeps AI memory, wikis, and status docs honest against what actually shipped, plus the one command that applies what it finds.

We built this after a routine cleanup turned up dozens of engineering notes stating things that were no longer true: features marked unshipped that had shipped, functions called by names they’d been renamed away from, decisions a later note had already reversed. Nothing was flagging the drift. Re-running that cleanup by hand doesn’t scale, so now it runs itself.

Why the notes drifted, and what changed besides the check

The AI’s memory is meant to hold fast-moving project state: what’s decided, what’s blocked, what’s in flight. That part works. The mistake was writing status as a flat assertion: “committed, not pushed” is only true the instant it’s typed, and every day after that it’s a bet nobody’s collecting on.

So two things changed alongside the audit:

Status became a pointer. A note citing a commit hash and a date ages into “here’s how to check.”
Old claims get re-checked before they’re repeated as fact.

The actual rule, if you want to copy it

before “Shipped 2026-08-15, not committed”

after “Shipped 2026-08-15, commit a1b2c3d, confirmed ancestor of origin/main as of 2026-09-11”

Generalized: never write committed / uncommitted / pending / shipped as a bare adjective. Pair every one with what to check it against (a commit hash, PR number, migration filename) and when you last checked, including the ones you’re marking resolved right now, since that claim will age exactly the same way.

Tell your Claude this, directly or in CLAUDE.md

When writing status into persistent memory, never state "committed" /
"uncommitted" / "pending" / "shipped" as a bare claim. Always attach:
  1. what to check it against — a commit hash, PR number, or migration filename
  2. the date you checked it

Bad:  "Shipped, not committed"
Good: "Shipped 2026-08-15 — commit a1b2c3d, confirmed merged to main
       as of 2026-09-11"

This applies when marking something resolved too — write what you verified
and when, not a fresh assertion that will go stale the same way.

Some drift is still unavoidable. Any note asserting a fact about a system that keeps changing needs a periodic re-check, for the same reason a cache needs invalidation or a wiki needs a link-checker. The audit is the permanent piece, and it has less to do now that the upstream change shrank its workload.

Setup

Two small pieces: a scheduled trigger, and two Claude Code commands that split the work in half: one finds problems, the other fixes them.

/audit-memory

Finds drift. Runs headless, on a schedule, and never writes anything.

/audit-memory-apply

Fixes drift. Runs interactively, only when a person triggers it.

A cron-style trigger on any machine that can already reach both the notes and the repo. We use macOS launchd firing once each morning; a scheduled cloud job works the same way for a shared/team setup.
A ground truth to check against: a commit hash, PR number, or ticket ID cited in the note. The check is only as good as what it can cross-reference.
Notes with no version control of their own, which is exactly why the audit never edits unattended, and a human triggers every write.

Audit

Runs once a day, unattended. Flags drift, verifies it against git, writes a report, and stops. It never edits a note itself.

Flag. Greps every note for stale-status language: unpushed, uncommitted, pending, OPEN, awaiting, and siblings. Durable style-guide notes are excluded; they don’t go stale just because the incident that prompted them shipped.

Verify. For every commit a flagged note cites: on the main branch means shipped and live; on a dev branch only means promoted but not released; on neither means still genuinely open, or unverifiable if the reference has since been rewritten.

Report. Findings become the command’s own reply, captured into a dated file by the wrapper. The session never writes the file itself, since a headless session refuses to write outside its own working directory anyway.

Stay quiet. A notification fires only when something new turns up. A clean day sends nothing, so the signal stays worth opening.

Apply

The audit only ever proposes. This is the whole confirmation step: one command, run by a person.

/audit-memory-apply

Loads the latest report.
Re-verifies every finding itself against git before accepting the report’s suggested fix, the same way you’d read the diff before merging a review comment.
Edits the notes directly, then prints one line per file changed.
Runs interactively only. A human always triggers the write, even though verifying and editing are automatic.

Example

A representative run: 48 notes matched the stale-language grep, four turned out to be genuinely wrong, and only one was about whether code had shipped. The build in Section 03 checks git; the other three needed a grep, a cross-referenced note, and a config diff. Same pattern, different ground truth each time.

48 notes flagged4 genuine findings~5–6 min runtime

project_rate_limiter_rollout.mdship status

Body already read “shipped, live in production”, while the summary line above it still said “not committed.” A prior fix had corrected the prose and missed the metadata sitting right above it.

checked via: git merge-base --is-ancestor on the cited commit

project_webhook_signature_verification.mdreference drift

Explained the flow by name-checking verifyWebhookSignature(); a later refactor renamed it to verifyInboundSignature() and moved it to a different file. This claim had nothing to do with commits or ship status. The function had simply been renamed.

checked via: grepped current source for both names

project_user_preferences_storage_approach.mdsuperseded decision

Recorded a real decision (preferences live in a JSONB column on the main user table) that a later note on the same topic explicitly reversed in favor of a dedicated table. The older note was never marked superseded, so on its own it still reads as current guidance.

checked via: cross-referenced against the later note

project_free_tier_limits.mdstale figure

Stated the free tier allows 3 projects per workspace; the actual limit in the plan-limits config had since changed to 5. No commit was cited to check. It was a plain fact that drifted the moment the config changed under it.

checked via: diffed against the current config value

Before you adopt this

Worth knowing

Costs a few minutes a day, and runs against the same usage budget as normal sessions, so schedule it away from heavy interactive use.
A team-shared version needs a different trigger than a personal cron job: a scheduled cloud job with repo read access.

Stays true on purpose

Audit-only is a starting posture. Promote to auto-apply once the report has a track record you trust.
No PR-style review exists for these notes, so the entire “confirm” is a human skim plus one command. That’s lower friction than code review, and a lower guardrail for exactly that reason.

We build tooling like this into how we work as well as into what we ship. Notes on the practice, as we refine it.