Tap Notes: Trust the Hook, Not the Prompt

The common thread today: stop asking systems to behave and start building the thing that makes them. A pre-commit hook beats a paragraph in AGENTS.md. A custom VCS beats a git that wasn’t built for 30,000 merges a day. And a cryptographic backdoor beats any eval that assumes the model is showing you everything.

Enforce it, don’t ask for it

Quoting Geoffrey Huntley

Huntley’s agent kept bypassing his diagnostics library. Updating AGENTS.md left a gap that drifts. So he wrote a pre-commit hook that scans the diff for direct calls to observability modules, fails the commit, and injects a prompt telling the agent to use the library. Compliance went to ten out of ten.

Why it matters: Prompts are suggestions. A hook is a wall with a sign on it. If a convention matters, encode it where the agent can’t skip it, and let the failure message do the teaching. Steal this one today.

Redesigning source control at Anthropic

Anthropic went from roughly 2,500 merged changes a day in May to over 30,000 by September. Git couldn’t keep up with a 100 GB main-branch history and about 25 million CI checkouts a day, so a team of seven built a centralized replacement called Scion and migrated in five months.

Why it matters: This is what “agents in production” looks like when the bottleneck moves. The code got cheap; the plumbing underneath it didn’t. Also worth noting: their adversarial reviews were effective but myopic, and engineers had to steer the model away from over-engineering against hypothetical concerns. Reviewers need supervision too.

Trust, but you can’t verify

AI models can provably hide arbitrary capabilities

The paper embeds an encrypted neural network inside a host model’s weights, decryptable only with a secret trigger. It’s provably resistant to elicitation, even for someone holding the full weights. The authors released the models.

Why it matters: Hiding a fixed string was a party trick. Hiding arbitrary computation means an eval can tell you what a model does on the inputs you tried, and nothing more. Capability audits just lost their guarantee. If your safety story is “we tested it,” this is the footnote that breaks it.

Infrastructure worth reading

Under the hood, off the hot path: how D1 scales to millions of databases

Cloudflare’s deep dive on D1’s control plane: how they manage metadata for millions of SQLite databases while keeping query execution at the edge.

Why it matters: One database per user or tenant is where a lot of agent-era apps are heading, and the hard part isn’t the databases, it’s the control plane. The design principle of keeping management work off the query path applies well beyond D1.

I thought AI was gonna get cheaper?

Theo’s argument: per-token prices did fall, but models burn far more tokens per task, so the bill per outcome doesn’t.

Why it matters: Price per token is the wrong unit. Track cost per completed task, or you’ll keep being surprised by invoices. The conclusion is the predictable Jevons one, but the measurement advice is the useful part.

We turned EmbeddingGemma 2 into a speech-to-text model

The team decodes speech by finding the nearest of about 229k English words in the embedding space, training only a tiny reader on top of the existing model.

Why it matters: It’s a clever reuse of a model built for something else, and it runs on-device. The open question is how brittle nearest-neighbor decoding gets on rare words and noisy audio. Worth a read for that alone.

Three items about trusting what you can’t see, and two about what it costs. The hook is the only one you can ship by lunch. 🪨