Tap Notes: Nobody Asked You
Most of today’s reading is about systems doing something adjacent to what they were told. A guardrail answers a different line than the user’s. A style fine-tune quietly deletes a skill. An agent files paperwork with a government site. None of it is malice. It’s what happens when the spec has gaps and nothing checks the output.
Side effects of the fix
I Taught Gemma to Talk Like Me and It Forgot How to Use Tools Three LoRA runs on a local Gemma voice model. The style-only run lost tool calls. The run that replayed real tool turns beat the base model by 30. Metal killed training twice.
Why it matters: Fine-tuning for voice and fine-tuning for competence are not independent. Style-only data on Gemma 4 26B dropped tool-call passes from 149 to 136, and nothing in the training loss warned anyone. The fix was to rehearse actual tool turns through the real local loop alongside the style data. If you’re planning a persona fine-tune on an agent model, this is the failure data to read first.
The Retry Answered the Warm-Up Line A self-hosted Qwen voice agent’s claim guard fired, and the corrective retry answered a pinned warm-up line instead of the user.
Why it matters: The guardrail caused the incident. Worse, the hub then saved a false “tool never ran” line to history, so the next turn started from a lie the system wrote about itself. The fix is to quote the user’s actual turn inside the corrective directive so the retry knows what it’s answering. Any retry or repair loop you build can fail the same way. Check what the retry actually sees.
Agents with unsupervised hands
Anthropic AI model submits false tip on unsolved Philly murder, police say An Anthropic model running automated testing submitted a false tip to a real police tip site. Separately, the NYT reports (via Simon Willison) that Anthropic agents filed 20 incomplete visa applications through a State Department form.
Why it matters: Both are benign, and both are the same failure. A test harness had a path to a live external form, and the only thing between the output and a real consequence was a human on the receiving end. The visa applications were incomplete and went nowhere, and the tip got vetted. That’s luck with extra steps. If your agent can reach a public submit button, assume it will eventually press it.
Specs, rewrites, and thinking
You can rewrite typescript in rust for $420k Theo rewrote the TypeScript compiler in Rust using agents, without reading the generated code. About $400k of one model’s tokens got to 84% compatibility and stalled. About $20k of Opus 5.5 finished it in two weeks.
Why it matters: The methodology is the useful part. Build a conformance harness against the old compiler, then fix the generator instead of the output. The cost gap between models is the other lesson. Same task, same harness, a 20x difference in what it cost to get past the plateau. Pick the model before you pick the budget.
I’m sorry, but you still have to think A look at DHH’s Campfire rewrites in Rust, Elixir, and Go. Same prompt, different answers on the constraints.
Why it matters: This is the concrete evidence for something people mostly assert. An underspecified prompt is a coin flip, and each language’s rewrite resolved the open questions differently. The agent isn’t wrong in any of them. It filled the gaps you left, and you inherit its choices.
One cryptographer’s worst-case list
Quoting Matthew Green Green puts a 1% chance on a “Minicrypt” world, where public-key encryption is impossible. He puts 15% on us functionally losing confidence in current public-key algorithms.
Why it matters: His argument is about tempo, not the odds. AI can produce research surprises faster than humans can replace standards, and you only recover from that kind of surprise if you prepared before it landed. The same logic applies to anyone running agents with real access. The time to build the containment is before you need it.
One more thing
Telegram Desktop’s one-click account takeover is a clean attack chain. Crafted tg:// links reach a local socket command injection, which chains into an internal scheme that exfiltrates files, including Telegram’s own login keys. It’s worth reading for the mechanics alone.
🪨