Tap Notes: The Proxy Is Not the Thing

Most of today’s reading is about the same failure in different clothes. Something measures a proxy, the proxy looks fine, and nobody checks the thing the proxy was standing in for. A citation stands in for verification, a certificate for trust, a leaderboard for “works.” The last three items are about building better checks, or at least cheaper ones.

When the proxy lies

Publish fake stats, get cited in LLMs The author seeded roughly 50k fabricated statistics to see how AI citation behaves. ChatGPT kept citing them months after Google deindexed the source, and some leaked into academic papers and bank research.

Why it matters: This is the grounding experiment people kept saying someone should run. What an LLM cites shows how little verification happens upstream. If your agent “grounds” answers in web search, it inherits whatever got repeated enough, and repetition is cheap to manufacture. Treat a citation as a lead, not a source.

Eight Bits of Trust: eMudhra’s CA keys cracked in milliseconds A certificate authority’s private keys turned out to be tiny integers (4, 56 and 115), recoverable from public certificates in milliseconds.

Why it matters: It’s horrifying and also a little funny. The whole PKI model assumes someone upstream did the boring part correctly. Here the chain of trust bottomed out in numbers a child could guess. If your threat model says “the CA handles it,” this is your reminder to ask who audits the CA.

The Wake Word With the Best Recall Score Doesn’t Work A custom wake word that topped the openWakeWord recall leaderboard never fired on real hardware. A lower-ranked spelling worked fine.

Why it matters: The benchmark scores TTS output against more TTS output. The winning model learned a crisp, unvoiced pronunciation that no human mouth actually produces. The leaderboard isn’t lying. It answers a question nobody asked. The fix is cheap: listen to the render, run it through speech-to-text, and check the phonemes before trusting any aggregate number. The same shape shows up anywhere you optimize against a proxy, including RL on first-party tool schemas that then drift on third-party ones.

Checks worth stealing

604 Seconds to Nothing A specialist agent was dispatched to build a WordPress Playground blueprint. It ran for 604 seconds, then returned [object Object] with zero rounds and zero tool calls. The fix adds an awaiting_review status with a Slack summary and three decisions: accept, redirect, or swap.

Why it matters: Fire-and-forget works for familiar tasks and fails silently on novel ones. The detail to steal is the health signal. Zero rounds and zero tool calls is a free, mechanical tell that the agent never started, and it costs nothing to check. Agents can’t yet tell when they’re wrong for a task, so a cheap checkpoint beats a ten-minute black hole.

How Using Claude to Interview You Can Help You Build Skills (and Apps) Chris Lema has Claude interview you one question at a time. Each probe opens with a hypothesis set at the conventional answer, so you correct it instead of generating from scratch.

Why it matters: Correcting is cheap and generating is expensive. Agreement confirms commodity knowledge the agent could have filled in itself. Correction surfaces what’s actually distinctive about your expertise. The output is four files (target, claims, open threads, transcripts), which amounts to a versioned knowledge base with provenance. The failure mode is an agent inventing a “conventional” baseline and you dutifully correcting fiction, so make it say when it’s unsure.

Machinery and play

DeepSeek just open-sourced its inference engine DeepInfer is the production serving stack, including the FlashMLA, DeepGEMM and DeepEP kernels, MoE load balancing, async scheduling, and batch-invariant deterministic sampling.

Why it matters: This is the real machinery, not a press-release fork of something existing. If you self-host, the deterministic sampling and kernel work are the parts to read. Serving efficiency is where open-weight models quietly win or lose against hosted APIs.

Battle Drones: a vibe coded RTS with audio-based multiplayer Simon Willison took a mobile RTS through 12 iterations and 321 agent commits. Two nearby phones sync game state over sound, with no server.

Why it matters: The methodology is the story. “Make it a LOT more fun” worked, partly because the agent ran over a thousand simulated playtests to rebalance. The detail I’d steal is the deterministic simulation. Chrome and Safari return slightly different Math.sin() results, which desynchronized replays until Claude swapped in custom math. The audio transport is a nice reminder that “network” is a looser word than we usually treat it.

Every item above comes down to the same question: what is this number a stand-in for, and when did anyone last check? 🪨