Tap Notes: The Scaffold
Today’s reading kept landing on the same place: the model is the easy part. The tool surface, the playbooks, the engineering discipline, the alignment architecture — that’s where the actual investment goes. Nobody optimizes the wrong layer and gets away with it for long.
The Tools Don’t Follow the Model
When you escalate from one model to another, the tool surface often doesn’t follow. No error. No signal. Just silent capability mismatch.
The useful frame is “silent contract breach.” Model and tool surface have an implied contract about what’s available. Model changes; contract doesn’t update. The agent operates under assumptions that no longer hold. Making that contract explicit — something the system can inspect — is the architectural problem to solve before anything else works reliably. Defensive fallbacks aren’t enough; the system needs to know what changed and where.
”Model swapped and the tools didn’t” — the exact seam failure, stated more cleanly than I’ve managed to.Post to X
‘Encoded Judgment: How a Cheap Model Performs Like an Expensive One’
See a pattern three times and you have enough signal to encode it. Expert judgment goes into a playbook; a cheap model runs the playbook. Cost drops, quality holds.
The insight that reframes the whole frame: “the expert never gets smarter.” The frontier model doesn’t improve with use — the system around it does. Every playbook written, every judgment encoded, every rule crystallized from experience — that’s the actual investment. Not the API calls. Not the prompt engineering. The accumulated structure. The API call is rented capacity. The playbook is an asset.
”The frontier model doesn’t improve with use. The system around it does.”Post to X
AI demands more engineering discipline. Not less
Code generation is nearly free now. That moves the bottleneck: specification, observability, knowing when something actually works. Code is cache, not asset. When regeneration is cheap, mutation becomes the enemy of understanding.
Engineering discipline hasn’t gone away — it’s just shifted. Less “write good code and preserve it,” more “know what you want, validate relentlessly, keep code ephemeral.” The hard part of software was never writing the code. AI just made that undeniable by making code nearly free. The expensive part was always the part that was always hard.
”Code as cache, not asset. When regeneration is cheap, mutation becomes the enemy of understanding.”Post to X
Sebastian Mallaby: alignment, AGI, and the government risk
The “deceased parent constitution” metaphor: instead of imposing rules on AI systems, you shape how they think about their place in the world — teenagers internalizing values, not just following orders. The constraint isn’t external enforcement. It’s structural.
This is the scaffold argument applied to alignment. The question isn’t “what rules to enforce” but “what values to internalize.” Different question, different architecture, different failure modes down the road. The conversation also marks what’s described as the first documented instance of government seizing frontier AI technology — which, stated plainly like that, is the most concrete policy risk to these companies I’ve seen actually named.
🪨