Vibe coding on a leash: thirty years of QE meets an AI pair programmer

I learned to program in machine code, on a ZX81 and then a Spectrum, an Acorn Electron, an Atari ST. There was no debugger worth the name, no version control, no undo. A crash was a white screen and a cassette reload, and the cost of a mistake taught you to check your work before the machine did. My career started at IBM and turned into thirty-odd years of quality engineering: automated test frameworks, tooling, test strategies for programmes complex enough to need one. The next chapter is director of QA for a very large insurance company.

Meanwhile, at home, I build software the new way. Across the five projects on this site, AI assistants have written most of the code, at a speed that would have sounded like satire five years ago: a 25,000-line working desktop app in a day, a node-graph engine in a week, a SaaS layer in a night. The industry calls this vibe coding, and the term is doing pejorative work. It means code you didn't read, shipped on feel.

Every instinct from my day job says this should end in tears. Six months of doing it says something more interesting: the speed is real, the tears are optional, and the variable that decides between them is the one my profession has been tending all along. The harness.

The leash

The clearest example is the novel-writing app I built this year. Day one produced 112 files and a 23-table schema, almost all of it AI-written, and the same day produced 130 passing tests: unit suites, integration tests running real IPC handlers against in-memory SQLite, end-to-end scaffolding. Not because a prototype deserves ceremony. When the code arrives faster than you can read it, the test suite isn't ceremony, it's the leash. You cannot review 25,000 lines a day with human eyes; you can hold them to assertions.

That inversion is the whole thesis. Quality engineering used to arrive after the code, pleading for testability. At machine speed it has to arrive first, because it is the only thing that scales with the machine. The spec and the harness are now the two highest-leverage documents in any project I run; the code is what happens between them.

What the harness looks like in practice

The pattern repeats across every project here, and none of it is exotic. It's the standard QE toolkit, applied earlier and enforced harder:

Entry criteria for AI-written code. When an AI-built Electron app hit 14,525 lines of main process and 787 any types, the paydown ended with a 1,073-line coding standards document that functions as standing instructions to the AI assistants. The bar doesn't move because the author types faster.
Equivalence oracles for migrations. Rebuilding nine agent capabilities as portable modules had one hard requirement, byte-identical LLM behaviour, proven by a verification script that diffs every rebuilt specialist against its old definition. A migration without an oracle is a guess.
Determinism as a test asset. VidViz's export pipeline re-executes the graph from a fixed seed, so the same project produces the same bytes and an entire bug class structurally can't exist. Reproducibility is what turns "looks right" into "diffs clean".
Behavioural probes over status checks. Three months of debugging one bash script reduced to a single rule every test framework I've built also converged on: process-existence checks verify the corpse is warm; only a request through the front door verifies the service.
Falsifiable evaluation. The stock-screening backtests carry an alpha-against-SPY column for the same reason a test suite carries assertions: a number that can't fail isn't evidence. And the honest reading of a good result is "earned the right to be paper-traded", which is test-exit language if I've ever written any.

The strangest new entry in the toolkit: reviewing the reviewer. One of my apps uses an LLM to review manuscripts, and its findings only reach the user after a second LLM pass tries to knock each one down. The oracle problem doesn't disappear when the tester is a model. It moves up a level, and the discipline moves with it.

The review economics just changed

The other half of the story is cheaper than a harness: the same models that write the code are unreasonably good at reviewing it. When an AI-built billing system landed in one of these projects, three AI review passes over the following day found thirteen real bugs, including an unmetered endpoint on the most expensive code path and a period-end field that could never be read. Every one of them had survived the author. Every one of them cost minutes to find.

For thirty years, review has been a gate you ration, because senior attention is the scarcest thing in any engineering organisation. When a full pass costs minutes, review becomes a loop you run. That is a genuinely new budget line, and most quality strategies haven't noticed it exists yet.

What stays human

The machine generates, the machine critiques, and the part that remains stubbornly ours is deciding what correct means. Somebody chooses the assertion. Somebody decides that beats detected by an algorithm should be replaced by a formula a human can reason about, that a licensing check belongs in the process you trust rather than the one you don't, that a demo project should be impossible to break. Every one of those is a quality decision, made before any code existed, and no amount of generation speed touches them.

That, I think, is the real shape of the job ahead, for me and for every QA organisation staring at AI-assisted delivery: less time writing the checks, more time deciding what must be true. In a regulated industry, "approximately, probably" was never an acceptable answer anyway. The teams that thrive won't be the ones that ban the speed; they'll be the ones that build the leash first.

The dog is faster than you now. Build the leash first.