Competent Enough: AI Agents Changed Who Writes the Code, Not Who Answers for It
TL;DR
The big leap in AI coding didn't come from bigger models — it came from agents that can search, run tests and try again. But a loop is only as good as the thing that tells it "wrong". Write that oracle — tests, specs, invariants — and the machine is competent enough. Skip it, and you're shipping confident slop.
Something odd has happened to the AI conversation among people who actually ship software. A year or two ago, every discussion started with which model. Today the people getting the most out of these tools barely care. They talk about tests, about specs, about what the agent is allowed to touch. The model has quietly become a commodity, and the interesting work moved somewhere else.
I think that shift is the most useful way to understand AI-assisted programming right now — what it’s genuinely good at, where it falls over, and what it leaves for us.
The model stopped being the story
Spend time with the current crop of frontier models and a pattern shows up: for everyday programming, they’re converging. The gap between the best and the merely good keeps shrinking, open models are close behind the closed ones, and the difference you feel between them in practice is smaller than the marketing suggests. How much room you give a model to think often matters more than which logo is on it.
So where did the leap everyone feels come from? From the harness. A model on its own has to answer in one pass, from whatever it memorised during training. An agent doesn’t. It can look things up when it needs them, run the code, read the error, and try again. The model becomes less of an encyclopedia and more of an operator. METR, a research group that tracks this, measures how long a task an AI agent can complete reliably — and that length has been doubling roughly every seven months for years. Much of that curve is tooling, not raw intelligence.
That reframes the hype. A lone model is an expensive parrot. Put it in a loop with tools and feedback, and it becomes something you can actually work with. Not a genius. Competent enough.
The loop needs an oracle
Here’s the catch, and it’s the part I wish every team adopting these tools would tattoo somewhere visible: a loop only converges if something tells it when it’s wrong.
An agent with a test suite is a different animal from an agent with a paragraph of instructions. Give it tests and it proposes, runs, fails, reads the failure and corrects — dozens of tiny fixes you never see, until the thing passes. Give it only prose and it does its best guess, declares victory, and moves on. Point it at a domain with no documentation and no tests at all and it simply goes in circles, confidently.
This also quietly kills a popular debate: which programming language is best for AI? Once an agent can run the code and check the result, the language matters far less than people expect. Terse syntax doesn’t magically save effort, and strong types don’t rescue a misunderstood requirement — the hard part is understanding the contract, and that looks the same in any language. The real variable isn’t Rust versus Ruby. It’s whether anyone wrote down what “correct” means.
Margaret Hamilton in 1969 with listings of the Apollo Guidance Computer software her team built at MIT. Public domain, via Wikimedia Commons.
None of this is new, by the way. Margaret Hamilton’s team didn’t put people on the Moon because they typed fast. They put them there because they obsessed over how the software could fail and what it had to do when it did. The discipline was the product. Agents just make that discipline matter more, because they multiply whatever you give them — rigour or the lack of it.
Proofs: scope, not coverage
If tests are good, are mathematical proofs better? It’s a fair question now that AI is getting surprisingly capable at writing machine-checked proofs — and the answer is a useful dose of realism.
Formal verification proves that a program matches a specification, under a model of the world. It doesn’t prove “no bugs”; it proves “no bugs of this kind, if these assumptions hold”. And it is brutally expensive. The classic example, the seL4 microkernel, is under ten thousand lines of C, yet its correctness proof runs to hundreds of thousands of lines and took on the order of twenty person-years. Change the code and the proof can break, in ways far harder to fix than a failing test. And everything outside the model — the compiler, the hardware, the libraries you call — sits outside the guarantee.
AI may well bend that cost curve. But the lesson isn’t “prove everything now”. It’s scope: pick the small, critical core where a failure would be catastrophic, specify it precisely, prove it, and test the rest. Anyone promising “bug-free code” is selling the wrong thing.
Cheap discovery changes the maths
The sharpest consequence of all this is about security. For decades the industry quietly accumulated technical debt — shallow reviews, decorative test coverage, over-privileged service accounts, secrets lying around in environment variables — and got away with it, because finding the problems was expensive. It took skill, time and patience. Mostly, nobody looked.
That assumption is gone. The same models that help us write code are very good at finding flaws in it, and they’re available to anyone with an account and an afternoon. The debt didn’t get bigger. It just got much easier to find.
The Harvard Mark II logbook, 9 September 1947: a moth found in a relay, taped in as the “first actual case of bug being found”. Public domain, via Wikimedia Commons.
The good news is that the lever works both ways. Tools that make attacks cheap also make audits cheap. When discovery is nearly free, not looking becomes the expensive choice — the same way HTTPS and two-factor authentication went from “nice to have” to “why don’t you have that?”
So what’s left for us?
A lot of front-end and glue work is now genuinely mechanical for an agent: turning a finished design into markup, fixing the responsive layout, wiring a form to an API, writing the boring tests. That’s real, and pretending otherwise doesn’t help anyone. Big systems full of undocumented rules and legacy integrations are a different beast — but turning a decision that’s already been made into code was always the mechanical part of the job.
What wasn’t mechanical is the list that’s left for us:
- Deciding what “correct” means — the tests, the specs, the invariants the loop runs against.
- Choosing the direction and rejecting the plausible-looking wrong answer.
- Knowing the fundamentals well enough to spot the time bomb that renders beautifully: networks, caching, databases, accessibility, security.
- Answering for the result. An agent can write the code. It can’t be accountable for it.
That last one is the whole game. The career ladder of “easy tickets” is shrinking, and it’s better to say so plainly: the market for people who only translate decisions into code gets smaller and more selective. But the people who thrive will be the ones who treat the agent the way Hamilton’s team treated the guidance computer — as something extraordinarily capable that must be told, precisely, what success looks like.
Competent enough is a high bar for a machine. The trick is making sure it’s not the only competent thing in the room.