AI development
A nightly crew of AI agents: maintenance that continues while you sleep
Every night a crew of agents runs across two codebases: removing dead code, filling test gaps, measuring search visibility. How it is set up, and why it is only safe with the right guardrails.
Maintenance is the work that always yields to something more urgent. Dead code stays. Tests lag behind the features. The translation for a new screen is missing in three of five languages. Nobody chooses that — it is simply what happens when real features also have to be built.
For a few months now that work has continued overnight at our end. A crew of AI agents picks up one task each night, finishes it, and delivers the result as a pull request. In the morning there is an email describing what happened and what went wrong. Here is how it works and why we think it is safe.
The crew
There are three kinds of agent, and the distinction matters more than the numbers.
Maintenance agents run in rotation. One is up each night: the dead-code sweeper, the duplication hunter, the test pruner, the framework optimiser. They may remove and rewrite code, but they may not change behaviour. That is the line everything else rests on.
Watch agents run every night, outside the rotation, because their source lives outside the repository. The most important one reads Search Console and Bing Webmaster Tools each morning for the public sites, spots regressions and applies the improvement itself.
Sync agents maintain the things that drift apart if left alone: the translations, and the knowledge base behind the in-app assistant.
Alongside those sits one agent that is allowed to change behaviour: the builder that picks work off the kanban board. It may do so only because a human approved the ticket. That is an organisational guardrail rather than a technical one, and it is the most important of the lot.
The guardrails
An agent alone in your codebase at night is exactly as safe as the gates around it. These four do the work.
A worktree per agent
Every agent gets its own git worktree and its own branch. Two agents therefore cannot touch each other’s files, and a failed run leaves no half-finished work in the main checkout. The worktree is discarded afterwards if nothing changed in it.
Gates you cannot route around
Tests green, typecheck green, build green. On the bookkeeping product that means nearly four thousand tests, most of them about money and tenant isolation. An agent that wants to get through has exactly one option: actually doing it properly.
The temptation to “just adjust” a failing test is as strong for an agent as for a human under deadline. So there is a separate check for whether a test was modified in the same run as the code it guards. That is not watertight, but it closes the easy route.
One task per night
No agent does two things. A run producing one scoped change can be reviewed in five minutes; a run with seven changes does not get reviewed, it gets approved. That difference is the whole point.
Automatic merging is only allowed when main is not live
This is the guardrail we nearly got wrong. On one product, merging to the main branch publishes nothing — the site only goes live on a manual deploy. So there the nightly crew may merge on its own: the worst that can happen is that a change is queued up which we revert in the morning.
On a product where the main branch does publish immediately, that is not allowed. We confused the two for a while because the configuration file in the repository claimed something different from the hosting platform itself. The rule is now explicit: automatic merging is permitted as long as merging is not the same thing as publishing.
What it delivers
The honest answer: less spectacular than it sounds, and more than we expected.
What it does not do is build features. It is not allowed to, and that should stay that way.
What it does do is hold back the slow decay. Over a few months: thousands of lines of dead code removed, dozens of missing translations filled in, tests added where money is calculated, and a series of small SEO improvements we would never have picked up ourselves because each one is individually too small to spend an evening on.
The most valuable part may be something else entirely: the morning email. A short summary every day of what happened, what went red and what was left undone. That is a daily health check on a codebase you are otherwise not looking at, and we would not have had it otherwise.
What we would do differently
Start with one agent, not eight. For the first fortnight, repairing the crew cost more time than the crew returned. One agent behind a good gate is worth more than six behind a mediocre one.
Let failure fail loudly. The first version caught errors politely and moved on to the next agent. The result was a green morning email while three runs had quietly done nothing. A run that fails should turn the email red.
Budget for maintaining the maintenance. The crew is software too, with the same tendency to decay. It now sits behind the same gates as everything else.
If you take one thing from this, make it this: what is interesting about agents running overnight is not that they are cheap. It is that they do work which otherwise does not happen at all — and that is a different kind of gain from doing faster what you were already doing.