What building a SaaS product with AI agents really costs

Plenty has been written about how fast you can build with AI agents. Very little about what it costs. We have built two products this way — Pilot-Next for flying clubs and booxx for Dutch bookkeeping — both in production, both with real users. Here is the bill.

What genuinely got faster

The first version of something is not three times faster. It is roughly ten times faster. A complete screen with a form, validation, error handling and tests takes an hour instead of a day. That is not a marginal gain, and it changes what you dare to try: when an idea costs an hour instead of a day, you also throw it away an hour later.

The second thing that genuinely got faster is the boring work you normally walk around. Error handling on every path. Keeping translations current in five languages. Tests for edge cases you can see but would not bother writing down. That work is not hard, it is just tedious — and tedious is exactly what an agent does without complaining.

What did not get faster

Deciding. Every choice about the data model, every trade-off between two ways of doing something, every question about whether it belongs in there at all — none of that speeds up, because it is about your understanding of the problem, not about typing.

In fact it gets slower, because you decide more often. If execution is ten times faster, the next decision arrives ten times sooner. We have had days where it felt like nothing happened because all we did was choose — and where three screens appeared anyway. The centre of gravity has moved from making to deciding, and that is more tiring than it sounds.

The bill: where it went wrong

The interesting part is not what worked. Here are three things that went wrong, all in ways a human team would also produce — only faster, and therefore more often.

1. A fix that looked right and did nothing

Pilot-Next got a rate limiter on its login page. The code was correct, the tests were green, the header politely said limit: 5. In production it did nothing: the number of remaining attempts bounced around — 3, 4, 4, 3, 2 — instead of counting down.

The cause was not in the code but underneath it. The limiter counted per IP address, and that address arrived through two proxy layers, so every request appeared to come from somewhere else. The agent had built exactly what was asked; the ask was simply wrong.

What we took from that is not “AI makes mistakes”. It is that a green test suite proves your code does what you wrote down, and nothing more. The only reason this surfaced is that we gave the real site five wrong passwords in a row and watched the responses. Measuring in production is not a luxury; it is the only place where your code’s assumptions meet reality.

The temptation afterwards was to fall back on the X-Forwarded-For header. We did not: an attacker fills that in themselves, so it is a brake you can reset by lying. A brake like that is worse than no brake, because it makes you feel protected.

2. A configuration file that lied

The same repository has a render.yaml declaring autoDeploy: true for three services. We relied on that for months, assuming a merge to master publishes the site.

It did not. For one of the three services the hosting platform itself had it switched off. The file in the repository is a blueprint for creating a service, not a live view of its settings; change something in the dashboard afterwards and you leave the file behind as a neatly documented untruth.

This is not an AI mistake, but AI pace makes it more dangerous: if ten times as many changes come through, the question “is this actually live” is relevant ten times as often. Since then we check the real setting through the API, not the file that claims to describe it.

3. A number that came from nowhere

The Pilot-Next site carried a rating of 4.9 out of 5 for years. The structured data held nine genuine reviews. When those reviews were finally rendered on the page as well, the average turned out to be 5.0 — all nine were five stars. The 4.9 could not be derived from anything.

It was not a lie someone invented; it was a number typed in once that nobody revisited. That is exactly the kind of error that scales when you build fast: not the spectacular one, but the small claim nobody checks any more. Now the average and the count are computed from the same list of reviews at build time. There is no longer a place where you can set the number by hand.

What it means on balance

Building with AI agents is not a matter of less work. It is a matter of different work. The typing largely disappears; what remains is architecture, judgement and verification — precisely the part seniority exists for.

The three things that make the difference:

1. Gates you cannot route around. Tests, typecheck and build green before anything merges. An agent that edits a test to make it pass should not get through.
2. Measuring in production. Every assumption about behaviour outside your own machine is a hypothesis until you have checked it on the real site.
3. One source per fact. Every number you display should be computed from wherever it actually comes from. A figure you can set by hand will drift eventually.

What it is not is a way to make software without senior knowledge. The agents are extraordinarily good at carrying out an instruction. The price of a wrong instruction has merely dropped from a week to an hour — which makes it more important, not less, that someone knows which instruction is the right one.