Building software with AI: seven questions to ask

Almost every software firm now says it uses AI. On its own that tells you nothing — the difference is in what is built around it. We build two of our own SaaS products entirely with AI coding agents and run them in production with paying users, so we know both the gain and the bill. These are the seven questions we would ask ourselves, with what a good answer sounds like.

1. Who owns the code, and where does it live?

The code should be yours: your repository, your infrastructure, your keys. That is not a given with AI-driven builders — some deliver on their own platform, where the “AI acceleration” sits in a place you cannot take with you.

A good answer is a repository URL and a list of accounts in your name. A bad answer is that you can “of course export it at any time”.

2. Who is responsible when the agent gets it wrong?

There is only one workable answer: the builder, exactly as with hand-typed code. An agent is a tool. Whoever chooses the tool carries the result.

Watch for phrasing that blames the model. “That is how the model generated it” is not an explanation but a transfer of liability, and it predicts how the conversation will go when something breaks in production.

3. How do you know the code is secure?

This is the question most needed and least asked. In the 2025 GenAI Code Security Report, Veracode tested the output of more than a hundred language models on security tasks: 45% of code samples failed the security test and introduced OWASP Top 10 vulnerabilities. For Java it was 72%. (Source: veracode.com, published 30 July 2025, retrieved 7 September 2026.)

That figure does not mean AI code is unusable. It means unreviewed AI code is unusable, and that the question is not “do you use AI” but “what stands between the agent and production”. A good answer names concrete gates: static analysis, dependency scanning, and a human who reads every change before it is merged.

4. How is it tested, and on what?

Green tests prove the code does what was written down, and nothing more. That sounds like a platitude until it happens to you: at Pilot-Next we built a rate limiter on the login page that was fully tested and did not work in production, because the IP address arrived through two proxy layers. The agent had built exactly what was asked; the question was wrong. That story is written out in what building SaaS with AI agents really costs.

So a good answer is about where the tests are, not how many there are. Money, permissions and integrations deserve coverage; chasing full coverage on the rest is waste. booxx has almost four thousand automated tests, concentrated on the bookkeeping invariants — a journal entry that does not balance simply must not be able to exist.

5. Can another firm take it over later?

This is the question that buys off the most future cost, and it is almost never asked while choosing a builder. AI code can work perfectly well and at the same time be structured so that nobody else can follow it.

A good answer is checkable without you being able to program: is there documentation a next developer can start from, does the deploy pipeline run on your account, and are the tests readable enough to serve as a description of the behaviour? When we take over existing software we often write the tests first, precisely because they are the documentation that was missing.

6. What does not get faster?

If a builder has no sharp answer here, they have not been doing it long enough. The honest answer is: deciding does not get faster. It gets slower, because you decide more often — if execution is ten times faster, the next decision arrives ten times sooner.

That has a consequence for you as the client, and it is the most important sentence here: the centre of gravity shifts from building to deciding, and therefore from the builder to you. A project with AI agents does not stall on capacity; it stalls on a client who has one day a week for it. So ask what is expected of you, and how often.

7. Where does our data go?

Agree up front what does and does not go to an AI vendor, and put it in writing that this is a per-feature choice rather than a default setting. With document extraction the cheap path is often also the safe one: pull the text layer out of the file first, and only consult a model when in doubt.

Also ask whether the model call sits behind a single layer. If it does, switching to another model — or to one that runs in Europe — is a configuration change rather than a rebuild.

The seven questions in one table

Question	What a good answer sounds like	Warning sign
Who owns the code?	Your repository, your infrastructure, your keys	”You can always export”
Who is liable?	The builder, as with hand-written code	”The model generated it that way”
How do you know it is secure?	Static analysis, scanning, human review before merge	”The agent tests itself”
How is it tested?	Coverage on money, permissions, integrations	A percentage with no location
Can another firm take over?	Docs, pipeline on your account, readable tests	”We sort that out at handover”
What does not get faster?	Deciding — and that costs your time	”Everything is ten times faster”
Where does our data go?	Agreed per feature, call behind one layer	”It’s GDPR-proof”

Why we can write this down

Because we paid the bill ourselves. Pilot-Next and booxx are not demos but running products with paying users, built with agents under our own review, including the times it went wrong. That is a different kind of knowledge from a presentation about AI acceleration.

If you want to know whether this fits what you want to build — send an email. Half an hour is usually enough to tell, and if it does not fit we will say so straight away.