AI Can Write the App. What Still Has to Happen Before It Can Ship?

AI can write the app. What still has to happen before it can ship?

The first draft arrived before the coffee did.

It had routes, a database model, an authentication screen and a polished dashboard. A few years ago, that much working surface would have represented several days of deliberate engineering. An AI coding system produced it in minutes.

That is real progress. It also changes where the difficult work begins.

Code generation compresses the distance between an idea and something that looks finished. It does not automatically compress the distance between looks finished and can be trusted with another person's data. Those are different milestones, and the gap between them is where teams now need better habits.

GitHub's 2024 Octoverse report counted 137,000 public generative-AI projects, up 98% year over year. The direction is clear: more software will begin with generated code, generated dependencies or generated changes. The useful question is no longer whether AI belongs in development. It is what evidence a team needs before the resulting application belongs in production.

A working screen is not a release

Generated applications often begin by optimising for the visible path: submit the form, receive a response, render the result. Production adds all the paths nobody demonstrates in a prompt:

• the same request arriving twice;
• a user changing organizations halfway through a session;
• an identity provider returning late or not at all;
• a database write succeeding while an email fails;
• a dependency disappearing or publishing a compromised update;
• a secret reaching a browser bundle, build log or generated example;
• a rollback restoring code but not the data shape it expects.

None of these failures means AI-generated code is inherently unsafe. Human-written software produces the same failures. The difference is velocity: generation can create more unreviewed surface faster than a team can build a mental model of it.

The answer is not to distrust every generated line. It is to make the release boundary stronger than the generation boundary.

1. Recover the intent before reviewing the syntax

Review starts with a short written contract:

1. Who may perform this action?
2. Which data may they read or change?
3. What must remain true if the operation fails halfway through?
4. What will an operator see when it fails?
5. How can the change be reversed?

An assistant can produce convincing code while misunderstanding any one of these answers. If the contract is absent, a reviewer is forced to infer product intent from implementation details. That is slow and unreliable.

A useful generated pull request therefore includes more than code. It names the trust boundary, failure behaviour and acceptance tests. The explanation does not need to be long. It needs to be specific enough that a second person can disagree with it.

2. Treat dependencies as part of the generated change

Generated code frequently solves a small problem by introducing a package. That package brings maintainers, release processes, install scripts and transitive dependencies into the application.

Before release, answer four questions:

• Is the dependency necessary, or could a platform or language primitive do the work?
• Is the exact version locked?
• Does installation execute code?
• Can the build run with the network restricted after approved dependencies have been resolved?

OWASP's 2025 Top 10 moved software supply-chain failures into its top three categories. A lockfile is not a complete supply-chain programme, but an unlocked build is an avoidable source of change. The artifact reviewed on Tuesday should not silently mean different bytes on Friday.

For higher-assurance releases, preserve a software bill of materials, build provenance and the digest of the artifact that passed review. SLSA describes provenance as verifiable information about where, when and how an artifact was produced. The practical benefit is simple: a team can connect deployed bytes to a controlled build instead of trusting a filename.

3. Re-establish identity and authorization

Generated interfaces are very good at hiding buttons. Hiding a button is not authorization.

Every privileged read or write must be checked at the server-side boundary using the current actor, tenant and resource. Avoid accepting an organization ID, price, role or ownership claim merely because the browser supplied it. The server should derive what it can from the authenticated session and verify the rest against authoritative state.

Authentication deserves the same care. OAuth and OIDC flows need one-time state, exact callbacks, short expiry and safe account linking. Email equality alone is not sufficient proof that two external identities belong to the same person.

Secrets require a separate pass because generated examples routinely use convenient placeholders. Search source, history, fixtures, logs, browser bundles and documentation. A value does not become safe because it is called EXAMPLE_KEY.

4. Design the unhappy paths

The most revealing production test is often an interruption at the least convenient moment.

Take a checkout flow. Payment succeeds, the process loses connectivity, and the response never reaches the browser. Retrying the request must not create a second charge. The usual tool is an idempotency key tied to a durable operation record, but the important idea is broader: retries are part of the protocol, not an afterthought.

For each external call, choose:

• timeout;
• retry policy;
• idempotency behaviour;
• terminal failure state;
• operator evidence;
• user-facing recovery.

Then test malformed input, missing data, provider throttling, stale sessions and duplicate delivery. A green happy-path test says very little about these conditions.

5. Separate build, release and runtime

A production release should be an identified artifact, not the current contents of a mutable directory.

The build turns reviewed source and locked dependencies into an artifact. The release associates that artifact with configuration and policy. Runtime executes the identified release. Keeping these stages distinct makes rollback meaningful: restore a known artifact and its compatible configuration rather than rebuilding old source with today's dependency graph.

BlinkHost applies this principle to managed backend modules by treating repository source as input, not as trusted executable output. A managed build produces an artifact, an independent boundary validates its supported imports and envelope, and deployment refers to the verified artifact identity. That is one implementation of a wider rule: never let the place where code was authored be the only place where it was trusted.

6. Observe the release without collecting the customer

Useful telemetry answers operational questions:

• Which release handled the request?
• How long did it take?
• Which dependency or downstream operation failed?
• Did the retry eventually succeed?
• Did usage remain inside its policy?

It should not answer those questions by copying passwords, authorization headers, uploaded documents or complete request bodies into logs. Structured event names, bounded identifiers and redaction make evidence more useful while reducing the blast radius of a logging mistake.

A practical release gate

Before generated work ships, require evidence in five small bundles:

Bundle	Minimum evidence
Intent	actor, resource, allowed outcome and failure outcome
Change	reviewed diff, locked dependencies and migration plan
Verification	tests for authorization, invalid input, retries and rollback
Artifact	controlled build, digest and provenance or equivalent trace
Operation	release identifier, bounded telemetry and recovery owner

This does not require a large compliance department. A two-person team can apply it with a pull-request template, automated checks and a short release record. What matters is that speed does not erase accountability.

AI has moved the bottleneck. Typing is cheaper. Establishing intent, controlling dependencies, proving identity and recovering safely are now a larger share of the craft.

That is good news. Those are also the parts of software engineering that make a product worth trusting.

Further reading

• GitHub Octoverse 2024: AI leads Python to the top language
• NIST Secure Software Development Framework 1.1
• SLSA provenance specification
• OWASP Top 10: 2025
• CISA Secure by Design