How to build an agent factory, stage by stage

An agent factory is not a faster version of the software development lifecycle. It is the same lifecycle closed into a loop, where every stage leaves a record and people move from doing the work to three decision points: they approve the description, confirm the verification and agree to the release. Agents prepare everything in between. A team that copies its old process one to one, with an agent in place of a developer, gets more code and not a better product.

This is the third post in a series. The first argued that factories will spread beyond IT; the second asked who owns what a factory learns. Here we describe the loop itself, in the terms we use for the whole programme: operate, repair, evolve, with a human envelope around all three.

The old lifecycle in one paragraph

The classic lifecycle is a relay of people. An analyst gathers requirements, an architect designs, a developer writes, another developer reviews, a tester tests, an operator deploys, somebody watches the monitoring. Between each pair there is a hand-over: a document, a ticket, a corridor conversation. The knowledge of why something was built a particular way lives mostly in heads, and leaves with them.

That model worked because the developer’s time was the scarcest resource, and hand-overs were the price of people working together. In a factory, execution becomes the cheapest resource and human attention the most expensive. That inverts the logic of the whole cycle.

The loop in eight stages

  +-------------------------------------------------------------+
  |                                                             |
  v                                                             |
1 INTAKE --> 2 TRIAGE --> 3 DESCRIPTION --> 4 EXECUTION --> 5 REVIEW
                            [PERSON                             |
                             approves]                          v
8 MONITORING <-- 7 RELEASE <---------------------------- 6 VERIFICATION
                  [PERSON                                  [PERSON
                   agrees]                                  confirms]

Above the loop sits a coordinator: an agent that receives a trigger (a ticket, a chat message, a monitoring alert), decides which stage the work is at, and starts the right agent with the right model and context. A cheap model for sorting. A stronger one for descriptions. For review, whichever scored best on your own test set. The coordinator is also the first thing the interrupt has to reach. If you can stop the coordinator from outside the loop, you can stop the factory.

Each stage below has three parts: what happens, why it matters, and what breaks without it.

1. Intake: work arrives in one place

What happens. An idea, a bug report, an alert or a request lands in a single queue.

Why it matters. A factory with six front doors does not know what is on its table. Neither throughput nor cost can be measured.

What breaks without it. Half the agent work happens outside the loop, on laptops, with no trace and no metrics.

2. Triage: the cheapest saving in the loop

What happens. An agent reads the ticket, tries to reproduce it and puts it in one of three bins: ready to execute (a clear, small fault), unclear and needs a description (a feature, a requirement open to several readings), parked and needs a person (missing data, a business decision).

Why it matters. In the old cycle sorting, estimating and first diagnosis eat hours of the most experienced people. Here they receive work already sorted, and well-described small faults go to execution without a person touching them.

What breaks without it. The executing agent receives a vague task and produces a lot of code that solves the wrong problem.

3. Description: where the quality of the loop is decided

What happens. For unclear work an agent drafts two documents. The product description says what the user will see and do, start to finish. The technical description says how the change fits the existing architecture. A person reads both, corrects them and approves them before a line of code exists.

Why it matters. This is the control point with the most leverage in the factory. Fixing a sentence in a description costs a minute. Fixing the same decision inside a thousand-line change costs a day. In the old cycle people reviewed finished code. In a factory they review intent.

What breaks without it. Review turns into archaeology: a person reconstructs from the code what the agent meant to do, and usually finds it is not what the team meant.

4. Execution: the least interesting stage

What happens. An agent takes a triaged ticket or an approved description and prepares the change.

Why it matters. Less than it seems. If intake, triage and description work, execution is the most predictable part of the loop. Everyone wants to start here, because this is where results are visible. That is the mistake.

What breaks if you start only here. A fast agent writes code for tickets nobody sorted or described, and people spend their days rejecting changes instead of approving them.

5. Review: the stage that learns

What happens. A reviewing agent goes through every change before a person sees it. It works in a sandbox with access to the code, so before suggesting a fix it can run it. It has the product and technical descriptions in front of it, so it judges the change against intent, not only style.

A second agent watches the comments people leave in their own reviews, collects them, and from time to time proposes a change to the reviewer’s instructions. That proposal is an ordinary change that a person accepts or rejects.

Why it matters. Team conventions do not need to be written in advance in a document nobody reads. They accumulate in the reviewer’s instructions, one comment at a time, and they stay with the firm when people move on.

What breaks without it. People receive raw changes and repeat the same five comments every week.

6. Verification: a person confirms instead of investigating

What happens. An agent gets a machine, starts the application and walks the user’s path end to end, as a tester would. It attaches screenshots or a recording to the change, and separately checks whether the result matches the description it started from.

Why it matters. Automated tests check what someone anticipated. Live verification catches what nobody did: a button hidden behind a notice, a form that works but cannot be filled in on a phone. The person’s job changes from reproducing the change to watching a short recording and saying yes or no.

What breaks without it. The last step before release again depends on whether someone had time to click through the application by hand.

7. Release: the decision that stays with a person

What happens. A person accepts the change and releases it.

Why it matters. Release is the moment risk passes from the team to customers. Somebody has to put a name to it. This is the same point we made in the last human in the loop is a signature: the signature is not a technical step, and it does not automate away.

8. Monitoring: the loop closes

What happens. An agent watches the product in operation, catches failures and odd behaviour, and opens a ticket that returns to stage one. A whole loop can run with no person until approval: monitoring detects a fault, triage reproduces it, execution prepares a fix, review and verification check it, and in the morning a person finds a finished change with a recording attached.

Why it matters. Without this stage the factory is an assembly line. With it, the product reports what needs fixing. In our terms this is the repair loop, and as with self-healing in general, it is only safe if every fix it proposes can be undone.

The second loop: a factory that improves the factory

Above the product loop runs a second one, less visible. An observing agent scores each run: what it cost, how many times a person had to correct something, whether the change came back later as a fault. From that it proposes changes to the factory itself: another model at triage, a sharper description template, a new skill for the reviewer.

This is the evolve loop, and it is where factories most often go wrong. For it to be safe, the factory definition has to be code: versioned, with a way to undo a bad change, to try a new version on part of the work, and to compare results on a fixed set of test tasks. That test set must sit outside the factory’s write path, for the reasons in a yardstick the system cannot edit. A change to the factory follows the same path as a change to the product: proposal, review, a person’s acceptance. A factory that rewrites its own instructions without a person’s consent will, within a few months, be doing things nobody understands.

The second loop is also a second job for the team. Engineers do not only build the product through the factory. They improve the factory, based on measurement rather than instinct.

Old lifecycle and factory: what really changes

Dimension	Old lifecycle	Agent factory
Shape	relay of linear hand-overs between people	loop: monitoring feeds new work
Where people look	at finished code	at the description before code and the recording after verification
Scarcest resource	developer time	human attention at the decision points
Team knowledge	in heads and an outdated wiki	in agent instructions, versioned with the code
Testing	tests written up front plus manual clicking	tests plus live verification against the description
Improving the process	a retrospective once a sprint	every run scored, changes to the factory proposed and approved
Record of decisions	scattered across tickets and chats	each stage writes to one ledger
Accountability for release	a person	still a person

The last row matters most. Requirements, architecture, approving the description and deciding to release stay with people. Everything between them changes.

Where a factory does harm

Automating work that only looks simple. “Bump the library version” sounds like a task for an agent, until the new version changes how rounding works in the invoicing module. Start where a mistake is cheap and immediately visible: small interface faults, obvious failures, updates that change no behaviour.

Skipping the description because the agent writes fast. Writing fast without an approved description is the most expensive way to discover that everyone understood the task differently.

Self-improvement without a brake. A change to the factory without a person’s acceptance is drift, not progress.

A factory without people. A team told that the factory will replace them will leave before the factory works. The people who like hard system problems tend to take to building the factory fastest, if it comes as more responsibility rather than as a verdict.

How to tell the factory is working

Vanity metric	Outcome metric
lines of code written by agents	times a person touched one change from ticket to release, falling over time
tokens consumed	cost of one closed change, and its trend month on month
number of agents running	share of tickets that passed the loop with no correction after approval
“agents write most of our code”	lead time for changes, deployment frequency, change failure rate and time to restore service (the four DORA measures)

A second model grading quality can supplement this, but only supplement it. If fixes after release are rising, no model’s score will cover the fact that the factory is producing worse work.

The order to build it in

Each step should leave a working system, so you can stop after any of them and still have something of value.

1. Triage. An agent sorts incoming tickets into three bins and drafts a first change for the well-described small faults. People stop losing hours to labelling and estimating.
2. Descriptions. For unclear work an agent drafts product and technical descriptions, and a person approves them before execution. Complex work enters the factory instead of stalling.
3. Review with a learning loop. A sandboxed agent reviews every change, and a second agent turns people’s comments into proposed changes to the reviewer’s instructions.
4. Live verification. An agent walks the user’s path, records the evidence and compares the result with the description. People confirm instead of investigating.
5. Join and measure. The coordinator links the stages, monitoring feeds intake, an observer scores runs, and the interrupt is tested from outside. Only now does the second loop start.

The same order works outside software. A loop of “request, sorting, description of the purchase, preparation, policy check, verification of amounts and approvals, release of the order, measurement” is the same factory in purchasing. In our own programme the loop currently runs only on non-regulated internal workloads, and that is the order we recommend: internal, non-regulated work first, and only then a wider scope.

This week, take the last twenty tickets from your tracker and, by hand in a spreadsheet, put each into one of the three triage bins. If more than half land in “unclear”, the first agent in your factory should not be the one that writes code. It should be the one that writes descriptions.

That leaves a decision. You can give the team an agent that writes code faster and watch the review queue grow. Or you can build a loop in which people stop writing and checking everything, and start approving the things that genuinely need their judgement.