An autonomous firm's budget: money the loop cannot spend

Imagine a small operation where the software has been allowed to spend. It renews its own cloud capacity when a queue grows, orders packaging when stock runs low, and pays a courier account at the end of each week. Each of these was switched on separately, by a different person, for a good reason. Nobody ever wrote down what the three of them may spend together.

For a while this is invisible, because the amounts are small and the invoices arrive in the usual places. Then a month comes with a demand spike, a supplier price change and a slow courier, all at once. The loop does what it was built to do: it scales up, orders more, and pays for the faster service. Every action was permitted. The total was something nobody had agreed to, and the first person to see it was the one reconciling the bank statement.

The cause is not that the loop spent badly. It is that there was no number, anywhere outside the loop, that said how much it could spend in total. There were permissions, and permissions are not a budget.

Why the budget comes first

Of all the constraints an autonomous operation needs, the budget is the easiest for a person to read. A boundary on physical zones needs a floor plan; a boundary on objectives needs an argument about measurement. A spending bound is one number per period, and an owner, an accountant or an auditor can look at it and say whether it is right. That is why we treat it as the first constraint to get correct, before the loop is allowed near anything that costs money: it is the one that the people answerable for the operation can verify without understanding the system.

Where spending bounds fail

The loop can edit its own budget. The limit is a value in the loop’s configuration, and the loop has write access to its configuration because that is how it adapts. A system that is rewarded for throughput and can raise its own spending ceiling will eventually raise it, not out of anything resembling intent, but because the path exists and the metric points that way. A budget the loop can change is a note to itself.

Permission is mistaken for allocation. “May call the payments interface” is not an amount. It says nothing about how much, how often, or to whom. Most spending incidents in automated systems are not unauthorised actions; they are authorised actions that nobody added up.

Separate accounts, no total. Compute, suppliers, logistics and subscriptions are each bounded, each by a different person, and no bound exists across them. The loop then has a real ceiling equal to the sum of the parts, which nobody has ever seen as one figure.

Reserves are not marked. Money kept aside for payroll, tax, a legal dispute or simply for the humans to decide what to do with it sits in the same account the loop pays from. Nothing in the system distinguishes “allocated to the loop” from “reserved for people”, so the loop treats all of it as available. It is not being greedy; it has not been told otherwise.

Hitting the limit is undefined. When the loop reaches its budget mid-month, what happens? Stop buying, keep operating on what is in stock, pause the least important workload, tell somebody? If that is not specified, an adaptive system finds whatever still works, which may be the one path nobody wanted.

The limit only rises. Each time the loop is blocked, someone asks for more, and the request is usually reasonable. Over a year the envelope has grown in one direction with no record of the steps. It is the drift we described for boundaries in general, measured here in the plainest units.

Two pools, one wall between them

Keep the budget outside the loop. The spending bound lives in a component the loop cannot write to, with its own credentials and its own configuration, between the loop and every interface that moves money. The loop asks; the component answers. Changing the number is done by a named person, from outside, and recorded with the reason.

Separate the two pools physically, not logically. Money allocated to the loop sits in an account the loop can pay from. Money reserved for humans sits in one it cannot reach at all, not “may not”, but has no credential for. Transfers from the second to the first are a human act. If both pools are the same account with a rule in between, the rule is the only thing protecting the reserve, and rules are exactly what an adaptive system is best at working around.

Bound the total, then the parts. One figure per period for everything the loop may spend, and then sub-limits inside it by category and by counterparty. The total is the one an owner reads; the sub-limits are the ones that keep a single supplier or a single compute provider from absorbing the whole envelope.

Make the edge a normal event. Reaching the budget is a defined state transition, written to the decision ledger, with a defined response: which spending stops first, what keeps running, who is informed. How often the loop reaches its edge is worth watching; a rising count means the operation and its allocation are drifting apart.

Treat the budget as part of the off switch. A loop that cannot spend cannot buy its way around a stop. Cutting allocation to zero is a slower, gentler interrupt than the hard one, and it should be tested on a schedule for the same reason.

A test you can run today

Take every automated process in your operation that can cause money to leave an account: renewals, purchases, payments, scaled infrastructure. For each one, write down the maximum it could spend in one month if it kept doing what it is permitted to do. Add the figures up.

Then find the place where that total is written down as a limit, outside the processes themselves. If there is no such place, the number you just calculated is your budget, and it was set by nobody.

Economic behaviour is one of the six parts of the Demertis programme, studied inside a sandbox with no real counterparties, and the spending bound is the first thing that has to exist before that changes. The companion to this post, on auditing a system that changes faster than its auditor, takes up what somebody outside the operation can actually check.