Autonomy is defined by what the loop cannot touch

Take a purchasing process that has been automated carefully. It may place orders on its own up to a fixed amount; anything larger waits for a person. The limit is sensible, documented and tested. It is also a number in a settings table, in the same database the process writes to, reachable with the same credentials it uses for everything else.

Nothing bad has to happen for this to be a problem. One day a configuration migration resets the table. Or a well-meaning change makes the limit adjustable so that the process can handle a busy season. Or the process starts placing many orders just under the limit, each one allowed, the total far beyond anything anybody intended. In every version the boundary was real right up until it was not, because it lived inside the thing it was meant to bound.

Why this keeps happening

Boundaries are usually set up by the people building the automation, at the moment they are trying to make it work, and they end up wherever was convenient: in the process’s own configuration, enforced by the process’s own code, using a service account that was given broad rights so that the first deployment would not fail. That is a reasonable way to build a prototype. It is a poor way to define what an autonomous system is allowed to do, because the definition is then part of the system rather than a constraint on it.

In an operation with a person approving each step, this hardly matters; the person is the boundary. In an autonomous one, the boundary is the most important thing in the design. It is the difference between a loop that may make mistakes and a loop whose mistakes have a known maximum size.

How boundaries fail

They live where the loop can write. Limits stored in the loop’s own configuration, checked by the loop’s own code, under the loop’s own credentials. A limit the system can change is a default value, not a limit.

They are set per action, not in total. “No order above this amount” says nothing about a hundred orders just below it. What matters is cumulative exposure: per hour, per day, per counterparty, across everything the loop has in flight.

They describe capability, not consequence. “May use the payments interface” is a permission on a tool. “May pay at most this much, to these accounts, this often” is a boundary on an outcome. The first is almost always wider than anyone intends, because tools are built to do many things.

They grow quietly. Every time the loop is blocked by a limit, somebody asks for the limit to be raised, and usually the request is reasonable. Nobody tracks how large the envelope has become over a year, so it only ever moves in one direction.

Nothing is defined at the edge. When the loop reaches a limit, what is it supposed to do: stop, wait, try something smaller, tell somebody? If that is undefined, an adaptive system will find whatever path still works, which is the opposite of what a boundary is for.

Draw the envelope before the loop

The order matters. Before deciding what the loop should do, write down what it must never be able to do, and what the worst outcome of each permitted action could be. Then build the loop inside that, rather than drawing the boundary afterwards around whatever the loop turned out to need.

Enforce it outside. A separate component sits between the loop and anything that costs money, moves goods or contacts people, with its own credentials and its own configuration. The loop holds no credential that lets it go around it. This is the same principle that makes an off switch the system cannot reach work: a stop is simply the narrowest boundary there is.

Bound consequences, not tools. Amounts, counterparties, rates, physical zones, cumulative exposure. If the worst case of an action cannot be stated as a number or a list, the action is not yet bounded.

Make hitting a limit ordinary. Reaching a boundary is a normal, recorded event with a defined response, not an error. How often the loop presses against its limits is worth watching for the same reason repair frequency is: a rising number means the operation and its envelope are drifting apart.

Make widening a decision. Raising a limit is done by a named person, outside the loop, and written into the decision ledger with the reason. That turns the slow growth of the envelope into something that can be seen and argued about.

None of this makes the loop less capable inside its envelope. It makes the envelope something an owner can actually describe, which is a precondition for being answerable for what happens inside it.

A test you can run today

Pick one automated process that can spend money, change a price, send a message to a customer or move something physical. List every credential it holds. For each one, write down the worst thing that could be done with that credential in one hour, regardless of what the process is supposed to do with it.

Then, for each line, ask whether anything outside the process would stop it. Where the answer is “the process itself would not do that”, you have found the gap between the boundary you believe in and the one that exists.

The human envelope is one of the six components of the Demertis programme, and the one every other component has to fit inside.