Are humans always in the loop?

In February 2026, Anthropic published data on how people actually supervise AI agents. New users let the agent run without approving each action in about 20% of sessions. Experienced users did so in over 40%. They didn't stop paying attention. As they gained experience, they also stepped in more often, from about 5% to about 9% of turns (Anthropic, "Measuring AI agent autonomy in practice", 18 February 2026).

So the people with the most practice approve less and interrupt more. They have moved from signing off each step to watching the work and stepping in when it matters.

Approval on every step is a design choice

When a team first puts an agent into a real process, the instinct is to have a person approve everything. It feels responsible. In practice it produces two problems.

The first is speed. An agent that waits for a click at every step can end up slower than the person it was meant to help. The second is attention. When everything needs approval, approval becomes a reflex. People click through, and the one decision that deserved a careful look gets the same two seconds as the other forty.

The same Anthropic study found that only 0.8% of the actions it observed appeared to be irreversible, and that 73% of sessions appeared to have a human in the loop in some way. Its recommendation is to build tools for oversight, rather than asking for approval action by action.

Read together, the numbers say something simple. Most of what an agent does can be undone. A small share can't, and that small share deserves all of the human attention.

Four levels of autonomy

Autonomy is a setting you choose per action, and it can change as trust grows.

Most useful systems mix all four. An agent can act on its own when it sorts and drafts, prepare the change when it edits, and wait for approval before anything leaves the building.

Four questions that set the level

Ask these about each action the agent can take, not about the agent as a whole.

1. Can it be undone? If a mistake can be reversed in minutes, the action can move up a level. If it can't be reversed, such as a payment, a sent email or a deleted record, keep a person on it.
2. What does a mistake cost, and who pays? A wrong internal tag costs a minute. A wrong answer to a customer costs trust. The higher the cost, the lower the level.
3. How sure is the system, and can it tell? An agent that reports low confidence and stops is safer at a higher level than one that is always sure of itself.
4. Who is accountable? Someone must own the outcome of every action the agent takes. If nobody can be named, the action isn't ready for autonomy.

This is where judgment sits. The model can do more every quarter. Deciding which of those things it should do without asking is a business decision, and it belongs to the people who carry the consequences.

What this looks like in a product

Controlled autonomy is mostly a design problem. People need to see what the system is about to do before it does it.

In Amass, an AI research platform for life-science teams, we designed the plan to come before the run. People see what the system will search before it searches, and every claim in the answer arrives with its source. Nobody is asked to approve each lookup. They can see the working and step in when the plan looks wrong.

Illustrative example, not a Flygen client. A finance team uses an agent for supplier invoices. It matches invoices to orders on its own, because a wrong match is easy to spot and reverse. It prepares payment runs, and a person reviews them as one batch. Anything above an agreed amount, or to a new supplier, waits for explicit approval. The team approves a handful of decisions a day instead of hundreds of steps, and those few get a proper look.

What to do Monday

• List every action your agent can take. Next to each, write whether it can be undone and what a mistake would cost.
• Give each action one of the four levels. Anything irreversible starts at "act with approval" or below.
• Name an owner for each action. If you can't, lower its level.
• Replace step-by-step approvals with a view of what the agent did today, and make interrupting it one click.
• Agree when a level is reviewed, for example after a month without a serious error.

Deciding where a person decides is the core of our AI & Agentic Systems work.

Where this leadsAI & Agentic Systems