A chatbot that answers questions about your data is useful and mostly harmless. An agent that can change that data — update a record, create a case, kick off a workflow — is a lot more useful and a lot more dangerous. On the kind of systems I work on, where the records are what the business actually runs on, I’ve had to be careful about where the line sits before I let an agent act on its own.
The reason is simple. A wrong answer is just a wrong answer; the user reads it and moves on. A wrong action leaves the system in a wrong state, and that state flows into every report, automation, and integration downstream that trusts the record. So here’s how I think about making an action-taking agent safe enough to turn on.
Let it act as the user, not as a superuser
The most important decision is whose permissions the agent uses. The tempting shortcut is to give it a powerful service account so it “just works.” That’s how you end up with an agent that happily does something for a user that the user could never do themselves.
If the agent acts inside the requesting user’s permissions, the platform’s existing access controls do the hard part for you. The agent simply can’t take an action the user wasn’t allowed to take. You get years of careful access-control work for free instead of rebuilding a worse version of it next to the model.
Make every action safe to repeat
Agents retry. The model times out, the loop runs again, a network blip triggers a second attempt, and now you’ve created two cases or sent the same update twice. On a system of record, a duplicate is corruption.
So every action an agent can take should be safe to apply more than once, with the same result as applying it once. Use an idempotency key, check before you create, lean on whatever natural deduplication the platform gives you. Assume the agent will repeat itself, because under real conditions it will.
Give it a short list of things it’s allowed to do
An agent with access to “any API call” is impossible to reason about and impossible to trust. The version that works has a small, defined set of tools, each one a deliberate capability with validated inputs, not a raw door into the platform.
There’s a real difference between “the agent can set the opportunity stage to one of these allowed values” and “the agent can run arbitrary updates.” The first you can test and trust. The second you’ll be explaining in an incident review. Keep the list of allowed actions short enough that each one is easy to defend on its own.
Put a person in front of anything you can’t undo
Not every action needs a human watching, but the consequential ones do. I tier it by how much damage a mistake does and how hard it is to reverse. The cheap, reversible things — drafting a summary, suggesting a next step, filling a scratch field — can run on their own. The things that touch money, or a customer, or close something out, ask for a person first.
The mistake is treating those the same. Full autonomy on trivial actions is fine. Full autonomy on a refund is a headline waiting to happen.
Log enough to explain what happened later
When an agent changes a record, you need to be able to answer “why did this happen?” weeks after the fact. That means logging the whole chain: what the user asked, what the agent retrieved, what it decided, which tool it called, and what came back.
This isn’t only for debugging. It’s the trail that makes the feature defensible to security, to compliance, and to the person whose record changed. An agent you can’t explain is one you shouldn’t have writing to a system of record in the first place.
None of these are about making the model smarter. They’re about giving it a place to operate where its mistakes are contained. Get that right and an agent on a real platform is genuinely useful. Skip it and you’ve built something that demos well and scares everyone who has to run it.
