Human-in-the-loop controls for AI agents
Decide who checks what your AI agents do before it reaches a customer or your bank account, with a log, a stop rule and a monthly review.
Ready to use. Buy it in your account: a one-time licence fee and a monthly fee, plus the AI it uses.
What it does
List what your AI agents or tools do, a line each, and the people who can review them. We rate each action's risk: high for money, deleting, contracts, decisions about people and messages to many customers at once; medium for sending, booking, changing records and publishing; low for drafts and suggestions a person uses. We then set its control. A person approves every high-risk action. Medium ones are approved for the first month, then sampled. Low ones are sampled. We also set what the decision log records, a stop rule anyone can use and a monthly error review, with the rules to check for the UK, the US and the EU. The AI writes what could go wrong with each action and what the reviewer checks, chooses a reviewer and a backup from your list, and writes when to escalate and a note to staff.
Who it is for
- Businesses using AI agents that reply, book, refund or update records
- Owners starting with help desk, CRM or automation AI
- Teams who want AI to save time without unchecked mistakes
Quick start
- 01List what the AI doesA line each, for example: Issues refunds up to £50 for late deliveries.
- 02List your reviewersA line each with their role, for example: Priya Shah (customer service lead).
- 03Run itIt takes a minute. You can leave the page: the result waits in your run history.
- 04Set it up in your toolsTurn on approval steps for high-risk actions, start the log, and share the note with staff.
What you give it
- What your AI agents or tools doRequired, up to 20 lines
- What your AI agents or tools do, a line each, as specifically as you can.
- People who can review itRequired, up to 15 lines
- The people who can review the AI, a line each with their role.
- The AI tools or platformsOptional, up to 10 lines
- The AI tools or platforms behind the actions, a line each. Optional.
- We operate in the UKOptional
- Tick if you operate in the UK, so its rules are given to check.
- We operate in the USOptional
- Tick if you operate in the US, so its rules are given to check.
- We operate in the EUOptional
- Tick if you operate in the EU, so its rules are given to check.
- Anything elseOptional, up to 1,500 characters
- Anything else, such as a mistake the AI made recently. Optional.
What you get
- Your AI actions and their controls
- Each action with its risk, its control, what could go wrong, what the reviewer checks, the reviewer and a backup. Downloadable as a spreadsheet.
- Escalation and stopping it
- When a reviewer passes something up and to whom, and how anyone pauses the AI.
- Decision log and monthly review
- What to record for every decision, and how to review errors each month.
- Note to staff
- A plain explanation of the controls for everyone who works with the AI.
- What to check
- Notes for you only: reviewers not on your list, the same person as reviewer and backup, and any promise about the AI.
You can edit the result, then copy it, print it or download it as Word, Markdown, plain text or data. Every run is kept in your run history until you delete it.
An example
Kestrel Outdoors is a made-up business. Below is what was entered, then the tool's real result, unedited, made on 8 November 2026. Subscribe to run it with your own details.
What was entered
From Kestrel Outdoors’s Business profile
- What you sell
- Outdoor clothing and kit sold online from Kendal
- What your AI agents or tools do
- Replies to customer emails automatically
- Issues refunds up to £50 for late deliveries
- Drafts product descriptions for staff to check
- Updates customer records in the CRM
- People who can review it
- Sam Patel (owner)
- Priya Shah (customer service lead)
- Tom Reid (warehouse manager)
- The AI tools or platforms
- Our help desk's AI agent
- We operate in the UK
Yes
3 optional details were left empty.
In your account this is a form you fill in, and the tool runs once you subscribe.
What the tool returned
Human-in-the-loop controls
In short
The refund action carries the most risk because it moves money, so a person approves every one and logs it. The customer email replies and CRM updates are medium risk and start with approval of each one, moving to weekly sampling only after a clean month. Product descriptions are lowest risk and are sampled weekly. The stop rule lets anyone pause the AI.
Your AI actions and their controls
Action 1
Action: Replies to customer emails automatically
Risk: Medium
Control
A person approves each one before it happens for the first month; after a month with no errors, a person checks 1 in 10 each week, and any error goes back to approving every one.
What could go wrong
A reply could give a wrong answer about an order, promise something the business has not agreed, or miss that the customer is complaining or upset. A reply could also go to the wrong person or include another customer's details.
What the reviewer checks
The reviewer checks that the reply answers the question, matches what the business can actually do, has the right recipient and shows no one else's details. They also check for any complaint or request that should be passed up.
Reviewer: Priya Shah (customer service lead)
Backup: Sam Patel (owner)
Action 2
Action: Issues refunds up to £50 for late deliveries
Risk: High
Control: A person approves every action before it happens, and every decision is logged.
What could go wrong
A refund could be paid for an order that was not late, paid twice, sent to the wrong customer or for the wrong amount. The AI could also refund more than the £50 limit.
What the reviewer checks
Before approving, the reviewer checks the order and delivery dates, that no refund has already been paid, the amount is within £50 and the right customer and payment method are used. They make sure the decision is logged.
Reviewer: Priya Shah (customer service lead)
Backup: Sam Patel (owner)
Action 3
Action: Drafts product descriptions for staff to check
Risk: Low
Control: A person checks 1 in 10 each week and logs any error.
What could go wrong
A description could contain wrong details about materials, sizing or features, or make claims about performance that the product does not support. It could also be published without staff checking it.
What the reviewer checks
Each week the reviewer samples 1 in 10 drafts and compares them with the product details held by the business. They check for unsupported claims and log any error.
Reviewer: Sam Patel (owner)
Backup: Priya Shah (customer service lead)
Action 4
Action: Updates customer records in the CRM
Risk: Medium
Control
A person approves each one before it happens for the first month; after a month with no errors, a person checks 1 in 10 each week, and any error goes back to approving every one.
What could go wrong
A record could be changed on the wrong customer, overwrite correct details with wrong ones, or record something about a person that is inaccurate. Errors could then affect later emails and refunds.
What the reviewer checks
The reviewer checks that the right customer is updated, that the change matches the source message or order, and that nothing correct is overwritten. They look for any change involving a person's personal details or a request about their data.
Reviewer: Priya Shah (customer service lead)
Backup: Sam Patel (owner)
When to escalate
- If a customer complaint, or a customer who is clearly upset, appears in any email or record, the reviewer passes it to Priya Shah (customer service lead), or to Sam Patel (owner) if Priya is the reviewer.
- If a refund is above £50, or the amount is unclear, the reviewer does not approve it and passes it to Sam Patel (owner).
- If anything concerns a person's rights over their data, such as a request to see, correct or delete it, the reviewer passes it to Sam Patel (owner) before anything is done.
- If the same error happens again, or an error is found in a sample, the reviewer pauses the AI under the stop rule and tells Sam Patel (owner) the same day.
- If a delivery or stock issue is behind a late-delivery refund, the reviewer asks Tom Reid (warehouse manager) to confirm the facts before approving.
Stopping it
Anyone who sees the AI do something wrong pauses it (or switches it to drafts only) and tells the reviewer the same day. It stays paused until the reviewer has found the cause and logged it.
The decision log records
- Date and time
- The action
- What the AI proposed
- Who reviewed it
- Approved, changed or refused
- What was changed and why
- What happened after
Monthly error review
- Count the errors, near-misses and refusals in the log for each action.
- Look at the three worst and agree what changes: the instructions, the data, the control or the tool.
- Move an action to a stricter control after any serious error, and to a lighter one only after a month with none.
- Note the decisions and who made them, and check the changes the month after.
The rules to check
- UK: there is no AI-specific disclosure law yet; UK GDPR gives people the right to be told about, and to challenge, decisions based solely on automated processing that have legal or similarly significant effects, and the ICO expects AI use to be explained in plain words. Advertising rules apply to claims about AI. Check the ICO's current guidance.
Note to staff
Our help desk's AI agent is allowed to help with emails, refunds, product descriptions and customer records, but people stay in charge. If you see the AI do something wrong, pause it, or switch it to drafts only, and tell the reviewer the same day. It stays paused until the reviewer has found the cause and logged it. Every decision is logged, including what the AI proposed, who reviewed it and what happened after. Priya Shah reviews customer emails, refunds and CRM updates. Sam Patel reviews product descriptions and is the backup for the others. Tom Reid helps confirm delivery facts when a refund depends on them. Emails and CRM updates are approved one by one for the first month, then sampled weekly, and any error puts them back to approving each one. Refunds are always approved first. If you are unsure, pause it and ask.
Only for you: not included when you copy, print or download
Check before relying on it
- Check each risk level fits how the AI is set up: if it can act without a person, its control must be in place before it runs.
What the AI does
It does
- Writes what could go wrong and what the reviewer checks for each action
- Chooses a reviewer and a backup from your list
- Writes the escalation rules and a note to staff
- Never changes a risk level or control, or invents people or limits
It does not
- Turn on approval steps in your tools for you
- Watch your AI agents as they run
- Give legal advice
- Know about actions you did not list
Please check itCheck each risk level fits how your AI is set up, and make sure high-risk actions cannot run without a person before you rely on them.
Limits
- What your AI agents or tools do: up to 20 lines.
- People who can review it: up to 15 lines.
- The AI tools or platforms: up to 10 lines.
- Anything else: up to 1,500 characters.
- Up to 20 actions and 15 reviewers a run.
- Not legal advice.
- Your workspace's monthly cap: a run that would take you over it pauses, and carries on where it stopped when you raise the cap.
Your data
- What you enter, the files you choose and the result are stored encrypted with your workspace's own key. Only people in your workspace can open them.
- To produce the result, what you give the tool (and the parts of your Business profile it uses) is sent to our AI provider, Anthropic. It uses it only to answer, and its terms do not allow it to train its models on it. We do not train on it either.
- Files you upload are deleted after 30 days unless you choose to keep them. You can delete a run at any time: what you entered, the result and the progress notes go; only the usage record stays, because it is part of your billing.
- Our service is hosted in the United States.
The full details are in our privacy notice.
Questions
How are runs charged?
A one-time licence fee of $800 (£800 in the UK), then $200 a month (£150 in the UK), including any tax that applies. On top, you pay for the AI each run uses, billed each month with your invoice and never more than the monthly cap you set. From your fourth tool there is no licence fee.
Can I change the result?
Yes. Choose Edit this result to change any wording or figure, and Restore the original result to go back. Copy, print and downloads use your edited version.
Is it ready to use now?
Yes. Sign in, open Tools in your account and choose Get started for this tool.
Who sets the risk levels?
We do, by fixed rules from what each action does: money, deleting, contracts, decisions about people and bulk messages are high; sending, booking and changing records are medium; drafts and suggestions are low.
Why are medium-risk actions sampled after a month?
Checking everything forever wastes the time the AI saves. A month of approving every one shows whether it can be trusted; after that a person checks 1 in 10 each week, and any error goes back to checking every one.
If something goes wrong
- An action's risk looks too low.
- Describe it more fully, for example 'sends refunds' rather than 'handles refunds', and run it again.
- A reviewer is not on your list.
- Add the person to the reviewers list, or choose someone from it in the result.
Still stuck? Use the enquiry form and say which tool and what you saw.