Why an agent saying “done” is not evidence that the action happened
Four levels of proof for a customer-impacting action, and why a transcript settles none of them.
In progressResources
Writing on release assurance
Not another list of AI risks. These are the arguments the category is missing: what counts as evidence that an agent did something, why a testing system needs a third verdict, and how to tell whether a change made an approved system worse.
In progress
We publish when a piece says something we actually believe and can defend. These are the six being written first — if one of them would be useful to you now, ask and we'll send the draft.
Pass, fail, or unverified: why agent QA needs a third state
A binary verdict forces a testing system to guess when evidence is missing. The guess is usually an accusation.
In progressThe problem with letting an agent’s traces define its own test
A suite derived from the implementation can only establish that the implementation does what it does.
In progressHow to compare an agent release with an approved behavioural baseline
What to hold constant, what to vary, and how to tell a regression from a stochastic wobble.
In progressWhat evidence should exist before an AI agent takes a customer-impacting action?
Eight primitives that apply to blocking a card, changing a policy, issuing a refund or moving a booking.
In progressThe Agent Release Assurance Checklist
The questions to answer before an agent version reaches customers — usable without buying anything.
In progressThe vocabulary
These terms are used consistently across the product, the docs and the reports. They are defined here so nobody has to guess what we mean by them.
- Release assurance
- Establishing, before release, whether a specific agent version behaves as the business approved.
- Behavioural control
- An approved, versioned control with an owner, a severity and the evidence it requires.
- Assurance coverage
- The share of approved controls this environment can actually evaluate.
- Approved baseline
- The last release whose behaviour the organisation accepted, and the thing a candidate is compared against.
- Evidence gap
- A required piece of evidence the environment does not expose — reported as inconclusive, never as failure.
Would you rather discuss the problem than read about it?
We're speaking with teams working out how to decide that an agent version is safe to release. No demo required.