Security
What DialogAssert touches, and what it doesn't
DialogAssert invokes your agent over the interface it already exposes and reads the telemetry you already produce. Nothing is embedded in your agent, no source code is required, and your traces stay where they are. Everything below is current capability — where the honest answer is “not yet”, it says so.
Data flow
DialogAssert invokes your agent over its existing interface and reads execution evidence from the trace or API surface you point it at. It sends the journey turns in, and reads back the conversation and the spans your platform already produces.
It does not require the agent's model, prompts, weights, knowledge base or source code.
Read-only evidence access
Where your telemetry already exists, DialogAssert queries it. It does not ask you to duplicate traces into another observability platform in order to test your agents, and it does not write to your trace backend.
Post-condition verification, where you enable it, is a read-only capability call against an interface you define and control.
Agent ownership
DialogAssert does not alter the customer agent. There is no SDK to embed, no decorator to add and no proxy in the production request path.
The consequence worth stating plainly: what DialogAssert can verify is bounded by what your environment already exposes. That is why every run begins by reporting its own assurance coverage.
Data minimisation
Journeys are authored against test identities and seeded fixtures. Trace payloads are decoded only to the depth a control requires, and evidence a control does not need is not retained in the report.
Published or exported reports are projected onto an explicit schema rather than dumping raw runner output.
LLM judging
Controls that can be settled deterministically are settled deterministically — ordering, arguments, entity identity and observable state involve no model judgement.
Where a control genuinely requires interpretation, a semantic evaluation is used and the finding says so. The judging model is configurable, and a deterministic-only mode operates with no external model call.
Testing an action-taking agent has side effects
This is the question we would ask first in your position, so it gets its own section rather than a footnote.
DialogAssert runs your real agent. If that agent can freeze a card, move money, send a message or change a record, then exercising the journey will do those things. There is no way to test whether an action actually happened without the action happening.
So destructive workflows should run against a sandbox or non-production dataset unless you have explicitly decided otherwise. Journeys are authored against test identities and seeded fixtures for exactly this reason, and a control that requires a real side effect names that requirement up front rather than discovering it mid-run.
Every major agent-testing product carries this constraint. We would rather state it than let you find out during a pilot.
Before a first run, decide three things
- Which environment. Sandbox, non-production copy, or a production system with test identities.
- Which actions are permitted. Controls requiring a destructive action are opt-in per journey.
- What gets cleaned up. Fixture reset between attempts, so a repeated run is comparable rather than cumulative.
Credentials, judging and retention
Credentials
Target-agent credentials and evidence credentials are configured separately, so the credential that can invoke your agent is not the credential that reads your traces.
Evidence credentials should be read-only. Verification capabilities, where you enable them, are read-only by design.
Secrets are supplied through your environment or secret manager. They are not stored in a control file and never appear in a report.
LLM judging
Controls settled deterministically involve no model call at all. Where a control needs interpretation, the conversation turns and the specific evidence for that control are sent to the judging model — not your whole trace, and not unrelated controls.
The provider and model are configurable, including a model hosted in your own account. A deterministic-only mode runs with no external model call, at the cost of the controls that genuinely require interpretation.
Every finding records whether it was settled deterministically or semantically.
Retention and residency
On a self-hosted runner, run evidence stays where you put it. We hold none of it, and there is nothing for us to delete on your behalf.
Reports are files you own. Deleting them deletes the evidence; there is no separate copy in a DialogAssert account.
We do not currently operate a hosted control plane, so we make no data-residency claim. When a VPC option ships, residency will follow your own cloud region and we will say so then.
Deployment
We list what exists today. Anything not yet available is marked as roadmap rather than described as if it ships.
| Option | Status | Notes |
|---|---|---|
| Local / self-hosted runner | Available | Runs inside your network. Agent traffic and trace queries never leave your perimeter. |
| CI runner in your pipeline | Available | Executes as a job in your own CI, with your own credentials and egress rules. |
| Customer VPC deployment | Roadmap | Managed control plane inside your cloud account. |
| Hosted multi-tenant SaaS | Roadmap | Not offered today. We would rather say so than imply an option we do not run. |
DialogAssert holds no security certification today. We do not claim SOC 2, ISO 27001 or HIPAA, and we will not until an audit has actually been completed. A precise architecture is worth more to you right now than a placeholder badge.
The human decision boundary
DialogAssert produces findings and evidence. It does not autonomously certify an agent as lawful or compliant.
A release decision belongs to the people accountable for it. What we provide is reproducible technical evidence about how the agent behaved, an explicit statement of what could not be verified, and a comparison with the release your organisation previously approved. Whether that evidence is sufficient for your release and regulatory controls is your organisation's judgement to make.
Reviewing us before a pilot?
Send the questions your third-party risk process asks and we'll answer them directly, including the ones where the answer is “not yet”.
Security issues: security@dialogassert.ai · Data requests: privacy@dialogassert.ai ·Privacy notice