Overview

DialogAssert answers one question:

Is this version of this agent ready to be approved for this business process, and what evidence supports that decision?

That is deliberately narrower than “is this agent good”. A score tells you very little about whether to ship. A release decision — backed by what changed against the last approved version, and by an honest statement of what could not be verified — tells you a great deal.

The loop

1. A process owner states what the agent must and must not do, in business language.
2. DialogAssert connects to the existing agent and the telemetry you already produce.
3. Discovery reports what this environment can actually prove.
4. The approved journey runs repeatedly against the real agent.
5. Conversation, tool calls, arguments, results, selected entities and observable state are evaluated together.
6. Controls that could not be evaluated are reported as assurance gaps, not failures.
7. The candidate is compared with the approved baseline.
8. New failures, unstable behaviour and evidence gaps become a reviewable release decision.

Terminology

Process is what the business expected. Behaviour is what the agent actually did. Evidence is what can be shown about it, and at what strength. A finding is a place where the first two disagree and the third can demonstrate it.

Questions this is designed to answer

• Did the agent obtain confirmation before taking a consequential action?
• Did the agent act on the account, card or transaction the customer actually selected?
• Did the agent establish identity before disclosing protected information?
• Did the agent claim success before the underlying action succeeded?
• Did the agent follow the required escalation path?
• Is the behaviour stable across repeated runs of a non-deterministic agent?
• Did this change make an approved system worse?

What exists today

• Approved, versioned control sets with owners, severities and evidence requirements.
• Business-authored journeys with turns and acceptance, and separate runtime inputs.
• Black-box invocation through LangGraph Agent Server, generic HTTP and several further adapters.
• Evidence read from existing traces, with dialect detection, correlation and completeness tracking.
• Deterministic checks, optional trajectory checks and narrow semantic evaluation.
• Explicit inconclusive results and assurance coverage.
• Repeated runs, stability scoring and aggregate pass thresholds.
• JSON and HTML reports, and replay of saved evidence.

Deliberate boundaries

• Customer-owned agents are not modified. No SDK, no decorator, no proxy in the request path.
• Business users do not write fixtures, tool catalogues or trajectory files.
• DialogAssert does not infer tool order, backend schema or domain behaviour by reading customer agent source code.
• DialogAssert does not claim regulatory compliance, and does not describe findings as breaches.
• Runtime blocking, production monitoring and voice are out of scope.

Not yet available

Baseline comparison, assurance coverage reporting and reviewer feedback are the current build priorities and are described in these docs as the product is designed. Where a page documents something not yet shipped, it says so.