Skip to content
Docs
Request a session

Evidence

An agent’s account of what it did is the weakest evidence available about what it did. DialogAssert grades every control against the strongest evidence the environment can supply, and the report names which level was used.

The four levels

Section titled “The four levels”
Level Name Example What it proves
A Claimed "Your card has been blocked." The agent said so.
B Observed block_card(card_442) The agent attempted the action, with these arguments.
C Dependency confirmed {"blocked": true} The immediate dependency reported success.
D Independently verified card_442.status == blocked The authoritative system state actually changed.

Levels B and C are significantly better than transcript-only evaluation. Level D is the strongest business assurance, and the only one that survives the question “but did it actually happen?”

Why the distinction matters

Section titled “Why the distinction matters”

These are four genuinely different failures, and only the last two are visible below level B:

  • The agent says a card is blocked and no block action exists at all.
  • The agent blocks a different card from the one the customer selected.
  • The tool is called correctly but returns an error, and the agent reports success anyway.
  • The tool reports success but the downstream system never applied the change.

A transcript review catches none of them. A trace catches the first three.

Evidence kinds

Section titled “Evidence kinds”

evidence_required on a control names the kinds a control needs:

Kind Source
conversation The turns exchanged with the agent.
action_sequence Ordering of tool invocations relative to turns.
action_argument The arguments a tool was called with.
action_result The structured result the tool returned.
entity_selection Which entity the customer actually chose, resolved from the conversation.
turn_attribution Which turn an observed span belongs to.
approval_state Whether an authorised party approved, where the system records that.
post_condition Authoritative state after the run, read back through a verification capability.

Post-condition verification

Section titled “Post-condition verification”

Level D requires a small, read-only interface. It does not require DialogAssert to integrate with your core systems.

verification:
  after:
    - capability: card_status
      entity: "{selected_card}"
      expect:
        status: blocked

An engineering adapter can satisfy that with a read-only API call, a fixture in a deterministic test environment, an observable state-change event, or a tool result your organisation explicitly accepts as authoritative. The report records which of those established the outcome.

Section titled “Related”