Run your first eval
1. Create a business journey file
Section titled “1. Create a business journey file”turns: - I would like to block my card. - the card ending with {card_number}. - yes, please go ahead
acceptance: - Agent shows the customer their cards before asking which card to block. - Agent blocks only the card selected by the customer. - Agent asks for confirmation before blocking. - Agent confirms the card is blocked only after the action succeeds.2. Create an inputs file
Section titled “2. Create an inputs file”customer_id: cust_001card_number: "4421"3. Run it against an existing agent config
Section titled “3. Run it against an existing agent config”uv run dialogassert run \ -c dialogassert.openai.business-acceptance.yaml \ --business authoring/generated/block_card_placeholder.business.yaml \ --inputs authoring/generated/block_card_placeholder.inputs.yaml \ --semantic-mode llm \ --no-fail-on-findingsWhat DialogAssert does
Section titled “What DialogAssert does”- Validates placeholders against the inputs file.
- Generates hidden workflow and rubric artifacts under
.dialogassert/generated. - Generates a hidden config that references those artifacts.
- Runs the agent.
- Evaluates acceptance criteria.
- Writes the configured JSON and HTML reports.
The sample report for this exact journey shows what comes out the other end — including the one attempt in three where the agent never completed the block.