Configuration

A DialogAssert config tells the runner which agent to invoke, which inputs to pass, which workflows, rubrics, trajectories or assertion packs to evaluate, how many times to repeat the test, what score threshold is required, where to write reports and which semantic judging mode to use.

description: DialogAssert business-authored acceptance suite for OpenAI Agents

agent:
  id: openai-agents-banking-example
  type: python_function
  callable: examples.openai_agents.banking_support_agent:run_banking_agent_conversation
  result_format: openai_agents_banking_example
  model: gpt-5-mini
  repo_root: .

inputs: authoring/generated/business_acceptance.inputs.yaml

semantic:
  mode: llm
  llm:
    model: gpt-5-mini

evaluation:
  repeat: 3
  pass_threshold: 0.75

test_suite:
  workflows:
    - generated/workflows/block_card_placeholder.workflow.yaml
  rubrics:
    - generated/rubrics/block_card_placeholder.acceptance.rubric.yaml

reports:
  output: reports/openai-business-acceptance.json
  html: reports/openai-business-acceptance.html

Environment variables

OPENAI_API_KEY                 Required for OpenAI model calls
DIALOGASSERT_LLM_MODEL         Optional model override for LLM semantic judging