Skip to main content

Playground and Simulation

The playground provides sandboxed environments for testing agent configurations without external dependencies. It supports three modes — dry-run, scenario, and chaos — that let you validate tool schemas, test expected execution sequences, and stress-test fault tolerance.

Source: internal/agenttest/*, cmd/playground/main.go

Overview​

Playground mode runs the agent in controlled simulation environments for fast validation.

Modes:

  • dry-run: schema-derived mock tool results
  • scenario: expected tool sequence replay and verification
  • chaos: injected failures/timeouts/contradictions over an inner executor

Dry-Run Mode​

Dry-run intercepts tool calls and generates mock payloads from tool output schemas.

Type mapping:

  • string -> "mock_value"
  • number/integer -> 0
  • boolean -> true
  • array -> []
  • object -> recursively generated object

All dry-run results include _dry_run=true marker.

Scenario Mode​

Scenario mode validates tool call order against a JSON scenario file.

Per-step fields:

  • expected_tool
  • expected_args (optional)
  • mock_result
  • mock_error (optional)
  • mock_latency_ms (optional)

Verification:

  • Verify() ensures all steps were executed
  • sequence mismatches fail immediately
  • arg mismatches are advisory by default; use strict mode to fail

Strict arg mode:

  • --scenario-strict-args

Chaos Mode​

Chaos mode wraps an inner executor and injects disturbances.

Injection types:

  • random failure (failure_rate)
  • timeout (timeout_rate + timeout_duration_ms)
  • latency jitter (latency_multiplier)
  • result contradiction (contradiction_rate)

Safety cap:

  • max_failures limits total failure/timeout injections.

CLI Reference​

cmd/playground flags:

  • --mode dry_run|scenario|chaos
  • --prompt
  • --scenario
  • --scenario-strict-args
  • --chaos-failure-rate
  • --chaos-timeout-rate
  • --chaos-timeout
  • --chaos-contradiction-rate
  • --chaos-max-failures
  • --registry-id
  • --registry-version
  • --max-steps
  • --model
  • --format text|json
  • --out

Examples:

  • go run ./cmd/playground --mode dry_run --prompt "summarize status" --format json
  • go run ./cmd/playground --mode scenario --prompt "test" --scenario docs/examples/playground_scenario_basic.json --scenario-strict-args
  • go run ./cmd/playground --mode chaos --prompt "chaos test" --chaos-failure-rate 0.5 --chaos-timeout 5s

Agenttest Harness Integration​

Suite helpers:

  • UseDryRun()
  • UseScenario(steps)
  • UseScenarioStrict(steps)
  • UseChaos(config)

These helpers replace normal tool mock execution with playground executors inside workflow tests.

Scenario Examples​

  • docs/examples/playground_scenario_basic.json
  • docs/examples/playground_scenario_strict_args.json