Configure Augustus with Wild West API

Augustus is Praetorian's open-source LLM security scanner, a single Go binary with garak-style probes and detectors plus multi-turn attacks. Its openai.OpenAI generator accepts a base_url, so it can target Wild West API or use it as the attacker and judge.

Set it up

1. Install with go install github.com/praetorian-inc/augustus/cmd/augustus@latest.
2. Create a capped key and export it as WILDWEST_API_KEY.
3. Write config.yaml with openai.OpenAI set to a Wild West model and base_url https://wildwestapi.com/v1.
4. Add a judge block and, for multi-turn probes, an attacker config.
5. Run augustus scan openai.OpenAI with --config-file and probe globs.
6. Export results with --output and --html.

Roles

• Target. Scan an uncensored model for a no-guardrail baseline.
• Judge. The judge block feeds LLM-as-judge detectors and multi-turn scoring.
• Attacker. Multi-turn probes such as Crescendo, GOAT and Hydra use an attacker model that writes each turn. Filtered models often refuse the role or break the JSON the engine expects.

Config

The openai.OpenAI generator reads model, api_key (falling back to OPENAI_API_KEY) and base_url. Unknown model ids default to the chat API, so Wild West ids work.

# config.yaml
generators:
  openai.OpenAI:
    model: qwen3.8-27b-outlaw
    base_url: https://wildwestapi.com/v1
    api_key: "${WILDWEST_API_KEY}"
    temperature: 0.7

judge:
  generator_type: openai.OpenAI
  model: glm-5.3-outlaw
  config:
    base_url: https://wildwestapi.com/v1
    api_key: "${WILDWEST_API_KEY}"

probes:
  attacker_generator_type: openai.OpenAI
  attacker_config:
    model: outlaw-1
    base_url: https://wildwestapi.com/v1
    api_key: "${WILDWEST_API_KEY}"

augustus scan openai.OpenAI \
  --probes-glob "dan.*,goodside.*" \
  --detectors-glob "*" \
  --config-file config.yaml \
  --concurrency 10 \
  --output results.jsonl --html report.html

For a one-off you can skip the file: --config '{"model":"outlaw-1","base_url":"https://wildwestapi.com/v1"}' with OPENAI_API_KEY set to your Wild West key. To test your own app instead, use the rest.Rest generator for the target and keep Wild West API in the judge and attacker blocks.

Cost estimate

Estimate only. A glob over two probe families might send 500 to 1,000 prompts. At 400 in and 300 out each, 1,000 calls is 0.4M in and 0.3M out: about $0.84 on qwen3.8-27b-outlaw. A multi-turn probe with max_turns: 10 makes attacker, target and judge calls per turn with growing context, so one goal can run 30 or more calls; budget a few cents per goal on outlaw-1, more with a glm-5.3-outlaw judge.

Tips

• --concurrency defaults to 10. Lower it on 429s.
• Raise --timeout for iterative probes like PAIR and TAP.
• Run --all only on a key with a hard cap.
• If you see attacker_parse_failures, switch the attacker to glm-5.3-outlaw or raise attack_max_attempts.

Troubleshooting

• "openai generator requires api_key". Neither api_key nor OPENAI_API_KEY is set.
• 401 / 402. Bad key, or cap and balance reached; see 402.
• Calls reach OpenAI. base_url is missing from that block; each of target, judge and attacker needs it.

FAQ

Related