datasetactive

tau-bench: A benchmark for tool-agent-user interaction

Sierra Research

Sierra Research's open benchmark for tool-using agents, simulating airline and retail customer-service tasks with an LLM user and rule-based APIs. Measures task success, adherence to policy, and consistency across repeated trials under realistic constraints.

At a glance

Published

2024

Jurisdiction

International

More in Evaluation and benchmarks

SWE-bench: Can Language Models Resolve Real-World GitHub Issues?

Jimenez et al. (Princeton) • 2023

WebArena: A Realistic Web Environment for Building Autonomous Agents

Zhou et al. • 2023

GAIA: A Benchmark for General AI Assistants

Mialon et al. • 2023

Related resources

Practices for governing agentic AI systems: OpenAI's seven safety principles

Governance frameworks • OpenAI

Taxonomy of Failure Mode in Agentic AI Systems

Risk taxonomies • Microsoft

EleutherAI LM Evaluation Harness

Assessment and evaluation • EleutherAI

Build your AI governance program

VerifyWise helps you implement AI governance frameworks, track compliance, and manage risk across your AI systems.

Explore the library Start free trial

tau-bench: A benchmark for tool-agent-user interaction

Tags

At a glance

More in Evaluation and benchmarks

Related resources

Build your AI governance program