Practices for Governing Agentic AI Systems

Summary

As AI systems increasingly operate with autonomy—scheduling meetings, writing code, browsing the web, and executing multi-step tasks—the question of governance becomes critical. OpenAI's interdisciplinary research team has produced this practical framework addressing a fundamental question: how do you govern AI systems that can act independently in the world? This paper doesn't just theorize about risks; it provides concrete practices that developers, deployers, and users can implement today to ensure agentic AI systems remain safe and accountable.

What Makes Agentic AI Different

Agentic AI systems are distinguished by their ability to pursue complex goals with limited direct supervision. Unlike chatbots that respond to individual prompts, these systems can:

• Execute multi-step tasks autonomously
• Make decisions based on environmental feedback
• Take real-world actions with real consequences
• Operate over extended time periods without human intervention

This autonomy creates new governance challenges. Traditional AI oversight models assume humans review outputs before they have effects. Agentic systems may take consequential actions before any human sees them.

The Seven Practices Framework

The paper organizes recommendations around seven core practices:

1. Clear Accountability Assignment

At least one human entity should be accountable for every uncompensated direct harm caused by an agentic AI system. This creates incentives to reduce the likelihood and severity of harms efficiently.

2. Action Ledgers

System deployers should provide users with a ledger of actions taken by the agent. This lighter-touch method gives users visibility into agent operations without substantially slowing them down.

3. Human Approval Gates

Significant decisions by autonomous systems should be reviewed by a human first. The paper provides guidance on which actions warrant approval versus logging.

4. Capability Boundaries

Agentic systems should operate within clearly defined capability boundaries that limit their potential impact, especially for early deployments.

5. Staged Deployment

New agentic capabilities should be rolled out gradually, with monitoring at each stage to catch unexpected behaviors before widespread deployment.

6. Reversibility Design

Where possible, agentic actions should be reversible, allowing recovery from errors without permanent harm.

7. Shutdown Capabilities

All agentic systems should have reliable shutdown mechanisms that can halt operations when necessary.

The Lifecycle Parties

The paper identifies distinct parties in the agentic AI lifecycle, each with different responsibilities:

• Developers: Those who build the underlying AI models and agent architectures
• Deployers: Organizations that configure and deploy agentic systems for specific use cases
• Users: Individuals who direct agentic systems to accomplish tasks
• Affected Parties: People impacted by agent actions who may have no direct relationship with the system

Clear role definitions help distribute responsibility appropriately and ensure no governance gaps exist.

Who This Resource Is For

• AI developers building autonomous agent capabilities who need safety design principles
• Product managers at companies deploying agentic AI systems in production environments
• Policy teams developing internal governance frameworks for autonomous AI
• Regulators seeking to understand industry perspectives on agentic AI oversight
• Researchers studying AI safety and governance in autonomous systems
• Enterprise IT leaders evaluating agentic AI tools for business deployment

Implementation Considerations

The paper acknowledges the tension between safety and utility. Requiring human approval for every action would negate the benefits of autonomous operation. The framework provides guidance on calibrating oversight levels based on:

• Potential harm severity of actions
• Reversibility of outcomes
• User trust and capability levels
• System maturity and track record

This risk-based approach allows agentic systems to operate efficiently while maintaining appropriate safeguards for high-stakes decisions.