Gateway settings

Overview

The Settings page covers 3 areas: API keys for LLM providers, monthly budget limits, and guardrail configuration. Changes take effect immediately.

API keys

API keys connect the gateway to LLM providers. Each key is encrypted at rest using AES-256-CBC and only decrypted at the moment a request is proxied to the provider. Keys are scoped to your organization and can be referenced by multiple endpoints.

Adding a key

1. Click "Add key"
2. Enter a name (e.g., "Production OpenAI key")
3. Select the provider from the dropdown. The top 10 providers are listed first, with 100+ additional providers below the divider.
4. Paste your API key
5. Click "Add key"

Budget

Budgets set a monthly spending limit across all endpoints. The budget section shows current spend with a progress bar, alert threshold, and hard limit status. Spend resets automatically on the 1st of each month.

Budget options

Budget alerts

When spend crosses the alert threshold percentage, the system logs an alert. Alerts are deduplicated per month (only 1 alert per threshold crossing per month). The budget spend resets automatically on the 1st of each month via a background job.

Guardrail settings

These settings control the global behavior of guardrail scanning. They apply to all guardrail rules regardless of type.

Error behavior

Controls what happens when the guardrail scanner itself fails (e.g., the AI Gateway service is temporarily unavailable):

• PII scan on error: Block (default): If PII scanning fails, all requests are blocked. Fail-closed. Recommended for regulated environments.
• Content filter on error: Allow (default): If content filtering fails, requests go through. Fail-open. Prevents a scanner outage from blocking all AI traffic.

Replacement text

When a guardrail masks content, these settings control the replacement text:

• PII replacement format: Default: "<ENTITY_TYPE>". The placeholder ENTITY_TYPE is replaced with the detected type (e.g., "<EMAIL_ADDRESS>", "<CREDIT_CARD>").
• Content filter replacement: Default: "[REDACTED]". A fixed string that replaces any matched keyword or regex pattern.

Audit log retention

Guardrail detection logs record every blocked or masked request for compliance auditing (EU AI Act Art. 12). The retention period controls how long these logs are kept. Default: 90 days. Use the "Purge old logs" button to immediately delete logs older than the retention period.