Integration guide
Step-by-step instructions for connecting your network proxy, SIEM, or custom integration to Shadow AI.
Overview
This guide walks you through connecting your network infrastructure to VerifyWise Shadow AI detection, from creating your first API key to verifying that events are flowing into the dashboard. There are two integration paths: the REST API (for custom integrations, browser extensions and SIEM webhooks) and syslog forwarding (for network proxies and firewalls).
Prerequisites
- Admin role in VerifyWise (only Admins can create API keys and syslog sources)
- Access to your network proxy, firewall, or SIEM configuration
- Network connectivity from your data source to the VerifyWise server
REST API integration
The REST API is the most flexible integration method. Use it when your data source can make HTTP requests, for example a SIEM webhook action, a custom script or a browser extension.
Step 1: Create an API key
- Navigate to Shadow AI → Settings
- In the "API keys" section, click "Create API key"
- Enter a label that identifies the data source (e.g., "Zscaler webhook" or "Browser extension prod")
- Click "Create"
- Copy the full key immediately and store it securely, it won't be shown again
Step 2: Send events
Send a POST request to the ingestion endpoint with your events. The endpoint accepts batches of up to 10,000 events per request.
Endpoint: POST /api/v1/shadow-ai/events
Headers:
| Header | Value |
|---|---|
| Content-Type | application/json |
| X-API-Key | Your API key from step 1 |
Example request body:
{
"events": [
{
"user_email": "alice@company.com",
"destination": "chat.openai.com",
"timestamp": "2026-02-09T14:32:00Z",
"uri_path": "/v1/chat",
"http_method": "POST",
"action": "allowed",
"department": "Engineering",
"job_title": "Senior Engineer",
"manager_email": "bob@company.com"
}
]
}Required fields: user_email, destination, timestamp. All other fields are optional. department feeds risk scoring and department-level reporting.
VerifyWise stores user_email, manager_email and destination in lowercase with surrounding spaces removed, so Alice@Company.com and alice@company.com count as one user. http_method is stored in upper case. An action of exactly blocked is stored as blocked; any other value, or none, is stored as allowed.
Step 3: Verify ingestion
A successful request returns a 200 response with the number of events ingested and tools matched:
{
"message": "OK",
"data": {
"ingested": 1,
"tools_matched": 1
}
}After sending events, verify they appear in the system:
- Go to Shadow AI → Insights and check that the summary cards update (unique apps, AI users)
- Go to Shadow AI → AI tools and confirm the destination domain appears as a detected tool
- Go to Shadow AI → User activity and confirm the user email appears in the users list
REST API error responses
| Status code | Cause | How to fix |
|---|---|---|
| 400 | No events array, a missing required field, or an invalid user_email | Send a body with an events array. The message names the index of the first bad event. Check that every event has user_email, destination and timestamp, and that user_email is an email address. The refused batch still counts against the hourly rate limit. |
| 401 | Missing or invalid API key | Ensure the X-API-Key header is set with a valid, non-revoked key. |
| 413 | Batch too large | Reduce to 10,000 or fewer events per request. |
| 429 | The batch would go over the hourly limit | The whole batch is refused. The hour starts with the first batch counted, not on the clock hour. Wait, send a smaller batch, or ask an Admin to raise the limit in Settings → Rate limiting. |
| 500 | An event cannot be stored: the timestamp cannot be read, or a field is too long (user_email, manager_email, department or job_title over 255 characters, destination over 512, http_method over 10) | The whole batch is refused and nothing is stored. Send ISO 8601 timestamps and shorten the field. The refused batch still counts against the hourly rate limit. |
An empty events array returns 200 with ingested: 0. A uri_path longer than 2,048 characters is left out; the event itself is stored.
Syslog integration
Use syslog when a network proxy or firewall (Zscaler Internet Access, Netskope, Squid, or anything that writes key=value lines) can forward its logs. VerifyWise runs a TCP syslog listener on port 6514. Each log line carries a secret key that tells VerifyWise which organization and which source it belongs to. Parsed lines go into the same pipeline as the REST API.
Step 1: Check that the listener is on
Open Shadow AI → Settings → Syslog sources. A line above the table shows the state. When it is on, it reads "Syslog listener is on. Send to host:6514 over TLS." (or "over plain TCP"). When it is off, it reads "Syslog listener is off. Set SHADOW_AI_SYSLOG_ENABLED=true on the server to turn it on." If it is turned on but could not start, it reads "Syslog listener failed to start. Check the server log."
To turn it on, set these environment variables on the VerifyWise backend and restart it. A boolean is on only for the exact value true.
| Variable | Meaning |
|---|---|
| SHADOW_AI_SYSLOG_ENABLED | Set to true to start the listener. Default: off. |
| SHADOW_AI_SYSLOG_PORT | TCP port. Default: 6514. |
| SHADOW_AI_SYSLOG_TLS_CERT_FILE | Path to the TLS certificate (PEM). Set together with the key file. |
| SHADOW_AI_SYSLOG_TLS_KEY_FILE | Path to the TLS private key (PEM). |
| SHADOW_AI_SYSLOG_ALLOW_PLAINTEXT | Set to true to accept unencrypted TCP. Needed if you set no certificate. Default: off. |
| SHADOW_AI_SYSLOG_ALLOW_IP_ONLY | Set to true to match lines without a key by sender IP. Self-hosted, single-tenant only. Default: off. |
The listener starts without a certificate only when plaintext is allowed. It does not start with SHADOW_AI_SYSLOG_ALLOW_IP_ONLY=true when MULTI_TENANCY_ENABLED=true. If it does not start, the server log names the variable to fix. The listener accepts TCP only. There is no UDP. TLS is detected on its own, so one port serves TLS and, if allowed, plain TCP. The certificate and key are read when the backend starts. After you renew or rotate them, including a Kubernetes Secret, restart the backend.
Publish the port to your proxy. With Docker Compose, publish 6514:6514 on the backend. With the Helm chart, set shadowAiSyslog.enabled=true and shadowAiSyslog.tlsSecretName.
Step 2: Add a source and copy its key
- Go to Shadow AI → Settings. You need the Admin role to manage syslog sources.
- In "Syslog sources", click "Add source".
- Choose the Parser type: Zscaler, Netskope, Squid proxy or Generic key-value.
- Optionally enter an Allowed sender IP. If you set one, lines from any other IP are dropped. Leave it empty to accept the key from anywhere.
- Click "Add". The "Your source key" dialog shows the key and a ready-made
vw_source=<key>line. Copy it now. You cannot see the key again.
Create one source per proxy or log format. Each source has its own key. To replace a key, click "Regenerate key" on the source and confirm. The old key stops working. The server you are connected to stops accepting it at once. Other server replicas can keep accepting it for up to 60 seconds. Removing a source works the same way.
Step 3: Add the key to your log lines and forward them
Configure your proxy to send logs over TCP to your VerifyWise host on port 6514, using TLS, and to include vw_source=<your key> in every line. Put the field at the start or the end of the line, separated from the rest by a space. A key anywhere else in the line is ignored, so text inside another field, such as a user agent or a URL, can never route a line. VerifyWise removes it before the line is stored. The examples below show one line per product, with the field added.
Each event takes its time from the line's own time field, else from the syslog header, else from the time VerifyWise received it. A time with no zone is read as UTC, so if your proxy logs in local time, include an offset (+02:00) or Z. A zone name VerifyWise does not know, such as IST, makes the line's time unreadable and the header or receive time is used. A time more than a year old or more than 24 hours ahead is ignored the same way. As with the REST API, user emails and destinations are stored in lowercase.
Zscaler Internet Access
Create a Nanolog Streaming Service feed that sends to your VerifyWise host over TCP. In the feed output format, include the fields user, dst, method, uri and action as name=value pairs, optionally department and timestamp, and add the literal text vw_source=<key>. Your feed must be able to add a fixed field to every line. If it cannot, a self-hosted server can use IP-only mode (see below).
user=alice@company.com dst=chat.openai.com method=POST uri=https://chat.openai.com/v1/chat action=allowed department=Engineering vw_source=YOUR_SOURCE_KEYNetskope
Send the events as JSON over syslog, one object per line. Text before the object, such as a syslog tag, is fine. The object needs user and url (if url has no usable host, domain or site is used), plus method and action (the policy verdict), optionally department and timestamp. allow, alert, useralert and bypass are stored as allowed; block, restrictToView, restrictAccess and disableDownload as blocked. activity (what the user did, such as Upload) is read only when there is no action. Put the key in the object as a top-level "vw_source" member (not inside a nested object), or write vw_source=<key> before or after the object, separated by a space. The "vw_source" member is read only when the line, after its syslog header, is the object alone; if your forwarder adds other text before the object, use vw_source=<key> at the start or end of the line instead. If your product cannot add either to every event, a self-hosted server can use IP-only mode (see below).
{"user":"alice@company.com","url":"https://chat.openai.com/v1/chat","method":"POST","action":"allow","department":"Engineering","timestamp":"2026-02-09T14:32:00Z","vw_source":"YOUR_SOURCE_KEY"}Squid proxy
The Squid parser reads the native access log format: time, elapsed, client, result code, bytes, method, URL, user. Append vw_source=<key> to the line with a custom logformat, then send it with an access_log line that points at your VerifyWise host and port. Squid must authenticate users so that the user field holds an email address.
1707489120.000 200 10.0.0.1 TCP_MISS/200 1024 POST https://chat.openai.com/v1/chat alice@company.com DIRECT/chat.openai.com vw_source=YOUR_SOURCE_KEYDENIED in it (such as TCP_DENIED/403) means Squid refused the request and is stored as blocked. Every other code is stored as allowed, including a 403 returned by the destination itself. If the user field is a dash (-), the line is dropped because it has no user email. Squid's access_log tcp:// output is not encrypted, so the server operator must allow plaintext, or you must put a TLS forwarder between Squid and VerifyWise.Generic key-value (CEF-style)
Use this parser for firewalls, DLP systems and custom forwarders that write key=value pairs or CEF extensions. In a line that starts with a CEF header (CEF:0|...|, optionally after a host name and tag) a value can contain spaces (department=Sales Ops) and runs until the next key=; in plain key=value lines a value ends at the first space unless you put it in double quotes. It reads these fields:
| Source field | Maps to | Required | Example |
|---|---|---|---|
| suser | user_email | Yes | alice@company.com |
| dhost (or host of request) | destination | Yes | chat.openai.com |
| request | uri_path (path of the URL) | No | https://chat.openai.com/v1/chat |
| requestMethod | http_method | No | POST |
| act | action | No | allowed |
| department | department | No | Engineering |
| timestamp (or CEF rt, end, start) | timestamp | No | 1770647520000 |
suser=alice@company.com dhost=chat.openai.com requestMethod=POST act=allowed vw_source=YOUR_SOURCE_KEYProxies that cannot add the key (IP-only mode)
On a self-hosted, single-tenant server where the operator set SHADOW_AI_SYSLOG_ALLOW_IP_ONLY=true, a line without a key is matched by the IP it was sent from. Enter that IP as the source's Allowed sender IP. If two sources list the same IP, neither can claim the line and both count it as dropped. IP-only mode is refused when multi-tenancy is on. Behind a load balancer or NAT the listener sees the balancer's address, so use a key instead.
Step 4: Check that lines arrive
- Generate some traffic through the proxy, for example by opening chat.openai.com.
- Wait a few seconds. VerifyWise writes events in batches, every 2 seconds or at 500 events.
- In Shadow AI → Settings → Syslog sources, check the source's row. Last received shows the time, Accepted counts stored events and Dropped counts lines that were refused.
- If Dropped goes up, read Last drop reason. The reasons are listed below.
- Open Shadow AI → Insights. Only events to a destination that matches a known AI tool count as AI usage. Other events are stored but do not appear in those counts.
| Last drop reason | What to check |
|---|---|
| No user email in the log line | The user field is empty, a dash, or not an email address. Check proxy authentication and the field names for the parser. |
| No destination host in the log line | The destination field (or URL) is missing from the line. |
| Line does not match the parser type | The Parser type does not fit the log format, or the line repeats a field such as user, dst, department or timestamp. Check that the parser type matches the log format. |
| Sent from an IP that isn't allowed or is shared with another source | The line came from an IP other than the source's Allowed sender IP, or two sources list the same IP in IP-only mode. Behind a load balancer, clear the Allowed sender IP. |
| Over the hourly event limit | The hourly rate limit in Settings was reached, or the server is writing events slower than they arrive. |
Lines with no valid key do not belong to any source, so they do not show in these counts. VerifyWise closes a connection that sends 100 such lines, or no valid line within 60 seconds.
Limits
- Message size: Up to 64 KiB per message. Larger messages are discarded.
- Connections: Up to 500 at the same time per server process. A connection with no traffic for 5 minutes is closed, and a TLS handshake must finish within 30 seconds.
- Framing: Newline-delimited and octet-counted messages are both accepted. An RFC 3164 or RFC 5424 header is removed when present.
Troubleshooting
| Symptom | Likely cause | Resolution |
|---|---|---|
| REST API returns 401 | Invalid or revoked API key | Verify the key is correct and has not been revoked in Settings → API keys. Create a new key if needed. Keys have the form vw_sk_<number>_<48 hex characters>; a key with spaces or a missing part is refused. |
| REST API returns 429 | Rate limit exceeded | Check Settings → Rate limiting. Increase the limit or wait until the next hour window. |
| Events sent but nothing in dashboard | Tool not matched to a known domain | The destination domain must resolve to a known AI tool. Check the AI tools page. If the tool shows as "Detected", events are being processed. Insights metrics may need a page refresh. |
| Syslog source shows "Never" under Last received | The listener is off, the port is not reachable, or the lines carry no valid key | Check the status line above the table in Settings. Confirm your proxy reaches the host on port 6514 over TCP and that every line starts or ends with vw_source=<key> (or carries it as a top-level JSON member) with the key you copied. If you lost the key, click "Regenerate key". Lines with no valid key are not counted anywhere. |
| Dropped count rises | Lines reach VerifyWise but cannot be turned into events | Read Last drop reason on the source row and follow the table in Step 4. |
| A key stopped working | The key was regenerated or the source was removed | Use the new key. A server replica other than the one you used can accept the old key for up to 60 seconds. |
| Syslog listener is off in Settings | SHADOW_AI_SYSLOG_ENABLED is not set to true | Set SHADOW_AI_SYSLOG_ENABLED=true, add a TLS certificate and key (or SHADOW_AI_SYSLOG_ALLOW_PLAINTEXT=true) and restart. The hosted service does not offer syslog yet. |
| Syslog listener failed to start in Settings | The listener is turned on but could not start: the port is in use, the TLS certificate or key cannot be read, or the settings conflict | Check the server log. It names the variable or the error that blocked the start. Fix it and restart the server. |
| Risk scores are all zero | The daily recalculation has not run yet | Risk scores are recalculated once a day, overnight. A new tool shows 0 until the next run. |
| Department data missing | Department field not included in events | The department field is optional but required for department-level reporting and sensitive department alerts. Include it in your event payload or syslog format. |
Best practices
- Use descriptive API key labels: Label each key with the source system and environment (e.g., "Zscaler prod" or "Browser ext staging") so you can quickly identify and rotate keys
- Include the department: The department field is optional, but risk scoring, department-level reporting and sensitive department alerts depend on it
- Start with a single source: Integrate one data source first, verify events are flowing, then add additional sources
- Set up alert rules early: Create at least a "New tool detected" rule immediately after integration so you are notified when new AI tools appear in your network
- Monitor the rate limit: If you are ingesting from a high-volume proxy, configure the rate limit in Settings to prevent accidental overload during initial rollout
- Rotate API keys regularly: Revoke and recreate API keys periodically. When rotating, create the new key first, update the data source config, then revoke the old key. With several backend replicas, a revoked key can keep working on the other replicas for up to 5 minutes