Risks of using 3rd party systems for AI governance

The EU AI Act puts a long list of documentation requirements on high-risk AI providers. The moment you store those documents in a third-party SaaS, a few privacy and security problems come with them.

As the people building the source-available governance platform VerifyWise, and from our work with on-premises platforms inside enterprises, we've seen these risks up close.

Here's how handing that information to a third-party system can expose sensitive intellectual property and trade secrets.

Disclosing sensitive technical documentation

The EU AI Act requires high-risk providers to create and maintain detailed technical documentation. That covers:

• System architecture and design specifications
• Data requirements and governance procedures
• Development methodologies and techniques
• Performance metrics and testing results

Most high-risk AI companies with strict privacy requirements do not disclose such information publicly. Financial institutions, insurance companies, healthcare providers and telcos typically use self-hosted, on-premises platforms to store relevant data.

Disclosing the risk registry

Providers also have to run a risk management system, and document:

• Risk identification and analysis
• Risk estimation and evaluation
• Risk control measures

This documentation may also contain sensitive information about the company's system vulnerabilities and mitigation strategies.

Documentation about training data

The Act also wants detailed records of your training, validation and testing datasets, including:

• Data sources and characteristics
• Preprocessing techniques
• Labeling procedures

Uploading documents with this information could risk exposing proprietary datasets or data handling methods to 3rd party SaaS platforms.

Logging capabilities

High-risk AI systems must have logging capabilities to record events and decisions. Documentation of these capabilities may include:

• Types of data logged
• Storage and retention policies
• Access control measures

This information could reveal system operations and data handling practices, which poses risks when sharing documents with a 3rd party AI governance provider. The risk compounds when employees adopt AI tools without IT approval, a problem that shadow AI detection is designed to address by surfacing unauthorized usage from your existing network logs.

Across all four areas, uploading this documentation to a third-party governance SaaS can expose sensitive operational detail, and that can chip away at an AI provider's competitive position.

There's a sharper edge too. If the information reaches the wrong hands, it can be used to target weak points in the AI system or in how the organization responds to incidents.

To keep that from happening, high-risk providers should look hard at the SaaS provider's security, set strict access controls and encrypt the most sensitive parts of the documentation. It's also worth considering a hybrid setup, keeping the critical material on-premises and using the SaaS platform only for the less sensitive pieces.

How VerifyWise approaches this

VerifyWise is source-available and can be installed on-premises, which gives high-risk AI providers a few real advantages on privacy and security:

• Data control: You have full ownership and control over sensitive documentation and data. There is no need to upload critical information to third-party systems, which reduces risk of unauthorized access or data breaches.

• Customization and integration: Since you have the full source code, you can tailor the platform to specific organizational needs and security requirements, and integrate with existing on-premises security infrastructure.

• Reduced exposure of trade secrets: Your sensitive AI system details and algorithms remain within the organization, lowering the risk of IP leakage.

Running governance on-premises with code you can read addresses most of the privacy and security concerns that come with third-party SaaS. You keep tighter control over your data and infrastructure and still get a structured way to govern your AI.

Want to see how that works in practice? Book a demo and we'll walk you through it.