Risks of using 3rd party systems for AI governance

Based on the EU AI Act, high-risk AI providers have numerous documentation requirements. When using a third-party SaaS document management system to store these documents, several privacy and security concerns arise.

As developers of the open source AI governance platform VerifyWise, and through our work with on-premises platforms in the enterprise, we know these risks well.

In this post we'll discuss how sharing this information with a third-party system could expose sensitive intellectual property and trade secrets.

Disclosing sensitive technical documentation

According to the EU AI Act, high-risk AI providers must create and maintain detailed technical documentation. This includes:

• System architecture and design specifications
• Data requirements and governance procedures
• Development methodologies and techniques
• Performance metrics and testing results

Most high-risk AI companies with strict privacy requirements do not disclose such information publicly. Financial institutions, insurance companies, healthcare providers and telcos typically use self-hosted, on-premises platforms to store relevant data.

Disclosing risk registry

According to the EU AI Act, providers must establish a risk management system, documenting:

• Risk identification and analysis
• Risk estimation and evaluation
• Risk control measures

This documentation may also contain sensitive information about the company's system vulnerabilities and mitigation strategies.

Documentation about training data

According to the EU AI Act, detailed records of training, validation, and testing datasets must be maintained, including:

• Data sources and characteristics
• Preprocessing techniques
• Labeling procedures

Uploading documents with this information could risk exposing proprietary datasets or data handling methods to 3rd party SaaS platforms.

Logging capabilities

High-risk AI systems must have logging capabilities to record events and decisions. Documentation of these capabilities may include:

• Types of data logged
• Storage and retention policies
• Access control measures

This information could reveal system operations and data handling practices, which poses risks when sharing documents with a 3rd party AI governance provider. The risk compounds when employees adopt AI tools without IT approval, a problem that shadow AI detection is designed to address by surfacing unauthorized usage from your existing network logs.

For all of the areas above, uploading detailed documentation to a third-party SaaS AI Governance platform could potentially expose sensitive operational information. This may compromise the AI provider's competitive advantage.

Additionally, if this information falls into the wrong hands, it could be exploited to target weaknesses in the AI system or the organization's incident response capabilities.

To mitigate these risks, high-risk AI providers should carefully assess the security measures of the SaaS provider, implement strong access controls, and consider encrypting particularly sensitive portions of the documentation.

They may also want to explore hybrid solutions that keep the most critical information on-premises while using the SaaS platform for less sensitive documentation.

VerifyWise: democratizing AI governance

VerifyWise's open-source AI governance platform can be installed on-premises and offers several advantages in terms of privacy and security for high-risk AI providers:

• Data control: You have full ownership and control over sensitive documentation and data. There is no need to upload critical information to third-party systems, which reduces risk of unauthorized access or data breaches.

• Customization and integration: Since you have the full source code, you can tailor the platform to specific organizational needs and security requirements, and integrate with existing on-premises security infrastructure.

• Reduced exposure of trade secrets: Your sensitive AI system details and algorithms remain within the organization, lowering the risk of IP leakage.

By offering an on-premises, open-source AI governance platform, VerifyWise provides high-risk AI providers with a solution that addresses many of the privacy and security concerns associated with using third-party SaaS systems. This approach allows organizations to maintain stricter control over their data and infrastructure while still benefiting from a structured AI governance framework.

Contact us now to see a demo of VerifyWise.