Risks of using 3rd party systems for AI governance

•
October 2, 2024

Risks of using 3rd party systems for AI governance

Based on the EU AI Act, high-risk AI providers have numerous documentation requirements. When using a third-party SaaS document management system to store these documents, several privacy and security concerns arise.

As developers of open source AI governance platform VerifyWise, together with our expertise working with onprem platforms in the enterprise, we are aware of those risks.

In this post we’ll discuss about the control items of which this information to a third-party system could expose sensitive intellectual property and trade secrets.

Disclosing sensitive technical documentation

According to the EU AI Act, high-risk AI providers must create and maintain detailed technical documentation. This includes:

System architecture and design specifications.
Data requirements and governance procedures.
Development methodologies and techniques.
Performance metrics and testing results.

Most of the time, a high risk AI company with stringent privacy considerations do not disclose such information to the public. Especially financial institutions, insurance companies, healthcare providers and telcos use self hosted, on-premises platforms to store relevant data.

Disclosing risk registry

According to the EU AI Act, providers must establish a risk management system, documenting:

Risk identification and analysis
Risk estimation and evaluation
Risk control measures

This documentation may also contain sensitive information about the company’s system vulnerabilities and mitigation strategies.

Documentation about training data

According to the EU AI Act, detailed records of training, validation, and testing datasets must be maintained, including:

Data sources and characteristics
Preprocessing techniques
Labeling procedures

Uploading documents including information above could risk exposing proprietary datasets or data handling methods to 3rd party SaaS platforms.

Logging capabilities

High-risk AI systems must have logging capabilities to record events and decisions. Documentation of these capabilities may include:

Types of data logged
Storage and retention policies
Access control measures

This information could reveal system operations and data handling practices, which poses several risks when sharing documents including relevant private data with a 3rd party AI governance provider.

For all of the areas above, uploading detailed documentation to a third-party SaaS AI Governance platform could potentially expose sensitive operational information. This may compromise the AI provider’s competitive advantage.

Additionally, if this information falls into the wrong hands, it could be exploited to target weaknesses in the AI system or the organization’s incident response capabilities.

To mitigate these risks, high-risk AI providers should carefully assess the security measures of the SaaS provider, implement strong access controls, and consider encrypting particularly sensitive portions of the documentation.

They may also want to explore hybrid solutions that keep the most critical information on-premises while leveraging the SaaS platform for less sensitive documentation.

VerifyWise: democratizing AI governance

VerifyWise’s open-source AI governance platform can be installed on-premises offers several key advantages in terms of privacy and security for high-risk AI providers:

Enhanced data control: You have full ownership and control over sensitive documentation and data. There is no need to upload critical information to third-party systems, which reduces risk of unauthorized access or data breaches.
Customization and integration: Since you have the full source code, you have the ability to tailor the platform to specific organizational needs and security requirements, and integrate with existing on-premises security infrastructure
Reduced exposure of trade secrets: Your sensitive AI system details and algorithms remain within the organization, lowering the risk of IP leakage.

By offering an on-premises, open-source AI governance platform, VerifyWise provides high-risk AI providers with a solution that addresses many of the privacy and security concerns associated with using third-party SaaS systems.

This approach allows organizations to maintain stricter control over their data and infrastructure while still benefiting from a structured AI governance framework.