MLflow integration

Overview

The MLflow integration synchronizes your MLflow model registry with VerifyWise's model inventory. This connection automatically imports registered models, their versions, lifecycle stages, training metrics, and experiment context into VerifyWise for governance tracking.

By connecting MLflow, you eliminate manual data entry, ensure production models are captured with their metrics, and maintain audit trails showing when models were synced and any changes detected.

Why integrate with MLflow

Integrating MLflow with VerifyWise provides several governance benefits:

• Automated inventory: Models registered in MLflow automatically appear in your VerifyWise model inventory
• Training evidence: Metrics, parameters, and experiment data are captured as evidence of model development
• Version tracking: All model versions and lifecycle stage changes are recorded
• Governance gaps: Identify models that may be missing bias assessments, security reviews, or privacy evaluations
• Audit trails: Maintain defensible logs of sync timestamps and data changes

Prerequisites

Before configuring the MLflow integration, ensure you have:

• Admin access to VerifyWise
• MLflow tracking server URL and connection details
• Authentication credentials if your MLflow server requires them
• Read-only access permissions on the MLflow account
• Background worker process running (for scheduled syncs)

Configuring the integration

To set up the MLflow integration:

1. Navigate to the Integrations page from the main menu
2. Locate the MLflow integration card
3. Enter your MLflow tracking server URL
4. Set the request timeout (increase if you have many models)
5. Select your authentication method
6. Enter credentials if required
7. Click Test Connection to verify connectivity
8. Once the test succeeds, click Save Configuration

Authentication methods

VerifyWise supports three authentication methods for connecting to MLflow:

• None: For MLflow servers with anonymous read access. No credentials required.
• Basic authentication: Username and password authentication. Use this for servers configured with HTTP basic auth.
• API token: Token-based authentication. Use this for MLflow deployments that issue API tokens.

Credentials are encrypted before storage and are never logged. Use a service account with read-only permissions rather than personal credentials.

Understanding status cards

After configuration, the integration page displays status cards showing the current state of your MLflow connection:

• Connection status: Shows Connected, Action required, or Error depending on the last test result
• Last error: Displays the most recent error message. Clears automatically after a successful test or sync.
• Last successful test: Timestamp of the most recent successful connection test
• Last scheduled sync: Timestamp of the most recent automated sync attempt
• Upcoming sync: Indicates when the next scheduled sync will occur

Scheduled sync behavior

Once configured, VerifyWise automatically syncs with MLflow on an hourly schedule. The sync process:

1. Queries MLflow for all registered models and their versions
2. Retrieves linked experiment runs with metrics and parameters
3. Updates the VerifyWise database with new or changed records
4. Skips records that have not changed since the last sync
5. Logs the sync attempt in the sync history table

Working with MLflow data

Synced MLflow data appears in the Model Inventory section under the MLflow data tab. This tab provides a read-only view of models imported from MLflow.

Summary cards

At the top of the MLflow data tab, summary cards display counts for:

• Models: Total number of registered models synced from MLflow
• Active: Models in the Production lifecycle stage
• Staging: Models in the Staging lifecycle stage
• Experiments: Number of unique experiments associated with synced models

Use the Refresh button to re-query the VerifyWise database. Note that this refreshes the display from local data, not from MLflow directly.

Data table

The main table displays synced models with the following columns:

• Model name: The registered model name from MLflow
• Version: Model version number
• Lifecycle status: Current stage (None, Staging, Production, Archived)
• Created date: When the model version was registered in MLflow
• Last updated: Most recent modification timestamp
• Description: Model description from MLflow
• Actions: View details button

Detail drawer

Click on any table row to open the detail drawer, which displays comprehensive information about the model:

• Model metadata: Version, lifecycle stage, run ID, creation timestamp
• Description: Full model description from MLflow
• Tags: Key-value tags attached to the model in MLflow
• Metrics: Training and evaluation metrics from the linked run
• Parameters: Hyperparameters and configuration used during training
• Experiment info: Experiment ID, name, and artifact storage location

Manual sync

If you need to sync immediately rather than waiting for the next scheduled run, use the Re-run Sync button on the Integrations page. This triggers an immediate sync from MLflow and updates the Last scheduled sync timestamp.

The Refresh button on the MLflow data tab only refreshes the display from the VerifyWise database. To pull fresh data from MLflow, use Re-run Sync on the Integrations page.

Troubleshooting

Connection succeeds but no data appears

If the connection test passes but no models appear in the MLflow data tab:

• Verify the background worker process is running
• Check that the service account has permissions to list models in MLflow
• Wait for the next scheduled sync (hourly) or trigger a manual sync
• Check API logs for BullMQ errors or sync failures

Unable to reach MLflow backend

If you see connection errors:

• Increase the Request Timeout value in the configuration
• Verify network connectivity between VerifyWise and the MLflow server
• Check that the MLflow URL is correct and includes the protocol (http:// or https://)
• Verify firewall rules allow outbound connections to the MLflow server

The error card on the Integrations page displays the most recent failure reason, which can help diagnose connectivity issues.

Data appears stale or outdated

If the MLflow data tab shows outdated information:

• Check the Last scheduled sync timestamp to confirm syncs are running
• Use Re-run Sync to trigger an immediate refresh from MLflow
• Verify the worker process has not stopped or encountered errors

Best practices

• Use a service account: Create a dedicated read-only account for VerifyWise rather than using personal credentials
• Monitor sync status: Regularly check the status cards to ensure syncs are completing successfully
• Keep worker running: Ensure the background worker process runs continuously for scheduled syncs
• Document models in MLflow: Add descriptions and tags in MLflow to enrich the data available in VerifyWise
• Review governance gaps: Use the synced data to identify models that need risk assessments or security reviews