VerifyWise 2.5: reporting, observability, and frameworks as extensions

Governance work produces evidence, and eventually somebody asks for it. An auditor wants the control coverage for a framework. A board wants a quarterly summary. A customer's procurement team wants proof before they sign. Getting that out of VerifyWise meant assembling it each time it was asked for.

VerifyWise 2.5 is about that gap. Where 2.4 added new modules for governing third-party AI, this release makes the platform something you can operate over a long stretch: get evidence out of it on a schedule, see what it is doing internally, and fit it to the frameworks you already follow instead of the ones we shipped.

What is new in 2.5

Three changes carry the release:

• Template-first reporting. Build a report from a template, put it on a schedule, and have it delivered, with content that knows which framework you are reporting against.
• OpenTelemetry observability. OTel instrumentation across the backend, EvalServer and AI Gateway, with a compose stack for running the collector.
• Frameworks as extensions. Custom frameworks and the framework plugins that shipped with the product both moved to an extension model.

Behind those, 2.5 carries a security-hardening pass, more work on AI Agents and Agent Control, a refactor of the policy editor and the LLM Evals experiment flow, and a typography and theme token foundation.

Template-first reporting

The reporting stack in 2.5 starts from templates rather than from a blank export. You define what a report contains once, then generate it repeatedly.

Four pieces make that work:

• Templates define the structure and content of a report, so the same shape produces a comparable document every time you run it.
• Schedules run a template on a cadence rather than when somebody remembers.
• Runs record each generation, so past reports stay retrievable instead of being replaced by the latest output.
• Delivery gets the output to the people who asked for it.

Reports are also framework-aware: you select the framework a report covers, and the content follows that selection. Reporting EU AI Act coverage to one stakeholder and ISO 42001 coverage to another means two templates against the same underlying evidence.

Run history is the part an audit leans on. A report generated on a date is a different artifact from the same report regenerated today against today's data. When an auditor asks what a control looked like in March, the regenerated version does not answer the question.

OpenTelemetry observability

Governance platforms usually ask their customers for transparency while offering little in return. If a scheduled evaluation silently stops running, or a gateway call takes twelve seconds, you find out when somebody notices the gap.

2.5 instruments the services with OpenTelemetry, covering the three places where governance work runs: the backend, the EvalServer that executes evaluations, and the AI Gateway that brokers model calls. The release ships a dedicated observability compose stack, so you can run a collector alongside the platform rather than assembling one.

OpenTelemetry is an open standard under the Cloud Native Computing Foundation, and being vendor-neutral is the reason to use it here. The platform emits telemetry in a standard format instead of a proprietary one, so it goes to whatever backend you already run. If your organization has standardized on a particular observability tool, VerifyWise reports into it rather than asking you to adopt another console alongside it.

The three instrumented services are the ones where silence is expensive. An evaluation that stops running still looks fine on a dashboard that only reports the last successful result. A gateway call that degrades from one second to twelve is invisible until somebody complains. Both of those are now signals rather than surprises.

For self-hosted deployments this is the more practical change. You own the uptime, which means you need the operational signals that come with that responsibility, and you should not have to instrument someone else's services yourself to get them.

Frameworks as extensions

VerifyWise ships support for frameworks like the EU AI Act and ISO 42001. Customers also bring their own: an internal risk standard, a sector framework, a customer-imposed control set.

Previously custom frameworks and built-in framework support were different kinds of things. A framework we shipped was part of the product; one you defined was handled separately. In 2.5 both are extensions, and the framework plugins that shipped with the product were migrated onto the same model that custom frameworks use.

Sharing one model has a downstream consequence. When your own framework runs on the same mechanism as the built-in ones, it gets the same treatment from everything downstream, including the reporting stack described above. A report can target your internal risk standard the way it targets ISO 42001, because the platform does not distinguish between them. Neither do the control mappings, the evidence links or the coverage views.

The change is architectural: framework support now sits behind the same extension boundary as customer frameworks, rather than being wired into the product. That boundary is what makes it possible to treat a framework as something added to a deployment instead of something a release has to hardcode. It earns its value on the frameworks nobody builds off-the-shelf support for: a sector regulator's guidance, a parent company's internal standard, a control set a large customer wrote into your contract.

The migration in this release moves the existing plugins onto the extension model. Custom frameworks you run today keep working. The change is that the built-in ones now work the same way.

The supporting changes

The rest of 2.5 is spread across the platform.

Security hardening. The substantive changes are worth naming: row-level security, HTTPS and DAST scanning wired into CI, a server-side request forgery fix, remediation of polynomial ReDoS alerts flagged by CodeQL, and a sweep of open code-scanning alerts. Administrative registration was locked down, and an organization can now have more than one superadmin, removing a single point of failure.

AI Agents and Agent Control. Both modules arrived in 2.4 and got a round of practical work here. You can add agents manually from the empty state instead of waiting for a sync to discover them, and the interface now distinguishes manual from synced agents. Agent Control gained per-agent policy scope, decision provenance and a per-agent activity view. The Runs page gained stat cards at run granularity.

Policy editor and LLM Evals. Both were refactored. The policy editor was split into co-located modules and gained optional team-member input. The new-experiment modal in LLM Evals was broken into per-step modules for dataset, model and judge. EvalServer models moved to Pydantic.

Design tokens. A typography token foundation, chip and badge tokens, plus theme spacing migrated to MUI spacing units. Invisible in any single screen, and the reason the interface stays consistent as it grows.

Fixes worth knowing. Dashboard metrics revalidate when you return to the dashboard rather than showing a stale cache. A project's current risk level now syncs to its computed residual, so the number on the project matches the number the calculation produced. Password reset works by stored email while preserving anti-enumeration behavior. Organization-level file access is authorized by organization. The AI Gateway prompt editor model selector is searchable, and litellm moved from 1.89.3 to 1.97.0.

Testing. Accessibility checks with axe-core, contract tests for raw SQL list endpoints, a critical-journey end-to-end suite, and work on flaky-test stabilization. Frontend line coverage was targeted to move from 30 percent to at least 50.

A note on the changelog itself: the release lists 173 merged pull requests, but 99 of those are automated dependency updates. The human-authored work is 74 changes. We would rather say that than let a large number stand in for a large release.

What this covers for compliance

Reporting is the piece with the most direct compliance value, though it is worth being precise about which obligation it serves. Article 11 of the EU AI Act requires technical documentation for high-risk systems to be drawn up before the system reaches the market and kept up to date. That is the obligation scheduled reporting helps with. Article 12 is a different thing: it requires the system itself to automatically record events over its lifetime, and Article 19 requires providers to retain those logs. Governance reports are not a substitute for system-level logging. They are the dated, non-regenerated record that keeping documentation current tends to require.

ISO 42001 follows the ISO management-system structure, in which top management reviews the system at planned intervals and retains documented information as evidence of the results. Scheduled reporting produces that kind of record as a side effect of running rather than as a separate exercise before the audit.

Framework extensions matter where you are governed by something without an off-the-shelf implementation. Sector regulators, internal standards, and customer-imposed control sets can be first-class rather than tracked beside the platform.

Observability supports compliance without being a requirement of it. Where you have to show that governance controls operate on an ongoing basis, evidence that the services ran is part of that.

How to upgrade

2.5 is available now. The release notes on GitHub list every change, including the dependency updates left out here.

Self-hosted deployments upgrade the same way as previous releases; the installation guide covers environment and database setup, and the observability compose stack is optional. If you drive VerifyWise over its API, the developer portal has the endpoint reference, including the new self-scoped preferences endpoint for reading and updating the current user's settings.