Methodology
How we test, and why it holds up
An audit is only worth what its method is worth. This is the testing we run, the principles that constrain how we run it, and the artefacts you get at the end.
Principles
Independence is structural, not asserted
We hold no equity and take no revenue share in the systems we audit, and we do not sell remediation consulting attached to an audit engagement. An auditor whose next contract depends on a clean result is not independent, whatever the report says.
The method is fixed before the data arrives
Thresholds, categories, exclusion rules and significance tests are agreed at scoping and written down. Deciding which test to use after seeing which one gives a better number is how audits lose their value.
Every number traces back to a record
Each figure in the report ties to the applicant-level data that produced it, with the transformation steps recorded. If a regulator asks how a ratio was derived, the answer is in the file rather than in someone's memory.
Review is separated from analysis
A second reviewer checks methodology and results independently of whoever ran the analysis. You see findings before the report is finalised, so nothing in the signed document arrives as a surprise.
Small groups are handled explicitly
Local Law 144 lets an auditor leave out a group that makes up under 2% of the audit data. Where we do that, we say so. Dropping a group quietly is a way to make a report look cleaner than the data is.
What we calculate
Which of these appear in your report depends on the jurisdictions in scope and on whether your tool ranks, scores or filters.
Selection rate
selected in group ÷ applicants in group
Computed for every category the applicable law names, not only the largest cohorts.
Impact ratio
group selection rate ÷ highest group selection rate
The headline figure Local Law 144 requires you to publish. Read against the 0.80 threshold.
Scoring rate
share of group scoring above the median
Used in place of selection rate where a tool outputs scores rather than pass or fail.
Intersectional ratio
impact ratio per sex-by-race combination
Catches disparity that single-axis testing misses. Fourteen combinations under the EEO categories.
Statistical significance
test of whether the observed gap is likely to be chance
A striking ratio on forty applicants carries different weight than the same ratio on forty thousand.
Proxy correlation
association between model features and protected class
Run where a jurisdiction names it, as Illinois does for zip code, and wherever geography drives the model.
The engagement, stage by stage
Scope
We identify which systems are in scope, which jurisdictions apply and which protected categories each of those jurisdictions names. Scope errors are the most expensive mistake in an audit, so this stage is deliberate.
Data hand-off
Applicant-level records arrive by secure upload or through an API pull on a schedule. We confirm completeness and flag gaps in demographic coverage before any testing begins.
Testing
Selection and scoring rates per group, impact ratios against the most-selected group, intersectional cross-tabulation, significance testing and proxy analysis where a jurisdiction calls for it.
Review
A second reviewer checks methodology and results independently of whoever ran the analysis. Findings are shared with you before the report is finalised, so nothing in the signed document is a surprise.
Report and publication
You receive the signed audit report, the methodology notes and a publication-ready summary formatted for NYC Local Law 144, currently the one jurisdiction that requires publication. Prior-year reports stay available for comparison.
How we handle your data
Audit data arrives by secure upload or by an API pull on a schedule you control. We ask for applicant-level records because aggregate counts cannot support intersectional testing or significance tests, and both are required by the laws that name them.
Where you hold no demographic data, which is common, we discuss what the applicable law expects before any testing starts. Options include a candidate self-identification flow and, in some jurisdictions, statistical inference with its limitations disclosed in the report.
Retention follows the longest applicable requirement, which is four years under the California regulations. What we hold, for how long and who can reach it is set out in the engagement terms rather than left to assumption.
What you receive
The signed audit report
Names VerifyWise as auditor of record, the system audited, the data used, the method applied and the results. This is the document a regulator asks for.
Methodology notes
The decisions taken at scoping, the thresholds applied and any group excluded from calculations with the reason it was excluded.
Publication-ready summary
Formatted for NYC Local Law 144, currently the one jurisdiction that requires public posting, ready to place on the employment section of your site.
Prior-year comparison
From the second cycle onward, movement in each ratio against the previous audit, so you can see whether changes to the system moved the numbers.
Questions about the method?
We would rather answer them before an engagement than after a report lands.