The language of bias auditing

Automated decision-making technology. Colorado's term under SB 26-189 for technology that processes personal data to generate outputs used to make, guide or assist a decision about a person.

See also: Automated-decision system, Consequential decision

A substantially different rate of selection that works to the disadvantage of a protected group, whether or not anyone intended it. Adverse impact is about outcomes, which is why a model that never sees race can still produce it.

See also: Disparate impact, Four-fifths rule

Automated employment decision tool. NYC Local Law 144's name for software that uses machine learning, statistics, data analytics or AI in a hiring decision. It counts if it does enough of the thinking to stand in for human judgement.

See also: Automated-decision system

Unlawful differential treatment or impact produced by an AI system on the basis of a protected characteristic. It was the organising term in Colorado's repealed 2024 AI Act, and remains the plain-language name for what a bias audit looks for.

Wider context: Algorithmic discrimination in the AI governance lexicon

The signed report naming the independent auditor, the system audited, the data used, the methodology applied and the results. The artefact a regulator asks for.

California's term in the FEHA regulations, hyphenated in the text, for a computational process that makes or materially informs an employment decision. Related state bills use the unhyphenated form, so quote the spelling of whichever law you are citing.

See also: AEDT, Consequential decision

The employer's defence once adverse impact is shown: that the selection procedure measures something genuinely required by the job. A plaintiff can still prevail by identifying a less discriminatory alternative.

See also: Disparate impact

The EU AI Act procedure by which a provider demonstrates a high-risk system meets its requirements before placing it on the market.

A decision with a material legal or similarly significant effect on access to employment, education, lending, housing, healthcare, insurance or an essential government service.

The Title VII rule that a practice can look neutral and still be unlawful. If it screens out a protected group at a much higher rate, it is unlawful unless the employer shows it is job-related and necessary for the business.

See also: Adverse impact, Business necessity

A screening test from the EEOC's 1978 Uniform Guidelines. If one group is picked at less than 80% of the rate of the most-picked group, that is usually treated as evidence of adverse impact. A screening heuristic, not a legal safe harbour.

See also: Impact ratio, Selection rate

Fundamental rights impact assessment. Article 27 of the EU AI Act requires one before certain high-risk systems go into use. It applies to a narrow group: public bodies, companies running public services and anyone deploying credit-scoring or life and health insurance pricing. A normal private employer does not owe one.

See also: Impact assessment

A documented review of a system's risks, data, performance and safeguards. Colorado's repealed 2024 AI Act required one annually; the replacement dropped it, leaving the EU's fundamental rights impact assessment as the main live example in this area.

See also: FRIA

Wider context: Impact assessment in the AI governance lexicon

A group's selection rate divided by the selection rate of the most-selected group. The headline number Local Law 144 requires you to publish for every category.

See also: Four-fifths rule, Selection rate

An auditor with no employment relationship, financial stake or consulting entanglement with the employer or with the vendor of the tool being audited. Independence is the requirement most often failed in practice.

Testing combinations of protected characteristics rather than one at a time. A tool can look balanced on sex and on race separately while disadvantaging a specific sex-by-race group, which is why Local Law 144 requires the cross-tabulation.

Colorado's scope test: the technology's output must be a non-de-minimis factor in the decision, such as by constraining, ranking, scoring, recommending or classifying. Incidental and clerical uses fall outside.

See also: ADMT

A characteristic that anti-discrimination law shields from adverse treatment. The list varies by jurisdiction and is broader in Illinois and California than the federal baseline.

A feature that correlates with a protected characteristic strongly enough to stand in for it. Zip code as a proxy for race is the case Illinois HB 3773 names outright.

The short version of the audit results that Local Law 144 makes you post on the careers part of your website. It shows the selection rates, the impact ratios and the date you started using the tool.

The point below which an auditor may leave a group out of the ratio calculations. Local Law 144 sets it at under 2% of the data used for the audit. Leaving a group out is a choice, not a rule, and the auditor has to say why, along with that group's headcount and rate.

Where a tool produces scores rather than pass or fail outcomes, the share of a group scoring above the sample's median score. Local Law 144 accepts this in place of selection rate for scoring tools.

See also: Selection rate

The share of applicants in a group who advanced, were scored positively, or were hired. The building block of every impact ratio.

See also: Impact ratio, Scoring rate

Whether an observed difference is large enough, given the sample, to be unlikely to arise by chance. A small applicant pool can produce a striking impact ratio that carries little evidential weight, which is why ratios are read alongside significance tests.