Algorithmic Impact Assessment

Summary

Canada's Algorithmic Impact Assessment (AIA) is the world's first mandatory government-wide tool for evaluating AI systems before deployment. This questionnaire-based assessment determines whether your automated decision-making system poses low, medium, or high impact risks—and what mitigation requirements you must meet. Unlike voluntary frameworks, this tool carries real regulatory weight: federal departments must complete it before implementing any automated decision-making system, making it essential reading for anyone working with AI in Canadian government contexts.

How the scoring works

The AIA uses a point-based system across four key dimensions: decision type, algorithm type, data inputs, and organizational capacity. Each question assigns points based on risk factors—for example, using machine learning adds more points than rule-based systems, while processing sensitive personal data significantly increases your score.

Your final score determines your impact level:

• Low impact (0-15 points): Minimal mitigation requirements
• Medium impact (16-35 points): Moderate oversight and documentation needed
• High impact (36+ points): Extensive governance, human oversight, and regular auditing required

The tool automatically calculates your score and generates specific compliance requirements based on your result, removing guesswork about what's needed for Treasury Board Directive compliance.

Who this resource is for

• Federal government employees developing or procuring automated decision-making systems
• Vendors and contractors building AI solutions for Canadian government departments
• Compliance officers ensuring adherence to the Treasury Board Directive on Automated Decision-Making
• International policy makers studying Canada's pioneering approach to mandatory AI assessments
• Researchers analyzing government AI governance frameworks in practice

What makes this tool different

Most AI risk assessments are voluntary guidelines—Canada's AIA has teeth. It's legally mandated under Treasury Board policy, meaning federal departments face real consequences for non-compliance. The tool also goes beyond technical risk to evaluate organizational readiness, asking whether your department has adequate staff training and governance structures.

The assessment covers the full AI lifecycle, from initial development through ongoing monitoring. It's designed for government decision-makers, not just technical teams, using plain language questions about business impact rather than algorithmic complexity. This makes it accessible to the program managers who actually deploy these systems.

Common implementation challenges

Timing confusion: Many teams complete the AIA too late in the development process. It should be done during planning phases, not after your system is built—high-impact scores may require fundamental design changes.

Vendor coordination: When using external AI services, determining who completes which sections can be unclear. Generally, the government department completes questions about decision context and organizational capacity, while vendors provide technical algorithm details.

Ongoing obligations: The AIA isn't a one-time checkpoint. Systems must be reassessed when materially changed, but the tool doesn't clearly define what constitutes a "material change"—leading to inconsistent re-assessment practices across departments.

Quick reference checklist

Before starting your assessment:

• [ ] Identify all stakeholders affected by the automated decision
• [ ] Gather technical documentation about your algorithm/system
• [ ] Review your department's existing governance structures
• [ ] Determine if you're using third-party AI services or building in-house
• [ ] Prepare to document data sources and types being processed
• [ ] Allocate time for implementing mitigation measures based on your likely score

The assessment typically takes 30-60 minutes to complete, but implementing the required mitigation measures for medium and high-impact systems can take months.