The Complete Guide to Canada's Algorithmic Impact Assessment

System and Algorithm Documentation

• •A clear description of what the system does and where it fits in the decision-making process
• •Documentation of the model, rules, or logic used by the system
• •A record of the system's inputs, outputs, and intended use
• •Clarity on who owns the system and who is accountable for its performance

Data Documentation

• •What data the system uses, including whether personal information is involved
• •Where the data comes from and how it is collected
• •Security classification and retention expectations
• •Any known limitations, quality issues, or bias concerns in the data
• •Any data sharing, matching, or transfer arrangements relevant to the system

Privacy and Legal Inputs

• •A completed or in-progress Privacy Impact Assessment, where applicable
• •Input from legal services on authority, compliance, and regulatory considerations
• •Confirmation of whether additional policy or legal authority may be required

Impact and Risk Inputs

• •A clear view of who may be affected by the system and how
• •Whether effects are reversible, long-lasting, or difficult to challenge
• •Whether the system affects vulnerable, equity-denied, or otherwise disproportionately impacted groups
• •Whether the system is novel, precedent-setting, or likely to attract public scrutiny

Consultation and Review Records

• •Records of consultations already completed or planned
• •GBA+ inputs where relevant
• •Notes from privacy, legal, technical, program, and operational reviewers
• •Any existing mechanisms for feedback, challenge, recourse, or escalation

Mitigation and Control Evidence

• •Documentation of existing safeguards such as audit logging, monitoring, human review, recourse, and testing controls
• •Evidence showing which controls are already operational versus still planned
• •Supporting materials that demonstrate how mitigation claims can be substantiated if later reviewed

The Algorithm

1. 1. Calculate Raw Impact Score – Sum the weighted answers from 65 risk questions across six risk areas. The maximum possible raw impact score is 169 points.

   Risk Area	No. of Questions	Maximum Score
   Project	10	22
   System	9	17
   Algorithm	9	15
   Decision	1	8
   Impact	20	52
   Data	16	55
   Raw Impact Score	65	169

   The Impact and Data areas together account for 107 of the 169 possible points. These are the most heavily weighted areas in the AIA.

2. 2. Calculate Mitigation Score – Sum the weighted answers from 41 mitigation questions across two mitigation areas. The maximum possible mitigation score is 75 points.

   Mitigation Area	No. of Questions	Maximum Score
   Consultations	4	10
   De-risking and Mitigation Measures	37	65
   Mitigation Score	41	75

3. 3. Apply Mitigation Deduction – If the mitigation score is 80% or more of the maximum attainable mitigation score, 15% is deducted from the raw impact score to produce the current score. If the mitigation score is below that threshold, the current score equals the raw impact score.

   For a 75-point mitigation score, the 80% threshold is 60 points.

   Example: A raw impact score of 90 with a mitigation score of 62 produces a current score of 76.5.

4. 4. Map to Impact Level – Convert the current score to a percentage of the maximum raw impact score of 169. The percentage determines the impact level: Level I (0%–25%), Level II (26%–50%), Level III (51%–75%), Level IV (76%–100%).

These scoring tables reflect the current structure of the public AIA tool and should be checked against the current official tool and Directive when preparing or updating an assessment.

Misclassifying System Scope

Including manual processes in the automated decision boundary inflates the risk score unnecessarily. The AIA assesses the automated component, not the entire business process. If a human officer makes the final call based on a system recommendation, the scope is the recommendation engine — not the officer's judgment. Define the boundary clearly before starting the assessment.

Answering Data Questions Without Evidence

The Data section carries the heaviest weight in the AIA (55 out of 169 possible points). Departments that claim they test for bias or validate data quality but cannot produce documentation for it will score poorly on mitigation. The mitigation questions ask about controls that are actually in place — not controls you plan to implement. Document your data governance before starting the AIA, not after.

Leaving Recourse and Explanation Too Late

Higher-impact systems require stronger explanation, human involvement, and review measures. Many teams discover late in the process that client-facing recourse, explanation, or escalation steps are underdesigned. Build these into the service design early rather than treating them as an afterthought.

Using an Outdated Questionnaire Version

The Directive on Automated Decision-Making and the AIA are reviewed periodically. Always confirm that you are using the current official questionnaire and current guidance before you start or update an assessment.

Treating the AIA as a One-Time Exercise

Published AIAs should be revisited on a scheduled basis and whenever the functionality or scope of the system changes. For higher-impact systems especially, the AIA should be treated as a living document. Any material change to the system — new data sources, expanded scope, algorithm updates — should trigger a re-assessment.

Missing the 80% Mitigation Threshold

The mitigation deduction is the single largest score adjustment available. If your mitigation score reaches 60 out of 75 points (80% of the maximum), 15% is deducted from the raw impact score. This can be the difference between Level III and Level II. But you only get credit for mitigation measures that are documented and in place — not planned or aspirational ones. Prepare your mitigation evidence before completing the assessment.