Canada’s Health-AI Checklist Starts Before the Contract
Matthew Leo · Published October 31, 2026 · Guides
A hospital should not discover how an artificial-intelligence tool handles patient data after the contract is signed.
Canada’s federal, provincial and territorial privacy regulators have published a joint framework for AI in health care and health research. The resolution, adopted September 15 and announced October 29, calls for systems that are privacy-protective, valid and reliable, safe, human-rights affirming, transparent and accountable.
The resolution is not a new national statute. It is a coordinated statement from the regulators who interpret and enforce privacy laws across Canada. Its value is practical: it shows what hospitals, clinics, vendors and researchers should expect to justify throughout an AI system’s life cycle.
1. Define the clinical need before evaluating a model
Start with the problem, affected patients, intended users and decision being supported. A transcription assistant, scheduling optimizer and diagnostic classifier do not carry the same risks.
- State the exact purpose and the outcome the organization expects to improve.
- Identify whether the tool creates recommendations, records, risk scores or consequential decisions.
- List every category of personal health information it will receive, infer, retain or export.
- Define what a human must verify before an output enters a chart or affects care.
If the need can be met without sending identifiable data to a new service, compare that option first. Data minimization is easier to enforce when it is designed into the procurement rather than negotiated after deployment.
2. Complete three assessments, not one
The regulators recommend integrating AI risk management with privacy impact assessments, threat-risk assessments and AI-specific reviews. Ontario’s Information and Privacy Commissioner provides a more detailed AI-scribe guide and checklist that apply the same approach to a common clinical tool.
- Privacy impact: Map collection, authority, consent, use, disclosure, retention, access rights and deletion.
- Security risk: Test identity controls, encryption, logging, vendor access, incident response and integration with electronic records.
- AI risk: Assess accuracy, bias, subgroup performance, explainability, human oversight and the consequences of an incorrect output.
Record assumptions and owners. A risk register that says “the vendor handles security” does not identify who verifies the control or responds when it fails.
3. Ask the vendor to draw the data flow
The vendor should be able to show where data enters, where it is processed, which subcontractors can access it, what is logged, where backups live and when each copy is deleted.
Ask whether prompts, recordings, notes or corrections are used to train any model. “We do not train on your data” is incomplete unless it covers analytics, debugging, abuse monitoring, human review and subcontractors. Ask whether the answer changes for optional features or future product tiers.
For a pan-Canadian starting point, Canada Health Infoway publishes an AI procurement toolkit and separate implementation guidance for AI scribes. Provincial rules still apply, and Quebec and Alberta may require jurisdiction-specific work.
4. Put operational promises into the contract
The contract should describe more than uptime and subscription price. Include provisions for:
- Permitted purposes and explicit prohibitions on secondary use or commercialization.
- Data location, cross-border access and every material subcontractor.
- Retention and verified deletion from production systems, logs and backups.
- Security controls, breach notification timing and cooperation during an investigation.
- Audit rights, evidence delivery and remediation deadlines.
- Advance notice before model, hosting, training-data or subprocessor changes.
- Performance commitments for the relevant clinical population and workflow.
- Export, transition assistance and deletion when the organization leaves.
Do not accept a term that lets a vendor change material data practices through an online policy without notice. Procurement staff should also check whether a limitation-of-liability clause leaves the health organization carrying nearly all privacy and patient-harm risk.
5. Build patient notice and challenge routes
The joint resolution says people should receive clear explanations of how their information is collected, used, disclosed and retained. When consent is required, communication must be meaningful. Where practical, providers should offer an opt-out before deployment.
Name a contact for concerns and create a way to challenge consequential decisions or inferences. A generic privacy inbox is not enough if staff cannot trace which model version produced an output or correct the record promptly.
6. Monitor the deployed system, not only the pilot
Accuracy can change when patient populations, workflows, model versions or integrations change. Track corrected outputs, near misses, privacy complaints, security events and differences among relevant demographic or clinical groups.
Set thresholds that pause use or trigger review. Keep human verification meaningful: a clinician who must approve hundreds of AI-generated notes without time to inspect them is not a reliable control.
Shadow AI also needs a response. Staff may adopt consumer tools when an approved system is slow or unavailable. Training, accessible alternatives and proportionate technical controls work better than a policy that simply says “do not.”
7. Treat research as a separate governance path
The regulators want national research-ethics guidance updated for AI-enabled studies and stronger training for research ethics boards. Researchers should disclose methods, assess group harms and share outcomes rather than assuming de-identification eliminates risk.
A clinical deployment, quality-improvement project and research study may have different legal authorities and oversight. Label the activity correctly before data is reused.
A short approval test
Before approving a health-AI system, the accountable executive should be able to answer five questions in plain language: What problem does it solve? What information does it touch? Who can access or reuse that information? How do people challenge an error? What evidence will cause the organization to stop or change the system?
If those answers are still “the vendor will tell us,” procurement is not finished.
Tags: AI governance, privacy, Canada, digital health