What does an AI audit actually check?
Four things, in order. Scope: which systems exist, what they decide and who owns them. Data: where training and input data came from, whether the company may use it, and whether it skews outcomes, the source of most AI bias. Behavior: accuracy, failure rates and robustness on realistic cases. Controls and records: guardrails, human review, access rights and logs that let a reviewer reconstruct a decision, which is what auditability means in practice. An auditor tests evidence that a control operated, not a policy saying it should.
Who performs an AI audit?
| Auditor | Typical trigger | What you get |
|---|---|---|
| Internal audit or risk team | Board request, annual audit plan | Findings and an action plan |
| Independent third party | Customer demand, or a law such as New York City’s hiring rule | An external report, sometimes published |
| Accredited certification body | Pursuing ISO 42001 | A certificate, with surveillance audits |
| Notified body under the EU AI Act | Some high-risk systems | A conformity assessment |
New York City shows what a mandated audit looks like: employers may not use an automated hiring tool unless it has been “subject to a bias audit within one year of the use of the tool”, with a summary of results made public (NYC DCWP, read 5 November 2026). The trigger decides the auditor, so ask who is requesting the audit before you scope it.
How does an AI audit relate to ISO 42001 and the EU AI Act?
ISO 42001 certification audits the management system: policies, roles, impact assessments and controls across all your AI, not one model. The EU AI Act works per system. For most high-risk uses listed in Annex III, points 2 to 8, Article 43 prescribes internal control, a procedure that “does not provide for the involvement of a notified body” (AI Act Article 43, read 5 November 2026). For most high-risk systems, the company audits itself, and its own evidence is what a market surveillance authority will read later. That is where AI assurance and AI compliance meet.
How do you prepare for an AI audit?
Start with the inventory, because an auditor cannot test what nobody listed. Build an AI inventory, classify each system’s risk, and name an owner for each. Then make the evidence exist before anyone asks: logs and an agent audit trail for systems that act, evaluation results on your own cases, approval records, and a residual risk register signed by the business owner. An audit finds gaps cheaply only if you run one on yourself first, which is why a dry run against the criteria is usually the first real step of AI governance.