Skip to content

Prepare for an Audit

An audit is coming—internal, external, or regulatory. This journey helps you get your AI documentation and evidence in order.

Step 1
Scope
Step 2
Gather
Step 3
Gap Analysis
Step 4
Remediate
Step 5
Present

Step 1: Understand the Audit Scope

Know what you're being audited against.

Types of AI Audits

Type Focus Typical Auditor
Compliance Regulatory requirements Regulator, external auditor
Security Vulnerabilities, data protection Security team, IRAP assessor
Ethics Fairness, transparency, accountability Internal review, ethics board
Performance Does it work as intended? Internal audit, business review
Financial Costs, value delivery Finance, internal audit
Vendor Third-party AI services Procurement, security

Key Questions

  • What standards/policies are in scope?
  • What time period is covered?
  • What evidence format is required?
  • Who are the auditors?
  • What's the timeline?
  • What are the consequences of findings?

Australian Government Context

Common frameworks audited against:

Reference: Compliance Checklist

Step 2: Gather Your Evidence

Collect documentation across all required areas.

Evidence Categories

Document Purpose
AI governance framework Shows oversight structure
Decision records Shows who approved what
Risk assessments Shows risk management
Roles and responsibilities Shows accountability
Meeting minutes Shows governance in action
Document Purpose
Model documentation Shows what was built
Training data records Shows data provenance
Testing results Shows validation
Performance metrics Shows ongoing monitoring
Change logs Shows evolution
Document Purpose
Privacy impact assessment Shows privacy consideration
Security assessment Shows security analysis
Bias testing results Shows fairness analysis
Human oversight procedures Shows human control
Incident records Shows response capability
Document Purpose
User guides Shows user support
Training records Shows capability building
Support logs Shows operational maturity
Monitoring dashboards Shows ongoing oversight
Feedback mechanisms Shows continuous improvement

Template Mapping

Requirement Toolkit Resource
Risk documentation Risk Register
Privacy assessment PIA FAQ
Security assessment Security Template
Model documentation Model Cards Guide
Bias testing Bias Testing Guide
Stakeholder management Stakeholder Register

Step 3: Conduct Gap Analysis

Compare what you have against what you need.

Gap Assessment Matrix

For each requirement:

Requirement Evidence Exists? Evidence Quality Gap Severity
Example: PIA Yes/No/Partial Strong/Weak Critical/Major/Minor

Common Gaps

Frequently missing

  • Documentation created after the fact - Auditors can tell
  • Decisions without records - "We discussed it" isn't evidence
  • Testing without results - "We tested it" needs proof
  • Governance without teeth - Framework exists but wasn't followed
  • Outdated documentation - System changed, docs didn't

Gap Prioritisation

Severity Definition Action
Critical Required evidence missing, no workaround Must fix before audit
Major Evidence weak or incomplete Strengthen if time allows
Minor Nice to have, not required Note for future

Step 4: Remediate Gaps

Fix what you can before the audit.

What You Can Do

Create missing documentation:

  • Risk assessments (if risks were actually considered)
  • Decision records (from emails, meeting notes)
  • Process documentation (if processes exist)

Improve weak evidence:

  • Add specificity to vague documents
  • Connect documents to actual practices
  • Update outdated information

Implement missing controls:

  • If governance is missing, establish it now
  • If monitoring is missing, set it up
  • If testing is missing, do it

What You Cannot Do

Don't do this

  • Backdate documents - Fraud
  • Create evidence of things that didn't happen - Fraud
  • Hide negative information - Will be found, destroys credibility
  • Claim compliance without evidence - Will be challenged

Honest Gaps

If gaps exist that can't be fixed:

  1. Document the gap
  2. Explain why it exists
  3. Show remediation plan
  4. Demonstrate commitment to fix

Auditors respect honesty and plans more than incomplete cover-ups.

Step 5: Present to Auditors

How you present matters.

Preparation

  • Evidence organised logically
  • Index/summary document created
  • Key personnel briefed on their role
  • Consistent narrative agreed
  • Known issues documented proactively

During the Audit

Do:

  • Answer questions directly
  • Say "I don't know, I'll find out" when you don't know
  • Provide evidence promptly
  • Take notes on requests and findings
  • Be cooperative and professional

Don't:

  • Volunteer information not requested
  • Argue with findings in the moment
  • Promise things you can't deliver
  • Speak for areas outside your knowledge
  • Get defensive

After the Audit

  • Address findings promptly
  • Document remediation actions
  • Update processes to prevent recurrence
  • Share learnings with team
  • Prepare for follow-up verification

Audit Readiness Checklist

Use this for ongoing audit readiness, not just when audits are announced:

  • Governance framework documented and followed
  • Risk register current and reviewed regularly
  • Model documentation complete and current
  • Privacy impact assessment completed
  • Security assessment completed
  • Bias testing performed and documented
  • Human oversight mechanisms in place
  • Monitoring active and reviewed
  • Incident response plan exists
  • Training records maintained
  • Change management followed