AI Governance Framework for Government
Ready to Use
Quick Reference
- What: Structures, policies, and processes for responsible AI governance
- Who: Executive sponsors, governance boards, project owners, ethics leads
- Key elements: Three lines model, risk tiering, ethics review, AI register
- Compliance: Aligned with Australian AI Ethics Principles
Purpose
This framework establishes the structures, policies, and processes for responsible AI governance in government agencies, ensuring accountability, transparency, and ethical AI use.
Executive Summary
AI governance ensures that AI systems are developed and deployed responsibly, ethically, and in compliance with relevant laws and policies. This framework provides a comprehensive approach to AI governance aligned with Australian Government requirements.
Governance Principles
Core Principles
| Principle | Description | Application |
| Accountability | Clear ownership and responsibility | Every AI system has a designated owner |
| Transparency | Openness about AI use and decisions | Publish AI use register, explain decisions |
| Fairness | Equitable treatment, no discrimination | Bias testing, fairness monitoring |
| Privacy | Protection of personal information | Privacy by design, minimal data use |
| Safety | Prevent harm, ensure reliability | Testing, monitoring, human oversight |
| Human Agency | Appropriate human control | Human-in-the-loop for high-stakes |
| Contestability | Ability to challenge AI decisions | Appeals processes, explanations |
Alignment with Australian AI Ethics Principles
| Government Principle | Framework Implementation |
| Human, societal and environmental wellbeing | Impact assessments, monitoring |
| Human-centred values | User research, co-design |
| Fairness | Bias testing, fairness metrics |
| Privacy protection and security | PIA, security assessment |
| Reliability and safety | Testing, monitoring, fallbacks |
| Transparency and explainability | Model cards, explanations |
| Contestability | Appeals process |
| Accountability | Governance structure |
Governance Structure
Three Lines Model
flowchart LR
subgraph L1["<strong>FIRST LINE</strong><br/>(Operational)"]
direction TB
L1A[Project Teams]
L1B[Business Units]
L1C[IT Operations]
end
subgraph L2["<strong>SECOND LINE</strong><br/>(Oversight)"]
direction TB
L2A[AI Governance Office]
L2B[Risk Management]
L2C[Legal/Privacy]
end
subgraph L3["<strong>THIRD LINE</strong><br/>(Assurance)"]
direction TB
L3A[Internal Audit]
L3B[External Audit]
end
L1 --> L2 --> L3
style L1 fill:#e3f2fd,stroke:#1976d2,stroke-width:2px
style L2 fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style L3 fill:#e8f5e9,stroke:#388e3c,stroke-width:2px
- First Line: Owns and manages AI systems
- Second Line: Provides oversight and guidance
- Third Line: Independent assurance
Roles and Responsibilities
| Role | Responsibilities |
| Executive Sponsor | Strategic direction, resource allocation, escalation |
| AI Governance Board | Policy approval, high-risk decisions, standards |
| Chief Data Officer | Data governance, data quality, data ethics |
| AI Ethics Lead | Ethics review, bias assessment, fairness |
| Privacy Officer | PIAs, privacy compliance, data protection |
| Security Officer | Security assessments, AI-specific security |
| Legal Counsel | Legal review, contracts, liability |
| Project Owner | Day-to-day accountability for AI system |
| Technical Lead | Model development, monitoring, maintenance |
AI Governance Board
Composition: - Chair: Deputy Secretary or equivalent - Chief Data Officer - Chief Information Officer - Chief Risk Officer - Privacy Officer - Legal Counsel - Business Unit Representatives - External Expert (optional)
Responsibilities: - Approve AI strategy and policies - Review high-risk AI proposals - Oversee AI portfolio - Address escalations - Report to executive
Meeting Frequency: Monthly or as needed
AI Classification System
Risk-Based Tiering
| Tier | Risk Level | Characteristics | Governance Requirements |
| Tier 1 | Low | Internal tools, no citizen impact, reversible | Standard review, annual audit |
| Tier 2 | Medium | Citizen-facing, recommendations only, human review | Ethics review, quarterly monitoring |
| Tier 3 | High | Affects rights/benefits, automated decisions | Full assessment, ongoing monitoring |
| Tier 4 | Critical | Safety-critical, law enforcement, essential services | Board approval, external audit |
Classification Criteria
| Factor | Low | Medium | High | Critical |
| Impact on individuals | Minimal | Moderate | Significant | Severe |
| Automation level | Assistive | Advisory | Delegated | Autonomous |
| Reversibility | Easily reversed | Reversible | Difficult | Irreversible |
| Scale | Small group | Department | Agency | Public |
| Sensitivity | Public data | Internal | Personal | Sensitive |
Classification Process
flowchart TB
START([Start Classification]) --> Q1{Does the AI make or materially<br/>influence decisions about people?}
Q1 -->|No| T1[<strong>Tier 1</strong><br/>Low Risk]
Q1 -->|Yes| Q2{Are decisions automated<br/>without human review?}
Q2 -->|No| T2[<strong>Tier 2</strong><br/>Medium Risk]
Q2 -->|Yes| Q3{Do decisions affect legal<br/>rights, benefits, or safety?}
Q3 -->|No| T3[<strong>Tier 3</strong><br/>High Risk]
Q3 -->|Yes| T4[<strong>Tier 4</strong><br/>Critical Risk]
style T1 fill:#c8e6c9,stroke:#388e3c,stroke-width:2px
style T2 fill:#fff9c4,stroke:#f9a825,stroke-width:2px
style T3 fill:#ffcc80,stroke:#ef6c00,stroke-width:2px
style T4 fill:#ef9a9a,stroke:#c62828,stroke-width:2px
style START fill:#e3f2fd,stroke:#1976d2,stroke-width:2px
Governance Processes
AI Project Lifecycle Governance
flowchart LR
subgraph I["<strong>IDEATE</strong>"]
I1[Use case proposal]
I2[Business case]
end
subgraph A["<strong>ASSESS</strong>"]
A1[Risk assessment]
A2[Privacy impact]
end
subgraph DEV["<strong>DEVELOP</strong>"]
D1[Ethics review]
D2[Security assessment]
end
subgraph DEP["<strong>DEPLOY</strong>"]
DP1[Go-live approval]
DP2[Readiness check]
end
subgraph O["<strong>OPERATE</strong>"]
O1[Monitoring review]
O2[Incident response]
end
subgraph R["<strong>RETIRE</strong>"]
R1[Retirement review]
R2[Data disposal]
end
I --> A --> DEV --> DEP --> O --> R
style I fill:#e3f2fd,stroke:#1976d2,stroke-width:2px
style A fill:#e8f5e9,stroke:#388e3c,stroke-width:2px
style DEV fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style DEP fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px
style O fill:#e0f2f1,stroke:#00796b,stroke-width:2px
style R fill:#fce4ec,stroke:#c2185b,stroke-width:2px
Stage Gate Requirements by Tier
| Stage Gate | Tier 1 | Tier 2 | Tier 3 | Tier 4 |
| Ideation | Manager approval | Director approval | SES approval | Board approval |
| Assessment | Self-assessment | Privacy screening | Full PIA | Full PIA + legal |
| Development | Standard review | Ethics checklist | Ethics review | Ethics board |
| Deployment | Tech lead sign-off | Business sign-off | Sponsor sign-off | Board sign-off |
| Operations | Annual review | Quarterly review | Monthly review | Continuous |
Ethics Review Process
flowchart TB
subgraph S1["<strong>1. SUBMISSION</strong>"]
S1A[Project team completes<br/>Ethics Assessment Form]
S1B[Use case, data, model,<br/>impact analysis]
end
subgraph S2["<strong>2. INITIAL SCREENING</strong><br/><em>3 days</em>"]
S2A[Ethics Lead reviews<br/>for completeness]
S2B[Determines review<br/>level required]
end
subgraph S3["<strong>3. REVIEW</strong>"]
S3A[Tier 1-2: Ethics Lead<br/><em>5 days</em>]
S3B[Tier 3: Ethics Panel<br/><em>10 days</em>]
S3C[Tier 4: Full Board<br/><em>15 days</em>]
end
subgraph S4["<strong>4. DECISION</strong>"]
S4A[Approved]
S4B[Approved with conditions]
S4C[Rejected]
end
subgraph S5["<strong>5. FOLLOW-UP</strong>"]
S5A[Conditions verified<br/>before deployment]
S5B[Post-deployment<br/>ethics check]
end
S1 --> S2 --> S3 --> S4 --> S5
style S1 fill:#e3f2fd,stroke:#1976d2,stroke-width:2px
style S2 fill:#e8f5e9,stroke:#388e3c,stroke-width:2px
style S3 fill:#fff3e0,stroke:#f57c00,stroke-width:2px
style S4 fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px
style S5 fill:#e0f2f1,stroke:#00796b,stroke-width:2px
Policies and Standards
Required Policies
| Policy | Purpose | Owner |
| AI Strategy | Strategic direction for AI | CDO |
| AI Ethics Policy | Ethical principles and requirements | Ethics Lead |
| AI Risk Policy | Risk classification and management | Risk Manager |
| Data Governance Policy | Data quality, access, privacy | CDO |
| Model Management Policy | Model lifecycle, versioning | Tech Lead |
| AI Security Policy | Security requirements for AI | CISO |
| AI Procurement Policy | Vendor assessment, contracts | Procurement |
Technical Standards
| Standard | Requirement |
| Model Documentation | All models require model cards |
| Testing | Bias testing mandatory for Tier 2+ |
| Explainability | Explanations required for decisions |
| Monitoring | Production monitoring for all deployed models |
| Audit Trails | Logging of all predictions and decisions |
| Version Control | Models versioned with full lineage |
AI Register
Every AI system must be registered with:
| Field | Description |
| System Name | Unique identifier |
| Owner | Accountable person |
| Purpose | What the AI does |
| Classification | Risk tier |
| Data Sources | What data is used |
| Users | Who uses the system |
| Review Date | Next scheduled review |
| Status | Active/Pilot/Retired |
Public Transparency
Consider publishing: - List of AI systems in use - Purpose of each system - How decisions can be contested - Contact for enquiries
Monitoring and Assurance
Continuous Monitoring Requirements
| Tier | Performance | Fairness | Drift | Compliance |
| 1 | Annual | Annual | Annual | Annual |
| 2 | Quarterly | Quarterly | Quarterly | Quarterly |
| 3 | Monthly | Monthly | Monthly | Monthly |
| 4 | Weekly | Weekly | Weekly | Weekly |
Internal Audit Program
| Audit Type | Frequency | Scope |
| AI Portfolio Audit | Annual | All systems |
| High-Risk System Audit | 6 months | Tier 3-4 systems |
| Ethics Audit | Annual | Sample of systems |
| Security Audit | Annual | All systems |
| Compliance Audit | As required | Specific systems |
| KPI | Target | Measurement |
| AI systems registered | 100% | Register completeness |
| Ethics reviews completed | 100% Tier 2+ | Review records |
| Fairness testing passed | 100% | Test results |
| Incidents resolved | <5 days average | Incident log |
| Model cards current | 100% | Documentation audit |
Incident Management
AI Incident Classification
| Severity | Description | Response Time | Escalation |
| P1 Critical | Safety impact, major harm | 1 hour | Executive |
| P2 High | Significant fairness breach, data breach | 4 hours | Governance Board |
| P3 Medium | Performance degradation, minor bias | 24 hours | AI Governance Office |
| P4 Low | Minor issues, documentation gaps | 5 days | Project Owner |
Incident Response Phases
- Detection: Alert or report received
- Triage: Classify severity, assign owner
- Containment: Stop ongoing harm
- Investigation: Root cause analysis
- Resolution: Fix issue
- Review: Post-incident review, lessons learned
Training and Capability
Training Requirements by Role
| Role | Required Training | Frequency |
| All staff | AI awareness | One-time |
| AI users | Responsible AI use | Annual |
| Project teams | AI governance process | Per project |
| Technical staff | Ethics and bias | Annual |
| Leadership | AI strategy and risk | Annual |
Capability Building
- AI ethics certification program
- Technical AI training
- Governance process workshops
- Lessons learned sharing
Implementation Roadmap
Phase 1: Foundation (0-3 months)
Phase 2: Operationalize (3-6 months)
Phase 3: Mature (6-12 months)
| Resource | Purpose | Link |
| AI Use Case Proposal | Initial proposal submission | Template |
| Ethics Assessment Form | Ethics review submission | Guide |
| Model Card Template | Model documentation | Template |
| Risk Assessment | AI risk evaluation | Template |
| Incident Report | Incident documentation | Template |
Review and Updates
| Activity | Frequency | Owner |
| Framework review | Annual | AI Governance Board |
| Policy updates | As needed | Policy owners |
| Process improvement | Quarterly | AI Governance Office |
| Metrics review | Quarterly | AI Governance Office |