AI Incident Response Plan¶

Template Download Template

Document Control¶

1. Scope and Objectives¶

1.1 Scope¶

This plan covers incidents involving: - [ ] AI/ML model failures - [ ] Biased or unfair outcomes - [ ] Data quality issues affecting AI - [ ] AI security breaches - [ ] Privacy violations from AI - [ ] AI-generated misinformation - [ ] Adversarial attacks on AI - [ ] Unintended harmful AI behaviors

AI Systems Covered:

1.2 Objectives¶

1. Detect - Rapidly identify AI incidents
2. Contain - Limit impact and prevent escalation
3. Communicate - Notify stakeholders appropriately
4. Remediate - Fix the root cause
5. Recover - Restore normal operations
6. Learn - Improve systems and processes

2. AI Incident Classification¶

2.1 Incident Categories¶

2.2 Severity Levels¶

2.3 Severity Assessment Matrix¶

3. Incident Response Team¶

3.1 Core Team Roles¶

3.2 Extended Team (As Needed)¶

3.3 RACI Matrix¶

4. Detection and Reporting¶

4.1 Detection Sources¶

4.2 Reporting Procedure¶

Anyone identifying a potential AI incident should:

1. STOP - If safe, stop the AI system from causing further harm
2. DOCUMENT - Note what happened, when, and what was affected
3. REPORT - Use the incident reporting form or contact:
4. Phone: [Number]
5. Email: [Email]
6. Portal: [Link]

4.3 Initial Report Information¶

5. Response Procedures¶

5.1 Response Workflow¶

flowchart LR
    D[Detection] --> T[Triage] --> DC[Declaration] --> C[Containment] --> I[Investigation]
    I --> RC[Root Cause]
    RC --> R[Remediation] --> REC[Recovery] --> CL[Closure] --> L[Learning]

    style D fill:#e3f2fd,stroke:#1976d2,stroke-width:2px
    style C fill:#ffcc80,stroke:#ef6c00,stroke-width:2px
    style RC fill:#ef9a9a,stroke:#c62828,stroke-width:2px
    style L fill:#c8e6c9,stroke:#388e3c,stroke-width:2px

5.2 Phase 1: Triage (First 15 minutes)¶

Objectives: Assess severity, activate response team

Actions: - [ ] Review incident report - [ ] Verify incident is genuine (not false positive) - [ ] Assess initial severity level - [ ] Determine incident category - [ ] Identify affected systems and users - [ ] Notify Incident Commander - [ ] Document triage findings

Decision Point: Declare incident and severity level

5.3 Phase 2: Containment (Severity-dependent)¶

Objectives: Stop harm, prevent escalation

Containment Options:

Containment Actions: - [ ] Select containment strategy - [ ] Implement containment measures - [ ] Verify containment effectiveness - [ ] Document containment actions - [ ] Communicate containment status

5.4 Phase 3: Investigation¶

Objectives: Understand what happened and why

Investigation Activities: - [ ] Collect and preserve evidence - [ ] Review system logs - [ ] Analyze model behavior - [ ] Review recent changes - [ ] Interview relevant staff - [ ] Examine data inputs - [ ] Check for external factors

Evidence Collection:

5.5 Phase 4: Root Cause Analysis¶

Objectives: Identify underlying cause

Root Cause Categories for AI:

5 Whys Analysis:

5.6 Phase 5: Remediation¶

Objectives: Fix the root cause

Remediation Options:

Remediation Planning: - [ ] Define remediation actions - [ ] Assign owners and deadlines - [ ] Assess remediation risks - [ ] Plan testing and validation - [ ] Document remediation plan

5.7 Phase 6: Recovery¶

Objectives: Restore normal operations

Recovery Steps: - [ ] Verify remediation complete - [ ] Test fixed system - [ ] Plan phased restoration - [ ] Monitor closely during recovery - [ ] Confirm normal operations - [ ] Update stakeholders

Restoration Sequence:

5.8 Phase 7: Closure¶

Objectives: Formally close the incident

Closure Checklist: - [ ] All remediation actions complete - [ ] System operating normally - [ ] Stakeholders informed - [ ] Documentation complete - [ ] Lessons learned captured - [ ] Incident report finalized - [ ] Closure approved by Incident Commander

6. Communication Protocols¶

6.1 Internal Communication¶

Communication Timeline:

Communication Channels:

6.2 External Communication¶

External Notification Matrix:

6.3 Communication Templates¶

Initial Notification (Internal):

Status Update:

7. Specific Incident Playbooks¶

7.1 Bias/Fairness Incident¶

Indicators: - Disparate outcomes across demographic groups - User complaints about unfair treatment - Bias monitoring alerts - Audit findings

Response Actions: 1. Immediately add human review to affected decisions 2. Preserve model and data for analysis 3. Engage Ethics Lead 4. Analyze outcomes by protected attributes 5. Quantify scope and impact 6. Consider whether affected decisions need review 7. Plan remediation (retraining, adjustment, replacement) 8. Notify affected individuals if significant harm

Escalation Triggers: - Multiple protected groups affected - Decisions involved significant consequences (benefits, enforcement) - Media or political interest

7.2 Data Breach Involving AI¶

Indicators: - Unauthorized access to training data - Model inversion attack detected - PII exposed in model outputs - Data exfiltration alerts

Response Actions: 1. Isolate affected systems 2. Engage Security Team and Privacy Lead 3. Assess data exposed (type, volume, sensitivity) 4. Determine if notifiable data breach 5. Preserve evidence for investigation 6. Notify OAIC if required (within 30 days) 7. Notify affected individuals 8. Implement additional security controls

Notifiable Data Breach Assessment: | Question | Answer | |----------|--------| | Personal information involved? | Yes/No | | Unauthorized access or disclosure? | Yes/No | | Serious harm likely? | Yes/No | | Can remedial action prevent harm? | Yes/No | | Notification required? | Yes/No |

7.3 Model Performance Degradation¶

Indicators: - Accuracy metrics below threshold - Increased prediction errors - User complaints about quality - Business outcome deterioration

Response Actions: 1. Assess current performance vs baseline 2. Check for data drift or quality issues 3. Review recent deployments or changes 4. Consider increasing human review 5. Evaluate fallback to previous model version 6. Plan model retraining if needed 7. Restore normal thresholds before full operation

7.4 Adversarial Attack¶

Indicators: - Unusual input patterns - Attempts to probe model behavior - Model extraction attempts - Poisoned data detected

Response Actions: 1. Engage Security Team immediately 2. Block suspicious sources if identifiable 3. Preserve attack evidence 4. Assess model compromise 5. Consider model replacement 6. Implement additional defenses 7. Report to Australian Cyber Security Centre if significant

7.5 AI Safety Incident¶

Indicators: - AI recommendation could cause harm - Dangerous content generated - Safety guardrails bypassed - Unintended real-world consequences

Response Actions: 1. IMMEDIATE: Disable AI system 2. Prevent further harmful actions 3. Engage executive sponsor 4. Assess actual harm caused 5. Support affected individuals 6. Conduct thorough safety review before restart 7. Implement enhanced safeguards

8. Documentation Requirements¶

8.1 Incident Documentation¶

Incident Record Template:

8.2 Timeline Log¶

8.3 Post-Incident Report¶

Required Sections: 1. Executive Summary 2. Incident Timeline 3. Impact Assessment 4. Root Cause Analysis 5. Response Evaluation 6. Remediation Actions 7. Lessons Learned 8. Recommendations 9. Appendices (evidence, logs)

9. Post-Incident Review¶

9.1 Review Process¶

Timing: - Severity 1-2: Within 5 business days - Severity 3-4: Within 10 business days

Participants: - Incident Response Team - System owners - Subject matter experts - Executive sponsor (Severity 1-2)

9.2 Review Agenda¶

1. Incident timeline review
2. Response effectiveness assessment
3. What went well
4. What could be improved
5. Root cause confirmation
6. Remediation status
7. Recommendations development
8. Action item assignment

9.3 Improvement Actions¶

10. Testing and Maintenance¶

10.1 Testing Schedule¶

10.2 Plan Maintenance¶

11. Training Requirements¶

11.1 Training Matrix¶

11.2 Exercise Participation¶

12. Metrics and Reporting¶

12.1 Incident Metrics¶

12.2 Reporting¶

13. Appendices¶

Appendix A: Contact List¶

Appendix B: Escalation Flowchart¶

flowchart TB
    DET[Incident Detected] --> TRI[Initial Triage]
    TRI --> S12[Severity 1-2]
    TRI --> S3[Severity 3]
    TRI --> S4[Severity 4]

    S12 --> EXEC[Exec + Full<br/>Response Team]
    S3 --> STD[Standard<br/>Response]
    S4 --> NRM[Normal<br/>Process]

    EXEC --> IMM[Immediate<br/>Containment]
    STD --> HR4[4-hour<br/>Response]
    NRM --> HR24[24-hour<br/>Response]

    style DET fill:#e3f2fd,stroke:#1976d2,stroke-width:2px
    style S12 fill:#ef9a9a,stroke:#c62828,stroke-width:2px
    style S3 fill:#ffcc80,stroke:#ef6c00,stroke-width:2px
    style S4 fill:#fff9c4,stroke:#f9a825,stroke-width:2px
    style IMM fill:#ef9a9a,stroke:#c62828,stroke-width:2px

Appendix C: Quick Reference Card¶

AI INCIDENT QUICK REFERENCE

Key Contacts: - Emergency: [Number] - Incident line: [Number] - After hours: [Number]

Appendix D: Glossary¶

14. Sign-Off¶