Incident Response Runbook

Incident response plan for [SYSTEM/SERVICE]:

**Severity Levels**:
- **P0 (Critical)**: Total outage, data breach
- **P1 (High)**: Major feature down, perf degradation
- **P2 (Medium)**: Minor feature issue
- **P3 (Low)**: Cosmetic, no user impact

**Response Timeline**:
- P0: Immediate response, 15min updates
- P1: 30min response, hourly updates
- P2: 4hr response, daily updates
- P3: Best effort

**Incident Response Steps**:

**1. Detection & Alerting** (0-5min)
- Monitor alerts trigger
- On-call engineer paged
- Acknowledge alert

**2. Assessment** (5-15min)
- Determine severity
- Identify affected systems/users
- Assemble incident team
- Create incident channel (#incident-123)

**3. Communication** (15-30min)
- Post status page update
- Notify stakeholders
- Start incident log (timeline)

**4. Mitigation** (Ongoing)
- Implement temporary fix
- Monitor metrics
- Update stakeholders regularly

**5. Resolution**
- Deploy permanent fix
- Verify resolution
- Monitor for regression
- Close incident

**6. Post-Mortem** (Within 48hr)
- Timeline of events
- Root cause analysis
- Impact assessment
- Action items (with owners)
- Lessons learned
- Preventive measures

**Communication Templates**:
- Initial: "We're investigating [issue]."
- Update: "Still working. ETA: [time]. Impact: [scope]."
- Resolved: "Issue resolved at [time]. RCA to follow."

**Tools**:
- Monitoring: [Datadog/New Relic]
- On-call: [PagerDuty/Opsgenie]
- Status: [StatusPage]
- Comms: [Slack/Teams]