<< All versions
Skill v1.0.0
currentTrusted Publisher100/100microsoft/pilotswarm/incident-response
──Details
PublishedOctober 19, 2026 at 06:42 AM
Content Hashsha256:8f1533af011b9e80...
Git SHA
──Files
Files (1 file, 1.8 KB)
SKILL.md1.8 KBactive
SKILL.md · 40 lines · 1.8 KB
version: "1.0.0" name: incident-response description: Domain knowledge for investigating production incidents — triage steps, correlation patterns, and escalation criteria.
Incident Response
Triage Checklist
When investigating an incident, follow this order:
- Identify the blast radius — which services are affected?
- Check health endpoints — are services responding?
- Review error rates — sudden spikes indicate the incident start time
- Check CPU/memory — resource exhaustion is a common root cause
- Search logs — look for exceptions, timeouts, connection errors
Common Root Causes
| Symptom | Likely Cause | Investigation Steps | |
|---|---|---|---|
| High error rate + normal CPU | Upstream dependency failure | Check downstream service health, look for timeout errors in logs | |
| High CPU + high error rate | Resource exhaustion | Check for memory leaks, runaway queries, or traffic spikes | |
| Intermittent errors | Network issues or partial deploy | Check if errors correlate with specific pods, look for connection reset logs | |
| Cascading failures across services | Shared dependency down | Identify the common downstream service, check database/cache health |
Correlation Patterns
- Time correlation: If service A errors spike 30s before service B, A likely caused B's issues
- Error type correlation: Same exception across services = shared dependency
- Traffic correlation: Error rate proportional to traffic = capacity issue
Escalation Criteria
- Error rate > 10% for 5+ minutes → P1 (page on-call)
- Service completely down → P0 (all-hands)
- Data integrity issues → P0 (immediate escalation)
- Performance degraded but functional → P2 (next business day)