Incident Response Playbooks

Incident response playbooks automate your team's response to security events and quality issues. When a guardrail violation, anomaly detection, drift alert, evaluation failure, or red team finding occurs, playbooks execute predefined actions to contain the issue and notify the right people.

Overview

Manual incident response is slow and error-prone, especially during off-hours. Playbooks codify your response procedures into automated actions that execute immediately when triggered. This reduces response time from minutes or hours to seconds.

Each playbook consists of a trigger condition and one or more actions. When the trigger fires, all configured actions execute in sequence.

Playbook Actions

Playbooks can execute the following automated actions:

Playbooks trigger a response workflow — a regular NodeLoom workflow that receives the incident event data as input. This gives you full flexibility: send Slack alerts via MCP, call webhooks, run remediation logic, or chain multiple actions together.

Example Response	How to Build It
Slack Alert via MCP	Use the built-in "Slack Incident Notification (MCP)" template. It formats the event into a rich Slack message with severity, action taken, and links to the workflow editor and execution logs.
Webhook Notification	Use an HTTP Request node to POST the event data to PagerDuty, Opsgenie, or any webhook endpoint.
Email Alert	Use a Gmail or SMTP node to send formatted incident emails to your on-call team.
Custom Logic	Combine Code nodes, conditionals, and AI nodes to build sophisticated response workflows (e.g. auto-investigate, generate incident reports, create Jira tickets).

Creating a Playbook

To create an incident response playbook:

Navigate to Settings → Playbooks.
Click Create Playbook and give it a descriptive name.
Select an event source (trigger type) from the dropdown.
Optionally filter by event sub-types (e.g. only PROMPT_INJECTION guardrail violations, or only DURATION_SPIKE anomalies).
Set the minimum severity threshold (LOW, MEDIUM, HIGH, or CRITICAL).
Optionally scope to a source workflow or leave blank for all workflows.
Select a response workflow to execute when the playbook triggers. Create one from a template or build your own.
Set a cooldown period (minutes) to prevent duplicate alerts for repeated events.
Save and enable the playbook.

Trigger Types

Playbooks support the following trigger types:

Event Source	Fires When	Sub-Type Examples
Guardrail Violation	A guardrail detects a violation (prompt injection, PII, toxicity, jailbreak, etc.). Content is blocked automatically.	PROMPT_INJECTION, PII_DETECTED, TOXICITY, JAILBREAK, CUSTOM_PATTERN
Anomaly Detection	Unusual patterns detected in workflow execution (error spikes, latency anomalies, data exfiltration).	DURATION_SPIKE, ERROR_RATE, DATA_EXFILTRATION, TOKEN_ANOMALY
Drift Alert	Model output drift detected — AI responses have changed significantly from baseline behavior.	OUTPUT_DRIFT, SCORE_DRIFT, LATENCY_DRIFT
Eval Failure	An LLM-as-Judge evaluation scores below the passing threshold.
Red Team Finding	A red team scan discovers a vulnerability.	PROMPT_INJECTION, JAILBREAK, PII_EXTRACTION, HALLUCINATION
Sentiment Alert	User sentiment drops below threshold — negative spike, score drop, or high volume of negative feedback.
Token Budget Exceeded	A workflow or model exceeds its configured token budget.
Rate Limit Violation	A request exceeds the configured rate limit. The request is automatically rate-limited.

Event Data

Every playbook dispatch includes standardized event data that the response workflow receives as input:

Field	Description
`eventType`	The event type (e.g. GUARDRAIL_VIOLATION, ANOMALY_DETECTION).
`severity`	Event severity: LOW, MEDIUM, HIGH, or CRITICAL.
`blocked`	Whether an action was taken to stop the event (true/false).
`actionTaken`	Description of the action taken (e.g. "Content blocked by guardrail").
`workflowId`	UUID of the affected workflow (if applicable).
`sourceWorkflowName`	Human-readable name of the affected workflow.
`timestamp`	ISO 8601 timestamp of when the event occurred.
`message / details`	Event-specific description or details.

Slack Notification Template

NodeLoom includes a built-in Slack Incident Notification (MCP) template that sends rich alerts to Slack when playbooks fire. To set it up:

Create a Slack app at api.slack.com/apps with chat:write and channels:read scopes.
Go to Settings → Credentials and create an MCP Server (Stdio) credential with your tokens:{"SLACK_BOT_TOKEN": "xoxb-...", "SLACK_TEAM_ID": "T..."}
Create a new workflow from the Slack Incident Notification (MCP) template.
In the MCP Tool node, select the credential you created.
In the Code node, replace REPLACE_WITH_CHANNEL_ID with your Slack channel ID and REPLACE_WITH_APP_URL with your NodeLoom URL.
Create a playbook pointing to this workflow as the response workflow.

The template formats alerts with event type, severity, action taken (blocked or not), and direct links to the workflow editor and execution logs.

Works with any MCP server

The stdio transport supports any MCP server, not just Slack. You can build response workflows using GitHub, Google Drive, or custom MCP servers the same way.

Execution History

Every playbook execution is logged with full details. View past playbook runs from the Monitoring > Playbooks section to see:

Which trigger caused the playbook to fire, including the originating workflow and execution ID.
The actions that were executed and their results (success or failure).
Timestamps for each step, showing the total response time from trigger to completion.
Any errors encountered during action execution.

Test your playbooks

Use the Test Run button to simulate a playbook trigger without waiting for a real incident. This helps you verify that notifications reach the right people and actions execute as expected.

Batch Evaluation

Overview