SOAR Playbook Design: Reactive Data Flow, Conditional Branching, and the Case for Dry-Run

TL;DR. A SOAR playbook is supposed to be the codification of what your best analyst does at 3 AM. In practice, most playbooks drift into one of two failure modes: they are too rigid to handle the variation real incidents produce, or they are so generic that they automate nothing of consequence. Four design choices, namely reactive data flow, a typed node contract, first-class conditional branching, and a non-destructive dry-run mode, separate playbooks that get used from playbooks that get rewritten every quarter.

The pitch for security orchestration is straightforward: codify the runbook so it fires automatically on an alert, let the platform do the mechanical work, and give the analyst room to think. Reality is messier. Real incidents do not match the runbook, because real incidents vary. The same phishing campaign reaches three users with slightly different mail headers; half the URLs are already neutralized by the time triage starts; the EDR has flagged two of the endpoints but not the third; the user in finance has approver privileges that change the containment calculus.

A playbook that cannot absorb that variation is really just a script, and a script that cannot be safely modified becomes a liability the moment it starts touching production systems. That is why so many SOAR projects quietly lose the team's trust after the first year.

This post walks through the four design choices that decide whether a playbook holds up under real traffic or ends up bypassed. They come out of building Krasper Suite's playbook designer. The treatment is technical and the examples are deliberately concrete.

Contents

1. Why most playbooks decay
2. Reactive data flow: why the editor matters
3. The node contract: typed ports, explicit failure modes
4. Conditional branching as a first-class primitive
5. A worked example: phishing response, end to end
6. Dry-run mechanics: testing without the blast radius
7. What changes when you have all four

1. Why most playbooks decay

The decay pattern is consistent across organizations. A playbook is authored against a clean mental model of the incident. It runs well in the demo, well in the first week, well on the incidents that look like the one the author had in mind.

Then the variations start arriving.

The mail header parser was written for the cloud mail gateway and breaks on the on-prem appliance. The IP-to-asset enrichment returns unknown for cloud workloads. The containment step assumes endpoint isolation works, which it does, until the device is offline. The notification step assumes the channel responder is on call, which they are not at 3 AM Saturday.

Each variation produces an exception. Exceptions accumulate as bolted-on conditional logic, deeply nested, increasingly unreviewable. After six months the playbook is a YAML file 2,000 lines long that one engineer understands and is reluctant to modify. The team starts working around it.

The root cause is rarely the logic itself. It is the representation. A playbook represented as a linear script with growing conditionals is a representation that fights its own evolution. A playbook represented as a typed directed graph with reactive data flow does not.

2. Reactive data flow: why the editor matters

The visual editor is often dismissed as a UX nicety. It is not. It is a forcing function for a specific data-flow model that linear scripts do not enforce.

In a reactive node-graph editor, every node declares its inputs and outputs as named, typed ports. Connections between nodes are explicit. The runtime walks the graph, evaluates a node when all its inputs are satisfied, propagates the outputs to downstream nodes. There is no hidden state; the graph is the program.

What this representation gives you that a script does not:

• The graph is the contract. A reviewer can read it, and a junior engineer can change one node without having to understand the whole.
• Side effects stay localized. A node that touches production is a single, identifiable shape on the canvas; the nodes around it are pure transformations.
• Diffing becomes structural. A change to the graph is a change to a defined set of nodes and edges, not a churn of shifting indentation in a thousand-line script, so pull requests against playbooks are actually reviewable.

The reactive part matters because the editor can show the current state of the data flowing through each port as the author builds the playbook against a sample alert. The author sees, in real time, what shape asset will have when it reaches the classifier, not what shape the documentation claims it will have. Most playbook bugs come from drift between documented and actual data shapes; reactive feedback collapses that gap.

3. The node contract: typed ports, explicit failure modes

The graph metaphor only buys you anything if the nodes inside it follow a tight contract. Our node contract has four required properties.

First, inputs and outputs are typed. Every port has a declared schema, and the runtime refuses to connect a port that outputs IpAddress to one that expects EmailMessage. That sounds obvious, but half of all playbook failures we have seen in audits trace back to runtime type mismatches the editor could have caught at design time.

Second, failure outputs are explicit. Every node that touches an external system has a success port and an error port, and the error port is mandatory: leave it unwired and the editor flags the node as incomplete. Of all the choices here, this one has prevented the most production incidents, because it forces the author to confront the unhappy path at design time rather than after the first 2 AM page.

Third, nodes are idempotent by default. Side-effect nodes carry an auto-generated idempotency key derived from the playbook execution ID and the node ID, so replaying the same execution against the same node deduplicates at the runtime level. Authors never have to think about it; the platform guarantees it.

Fourth, timeouts are deterministic. Every external call declares a timeout, and a node that exceeds it routes to the error port instead of hanging the execution. There is no "stuck waiting on the EDR API forever" state, because the runtime kills the node before it can happen.

A node that satisfies all four properties composes cleanly. One that satisfies none of them is exactly how playbooks decay.

4. Conditional branching as a first-class primitive

Most early SOAR products treated conditionals as an afterthought, a node with an "if" expression on it, output routed to a single downstream node. That works for one or two branches. It collapses at scale.

A first-class branching primitive looks like this:

• A switch node with N declared outputs, each guarded by a typed predicate over the inputs.
• The runtime evaluates predicates in declared order, routes to the first match, and short-circuits the rest.
• A required default output catches anything that matches no predicate. The editor refuses to save the playbook without it.
• The output ports of the switch carry narrowed types: downstream of the "alert is from cloud mail gateway" branch, the alert port type narrows to CloudMailAlert, and the downstream nodes only see the fields they can rely on.

Type narrowing is the part that distinguishes a real branching primitive from a cosmetic one. With it, each subgraph can be authored against a tighter contract and the editor catches integration mistakes at design time. Without it, every subgraph has to defensively re-check every field, which is exactly the failure mode that bloats playbooks into unreadable conditionals.

The default branch is not optional for a reason. Unhandled cases in production at 3 AM produce no value. Either the playbook decides what to do, or it explicitly escalates to a human. Both are acceptable; silent fall-through is not.

5. A worked example: phishing response, end to end

Phishing is the standard example because every SOC runs it and the variation surface is large. The skeleton of a real playbook looks like this:

A few things to notice about this shape.

The enrichment step is one node, not three; the platform handles parallel fan-out internally. The author does not write concurrency code; they declare that the classifier needs sender_reputation, domain_age, and url_reputation, and the runtime resolves the dependencies in parallel where it can.

The "for each recipient" loop is a single iterator node with a subgraph attached. The subgraph is itself reviewable, diffable, testable in isolation. The author does not write loops; they declare "this subgraph runs once per recipient."

The privileged-user check is a switch with two outcomes, both explicitly handled. There is no implicit fall-through where a privileged user accidentally gets the standard notification path.

The endpoint isolation step has an error port (not shown above) that routes to an analyst alert if isolation fails, for instance because the endpoint is offline. The playbook never silently assumes its containment action succeeded.

6. Dry-run mechanics: testing without the blast radius

The single feature that determines whether engineers actually iterate on playbooks is dry-run.

A dry-run executes the entire graph against a real or sampled alert, walks every node, evaluates every predicate, propagates every value, but does not call the side-effect APIs. Nodes that would quarantine mail, isolate endpoints, force password resets, or open tickets instead emit a record of what they would have done and pass through the synthetic success output.

The implementation pattern is straightforward: every side-effect adapter checks an execution_mode flag at entry. In live mode it calls the upstream API. In dry_run mode it returns a deterministic synthetic response that matches the schema of a real success, and records the intended call into the execution trace.

The benefits compound:

• Authors can run a playbook against last week's actual phishing alert and see the full execution trace before pushing the change.
• CI can replay a corpus of historical alerts against the new playbook on every change, and fail the build if any execution diverges from the previous version in unexpected ways.
• Incident retros can replay the exact alert against the current playbook to confirm the fix lands.

There is one design discipline required: dry-run integrity is a platform guarantee, not a per-node one. Every side-effect adapter that ships must honor the mode flag, and the runtime enforces it at the adapter boundary, not at the author's discretion. A single adapter that ignores dry-run and quietly calls production turns the entire mode into a liability. Adapter test suites verify both modes on every release.

7. What changes when you have all four

Reactive data flow, the typed node contract, first-class branching, and dry-run each help on their own, but together they change the picture. The shift shows up in two places you can actually measure.

The first is mean time to respond. When the playbook handles variation natively, the manual handoff steps that used to gate execution simply disappear. For a phishing playbook shaped like the one above, what we typically see across operating environments is that the automatable portion of triage drops from tens of minutes when a human drives it to seconds when it runs end to end, because the analyst no longer has to make the early classification call by hand. The human time that remains lands on the genuinely ambiguous cases the playbook escalates on purpose, which is where analyst attention belongs anyway.

The second is mean time to change, the metric most teams forget to measure at all. A playbook that takes a week to modify safely is a playbook the team quietly avoids modifying. With dry-run and structural diffs, that loop usually shrinks from days to under an hour: edit the graph, dry-run it against a corpus, review the structural diff, ship. The consequence is that the playbook keeps up with how attacks are actually evolving instead of drifting further behind them.

The numbers depend heavily on the baseline and on how well the surrounding stack is integrated. Anyone quoting a single before/after multiple is selling a slide, not measuring an outcome. Across the environments we have observed, the direction holds steady even though the size of the effect does not.

Closing

A SOAR product lives or dies on its authoring substrate, not on the playbooks it happens to ship with: the editor, the node contract, the branching primitives, and the dry-run guarantee are what actually matter. The playbooks are simply how that substrate gets used.

If the substrate is sound, the playbooks evolve. If the substrate is not, the playbooks ossify, the team works around them, and the investment quietly stops returning value.

The next post in this series goes one layer down, into the runtime itself: how executions are persisted, how partial failures are recovered, and why the execution log is the most important audit artifact the platform produces.

Further reading

• NIST SP 800-61r3: Computer Security Incident Handling Guide
• MITRE D3FEND: Detection, Denial, Disruption framework
• Eric Evans: Domain-Driven Design, chapter on intention-revealing interfaces