czlonkowski/n8n-skills/n8n-workflow-patterns

n8n Workflow Patterns

Proven architectural patterns for building n8n workflows.

The 6 Core Patterns

Based on analysis of real workflow usage:

1. [Webhook Processing](webhook_processing.md) (Most Common)

• Receive HTTP requests → Process → Output
• Pattern: Webhook → Validate → Transform → Respond/Notify

1. [HTTP API Integration](http_api_integration.md)

• Fetch from REST APIs → Transform → Store/Use
• Pattern: Trigger → HTTP Request → Transform → Action → Error Handler

1. [Database Operations](database_operations.md)

• Read/Write/Sync database data
• Pattern: Schedule → Query → Transform → Write → Verify

1. [AI Agent Workflow](ai_agent_workflow.md)

• AI agents with tools and memory
• Pattern: Trigger → AI Agent (Model + Tools + Memory) → Output

1. [Scheduled Tasks](scheduled_tasks.md)

• Recurring automation workflows
• Pattern: Schedule → Fetch → Process → Deliver → Log

1. Batch Processing (below)

• Process large datasets in chunks with API rate limits
• Pattern: Prepare → SplitInBatches → Process per batch → Accumulate → Aggregate

Pattern Selection Guide

When to use each pattern:

Webhook Processing - Use when:

• Receiving data from external systems
• Building integrations (Slack commands, form submissions, GitHub webhooks)
• Need instant response to events
• Example: "Receive Stripe payment webhook → Update database → Send confirmation"

HTTP API Integration - Use when:

• Fetching data from external APIs
• Synchronizing with third-party services
• Building data pipelines
• Example: "Fetch GitHub issues → Transform → Create Jira tickets"

Database Operations - Use when:

• Syncing between databases
• Running database queries on schedule
• ETL workflows
• Example: "Read Postgres records → Transform → Write to MySQL"

AI Agent Workflow - Use when:

• Building conversational AI
• Need AI with tool access
• Multi-step reasoning tasks
• Example: "Chat with AI that can search docs, query database, send emails"

Scheduled Tasks - Use when:

• Recurring reports or summaries
• Periodic data fetching
• Maintenance tasks
• Example: "Daily: Fetch analytics → Generate report → Email team"

Batch Processing - Use when:

• Processing large datasets that exceed API batch limits
• Need to accumulate results across multiple API calls
• Nested loops (e.g., multiple categories × paginated API calls per category)
• Example: "Fetch products for 4 markets × 1000 per API call → Aggregate all results"

Common Workflow Components

All patterns share these building blocks:

1. Triggers

• Webhook - HTTP endpoint (instant)
• Schedule - Cron-based timing (periodic)
• Manual - Click to execute (testing)
• Polling - Check for changes (intervals)

2. Data Sources

• HTTP Request - REST APIs
• Database nodes - Postgres, MySQL, MongoDB
• Service nodes - Slack, Google Sheets, etc.
• Extract From File - CSV / XLSX / PDF from a downloaded or uploaded file

3. Transformation (native first)

• Filter / IF / Switch - Drop items, route
• Edit Fields (Set) - Named steps: constant items, payload objects, error and return items
• Summarize / Aggregate / Split Out - Group and total, collect into an array, explode an array
• Merge / Compare Datasets - Join two lists, diff them for a sync
• Remove Duplicates / Sort / Limit - Dedupe (also across runs), top-N
• Date & Time, XML, HTML, Crypto - Dates, markup, hashes and signatures
• Code - Only the closed list (loops or recursion no node expresses, state across runs, parsing no node handles), with a Why Code: note

The job-to-node table in using-n8n-mcp-skills maps jobs to these nodes; configs are in n8n-node-configuration → NATIVE_NODES.md. Reading a filtered slice of a nested API response for one field? A single {{ $jmespath($json, "…") }} in that field does it without extra nodes (quoting rules in n8n-expression-syntax).

4. Outputs

• HTTP Request - Call APIs
• Database - Write data
• Communication - Email, Slack, Discord
• Storage - Files, cloud storage

5. Error Handling

• Error Trigger - Catch workflow errors
• IF - Check for error conditions
• Stop and Error - Explicit failure
• Continue On Fail - Per-node setting

Workflow Creation Checklist

When building ANY workflow, follow this checklist:

Planning Phase

• [ ] Identify the pattern (webhook, API, database, AI, scheduled)
• [ ] List required nodes (use search_nodes)
• [ ] Understand data flow (input → transform → output)
• [ ] Plan error handling strategy

Implementation Phase

• [ ] Create workflow with appropriate trigger
• [ ] Add data source nodes
• [ ] Configure authentication/credentials
• [ ] Add transformation nodes (native first: Filter, IF, Summarize, Merge…; Code only with a Why Code: note)
• [ ] Add output/action nodes
• [ ] Configure error handling

Validation Phase

• [ ] Validate each node configuration (validate_node)
• [ ] Validate complete workflow (validate_workflow)
• [ ] Test with sample data
• [ ] Handle edge cases (empty data, errors)

Deployment Phase

• [ ] Review workflow settings (execution order, timeout, error handling)
• [ ] Activate workflow using activateWorkflow operation
• [ ] Monitor first executions
• [ ] Document workflow purpose and data flow

Workflow lifecycle: validate, verify, test before activating

Building the nodes is the start, not the finish. Before a workflow goes live, run it through four gates — and remember the headline rule: validation passing is necessary, not sufficient. A workflow can validate clean and still drop items, pick the wrong Merge input, or post Slack messages as plain text. Clean validation means the shapes are right, not that the logic is.

1. Validate. Run validate_workflow on the full JSON during build, or n8n_validate_workflow({ id }) once the workflow exists on the instance. Fix every error and re-validate. This catches schema, node-config, expression, and reference errors — the structural layer.
2. Verify the connections. Pull the workflow with n8n_get_workflow({ id }) and read the connections object directly. Validation confirms connections aren't broken; it doesn't confirm they're correct. This is where you catch the valid-but-wrong wiring: a Merge whose useDataOfInput doesn't line up with the connection slot, a Switch fallback that connects to nothing, a fan-out branch that was never wired onward, an error output that goes nowhere. (See the n8n Node Configuration skill's NODE_FAMILY_GOTCHAS.md for the silent ones.)
3. Test. Run n8n_test_workflow and inspect the output via n8n_executions. Confirm the output shape matches what consumers expect, fan-outs all produced data, and (for webhook APIs) the status/body/headers are right. Real side effects fire during a test — writes commit, messages send, external APIs are called. If any node has a user-visible side effect, confirm with the user before running, or test against safe data first.
4. Activate only after the first three pass — using n8n_update_partial_workflow with the activateWorkflow operation. Don't activate straight off a clean validation; an active workflow that drops data or double-sends is worse than one that never started.

Skipping any gate trades a few minutes now for debugging a live, possibly stateful, possibly traffic-bearing workflow later. The trade is never worth it.

Data Flow Patterns

Linear Flow

Trigger → Transform → Action → End

Use when: Simple workflows with single path

Branching Flow

Trigger → IF → [True Path]
             └→ [False Path]

Use when: Different actions based on conditions

Parallel Processing

Trigger → [Branch 1] → Merge
       └→ [Branch 2] ↗

Use when: Independent operations that can run simultaneously

Loop Pattern

Trigger → Split in Batches → Process → Loop (until done)

Use when: Processing large datasets in chunks

Error Handler Pattern

Main Flow → [Success Path]
         └→ [Error Trigger → Error Handler]

Use when: Need separate error handling workflow

Batch Processing Pattern

SplitInBatches Loop

The SplitInBatches node splits a large dataset into smaller chunks for processing. Understanding its outputs is critical:

• main[0] = done — fires ONCE after all batches complete
• main[1] = each batch — fires per batch (this is the loop body)

Prepare Items → SplitInBatches → [main[1]: Process Batch] → (loops back)
                                  [main[0]: Done] → Limit 1 → Aggregate

Always add a Limit 1 node after the done output.

Choosing batchSize (the cost lever)

A SplitInBatches loop re-runs its whole body once per iteration — ~0.8 ms/iteration of engine overhead plus the body's own cost — so total ≈ ⌈items / batchSize⌉ × (overhead + body). batchSize is a direct speed dial:

• Pick the largest batch your real constraint allows (API page size, rate limit, memory). Bigger batches = fewer iterations = less overhead; the body still sees every item.
• batchSize: 1 is the expensive extreme — one full engine pass per item. Use it only when you must act on a single item at a time (nested-loop control, or an API that takes exactly one id).
• If you're looping only to "go over the items" with no external constraint, you don't need the loop: connect the source to the consumer. Per-item execution handles it, and native transform nodes process all items in one call.

Cross-Iteration Data

After the loop, $('Node Inside Loop').all() returns ONLY the last batch's items. To accumulate across all iterations, use $getWorkflowStaticData('global') in a Code node inside the loop. See the n8n Code JavaScript skill for the full pattern.

Nested Loops

When processing N categories × M items per category (where an API has a batch limit):

Define Categories (N items)
  → Outer Loop (SplitInBatches, batchSize=1)
    → Prepare category data
    → Inner Loop (SplitInBatches, batchSize=1000)
      → API Call → Verify → (loops back to Inner Loop via main[1])
    → Inner done[0] → Rate Limit Delay → back to Outer Loop
  → Outer done[0] → Limit 1 → Final Aggregate

Wiring gotcha: The inner done[0] must connect back to the OUTER loop input, not to the aggregate. The outer done[0] feeds the final aggregate.

API Pagination

Paging through one endpoint is HTTP Request's pagination option, not a loop (page-number, offset, cursor and next-URL APIs, with a page cap — see n8n-node-configuration → NATIVE_NODES.md). The loop below is for sweeps the option can't express, such as an API without multi-ID filtering that you walk with id_from + date windows, each window its own query:

Schedule → Set Date Window → Fetch Page → Process
  → IF has more? → [true] Update id_from → Fetch Page (loop)
                  → [false] → Aggregate → Output

Dry-Run / Verification Tolerance

When testing with API write nodes disabled (for dry runs), downstream verification nodes receive the request body instead of the response. Make verification tolerant:

// In verification Code node
const body = $input.first().json;
const looksLikeRequest = body.method && body.parameters && !body.status;
if (looksLikeRequest) {
  return [{ json: { status: 'SKIPPED', message: 'Upstream disabled for testing' }}];
}
// Normal response verification below...

Performance on the hot path

When a workflow processes thousands of items with little I/O, its speed is set by how many times n8n crosses a per-item / per-iteration boundary — each crossing sets up an execution context and copies the items. Four architecture choices dominate: