calesthio/generative-media-skills/comfyui-media-workflows

ComfyUI media workflow production

Use this skill to turn a creative or technical brief into a production-ready ComfyUI workflow package. Treat ComfyUI as a graph-based media production runtime, not as a prompt box. Your output should make the graph reproducible, reviewable, safe to run, and easy for another operator or system to execute.

This is provider-independent. Do not assume a specific checkpoint, hosted Comfy service, custom node set, or cloud vendor unless the user or the existing project explicitly names one. Re-check ComfyUI, custom-node, model, license, and cloud API facts at production time because they change frequently.

Operating principles

• Start from the deliverable and constraints, then choose graph topology, models, nodes, and runtime. Do not start by pasting a popular workflow.
• Separate documented facts from production heuristics. Documented facts come from current docs, model cards, licenses, API schemas, and direct tests. Heuristics are your production judgment.
• Keep two workflow artifacts when possible:
• workflow.json: frontend/save-format graph with layout, grouping, notes, and operator readability.
• workflow_api.json: API-format graph for programmatic execution.
• Keep enough metadata to recreate a run: workflow JSON, input files with hashes, model identifiers and hashes, custom-node repo revisions, ComfyUI version, parameters, seed policy, hardware/runtime, output paths, and QA notes.
• Prefer fewer custom nodes and fewer model dependencies unless the creative requirement genuinely needs them. Every custom node is code execution and supply-chain risk.
• Never treat model availability as rights clearance. A file being downloadable does not mean the project can use it commercially, use its outputs freely, train on outputs, or use a person's likeness.
• Decide intentionally whether workflow metadata should be embedded in outputs. Embedded graphs aid reproducibility, but may leak prompts, model names, private paths, client references, API node titles, or workflow IP.

Required first pass

Produce a short production brief before building or modifying a graph:

1. Deliverable: media type, count, dimensions/aspect ratio, duration/fps if video, file formats, naming, platform or downstream editor.
2. Creative intent: subject, style, constraints, references, negative constraints, must-preserve elements, brand rules.
3. Inputs: text prompts, images, masks, video, audio, depth/pose/control maps, brand assets, identity/likeness permissions, source ownership.
4. Runtime: local ComfyUI, Comfy Cloud, another hosted Comfy-compatible service, or handoff-only; available GPU/VRAM, concurrency, storage, network and privacy constraints.
5. Model inventory: intended checkpoint/diffusion model, VAE, text encoders, LoRAs, ControlNets, adapters, upscalers, video models, audio/3D models if any.
6. Custom-node inventory: every non-core node package, install source, version/commit, license, dependency impact, trust level, and replacement plan.
7. Rights and safety: commercial status, client/counsel approval needs, likeness/voice consent, minors, regulated domains, impersonation/deception, prohibited content, platform restrictions.
8. Reproducibility target: exact rerun, near-match, or exploratory batch. Exact rerun requires stricter seeds, versions, hashes, and deterministic caveats.
9. Handoff target: another artist, an API service, a cloud deployment, a batch render queue, or an archive.

If any of these are missing, make a bounded assumption only when it does not change the workflow design materially. Otherwise ask.

Workflow formats and API boundaries

ComfyUI workflows are JSON graphs. The frontend save format and API format are not equivalent:

• Save format is for editing in the UI. It includes layout, colors, groups, node sizes, and similar visual metadata.
• API format is for execution. It uses node IDs as keys with class_type, inputs, and optional _meta; it omits most UI layout data.
• For programmatic execution, export from the frontend with File -> Export Workflow (API) or otherwise use a conversion method verified against the target runtime.
• Before submitting an API workflow, validate that each class_type exists in the runtime's node registry and that each exposed input name matches the current node definition.

When you need a static preflight of an API-format workflow, use the bundled Python 3.11+ stdlib helper before runtime validation:

python scripts/inspect_workflow.py workflow_api.json --approved-classes approved_classes.txt --unknown-class warning --pretty

The helper inspects only local JSON. It validates the API-format shape, node IDs, class_type, inputs, optional _meta, link inputs shaped as [node_id, output_index], dangling links, nonnegative integer output indexes, graph cycles, and warnings for secret-like keys/values or absolute/private-looking path strings. It emits a stable normalized JSON inventory with node, edge, class, and finding counts. Exit codes are 0 for no validation errors, 2 for validation errors, and 3 for parse or operational failures. If you provide a newline-delimited approved class list, class types absent from that list can be reported as warnings, errors, or ignored with --unknown-class.

This helper is not a substitute for ComfyUI runtime validation. It never executes the graph, submits to /prompt, installs missing custom nodes, loads models, verifies input names against /object_info, or proves that a workflow will fit VRAM or run on a target server.

For local ComfyUI server execution, the core pattern is:

1. Upload needed inputs through the runtime's upload route or place them in the expected input directory.
2. Submit the API-format graph to the prompt queue.
3. Monitor status through WebSocket messages or polling.
4. Retrieve outputs from the run history or view/download route.
5. Archive run metadata and outputs outside transient runtime directories.

For cloud execution, do not assume local-route semantics. Confirm authentication, endpoint prefixes, concurrency limits, storage behavior, signed URL behavior, timeouts, and partner-node key handling in the current cloud documentation.

The production workflow card

For every non-trivial workflow, maintain a workflow card in the project notes or handoff message. It should be concise but complete:

workflow_name: product-hero-sdxl-controlnet-v003
purpose: "Generate 6 square product hero image candidates with locked silhouette and color palette."
formats:
  save_graph: workflow.json
  api_graph: workflow_api.json
runtime_verified:
  comfyui_version: "record exact version/commit"
  execution_target: "local RTX 4090 / Comfy Cloud / other"
  verified_at: "YYYY-MM-DD"
models:
  - role: checkpoint
    id: "repository-or-file-name"
    file: "model.safetensors"
    sha256: "..."
    source_url: "..."
    license: "..."
    license_verified_at: "YYYY-MM-DD"
    approval: "approved / needs counsel / noncommercial only / unknown"
custom_nodes:
  - package: "repo-or-registry-id"
    version_or_commit: "..."
    source_url: "..."
    license: "..."
    risk: "low / medium / high"
    reason: "why needed"
exposed_inputs:
  positive_prompt_node: "6.inputs.text"
  negative_prompt_node: "7.inputs.text"
  seed_node: "3.inputs.seed"
  width_height_node: "5.inputs.width|height"
seed_policy: "fixed seeds for rerun; randomized only in variants manifest"
input_custody:
  source_assets: "paths, hashes, consent notes"
output_custody:
  destination: "..."
  metadata_policy: "embed workflow / strip workflow / add C2PA externally"
qa_status: "not run / smoke passed / approved / blocked"
known_limits:
  - "..."

Do not invent hashes, versions, license status, or approvals. Mark unknowns plainly.

Model and license inventory

Inventory every model-like artifact, not only the main checkpoint:

• checkpoint or diffusion model
• VAE
• text encoder(s)
• LoRA, LyCORIS, embedding, textual inversion
• ControlNet, T2I adapter, IP/reference adapter, pose/depth/segmentation model
• upscaler, face restoration model, background remover
• video generation model, interpolation model, frame enhancer
• audio, 3D, captioning, or LLM/VLM model used by nodes

For each, record:

• exact file name and folder category expected by the loader node;
• source URL or repository ID;
• version, revision, or download date;
• file hash when available or computed locally;
• model card or license URL;
• license terms relevant to this job: commercial use, attribution, output use, derivatives/fine-tunes, redistribution, synthetic-data/training restrictions, content-use restrictions;
• known runtime requirements: dtype/quantization, minimum VRAM, required text encoders or VAE;
• who approved it and when, or what remains unresolved.

Use model-card metadata as a starting point, not as the full clearance. Hugging Face supports license metadata and model cards, but missing or ambiguous metadata is common enough that you must inspect the linked license or repository files for consequential work. If the model is a derivative or merge, verify upstream license compatibility when the project is commercial, public, client-facing, or redistributed.

Escalate to the client, rights owner, platform owner, or counsel when:

• the license is absent, contradictory, noncommercial, research-only, unclear about outputs, or incompatible with the intended use;
• the model is a merge or derivative with unknown upstreams;
• the workflow uses a person's likeness, a private reference, a living artist/style request, a trademarked character, a brand asset, or regulated-domain content;
• outputs will be sold, advertised, syndicated, used for political persuasion, used in a regulated domain, or used as training data;
• the workflow or custom-node package will be redistributed to others.

Do not provide legal advice. State the risk and the decision needed.

Custom-node risk handling

Custom nodes are extensions that execute code inside the ComfyUI environment. Treat them as supply-chain dependencies.

Risk-rank each custom node:

• Low: official/core node, current registry package from a known publisher, active maintenance, clear license, no network behavior beyond documented model/API calls, pinned version.
• Medium: popular community node with active repo but broad dependencies, unclear release pinning, limited docs, or non-critical network/file behavior.
• High: obscure repo, no license, no pinned release, unreviewed install script, binary wheels from unknown source, broad filesystem/network access, credential handling, code obfuscation, stale issues about breakage/security, or required only for a minor convenience.

Required handling:

• Prefer core nodes and official templates when they satisfy the brief.
• If a custom node is necessary, pin a release or commit and record the install source. Avoid "latest" for production.
• Read the node README and installation requirements; check whether it requires models, external APIs, credentials, ffmpeg, CUDA extensions, compiler toolchains, or system packages.
• Install dependencies only into the intended ComfyUI environment. Do not contaminate system Python or unrelated projects.
• Do not run untrusted install scripts or post-install commands without approval.
• Do not pass secrets through node parameters that can be saved into workflow JSON, output metadata, logs, or screenshots unless the node's docs explicitly require that pattern and the user accepts it.
• When using a workflow from an external source, inspect unknown nodes before executing. A missing-node error is safer than blindly installing everything.

If a required custom node is high-risk and no safe replacement exists, stop and ask for approval with options: accept risk in a sandbox, replace the creative approach, use a hosted provider, or pause.

Graph design patterns