jswortz/my-skills/dssib-vertex-pipelines
Claude Code and agent skill by jswortz/my-skills
- community
Checking that the SKILL.md file is still in the repository.
-- name: dssib-vertex-pipelines description: Guide for deploying Dynamic Swarm (DSSIB) Fan-Out agents on Vertex AI Pipelines with ML Metadata integrat...
Install
npx vskill@latest install jswortz/my-skills/dssib-vertex-pipelinesSKILL.md
The same SKILL.md was published earlier as jswortz/my-skills/dssib-vertex-pipelines.
-- name: dssib-vertex-pipelines description: Guide for deploying Dynamic Swarm (DSSIB) Fan-Out agents on Vertex AI Pipelines with ML Metadata integration.
Deploying DSSIB Agents on Vertex AI Pipelines
Use this skill when deploying or architecting a multi-agent fan-out orchestration system (like NovaStorm) on Google Cloud. Instead of using Cloud Run for long-running asynchronous background agents, Vertex AI Pipelines is the preferred native orchestrator.
Core Concepts
- Vertex AI Pipelines (`kfp`): Use Kubeflow pipeline definitions to orchestrate the ReAct loop. It handles the parallelization gracefully.
- Parallel Fan-out: Rather than spawning async
asynciotasks in an HTTP container, usekfp.dsl.ParallelForto achieve robust parallelism for the analytical agents querying BigQuery in parallel. - ML Metadata (MLMD): The most important benefit is native lineage tracking. Use
kfp.dsl.Metricsandkfp.dsl.Artifactto track all generated outputs (insights, visual charts, mutated agent scripts/skills) into Vertex ML Metadata. - Dataplex Integration: The ML Metadata naturally feeds back into the Dataplex Universal Catalog, ensuring that the insights generated by the swarm are deeply integrated into the data governance layer.
- Scheduled Execution: Use the
PipelineJobSchedule.createAPI to orchestrate hourly or daily pipeline runs, streamlining the architecture by removing the need for external Cloud Scheduler to Cloud Run HTTP triggers. - GEPA (Generative Evolutionary Prompt Augmentation): Use dynamic LLM temperature-dilated mutation to evolve analytical agents' "Skill DNA" over extended simulations (e.g., 50 epochs), hybridizing crossover skills.
- Secure Secret Injection: Never pass API keys or credentials as plain text pipeline arguments. Always mount them dynamically inside the component via Secret Manager at runtime.
Implementation Example Pattern
Advantages Over Cloud Run + HTTP Triggers
- Automatic UI visualization of the 10+ parallel fan-out steps in Vertex AI.
- Native metadata logging (tracking LLM prompts, token counts, and output insights as pipeline artifacts).
- First-class Google Cloud integrations with BigQuery, Dataplex, and TimesFM models.
Security
- Security scanPassed, 95 of 100Pattern checks on the skill's files.
- Model reviewPassed
Quality evaluation
- Task evaluationNot runNo evaluation yet. Run one in Skill Studio.
Source
- SKILL.md in the repositoryCheckingdssib-vertex-pipelines/SKILL.md
Popularity
- Repository stars
- 2
- Repository forks
- 2
- 7-day trend score
- 10
Stars and forks are for the whole GitHub repository, not this skill alone. Not refreshed yet.
Works with
Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.
- Claude Code
- Cursor
- GitHub Copilot
- Windsurf
- Codex
- Gemini CLI
- Cline
- Amp
- Roo Code
- Goose
- Continue
- Kilo Code
- Junie
- Trae
- OpenHands
- Qwen Code
- AI Agent
- Tabnine Chat
- Sourcegraph Cody
- Amazon Q
- Firebase Genkit
- Sweep
- Devin
- Phind
- Mentat
- Claude Desktop
- Aider
- GPT Pilot
- Plandex
- bolt.new
- v0
- Replit Agent
- Anse
- TypingMind
- BetterChatGPT
- LibreChat
- Warp AI
- Pieces
- Fig AI
Embed badge
[](https://verified-skill.com/skills/jswortz/my-skills/dssib-vertex-pipelines)