luum-home/luum-cognitive-os/cognitive-os-benchmark

Name: cognitive-os-benchmark
Author: luum-home

Trust tier 3

community

By luum-home
Coding
luum-home/luum-cognitive-os
packages/verification-audit/skills/cognitive-os-benchmark/SKILL.md
Updated 24 Oct 2026
Version 1.0.01 version recorded

Source check pending

Checking that the SKILL.md file is still in the repository.

Run benchmark comparisons between Cognitive OS and BMAD METHOD v6

Install

npx vskill@latest install luum-home/luum-cognitive-os/cognitive-os-benchmark

SKILL.md

Cognitive OS Benchmark Skill

Run standardized benchmarks to compare Cognitive OS vs BMAD METHOD v6 on 5 coding tasks.

Usage

/benchmark                    # Run all benchmarks with Cognitive OS
/benchmark --system bmad      # Run all benchmarks with BMAD v6
/benchmark --task fix-bug     # Run a single benchmark task
/benchmark --dry-run          # Preview what would run

What It Does

Reads task definitions from .cognitive-os/tests/benchmark/benchmark-config.yaml
For each task, creates an isolated git worktree
Runs Claude Code headless with the task prompt
Measures: time, tokens, files created, tests created, compilation success
Runs LLM evaluation for architecture/pattern compliance scores
Saves results to .cognitive-os/metrics/benchmark-results.jsonl
Generates a markdown report

Benchmark Tasks

ID	Description
`create-go-service`	Create a new Go microservice with clean architecture
`fix-bug`	Fix a validation bug and add regression tests
`add-endpoint`	Add an endpoint following existing patterns
`refactor-code`	Refactor controllers to use cases
`cross-service-feature`	Implement a feature spanning multiple services

Commands

Run All Benchmarks

bash

bash .cognitive-os/tests/benchmark/run-benchmark.sh

Run Cognitive OS Only

bash

bash .cognitive-os/tests/benchmark/run-benchmark.sh --system cognitive-os

Run BMAD Only

bash

bash .cognitive-os/tests/benchmark/run-benchmark.sh --system bmad

Run Single Task

bash

bash .cognitive-os/tests/benchmark/run-benchmark.sh --task create-go-service

Dry Run (preview)

bash

bash .cognitive-os/tests/benchmark/run-benchmark.sh --dry-run

Keep Worktrees After Run

bash

bash .cognitive-os/tests/benchmark/run-benchmark.sh --no-cleanup

Options

Flag	Description	Default
`--system`	`cognitive-os` or `bmad`	`cognitive-os`
`--task`	Run single task by ID	all tasks
`--model`	Override Claude model	from config
`--max-turns`	Override max turns	from config
`--dry-run`	Show tasks without executing	false
`--no-cleanup`	Keep worktrees after run	false

Output

JSONL results: .cognitive-os/metrics/benchmark-results.jsonl
Markdown report: .cognitive-os/metrics/benchmark-report-{run-id}.md
Comparison guide: .cognitive-os/tests/benchmark/compare-with-bmad.md

Dependencies

claude CLI
yq (YAML parser)
jq (JSON processor)
git (for worktree isolation)

Install missing deps: brew install yq jq

Metrics Collected

Metric	Type	Description
`time_seconds`	duration	Wall-clock execution time
`tokens_used`	count	Total input + output tokens
`files_created`	count	New/modified files
`tests_created`	count	Test files created
`compilation_success`	boolean	Code builds without errors
`architecture_score`	llm_eval	Clean architecture compliance (0-10)
`follows_patterns`	llm_eval	Codebase pattern consistency (0-10)
`architecture_improved`	llm_eval	Refactoring quality (0-10)

Scoring

Maximum score: 50 points across all 5 tasks. Weights defined in benchmark-config.yaml under scoring.weights. Time and token usage apply small negative penalties to reward efficiency.

Security

Security scanPassed, 90 of 100Pattern checks on the skill's files.
Model reviewPassed

See the full security report

Quality evaluation

Task evaluationNot runNo evaluation yet. Run one in Skill Studio.

See evaluation results

Source

SKILL.md in the repositoryCheckingpackages/verification-audit/skills/cognitive-os-benchmark/SKILL.md

Popularity

7-day trend score: 0

Not refreshed yet.

Works with

Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.

Claude Code
Cursor
GitHub Copilot
Windsurf
Codex
Gemini CLI
Cline
Amp
Roo Code
Goose
Continue
Kilo Code
Junie
Trae
OpenHands
Qwen Code
AI Agent
Tabnine Chat
Sourcegraph Cody
Amazon Q
Firebase Genkit
Sweep
Devin
Phind
Mentat
Claude Desktop
Aider
GPT Pilot
Plandex
bolt.new
v0
Replit Agent
Anse
TypingMind
BetterChatGPT
LibreChat
Warp AI
Pieces
Fig AI

Embed badge

[![vskill VERIFIED](https://verified-skill.com/api/v1/skills/luum-home/luum-cognitive-os/cognitive-os-benchmark/badge)](https://verified-skill.com/skills/luum-home/luum-cognitive-os/cognitive-os-benchmark)

Something wrong with this skill or its page? Tell us.