9thlevelsoftware/legion/review-evaluators

Trust tier 2
  • community
LLM review pending
Source check pending

Checking that the SKILL.md file is still in the repository.

Multi-pass specialized evaluators for deep code quality, UI/UX, integration, and business logic review

Install

npx vskill@latest install 9thlevelsoftware/legion/review-evaluators

SKILL.md

Review Evaluators

Provides multi-pass specialized evaluators for deep review beyond what the standard review-panel covers. Each evaluator type runs as a single agent invocation with a structured rubric containing multiple passes. Works alongside review-panel (agent selection and finding synthesis) and review-loop (fix cycles).

Only active when settings.review.evaluator_depth = "multi-pass". When inactive, review-panel handles all review duties with its standard per-criterion rubrics.


Section 1: Evaluator Selection

Auto-selects one or more evaluator types based on the phase type and files modified. When multiple types apply, all selected evaluators run (sequentially to control cost).

1.1 Activation Gate

Check settings.review.evaluator_depth before proceeding:

If settings.review.evaluator_depth != "multi-pass":
Skip this skill entirely.
Use review-panel standard rubrics instead.
Log: "Evaluator depth: standard — multi-pass evaluators not active"
If settings.review.evaluator_depth == "multi-pass":
Proceed to Section 1.2 (Phase Type Mapping)
1.2 Phase Type Mapping
Input: phase_type (from CONTEXT.md frontmatter), files_modified list
Step 1: Determine base evaluator set from phase_type
code | api → [Code Quality Evaluator, Integration Evaluator]
design | frontend → [UI/UX Evaluator]
business → [Business Logic Evaluator]
security → [Security Evaluator]
full-stack → [Code Quality Evaluator, Integration Evaluator, UI/UX Evaluator]
(default) → [Code Quality Evaluator]
Step 2: Augment from files_modified signals
**Matching rules — path-anchored globs with word-boundary tokens:**
- Substring globs (`*api*`) are replaced with path-anchored globs (`**/api/**`) so `rapi-tools.ts` no longer triggers Integration.
- Extensions (`.ts`, `.js`, `.jsx`, `.tsx`, `.svelte`, `.vue`, `.astro`, `.swift`, `.kt`, `.java`, `.cs`, `.cpp`, `.sql`, `.yaml`, `.yml`, `.json`, `.toml`) anchor on the filename suffix only.
- Canonical source: `.planning/config/intent-teams.yaml` → `evaluator_file_patterns` (add this key per LEGION-47-169). The rules below are a mirror; when patterns drift, edit the YAML first.
For each file in files_modified:
If file matches **/*.ts | **/*.js | **/*.jsx | **/*.tsx | **/*.py | **/*.go | **/*.rs | **/*.java | **/*.kt | **/*.cs | **/*.cpp | **/*.swift → add Code Quality Evaluator (if absent)
If file matches **/*.css | **/*.scss | **/*.vue | **/*.svelte | **/*.astro | **/*.jsx | **/*.tsx → add UI/UX Evaluator (if absent)
If file path matches **/api/** | **/routes/** | **/controllers/** | **/endpoints/** | **/handlers/** → add Integration Evaluator (if absent)
If file path matches **/services/** | **/domain/** | **/use-cases/** | **/business/** | **/core/** → add Business Logic Evaluator (if absent)
If file path matches **/auth/** | **/security/** | **/permissions/** | **/session/** | **/middleware/** OR filename contains token `auth|token|jwt|password|credential|secret|encrypt|crypto|rbac|acl` separated by `[._/\-]` boundaries → add Security Evaluator (if absent)
**Overlap resolution:** When a file triggers multiple evaluators, keep all. Cap at 3 evaluators per phase; if more than 3 trigger, keep Security (if present) + top-2 by file count and log cost warning.
Step 2b: Always add Completeness Evaluator
The Completeness Evaluator runs for ALL phase types. It evaluates whether error handling,
edge cases, and state coverage are adequate regardless of domain.
Step 3: Deduplicate evaluator list (preserve order: CQ → UIUX → INT → BL → SEC → COMP)
Step 4: Report selection
Log: "Evaluators selected: {evaluator_list}"
Log: "Trigger: phase_type={phase_type}, {N} file signals"
1.3 Multi-Type Phase Handling

When more than one evaluator is selected:

- Run evaluators in order: Code Quality → UI/UX → Integration → Business Logic → Security → Completeness
- Each evaluator produces its own structured result (one result per evaluator)
- Deduplication runs across all evaluator results (Section 9)
- Each evaluator's findings are tagged with its evaluator type for traceability
- Cost warning: N evaluators ≈ N × single-pass review cost (still one invocation each)
1.4 Evaluator Summary Table
EvaluatorPhase TypesDispatch TargetPass Count
Code Qualitycode, api, full-stackCodex (code_review)6
UI/UXdesign, frontend, full-stackGemini (ui_design)7
Integrationcode, api, full-stackInternal6
Business LogicbusinessInternal6
Securitysecurity, api, full-stackInternal (engineering-security-engineer)10 (OWASP)
Completenessall typesInternal6

Section 2: Code Quality Evaluator

Deep code quality analysis across 6 evaluation passes, run as a single agent invocation. Default dispatch target: Codex (capability: code_review).

2.1 Rubric Prompt Template

Inject the full rubric below as the agent's evaluation instructions. All 6 passes run within this single invocation — the agent works through each pass in sequence, producing a structured finding set per pass.

## Code Quality Evaluation — Multi-Pass Rubric
You are conducting a structured multi-pass code quality review. Work through all 6 passes
in sequence. For each pass, evaluate ONLY the criteria listed for that pass. Do not blend
criteria between passes.
For each finding, use this format:
- **Pass**: {N} — {pass_name}
- **File**: {file_path}:{line_number}
- **Severity**: BLOCKER | WARNING | INFO
- **Confidence**: HIGH (80-100%) | MEDIUM (50-79%) | LOW (<50%)
- **Issue**: {one-sentence description}
- **Fix**: {concrete suggested fix}
Only report HIGH and MEDIUM confidence findings. Discard LOW confidence findings.
---
### Pass 1: Build Integrity
Check that the codebase compiles and all imports resolve without error.
What to check:
1. All imported modules, files, and packages exist at the referenced path
2. No circular imports that would cause build failure
3. Export/import symmetry — everything imported is actually exported by the source
4. No syntax errors or malformed constructs visible in the diff
5. Build configuration (tsconfig, webpack, vite, etc.) is consistent with source changes
Severity guidance:
- BLOCKER: Missing import target, circular import in critical path, syntax error
- WARNING: Import path uses relative traversal that may break on refactor, build config inconsistency
- INFO: Unused import (lint will catch but noting for completeness)
Example findings:
- BLOCKER: `src/auth/token.ts:12` imports `../utils/crypto` which does not exist
- WARNING: `vite.config.ts:8` alias `@components` not updated to reflect new directory structure
- INFO: `src/index.ts:3` imports `lodash` but no usage detected in the file
---
### Pass 2: Type Safety
Evaluate type correctness, type coverage, and unsafe casts.
What to check:
1. No `any` types without explicit justification comment
2. All function parameters and return types are declared (strict mode compliance)
3. Type assertions (`as SomeType`) are safe and not masking a genuine type mismatch
4. Generic types are constrained appropriately, not open-ended where specific types exist
5. Null/undefined handling is explicit — no implicit null deref risk
Severity guidance:
- BLOCKER: Unsafe cast hiding a type error, unchecked null access in critical path
- WARNING: Untyped parameter in exported function, open-ended generic where specific type exists
- INFO: Minor type narrowing opportunity that doesn't affect correctness
Example findings:
- BLOCKER: `src/api/handler.ts:44` casts response as `UserRecord` without shape validation
- WARNING: `src/utils/parser.ts:18` parameter `data` is typed as `any` with no justification
- INFO: `src/models/order.ts:67` generic `Array<T>` could be narrowed to `Array<OrderItem>`
---
### Pass 3: Code Patterns and State Management
Verify that new code follows established project conventions and handles state correctly.
What to check:
1. New code follows the patterns and abstractions already used in the codebase
2. State mutation is controlled — no accidental shared mutable state between modules
3. Side effects are isolated and not hidden inside pure-looking functions
4. No duplicated logic that should be extracted into a shared utility
5. Abstraction level is consistent — not mixing low-level and high-level concerns in the same function
Severity guidance:
- BLOCKER: Shared mutable state that can produce race conditions or data corruption
- WARNING: Pattern deviation that will confuse future maintainers, hidden side effects
- INFO: Duplication that's a candidate for extraction but not harmful yet
Example findings:
- BLOCKER: `src/store/userSlice.ts:88` directly mutates `state.users` array outside a reducer
- WARNING: `src/components/Dashboard.tsx:120` fetches data inside a render function instead of using the established hook pattern
- INFO: `src/utils/format.ts:34` duplicates date formatting logic already in `src/utils/date.ts:12`
---
### Pass 4: Error Handling
Assess the completeness and correctness of error handling across the changed code.
What to check:
1. All async operations have `.catch()` or `try/catch` — no unhandled promise rejections
2. Error messages are descriptive and actionable, not generic "something went wrong"
3. Errors are not silently swallowed — at minimum logged, preferably surfaced to caller
4. External API/service calls handle failure modes: timeouts, 4xx, 5xx, network errors
5. Error boundaries or fallback states exist where user-facing failures can occur
Severity guidance:
- BLOCKER: Unhandled promise rejection, silent error swallow in critical path
- WARNING: Generic error message that won't help debugging, missing timeout handling
- INFO: Error logged but not surfaced to caller where caller could reasonably handle it
Example findings:
- BLOCKER: `src/services/payment.ts:55` `await stripe.charge()` has no try/catch — rejection will crash the process
- WARNING: `src/api/users.ts:89` catch block logs `"Error"` with no context about which user or operation failed
- INFO: `src/utils/cache.ts:32` swallows cache miss errors — caller could retry but isn't given the opportunity
---
### Pass 5: Dead Code and Cleanup
Identify code that is no longer needed or was left in from development.
What to check:
1. No commented-out code blocks left from development (use version control for history)
2. No `console.log`, `debugger`, or `TODO` statements in production paths
3. No unreachable code after returns, throws, or break statements
4. Exported functions, types, or variables that are no longer referenced anywhere
5. Feature flags or temporary workarounds that should have been removed
Severity guidance:
- BLOCKER: `debugger` statement in production code path
- WARNING: Commented-out code block (3+ lines), orphaned export with no references
- INFO: `console.log` in non-critical path, TODO comment without issue tracker reference
Example findings:
- BLOCKER: `src/auth/middleware.ts:203` contains `debugger;` statement
- WARNING: `src/legacy/v1-compat.ts:12-45` is fully commented out — remove or document why preserved
- INFO: `src/types/index.ts:88` exports `LegacyUserShape` which has zero references in the codebase
---
### Pass 6: Test Coverage
Evaluate whether the changes are adequately covered by tests.
What to check:
1. All new public functions have at least one corresponding test
2. Error paths are tested, not just happy paths
3. Existing tests still pass — no test breakage from the change
4. Edge cases (empty input, null, boundary values) have test coverage
5. Integration points (API calls, DB queries) have tests or are explicitly mocked

This is the start of the file. Read the full SKILL.md on GitHub.

Security

  • Security scanPassed, 93 of 100Pattern checks on the skill's files.
  • Model reviewNot runA model review is scheduled for the next rescan.

Quality evaluation

  • Task evaluationNot runNo evaluation yet. Run one in Skill Studio.

Source

Popularity

Repository stars
62
Repository forks
7
7-day trend score
21

Stars and forks are for the whole GitHub repository, not this skill alone. Not refreshed yet.

Works with

Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.

  • Claude Code
  • Cursor
  • GitHub Copilot
  • Windsurf
  • Codex
  • Gemini CLI
  • Cline
  • Amp
  • Roo Code
  • Goose
  • Continue
  • Kilo Code
  • Junie
  • Trae
  • OpenHands
  • Qwen Code
  • AI Agent
  • Tabnine Chat
  • Sourcegraph Cody
  • Amazon Q
  • Firebase Genkit
  • Sweep
  • Devin
  • Phind
  • Mentat
  • Claude Desktop
  • Aider
  • GPT Pilot
  • Plandex
  • bolt.new
  • v0
  • Replit Agent
  • Anse
  • TypingMind
  • BetterChatGPT
  • LibreChat
  • Warp AI
  • Pieces
  • Fig AI
Embed badge
vskill VERIFIED badge
[![vskill VERIFIED](https://verified-skill.com/api/v1/skills/9thlevelsoftware/legion/review-evaluators/badge)](https://verified-skill.com/skills/9thlevelsoftware/legion/review-evaluators)

Something wrong with this skill or its page? Tell us.