<< All versions
Skill v1.0.1
currentAutomated scan100/100simhacker/moollm/evaluator
5 files
──Details
PublishedOctober 25, 2026 at 09:08 PM
Content Hashsha256:0da6aadf42d77d1a...
Git SHAb38fddeee451
Bump Typepatch
──Files
Files (1 file, 5.5 KB)
SKILL.md5.5 KBactive
SKILL.md · 249 lines · 5.5 KB
version: "1.0.1" name: evaluator description: Independent assessment without debate context — adversarial loop prevents gaming allowed-tools:
- read_file
- write_file
tier: 1 protocol: INDEPENDENT-EVALUATOR tags: [moollm, evaluation, adversarial, scoring, review] credits: "Mike Gallaher — independent evaluator pattern" related: [adversarial-committee, rubric, roberts-rules, room]
Evaluator
"Fresh eyes, no bias, just the rubric."
Committee output goes to a separate model instance with NO debate context.
The Separation
yaml
evaluation:principle: "Evaluator has NO access to:"- debate_transcript- speaker_identities- amendment_history- voting_patterns- minority_dissentsevaluator_sees_only:- final_output- rubric_criteria- subject_matter_context
Room Architecture
yaml
# committee-room/# ROOM.yml# debate.yml# output.yml# outbox/# evaluation-request-001.yml# evaluation-room/# ROOM.yml# rubric.yml# inbox/# evaluation-request-001.yml # Landed here# evaluations/# eval-001.yml
Evaluation Request
yaml
# Thrown from committee to evaluatorevaluation_request:id: eval-req-001from: committee-roomtimestamp: "2026-01-05T15:00:00Z"subject: "Client X Engagement Decision"output_only: |Recommendation: Accept Client X with:- Explicit scope boundaries- Milestone-based billing- Quarterly scope reviewConfidence: 0.65Key considerations:- Revenue opportunity aligns with growth goals- Risk mitigated by contractual protections- Capacity impact manageablerubric: client-evaluation-v1# Note: NO debate context included
Evaluation Process
yaml
evaluation:request: eval-req-001evaluator: "fresh model instance"context_loaded: false # Critical!steps:1. load_rubric: client-evaluation-v12. read_output: evaluation_request.output_only3. score_each_criterion: independently4. calculate_weighted_total: true5. generate_critique: if score < threshold
Evaluation Output
yaml
# evaluation-room/evaluations/eval-001.ymlevaluation:id: eval-001request: eval-req-001timestamp: "2026-01-05T15:05:00Z"rubric: client-evaluation-v1scores:resource_efficiency:score: 4rationale: "Output indicates capacity is manageable"risk_level:score: 3rationale: "Mitigations proposed but not detailed"confidence: "Would score higher with specific terms"strategic_alignment:score: 4rationale: "Growth goals mentioned, seems aligned"stakeholder_impact:score: 3rationale: "Not explicitly addressed in output"flag: "Committee should consider stakeholder effects"weighted_total: 3.45threshold: 3.5result: REVIEW # Just below acceptcritique:summary: "Close to acceptance threshold"gaps:- "Risk mitigation lacks specifics"- "Stakeholder impact not addressed"- "Confidence of 0.65 seems low for recommendation"suggestions:- "Detail the milestone structure"- "Explain how scope boundaries will be enforced"- "Address impact on existing clients and team"if_addressed: "Score could reach 3.7+ (accept)"
Revision Loop
yaml
revision_loop:max_iterations: 3flow:1. committee_outputs: recommendation2. evaluator_scores: against rubric3. if score >= threshold: ACCEPT4. if score < threshold:- evaluator_generates: critique- critique_thrown_to: committee inbox- committee_revises: based on critique- goto: step 15. if max_iterations reached: ESCALATE to human
Adversarial Properties
yaml
adversarial_separation:why: "Prevents committee gaming metrics"committee_cannot:- see_evaluator_reasoning- predict_exact_scores- optimize_for_rubric_loopholesevaluator_cannot:- be_influenced_by_debate_dynamics- favor_particular_speakers- weight_majority_over_minorityresult: "Genuine quality signal"
Commands
| Command | Action | |
|---|---|---|
EVALUATE [output] | Send to independent evaluator | |
APPLY RUBRIC [name] | Score against criteria | |
CRITIQUE | Generate improvement suggestions | |
REVISE | Committee addresses critique | |
ESCALATE | Send to human decision maker |
Integration
mermaid
graph TDC[Committee] -->|output| T[THROW to outbox]T -->|lands in| I[Evaluator inbox]I --> E[Evaluate]R[RUBRIC.yml] --> EE --> S{Score}S -->|≥ threshold| A[✅ Accept]S -->|< threshold| CR[Generate Critique]CR -->|THROW back| CI[Committee inbox]CI --> REV[Revise]REV --> Csubgraph "No Context Crossing"IERend
Model Instance Separation
For true independence:
yaml
implementation:option_1:name: "Fresh conversation"method: "New chat with no history"option_2:name: "Separate model"method: "Different API call, different instance"option_3:name: "System prompt separation"method: "Explicit instruction: 'You have no prior context'"key_principle: |The evaluator must NOT have access to:- How the committee reached the conclusion- Who said what- What alternatives were considered- Why certain risks were dismissedOnly the final output matters.