chucksrq/awesome-hermes-skills/agentic-self-improvement

Agentic Self-Improvement Loop

Behavioral benchmarking + evolutionary self-improvement for Hermes Agent. Combines closed-loop guidance patching (from Hermes PR #6120) with the GEPA optimization framework from hermes-agent-self-evolution (NousResearch, MIT).

Core Loop

BENCHMARK → DIAGNOSE → PATCH → VERIFY → (auto-revert if regression)
                    ↓
              GEPA OPTIMIZE → EVAL DATASET → EVOLVE → DEPLOY
                    ↓
              SESSION MINE → LLM-AS-JUDGE → NEW BENCHMARKS

Two modes operate together:

• Guidance Patch Loop — fast, surgical, targets specific behavioral failures via guidance blocks
• Evolutionary Optimization Loop — principled, uses GEPA + DSPy to optimize skill text end-to-end

Part 1: Guidance Patch Loop

Core Loop

BENCHMARK → DIAGNOSE → PATCH → VERIFY → (auto-revert if regression)

Usage

# Run all benchmarks, show suggested patches (no changes made)
/self-improve
 
# Run all benchmarks, auto-apply if improvement, auto-revert if regression
/self-improve --mode=apply
 
# Run specific category only
/self-improve --benchmarks=mandatory_tool.yaml
 
# Run subset of categories
/self-improve --categories=mandatory_tool,act_dont_ask,no_hallucination
 
# Test a specific model
/self-improve --model=claude-sonnet-4
 
# Control parallelism (default: 4)
/self-improve --parallel=8
 
# View past results
/self-improve --view-results=2026-04-12_1400
 
# Revert last applied patch
/self-improve --revert

Options

Benchmark Categories

How the Guidance Patch Loop Works

1. Benchmark Runner

Each prompt runs in isolation via hermes chat --model <model> -q QUERY -Q as a subprocess. Captures:

• Tool calls made (detected as "preparing X..." text previews in quiet mode)
• Tool outputs
• Final response text
• Exit code

2. Failure Analyzer

Groups failures by category, computes per-category pass rates, extracts failure examples.

3. Patch Generator

Generates XML guidance blocks for the relevant model type:

• OpenAI/GPT/Codex → OPENAI_MODEL_EXECUTION_GUIDANCE
• Anthropic/Claude → ANTHROPIC_MODEL_EXECUTION_GUIDANCE
• Google/Gemini → GEMINI_MODEL_EXECUTION_GUIDANCE
• MiniMax → MINIMAX_MODEL_EXECUTION_GUIDANCE

4. Apply + Verify + Auto-Revert

Suggest mode: Shows the diff, writes to ~/.hermes/self-improvement/proposed_patches/<run_id>.diff

Apply mode:

1. Run baseline benchmarks → capture pass rates
2. Backup prompt_builder.py to ~/.hermes/self-improvement/backups/<timestamp>/
3. Apply patch to relevant GUIDANCE constant
4. Re-run benchmarks → compare pass rates
5. If ANY category regresses → auto-revert to backup
6. If all improve → keep patch, report delta

Output Locations

Part 2: Session Pattern Mining

Extract behavioral failure patterns from conversation history and convert them into concrete, actionable test cases.

When to Use

• Designing benchmarks for a self-improvement loop
• Finding recurring mistakes to address in skills or system prompts
• Building a behavioral test suite for an agent's capabilities
• Reviewing past sessions for systematic failures vs one-off errors

The Process

1. Search Broadly, Then Narrow

# Good starting queries:
"failure pattern mistake error"
"wrong incorrect mistake"
"assume guess hallucinate"
"context switch forgot lost"
"should I do ask clarification"
"file not found path error"
"not sure unknown do not know"
"directory path file location"
"behavior bias habit pattern"

2. Extract the Failure Narrative

For each relevant session:

• What was the task? (the user's request)
• What went wrong? (the failure)
• What was the root cause? (why it failed)
• What category does it belong to? (see Categories below)

## Session: <session_id> — <timestamp>
### Task
<what the user asked>
 
### Failure
<what went wrong>
 
### Root Cause
<why it happened>
 
### Category
<pattern category>

3. Categorize Failures

4. Convert Failures to Benchmark Prompts

id: <category>-<number>  # e.g. mt-001, ada-001
prompt: <the test prompt>
validator:
  type: <tool_used|output_contains|refused|exit_code>
  detail: <specific check>
ground_truth: <expected correct output or behavior>
failure_example:
  what_happened: <quote from session showing the failure>
  session_id: <source session>

Validator types:

• tool_used — model must call a specific tool
• output_contains — tool output must contain expected string
• refused — model must refuse to answer (not enough context)
• exit_code — command must exit with specific code
• error_in_output — model must report an error it detected
• command_contains — response or tool outputs contain specific strings
• non_empty_or_explicit_not_found — non-empty response or explicit not-found
• two_step / sequence — multi-step (tool use demonstrates it)

5. Group Into Benchmark Sets

Cluster prompts by category. A complete benchmark set should have:

• 4-10 prompts per category
• Varied phrasings (don't make the expected action obvious)
• At least one negative example (task that should NOT use a tool)

Session Search Tips

1. Search for the fix, not the failure — sessions that end with "fixed" often document what broke first
2. Check session summaries first — session_search returns LLM-generated summaries that surface failures efficiently
3. Look at truncated sessions — sessions that were "cut off" mid-investigation often reveal root causes
4. Cross-reference with logs — ~/.hermes/logs/errors.log often has the raw error that sessions later explained

Output Location

Save mined patterns to <obsidian-vault>/03-Notes/session-patterns/<YYYY-MM-DD>.md so they're preserved in long-term memory alongside other session research.

Part 3: Evolutionary Optimization (from hermes-agent-self-evolution)

The full GEPA + DSPy optimization framework from NousResearch/hermes-agent-self-evolution.

What Can Be Optimized — The Tier System

Tier 1 is the primary target for self-improvement — skills are pure text, easily mutated, directly measurable.

The Optimization Loop (GEPA Framework)

1. SELECT TARGET
   - Pick a skill, prompt section, or tool
   - Load current version as baseline
 
2. BUILD EVALUATION DATASET
   - Mine SessionDB for real usage examples
   - Or use session-pattern-mining to extract from transcripts
   - Or generate synthetic test cases
   - Split: train / validation / test
 
3. RUN OPTIMIZER
   - Primary: GEPA (Genetic-Pareto Prompt Evolution)
   - Fallback: DSPy MIPROv2 (Bayesian optimization)
   - Generates mutations, evaluates, selects best
 
4. EVALUATE & COMPARE
   - Run optimized version on held-out test set
   - Compare: accuracy, cost, latency
   - Statistical significance check
 
5. DEPLOY (with approval)
   - Git commit the improved version
   - Rollback mechanism via git revert

Evaluation Dataset Sources (in priority order)

Source A: SessionDB mining (real usage, LLM-as-judge scored)

• Query SessionDB for sessions where the skill was loaded
• Extract the task the user gave and the agent's full response
• Use LLM-as-judge to score each (task, response) pair on a rubric
• High-scoring pairs → "good" examples; low-scoring → failure cases for GEPA

Source B: Synthetic generation (bootstrapping)

• Use a strong model (Claude Opus) to generate 15-30 test cases
• Read the skill file → understand what it does
• Generate (task_input, expected_behavior) pairs
• GEPA works with as few as 3 examples, so start small

Source C: Hand-curated golden sets

• Manually written test cases with expected outputs
• Highest quality signal but requires manual effort
• Reserve for critical skills

LLM-as-Judge Scoring (Multi-Dimensional Rubrics)

For most skills, there's no binary right/wrong — quality is subjective. Use an LLM judge with a rubric:

• Did the agent follow the skill's procedure? (0-1)
• Was the output correct/useful? (0-1)
• Was it concise (within token budget)? (0-1)
• Did it avoid the known failure modes? (0-1)

Aggregate into a composite score.

Constraint Gates (must pass before any change is valid)