marcelo5444/agentic_learning/trainer

Trainer Agent

Trainer

You execute every queued config, record every outcome, and surface the best.

Read first

AGENTS.md in the repo root. runs/results.tsv is append-only — never rewrite past rows, even to fix typos. Append a corrective row instead.

Invocation

You are launched as a Hermes profile, one shot per session, by the operator:

hermes -p trainer chat -t terminal,file -q "Drain runs/queue/" --yolo

Toolsets: terminal (to invoke scripts/py scripts/train.py), file (to read configs, append ledger, write models). Working directory: the repo root.

Core Directive

One config in, one row out. Every config gets exactly one run; every run gets exactly one row in results.tsv regardless of success or failure.

Default Scope

• read data/clean/clean.parquet and every file in runs/queue/
• append rows to runs/results.tsv — never rewrite
• write trained artifacts to models/<run_id>.pkl
• overwrite runs/live/best.json once at the end
• never edit scripts/train.py

Pre-Execution Checklist

• confirm data/clean/clean.parquet exists and is non-empty
• confirm runs/queue/ has at least one config
• confirm runs/results.tsv exists with the header row from AGENTS.md (create it if missing)
• detect GPU availability once via scripts/py -c "import torch; print('gpu' if torch.cuda.is_available() else 'cpu')" (or the script's built-in detection); pass the result to train.py --device <gpu|cpu>

Execution Contract

For each config file in runs/queue/, in lexicographic order:

• run scripts/py scripts/train.py --config <path> --data data/clean/clean.parquet --models-dir models/ --device <gpu|cpu>
• always invoke scripts/py (not bare python) so the venv is picked up regardless of shell activation
• the script handles: load → fit → eval → save model → emit a one-line JSON summary on stdout
• parse the summary; append one row to runs/results.tsv per the schema in AGENTS.md
• on script failure: append a row with status: failed and the stderr tail in secondary_metrics_json; do not retry
• move the processed config from runs/queue/ to runs/done/ (create if missing)

After the queue is drained:

• read runs/results.tsv, filter status=ok, pick the row with the best primary_metric_value (max for roc_auc/f1, min for rmse/mae)
• write that row's contents to runs/live/best.json
• if no ok rows, do not write best.json — surface the all-failed condition

Required Final Report

Your final stdout (consumed by the operator):

• count of runs attempted, succeeded, failed
• the winning run_id and its primary metric value
• one line per failure with the stderr tail (if any)
• absolute path to the winning model under models/ — resolve via `realpath models/<run_id>.pkl` and quote that exact output; never type a path you did not just print

Escalation Triggers

Stop and report back instead of improvising if:

• clean.parquet is missing, empty, or schema-mismatched against what the configs expect
• train.py exits non-zero before producing any output for a config (you still append a failed row, but stop the queue)
• every run failed (no best.json written; operator decides next step)
• a row you are about to append would duplicate an existing run_id — the queue collided; abort