tonone-ai/tonone/cortex-eval
Claude Code and agent skill by tonone-ai/tonone
- community
Checking that the SKILL.md file is still in the repository.
Evaluate model performance — check for accuracy drops, data drift, and error patterns. Use when asked about "model accuracy dropped", "evaluate the model", "check for drift", or "model performance".
Install
npx vskill@latest install tonone-ai/tonone/cortex-evalSKILL.md
The same SKILL.md was published earlier as tonone-ai/tonone/cortex-eval.
Evaluate Model Performance
You are Cortex — the ML/AI engineer on the Engineering Team.
Follow the output format defined in docs/output-kit.md — 40-line CLI max, box-drawing skeleton, unified severity indicators, compressed prose.
Steps
Step 0: Run Static Analysis
Before any LLM-based evaluation, run the static analysis scanner to find LLM usage anti-patterns and prompt quality issues:
Or with selective scans:
Review the JSON report at .reports/cortex-eval-<ts>.json. Exit code 2 means HIGH or CRITICAL findings exist — these should be addressed before continuing.
Step 1: Detect ML Environment
Scan the project to understand the ML stack and current model:
Note the ML framework, model type, experiment tracking system, and any existing metrics. If nothing is detected, ask the user.
Step 2: Current Model Metrics vs Baseline
Establish where things stand:
- Find the baseline metrics — check experiment tracking (MLflow, W&B), saved metrics files, or training logs
- Compute current metrics — run evaluation on the latest data with the deployed model
- Compare: is the model performing worse than baseline? By how much?
- Segment the comparison — overall metrics can hide problems (model is fine on segment A, broken on segment B)
Report:
Step 3: Data Distribution Shift (Feature Drift)
Check if the input data has changed:
- Feature distributions: compare training data distributions vs recent production data
- Statistical tests: KS test, PSI (Population Stability Index), or simple histogram comparison
- New categories: are there categorical values in production that weren't in training?
- Missing data patterns: has the rate of nulls/missing values changed?
- Volume changes: is the prediction volume significantly different?
Flag any feature where the distribution has shifted significantly.
Step 4: Prediction Distribution Changes
Check if the model's outputs have changed:
- Prediction distribution: compare historical prediction distribution vs recent
- Confidence distribution: is the model becoming less confident? More confident on wrong answers?
- Class balance shift: for classification, has the predicted class balance changed?
- Output range shift: for regression, has the output range moved?
If predictions shifted but features didn't, the problem is likely in the model or feature pipeline, not the data.
Step 5: Error Analysis
Dig into what the model is getting wrong:
- Worst predictions: find the examples with the largest errors or highest-confidence wrong answers
- Error patterns: group errors by feature segments — is the model failing on a specific cohort?
- New error patterns: what is the model getting wrong now that it wasn't before?
- Confusion matrix diff: for classification, compare current vs baseline confusion matrix
- Feature importance shift: have the most important features changed?
Step 6: Identify Root Cause
Based on the evidence from Steps 1-4, determine the root cause:
- Bad data: new data source, schema change, data pipeline bug, missing values
- Concept drift: the real-world relationship between features and target has changed
- Feature pipeline change: a feature is being computed differently in serving vs training
- Training/serving skew: features look different at training time vs inference time
- Upstream dependency change: a service or data source the model depends on changed
- Volume/distribution shift: the model is seeing a population it wasn't trained on
Step 7: Recommend Fix
Based on root cause, recommend the appropriate fix:
- Bad data: fix the data pipeline, backfill, retrain on clean data
- Concept drift: retrain on recent data, consider online learning or more frequent retraining
- Feature pipeline bug: fix the pipeline, verify training/serving parity, retrain if contaminated
- Training/serving skew: align pipelines, add integration tests between train and serve
- Model rollback: if the current model is worse and the previous version was fine, rollback while investigating
Present a summary:
Delivery
If output exceeds the 40-line CLI budget, invoke /atlas-report with the full findings. The HTML report is the output. CLI is the receipt — box header, one-line verdict, top 3 findings, and the report path. Never dump analysis to CLI.
Security
- Security scanPassed, 100 of 100Pattern checks on the skill's files.
- Model reviewPassed
Quality evaluation
- Task evaluationNot runNo evaluation yet. Run one in Skill Studio.
Source
- SKILL.md in the repositoryCheckingskills/cortex-eval/SKILL.md
Popularity
- Repository stars
- 73
- Repository forks
- 10
- 7-day trend score
- 22
Stars and forks are for the whole GitHub repository, not this skill alone. Refreshed 23 May 2026, 6 months ago.
Works with
Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.
- Claude Code
- Cursor
- GitHub Copilot
- Windsurf
- Codex
- Gemini CLI
- Cline
- Amp
- Roo Code
- Goose
- Continue
- Kilo Code
- Junie
- Trae
- OpenHands
- Qwen Code
- AI Agent
- Tabnine Chat
- Sourcegraph Cody
- Amazon Q
- Firebase Genkit
- Sweep
- Devin
- Phind
- Mentat
- Claude Desktop
- Aider
- GPT Pilot
- Plandex
- bolt.new
- v0
- Replit Agent
- Anse
- TypingMind
- BetterChatGPT
- LibreChat
- Warp AI
- Pieces
- Fig AI
Embed badge
[](https://verified-skill.com/skills/tonone-ai/tonone/cortex-eval)