a2ui-project/a2ui/inference-format-optimizer

Inference Format Optimizer

This skill provides procedural workflows, CLI orchestrators, decision guardrails, and subagent protocols for iteratively optimizing A2UI inference formats.

Quick-Start CLI Cheatsheet

All execution scripts live under scripts/ in this skill:

Detailed References

• Scoring & Decision Rules: See references/scoring_model.md for $S_{\text{opt}}$ formula, correctness guardrails, and efficiency caps.
• Subagent Worktree Protocol: See references/subagent_protocol.md for launching subagents in isolated git worktrees.
• Subagent Prompt Template: See templates/subagent_prompt.md for launching pass tasks via invoke_subagent.

The 6-Step Optimization Workflow

1. Analyze History: Inspect past runs in eval/iterative_format_optimizer/history/<format>/ and read eval/iterative_format_optimizer/history_summary.md to avoid repeating past reverted hypotheses.
2. Implement Hypothesis: Modify compiler.py, prompt_generator.py, or parser.py under the format's package: python/a2ui_agent/src/a2ui/inference_formats/express/ for Express, or python/a2ui_agent/src/a2ui/inference_formats/experimental/<format>/ for an experimental format.
3. Run Unit Conformance Tests: Verify code changes pass pytest unit tests.
4. Execute Benchmark Evaluation: Run python scripts/optimize_format.py --format <format>.
5. Evaluate Decision Rules:

• Must pass Pytest and maintain baseline accuracy.
• Code Output Tokens must NOT expand $> +5\%$.
• Keep change if composite score $S_{\text{opt}}$ improves; revert otherwise (git reset --hard HEAD).

1. Archive & Synchronize: Archive run with --archive and update history index using python scripts/sync_history.py.