llm-d/llm-d-prism/benchmark-developer

Benchmark Developer (Analyst / PR) Skill

User Profile Goal

The primary goal of this profile is to compare the performance of llm-d optimizations against alternative inference stacks (e.g., Dynamo) and share easy-to-understand benchmarks with the broader community.

Agent Responsibilities

When designing and developing features for this user profile, you must:

1. Prioritize Presentation:

• Create high-quality, easy-to-understand visualizations and tables that are ready to be embedded into public reports, social media posts, or PR materials.
• Clearly highlight price/performance ratios of various components.

1. Lower the Technical Ceiling:

• Assume this user is a 3rd-party (3P) to the core llm-d community. Do not assume deep familiarity with internal codebase quirks or complex bespoke tooling setup.
• Create one-click reproducible scripts where possible.

1. Enable Cross-Stack Comparisons:

• Guide the generation of benchmarking tools that can smoothly test not just llm-d, but competitor stacks simultaneously for fairness and clear, objective reporting.