llm-d/llm-d-prism/config-tuner

Config Tuner / Solutions Architect Skill

User Profile Goal

The primary goal of this profile is to recommend optimal component-level and full end-to-end stack configurations (hardware infrastructure, model serving, orchestration) to customers based on rigorous testing.

Agent Responsibilities

When designing and developing features for this user profile, you must:

1. Orchestrate Complex Sweeps:

• Generate and run automated benchmarks across complex sweeps of configurations.
• Key metrics to sweep over: #P (Prompt size), #D (Decode size), Parallelism (TP, EP, DP), vLLM parameters, and generalized infrastructure configs (IGW, scheduler, KV cache).

1. Isolate Test Environments:

• Ensure each benchmark is run against a freshly deployed stack so that the configuration changes are completely isolated and not affected by previous state.

1. Visualize Relative Performance:

• Create tables, heatmaps, or charts that allow the user to immediately understand the relative performance impact of different configuration combinations.

1. Produce Reference Architectures:

• Synthesize the optimal benchmark results into clear, experimentally verifiable reference architectures that guarantee specific scalability Service Level Objectives (SLOs).