kwongfuk/codex-skills/palmetto-gpu-usage-advisor

Palmetto GPU Usage Advisor

Use this skill when the user wants a current Palmetto GPU view and a plain recommendation such as:

• which card type is least congested now
• whether 1, 2, 4, or 8 GPUs are realistic right now
• which nodes currently have the most free GPUs
• whether H100, H200, or A100 is the best current target

Workflow

1. Run the usage inspector:

python /home/gguo/.codex/skills/palmetto-gpu-usage-advisor/scripts/inspect_gpu_usage.py

1. Read the output in two layers:

• per-GPU-type usage summary
• final Current card suggestions section

1. Explain the result in user terms:

• cluster-wide usage by card type
• whether the cluster is fragmented or has idle nodes
• what to request now for 1, 2, 4, or 8 GPU jobs

Output Interpretation

• idle: at least one node of that GPU type is completely free
• mixed: nodes exist with some GPUs free, but those nodes are fragmented
• full: no GPUs free on those nodes
• max_free_on_one_node: the largest single-node GPU block currently visible for that card type

Decision Guidance

• Prefer card types with more idle nodes for new training jobs.
• Prefer card types with larger max_free_on_one_node for multi-GPU single-node jobs.
• For 4 or 8 GPU jobs, treat fragmentation as a real blocker even if total free GPUs across the cluster look high.
• If the user needs an exact job-fit decision for a specific sbatch script, switch to $palmetto-slurm-fit-check.

Notes

• The inspector handles the local scontrol shared-library issue automatically.
• The summary is partition-specific and defaults to work1.
• This skill gives a current cluster recommendation, not a guaranteed start time.