amd-agi/primus/backend-gap-report

Backend Gap Report & Shared Dashboard

Use this skill when the user asks to compare a Primus backend with upstream code and wants stable deliverables instead of ad hoc notes, or when integrating a different content type (e.g. weekly engineering reports) into the shared Primus engineering dashboard that lives under tools/backend_gap_report/.

Default Outputs

Unless the user explicitly asks otherwise, produce:

• detailed report: docs/backend-gap/reports/<backend>/<target>/report.md
• one-page summary: docs/backend-gap/reports/<backend>/<target>/summary.md
• publish-time PDF copies of both reports with the same basename
• Dashboard metadata: docs/backend-gap/dashboard-data/reports/<backend>-<target>.json
• Refreshed dashboard index: docs/backend-gap/dashboard-data/index.json

(generated build artifact, not committed)

If legacy report files already exist, update them in place instead of renaming them. For new report series, keep the default artifact names unsuffixed. Only add a language suffix when a non-default variant coexists or when a legacy file pattern already established one.

Required Inputs

Resolve these inputs before writing:

1. backend
2. Local source path or submodule path
3. Upstream repository and comparison ref (main, tag, release commit, etc.)
4. Authoritative dependency evidence files
5. Primus integration directories for that backend

If any of these are ambiguous, ask the user before proceeding.

Workflow

1. Establish the Comparison Baseline

Verify:

• local version or pinned commit
• upstream target commit
• commit dates
• commit gap
• merge-base relation
• diff size

Use git facts from the actual local checkout. If the upstream ref might be stale, fetch it first.

2. Verify Dependency Facts

Prefer authoritative sources in this order:

1. package metadata such as pyproject.toml
2. runtime or CI requirements files
3. workflow install commands
4. release notes or release docs
5. README install examples

Do not treat README examples as stronger evidence than workflow or package metadata.

3. Verify Directory and Capability Changes

Check the actual tree or diffs for areas such as:

• model directories
• distributed runtime
• experiments
• components
• docs
• workflows
• tests

Only write facts you can confirm from the repository state.

4. Verify Primus Coupling

Identify direct Primus dependencies on upstream internal paths, such as:

• imports from backend internals
• monkey patches
• trainer or adapter coupling
• config object dependencies

If the report discusses upgrade cost or blast radius, ground it in these concrete coupling points.

5. Write the Reports

Default report set:

• detailed comparison report
• one-page summary

When continuing an existing report series, preserve the established naming pattern and structure. Keep summary and detailed versions factually consistent.

6. Export PDFs

Use the shared stylesheet at tools/backend_gap_report/templates/pdf-report.css.

Preferred command pattern:

pandoc "docs/backend-gap/reports/<backend>/<target>/<report>.md" --from gfm --standalone \
  --css "tools/backend_gap_report/templates/pdf-report.css" \
  --metadata pagetitle="<title>" \
  --pdf-engine=weasyprint \
  -o "/tmp/backend-gap-pdf/<backend>/<target>/<report>.pdf"

Use pagetitle, not title, to avoid duplicate visible titles in the PDF.

Note:

• Markdown reports are the tracked source artifacts in the repository.
• PDF files are generated for publishing and are not tracked in the repository.

7. Emit Dashboard Metadata

Create a metadata JSON file under docs/backend-gap/dashboard-data/reports/.

The metadata must:

• be relative to the docs/ root
• reference publish artifact paths that can be generated
• map to markdown source files that exist in the repo
• include backend identity, refs, stats, highlights, and artifact links
• include dashboard_summary with concise homepage-ready facts when the report has enough evidence

Use the schema and examples in reference.md and examples.md.

8. Refresh the Dashboard Index

Run:

python3 tools/backend_gap_report/build_dashboard_index.py

This validates metadata files and rewrites docs/backend-gap/dashboard-data/index.json. That index is a generated build artifact — do not commit it; commit only the per-report metadata under dashboard-data/reports/.

8.5 Build the Standalone Site Bundle

For publishing or local preview, build the standalone dashboard bundle from the site templates plus generated artifacts:

python3 tools/backend_gap_report/build_site_bundle.py --output-dir /tmp/backend-gap-site

This assembles a publishable site root from:

• tools/backend_gap_report/site/
• docs/backend-gap/dashboard-data/
• docs/backend-gap/reports/

This command rebuilds dashboard index, generates PDF files from markdown report sources, builds the standalone bundle, and validates bundle integrity in one run.

9. Final Verification

Before finishing:

• verify the Markdown files exist
• verify the metadata JSON exists
• run the site bundle build successfully
• check lints for edited files when applicable

Update Semantics

• For the same backend + target, update the existing files in place.
• Re-running the same report series should overwrite or refresh:
• report.md
• summary.md
• docs/backend-gap/dashboard-data/reports/<backend>-<target>.json
• docs/backend-gap/dashboard-data/index.json
• PDF artifacts are regenerated during standalone site bundle build.
• Create new sibling paths only when the backend or target changes.
• Running the skill does not trigger background automation by itself; updates happen only when the agent is explicitly asked to run the workflow in a task.

Output Rules

• Keep the default report set as the primary artifact set.
• Do not over-emphasize the default language in filenames, labels, or dashboard copy.
• Add language suffixes or labels only when needed to distinguish a non-default variant or preserve a legacy series.
• Keep facts synchronized across detailed and summary reports.
• Prefer concise factual wording over long explanations.
• Do not invent missing versions or release claims.
• If a file contains comments only, do not call it "empty".

Dashboard Rules

• Dashboard source templates live under tools/backend_gap_report/site/.
• docs/backend-gap/ stores generated data and report artifacts, not the site templates.
• The deployed site root is a generated standalone bundle.
• Dashboard source data lives under docs/backend-gap/dashboard-data/reports/.
• docs/backend-gap/dashboard-data/index.json is a generated build artifact —

not hand-maintained and not committed (only the per-report metadata under dashboard-data/reports/ is tracked).

• Artifact paths in metadata are relative to the standalone published site root.
• Backend deep-dive cards should render structured dashboard_summary fields when present. Do not make the frontend infer important conclusions from Markdown prose.

Backend Dashboard Summary

When generating backend metadata, add a dashboard_summary object whenever the comparison has confirmed facts worth surfacing on the homepage:

• headline: one sentence explaining why this backend comparison matters
• recommendation: concise action label such as monitor, plan sync, or urgent sync
• why_it_matters: 3-5 concise facts or risks
• feature_deltas: notable upstream capabilities added since the Primus pin
• dependency_deltas: dependency, runtime, CUDA, ROCm, or package-channel differences
• integration_risks: Primus-specific coupling points or upgrade risks

All dashboard_summary content must be derived from the same evidence used in the detailed report. Do not invent features, versions, or risks to make the dashboard look richer.

Periodic Engineering Report Integration (Weekly + Monthly)

The shared dashboard under tools/backend_gap_report/ surfaces two content types from a single static site:

1. Backend gap reports — owned by this skill, stored under docs/backend-gap/.
2. Periodic engineering reports — the automated Primus reports, on a

weekly cadence under docs/weekly_reports/ and a monthly cadence under docs/monthly_reports/.

Weekly and monthly reports share one schema, one validation core (tools/backend_gap_report/periodic_reports.py), and one combined dashboard index, so the dashboard renders both through a single generic path — cadence is just a data field. Routine report runs should update report data only and should not redesign the dashboard.

Periodic-report data plane

For each cadence directory (weekly_reports or monthly_reports):

• Per-report metadata: docs/<cadence>/dashboard-data/reports/{report_id}.json
• Per-cadence index: docs/<cadence>/dashboard-data/index.json (generated build

artifact, not committed)

• The Markdown report lives at

docs/<cadence>/{report_id}-primus-{weekly|monthly}.md and is not bundled into the published site — the dashboard links to the GitHub-rendered Markdown via report_github_url in each metadata file.

• report_id uses YYYY-Www (weekly, e.g. 2026-W17) or YYYY-MM (monthly,

e.g. 2026-06).

• The combined reports-data/index.json consumed by the frontend is assembled

at bundle-build time from both cadences; it is generated, not committed.

• A cadence directory that does not exist yet is treated as zero reports, so the

monthly plane activates automatically once the first monthly report lands.

Required fields in each report metadata JSON (identical for both cadences):

• report_id, content_type (weekly-report or monthly-report), title
• report_path, report_github_url
• time_window (timezone, start, end)
• generated_at, merged_pr_count, category_breakdown
• megatron_status, torchtitan_status, primus_turbo_status
• recommendations (keys: megatron, torchtitan, primus_turbo)
• key_findings (non-empty list of short, factual strings)

Index builders

Refresh a single cadence index directly:

python3 tools/backend_gap_report/build_weekly_reports_index.py
python3 tools/backend_gap_report/build_monthly_reports_index.py

Run the full site bundle (which assembles the combined index and validates every plane) to publish:

python3 tools/backend_gap_report/build_site_bundle.py --output-dir /tmp/primus-dashboard-site

Presentation rules for the shared dashboard