tiiuae/ghaf/ghaf-hw-test

Ghaf Hardware Test Automation Skill

This skill automates the hardware testing workflow for Ghaf Framework devices using the ci-test-automation Robot Framework test suite. It supports running tests on locally connected devices, flashing images, analyzing test failures, and proposing code fixes.

When to Use This Skill

Use this skill when the user wants to:

• Run hardware tests on a connected Ghaf device (darter-pro, jetson-agx, lenovo-x1, etc.)
• Flash a new Ghaf image to a device
• Analyze test failures and identify root causes
• Generate fix proposals for failing tests
• Run a test-fix loop until all tests pass (with optional auto build+flash)

Supported Devices

The two columns are different names for a reason. x86 laptops share the generic intel-laptop-* image — modules/reference/hardware/intel-laptop/ passes display, network and audio through by PCI class wildcard instead of per-device IDs, so one image covers the fleet. The Robot suite, however, still branches on the physical machine: Robot-Framework/ config/variables.robot compares DEVICE_TYPE against the exact lowercase strings above to decide IS_LAPTOP, so a mismatched value silently changes which tests apply.

Jetson entries name the -from-x86_64 variants because the native aarch64 attributes are not exposed in packages.x86_64-linux; naming them from a build host resolves to nothing.

Available Commands

Run these commands from the Ghaf repository root:

# Check device connectivity and readiness
.github/skills/ghaf-hw-test/ghaf-hw-test status --device darter-pro --ip <IP>
 
# Run tests on a device
.github/skills/ghaf-hw-test/ghaf-hw-test test --device darter-pro --ip <IP>
.github/skills/ghaf-hw-test/ghaf-hw-test test --device Orin-AGX --ip <IP> --tag boot
.github/skills/ghaf-hw-test/ghaf-hw-test test --device darter-pro --ip <IP> --tag pre-merge
 
# Flash a device with a new image
.github/skills/ghaf-hw-test/ghaf-hw-test flash --device darter-pro --drive /dev/sda
.github/skills/ghaf-hw-test/ghaf-hw-test flash --device Orin-AGX --recovery
 
# Analyze test results and propose fixes
.github/skills/ghaf-hw-test/ghaf-hw-test analyze
.github/skills/ghaf-hw-test/ghaf-hw-test analyze --results /tmp/test_results
.github/skills/ghaf-hw-test/ghaf-hw-test analyze --propose-fixes
 
# Full workflow: test → analyze → fix → build → flash → retest
.github/skills/ghaf-hw-test/ghaf-hw-test run --device darter-pro --ip <IP> --fix-loop --drive /dev/sda
 
# Non-interactive mode (for AI agents)
.github/skills/ghaf-hw-test/ghaf-hw-test -y run --device darter-pro --ip <IP> --fix-loop --drive /dev/sda

Global Options

Typical Workflow

1. Check Device Connectivity

Before running tests, verify the device is reachable:

.github/skills/ghaf-hw-test/ghaf-hw-test status --device darter-pro --ip <IP>

This checks SSH connectivity and reports device readiness.

2. Run Tests

Run the pre-merge test suite (default):

.github/skills/ghaf-hw-test/ghaf-hw-test test --device darter-pro --ip <IP>

Or run specific test tags:

# Boot tests only
.github/skills/ghaf-hw-test/ghaf-hw-test test --device darter-pro --ip <IP> --tag boot
 
# Performance tests
.github/skills/ghaf-hw-test/ghaf-hw-test test --device darter-pro --ip <IP> --tag performance

3. Analyze Failures

If tests fail, analyze the results:

.github/skills/ghaf-hw-test/ghaf-hw-test analyze --propose-fixes

This parses /tmp/test_results/output.xml and:

• Identifies failing tests
• Maps failures to Ghaf module paths
• Generates actionable fix proposals

4. Apply Fixes and Re-run

Review the proposed fixes, apply them, then re-run tests:

# Edit the files suggested in the fix proposals
# Then re-run tests
.github/skills/ghaf-hw-test/ghaf-hw-test test --device darter-pro --ip <IP>

5. Fix Loop Mode (Automated)

For automated test-fix iterations with build+flash:

.github/skills/ghaf-hw-test/ghaf-hw-test run --device darter-pro --ip <IP> --fix-loop --drive /dev/sda

This will:

1. Run tests
2. If failures occur, analyze and output fix proposals
3. Wait for fixes to be applied (or proceed immediately with --yes)
4. Build and flash the new image (if --drive or --recovery was specified)
5. Wait for the device to boot and become reachable via SSH
6. Re-run tests
7. Repeat until all tests pass or max iterations (5) reached

Without --drive/--recovery, the loop skips the build+flash step (useful when using nixos-rebuild externally).

Flashing Devices

x86 Devices (darter-pro, lenovo-x1, dell-7330, nuc)

# Build and flash in one step
.github/skills/ghaf-hw-test/ghaf-hw-test flash --device darter-pro --drive /dev/sda

Jetson Devices (Orin-AGX, Orin-NX)

# Put device in recovery mode (hold recovery button while powering on)
# Then run guided flash
.github/skills/ghaf-hw-test/ghaf-hw-test flash --device Orin-AGX --recovery

Test Tags Reference

Configuration

Device profiles and defaults are stored in .github/skills/ghaf-hw-test/config.yaml.

That file is shared, so it holds only what is true for the project: which image a device takes, which name the Robot suite expects, how it is flashed. Anything specific to the hardware on your desk -- host_ip, ssh_identity, netboot_iface, netboot_mac, flash_drive, serial_device -- belongs in `config.local.yaml` beside it, which is gitignored. Copy config.local.yaml.example to get started, or point GHAF_HW_TEST_LOCAL_CONFIG elsewhere.

The local file is merged over the shared one per device and per field. A null in the local file does not blank a value in the shared one, so a partly-filled override is safe, and you can add devices that do not appear in config.yaml at all.

You can override defaults via command-line arguments or by editing the config file. Each device has a boot_wait_seconds setting that controls how long wait_for_device polls SSH after flashing.

Output Files

Test results are stored in /tmp/test_results/:

• output.xml - Robot Framework results (parsed by analyze)
• log.html - Detailed test log
• report.html - Test summary report

Integration with smoke-test Devshell

This skill wraps the existing robot-test command from the smoke-test devshell. You can also use the devshell directly:

nix develop .#smoke-test
robot-test -i <IP> -d darter-pro -p ghaf

Tips for AI Agents

Important: The --ip address is the device's external/reachable IP, not an internal VM address. Always ask the user for the device IP before running commands — do not guess or use placeholder addresses.

When operating in fix-loop mode:

1. Use `--yes` to skip interactive prompts: ghaf-hw-test -y run ...
2. Use `--drive` to auto-rebuild and flash after applying fixes
3. Read the fix proposals generated by analyze --propose-fixes
4. Locate the relevant Ghaf modules using the paths in the proposals
5. Make minimal, targeted fixes - change only what's needed
6. Re-run the specific failing tests first before running the full suite
7. Check the test code in ci-test-automation if the failure is unclear

Common Failure Patterns

Prerequisites

• Ghaf repository cloned
• Device connected via network (SSH) or serial
• For flashing: physical access to device
• Nix with flakes enabled