amdresearch/aup-learning-cloud/deploy-aup-learning-cloud

Deploy AUP Learning Cloud

Stand up a multi-node AUP Learning Cloud cluster with Ansible, AMD GPU access, shared storage, and the JupyterHub Helm chart.

Use the skill scripts guide as the source of truth for the complete generator-first command sequences and generated files. Use deploy/README.md for the human direct-edit workflow, operational background, and troubleshooting. This skill defines the interview and safety gates around the generated procedure.

Prerequisites

• A checkout of aup-learning-cloud on the operator machine.
• Ubuntu 24.04, a reserved controller IP, internet access, and Ansible.
• Physical node, network, storage, and authentication details from the user.
• Passwordless root SSH to every managed host in the SSH topology.

Site values and secrets don't ship in the repository. Generate them locally and never put tokens, private keys, or credentials in tracked files.

Phase 1: Interview

Ask for an explicit topology choice before collecting other details or touching machines. Never infer the choice from the hardware.

Then collect and confirm:

1. Courses and notebook resources.
2. Controller hostname, static IP, subnet, gateway, and DNS.
3. For SSH, every managed hostname and IP. Don't ask for a GPU host list;

generation discovers GPU hosts over SSH.

1. For PXE, the controller NIC, web port, rootfs SSH public key, and whether

diskless agents have AMD GPUs. This explicit yes or no is the sole PXE GPU policy input because agent hardware can't be inferred from the controller.

1. Shared storage location and the Hub access method.
2. Authentication providers: auto-login, dummy, native, GitHub, or native plus

GitHub. The canonical multi-node example uses native plus GitHub.

Confirm detected GPU product labels before mapping them to accelerator keys in the runtime values.

Phase 2: Generate

Create a fresh schema and fill only its current fields. Run the generator rather than writing inventory or GPU policy by hand.

The schema temporarily accepts auth_mode as a one-release generator compatibility input. It isn't Helm configuration. The generator always writes the selected providers as canonical custom.auth flags; see the migration table in reference.md.

For SSH, generation performs read-only discovery on every managed host and publishes canonical artifacts after GPU evidence is consistent.

For PXE, generation writes the canonical inventory, PXE vars, runtime overlay, and GPU resolution report directly as desired deployment inputs. Their existence does not prove rootfs provisioning succeeded. Review, install, and validate those files, then run the controller playbook with the canonical inventory and PXE vars; the playbook must complete successfully before proceeding.

The generated runtime overlay includes canonical custom.auth, custom.runtimeLimitEnabled: true, and custom.quota.enabled: true. It also maps detected GPU labels to accelerator selectors, defines notebook images and shared storage, and keeps resource visibility tied to custom.teams.mapping and its fallback groups regardless of the selected authentication providers.

Follow the complete topology command sequence in the skill scripts guide. Don't substitute the human direct-edit SSH workflow from deploy/README.md; the skill's SSH path remains generator-first and discovers GPU policy from managed-host evidence.

Phase 3: Validate and execute

Install the canonical generated inventory and runtime overlay into the checkout, then run the topology's exact validator command from the skill scripts guide. The validator inputs are:

• --repo
• --topology
• --inventory to validate generated host booleans
• --gpu-resolution with --inventory for generated-artifact consistency
• both --values files
• --pxe-vars for PXE only

A human direct inventory can be validated by itself with unquoted auto, true, or false. A resolution report requires an inventory, and that pairing accepts only generated boolean values. Supply both in this generator-first workflow so the validator checks their consistency. The skill resolves every host to true or false and never generates auto.

Stop on validation failure. After a clean result, continue with the topology's Ansible, device plugin, and Helm commands in the skill scripts guide. Treat the AMD device plugin and ROCm node labeller as infrastructure prerequisites owned outside AUPLC. Verify both existing DaemonSets and advertised GPU capacity before Helm; do not install these privileged components as part of the AUPLC procedure.

Keep the GPU contract distinct from storage configuration. The installer, Ansible role, and PXE controller install AMD's amdgpu-insecure-instinct-udev-rules package at the pinned version 30.30.4.0-2341068.24.04. Its rule sets mode 0666 only on /dev/kfd and DRM renderD* nodes. It does not change card*, which retains normal system policy, observed as root:video 0660.

Device-plugin allocation is a separate visibility layer. Only amd.com/gpu requests receive allocated GPU devices, and the plugin does not change Unix inode permissions. AUPLC Hub adds no GPU supplemental group; none is required for the tested ROCm compute path. singleuser.fsGid: 100 is for shared storage only.

Phase 4: Verify

Check that all expected nodes are Ready, the GPU labels and allocatable resources match the generated policy, the storage class is available, and JupyterHub pods are healthy. Open the Hub, start a CPU notebook, verify persistence, then start a GPU notebook and confirm it schedules on a GPU node.

Safety

Pause for explicit user confirmation before rebuilding a PXE rootfs, changing NFS exports, changing firmware boot settings, resetting a cluster, deleting a node, or uninstalling a Helm release.

Never commit or push deployment secrets. Preserve the four AUP Learning Cloud attribution layers described in the project AGENTS.md if Hub or chart sources are changed.

Reference

• Complete skill command sequences
• Human deployment and troubleshooting
• Skill-specific summary