walrusquant/sports-analytic-skills/environment-setup

Environment Setup

Outcome

Produce an isolated environment, explicit dependency record, and machine-readable verification report for the selected analysis. Install only the public scientific libraries and loaders the work actually requires.

Read the verification checklist while verifying a new environment and the troubleshooting guide when an install, import, output, or loader probe fails.

When to Use This Skill

Use when:

• onboarding a new machine or project for sports analysis;
• diagnosing broken imports, wrong interpreters, or conflicting packages;
• locking a reproducible environment before serious modeling;
• verifying that selected skill helpers and loaders work offline at --help.

Do not use this skill to:

• install the optional repository toolkit / bridge path → sports-ds-bridge;
• choose a public data source → data-sources;
• run EDA or modeling once the environment already works.

Plan before installing

Record operating system/architecture, Python version and executable, analysis tasks, named loaders, required file/plot formats, CPU/memory/storage constraints, network/proxy constraints, and whether exact reproduction or flexible minimum versions are needed. Separate core packages from optional loaders and dev tools.

Create an isolated environment

python3 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install numpy pandas scipy scikit-learn statsmodels matplotlib pyarrow

On Windows PowerShell use .venv\Scripts\Activate.ps1. Use python -m pip so the installer follows the active interpreter. Do not modify the system Python.

Install public loaders only when the source plan requires them:

python -m pip install nflreadpy
python -m pip install pybaseball
python -m pip install sportsdataverse

Do not install every loader “just in case”; optional native/transitive dependencies add conflicts and weaken reproducibility.

Verification ladder

Work from cheapest/local to bounded/networked. Stop at the first unexplained failure; later successes do not erase it.

1. Record python --version, executable path, platform, and python -m pip --version.
2. Confirm Python and pip resolve inside the same environment.
3. Import the minimum scientific packages and record resolved versions.
4. Run every selected helper or loader's --help; help must not make a network call.
5. Test required local outputs: JSON, Parquet, and images as applicable.
6. If network is authorized, run one small bounded public-data sample.
7. Save the verification report; freeze dependencies only after success.
8. Recreate from the lock in a clean environment for high-stakes reproducibility.

python /path/to/environment-setup/scripts/verify_install.py
python /path/to/environment-setup/scripts/verify_install.py \
  --packages numpy,pandas,sklearn --out data/environment.json
python -m pip freeze > requirements-lock.txt

The portable verifier accepts comma-separated import names through --packages. Import names can differ from distribution names (sklearn versus scikit-learn); document both when that matters.

The verifier checks the current interpreter, python -m pip, and the requested imports only. Its JSON lists helper, output, network, and lock-recreation checks as not run; OK is therefore not full environment sign-off. A standard-library venv is detected from Python prefixes, but other environment managers may not be. Run and document the remaining ladder steps separately. On a completed probe, stdout is exactly one JSON document; package import output is captured inside the relevant package result. With --out, the same JSON is also written to that path, and no human status line is mixed into stdout.

Verification matrix

Troubleshooting by layer

Capture the complete error, command, interpreter, package versions, and minimal reproduction before changing dependencies. Never silently substitute a package or broaden versions until something happens to install.

Reproducibility policy

Keep a human-edited dependency specification separate from a resolved lock. Record Python minor version, OS/architecture, package versions, loader versions, and relevant environment variables without secrets. Treat caches as performance artifacts, not provenance; data snapshots need their own source metadata.

Lock files are platform-sensitive. For multiple supported platforms, maintain and test explicit platform locks or a reproducible resolver workflow. A freeze from a contaminated environment is not a dependency design.

Worked example

For a new NFL analysis: create .venv; install scientific packages and nflreadpy; run the verifier for numpy,pandas,sklearn,nflreadpy; run every selected helper's --help; perform a one-season or smaller authorized sample; write environment.json; inspect it; freeze dependencies; then repeat the local checks in a clean environment before claiming setup is reproducible.

Output contract and integrity rules

The complete sign-off should report Python version/executable, platform, requested distributions and import names, resolved versions, helper/loader probes, skipped network checks, output round trips, exact failures/remediations, and paths to report and lock. The portable verifier emits only the subset stated above and enumerates the checks it did not run.

1. Use an isolated environment and python -m pip.
2. Never hide failed imports, skipped checks, or platform limitations.
3. --help must remain offline; network samples must be bounded and authorized.
4. Freeze only after verification; verify the lock by recreation when required.
5. Use verify_checklist.md for sign-off and troubleshooting.md for failure routing.