timewarpengineering/timewarp-nuru/tw-nuru-verify

Nuru consumer verification

Repo tests build against project references, with this repo's Directory.Build.props. A consumer gets none of that. They run dotnet add package TimeWarp.Nuru and copy a snippet from the README. This skill does the same thing and records what happened. (why: task 523, Steven approved the consumer-use check 2026-10-23) (why: Steven 2026-10-23 docs are not the source of truth) (why: task 527 2026-10-25, logging probes follow the removal of UseLogging/UseConsoleLogging)

The source is the authority

The source code, with the #region Purpose and #region Design blocks in each file, defines what Nuru is meant to do. The docs and the README do not. So:

• Probe coverage comes from the shipped public API (public builder and extension methods in

source/, as compiled into the package), plus Purpose/Design regions where they exist.

• A doc snippet that fails to build is a doc defect: the docs are out of date with the source.

It is reported in its own section and does not fail the release verdict.

• The exception: when the source public API says the snippet should work, the failure is a

code defect. The probe for that API shows it and fails the verdict. Example on 4.0.0-beta.5: AddCheckUpdatesRoute() is public in source but the package gives CS0122; UseLogging and UseConsoleLogging were public in source but the generator answered NURU_S999. Task 527 removed them from the public API, so their probes went with them.

It exists because of three failures that got past tests, the REPL and --capabilities:

Run it

skills/tw-nuru-verify/scripts/nuru-consumer-check.sh                         # latest on nuget.org
skills/tw-nuru-verify/scripts/nuru-consumer-check.sh --version 4.0.0-beta.5  # a given release

Run it from a timewarp-nuru checkout with tags fetched (git fetch --tags). The docs step reads the docs at tag v<version>, which are the docs a consumer of that version reads. The full run takes about 90 seconds after the first restore.

Exit codes: 0 means launch, cache, probes and cli all ran and passed. Doc defects do not change the exit code unless you pass --docs-strict, which makes them failing too. 1 means one of those checks failed. 2 means doctor refused to run. No check is skipped: a check that cannot run in this environment is a doctor refusal.

What it does

Doctor. Before it runs, it refuses (exit 2) when:

• a Directory.Build.* or Directory.Packages.props sits above the scratch folder. MSBuild would

import it and turn the consumer build into a repo build. A stale /tmp/Directory.Build.props from an agent session has done this before. Move the file or set NURU_VERIFY_WORK.

• --source is a local feed and the version is already on nuget.org. That is the state#647

pattern. Pack candidates under a unique prerelease version instead.

• the version is not on nuget.org, or the docs tag is missing.
• a selected check cannot run: script(1) is missing for the REPL (pty) cases, nc is missing

for the telemetry (otlp) case, or cli is selected without probes (it searches the core-routing probe app).

Launch. It creates a scratch folder under ~/.cache/nuru-verify/ with its own nuget.config (<clear/> plus one source), NUGET_PACKAGES and HTTP cache. Then it runs dotnet new console and dotnet add package TimeWarp.Nuru --version <v>, and records which source the package came from. Nothing in the run writes to the global cache.

Drive.

• cache: compares the content hash of every package in the private cache (TimeWarp.Nuru and its

whole dependency graph) with the global cache entry for the same id and version. A mismatch is reported as POISONED, with the global entry's source and the folder to delete.

• probes: builds each probes/*.cs as its own consumer project, in one parallel solution build.

Then it runs the cases in probes/cases.tsv. Modes: run (plain process), pty (script(1) types into the REPL with -i), otlp (nc listens on the probe's literal endpoint and the capture must contain the service name).

• cli: dotnet tool install TimeWarp.Nuru.Search into the scratch folder, nuru --version, then

nuru index rebuild and nuru search against the core-routing probe app.

• docs (a doc-defect report, not part of the release verdict): extracts every C# block that contains CreateBuilder from readme.md and

documentation/user/**, and builds each one as a program. A block that calls builder.Map("pattern", handler) without CreateBuilder is built too, after a var builder = NuruApp.CreateBuilder(); line, so the pre-3.0 two-argument Map meets a real receiver (context = builder in snippets.tsv). When a block is followed by dotnet run -- <args> and # Output: <text>, it runs the app and checks the output. Each snippet gets one of three results, and broken is a DOC-DEFECT row with file:line:

• compiles
• broken: the build fails on a Nuru API or on the package. Examples: a NuruAppBuilder member

that does not exist, NURU_S999 Unrecognized DSL method, a route the analyzer rejects, an inaccessible package type, or a call shape that no longer exists. CS1501 (wrong argument count, such as Map with 2 arguments) is always broken. CS1061 and CS0117 (missing member) are broken unless the receiver is a host extension point such as IServiceCollection or ILoggingBuilder, where docs call their own or another package's extensions.

• fragment: it only lacks names it never defined, such as its own handlers, services or types.

Generator errors that follow from those missing names count as fragment too.

Evidence. By default it writes to artifacts/nuru-verify/<version>/:

summary.md                    Release verdict table (launch, cache, probes, cli), then a separate
                              "Docs out of date with source" section: DOC-DEFECT rows with file:line
environment.txt               version, source, docs ref, dotnet, scratch paths
launch/*.log                  dotnet new / dotnet add package transcripts
launch/timewarp.nuru.nupkg.metadata
cache/global-cache-audit.tsv  per package: match / POISONED / not-cached, both hashes
probes/build.log              the solution build
probes/NN-<probe>.log         per case: "$ command", stdout, stderr, exit code
probes/<probe>.build-errors.txt
cli/*.log
docs/build.log, docs/snippets.tsv, docs/<snippet>.run.log

Every log has the command, stdout, stderr and the exit code. Paste summary.md into the PR or the release notes.

Cleanup. It deletes the scratch folder unless you pass --keep. It does not change the global cache. When the cache step reports POISONED, delete the folder it names, then restore again.

Builder API coverage

Coverage comes from the shipped public API in source/timewarp-nuru/, not from the docs. Each probe cites the source type and method it exercises. A public API without a probe is listed as uncovered below, never silently omitted.

Uncovered public APIs (no probe yet):

When a source file has #region Purpose / #region Design, read it before adding or judging a probe: it says what the API is meant to do.

To add a probe, drop probes/<name>.cs (a complete Program.cs) and add rows to probes/cases.tsv:

probe  api  mode(run|pty|otlp)  env(KEY=VAL|-)  args  stdin(\n-separated|-)  exit  contains  not_contains

Where it is invoked