ripmilla/netwalk/netwalk-scan

netwalk-scan

Part of the netwalk read-only network survey toolkit. Toolkit lives at {{TOOLKIT}}.

The goal is a complete, honest picture of the target network — and honesty includes writing down what you could not reach. A survey that quietly stops at the first ring of neighbours and presents itself as complete is worse than one that says "6 of 9 devices reached, here is why".

Read-only, and it is enforced

Every command goes through netwalk_exec.py, which checks it against a per-vendor read-only allowlist before it leaves the machine. Config writes, counter clears, service restarts, reboots and shell metacharacters are refused by the tool. You never SSH to a surveyed device directly — always go through the wrapper, so the guarantee holds and every command lands in the evidence log.

If the gate blocks something you believe is genuinely read-only, do not work around it. Either pick a different command, or add it to the allowlist in scripts/netwalk_policy.py, run python3 {{TOOLKIT}}/tests/test_policy.py, and note the change. Never bypass the wrapper.

Before you touch anything

Ask, and do not guess:

1. What is the target? An entry device (IP), a subnet, a site name, "my whole office"? You need

at least one device you can log into — netwalk crawls from a device. It can also sweep an address range, but only one the owner has explicitly authorised; see Sweeping a range below.

1. Whose network is it? If it is a customer's, confirm the user is authorised to log into this

equipment today. Record what they say in site.scope_note — it goes in the report.

1. Anything off limits? Production boxes that must not even be logged into, a maintenance

window, a device that falls over when you open a session. Respect it and record it under coverage.not_covered.

1. How far? Default is exhaustive: keep hopping until every reachable neighbour has been

visited. On a big site, say up front roughly how many hops that might be and check in.

Pick a site slug (acme-hq). Everything for the engagement lands in ~/.netwalk/sites/<slug>/ — outside the installed toolkit, so an upgrade cannot delete it.

The shape of the whole thing

The crawl and the login form are one loop, not two phases. Every hop turns up devices nobody mentioned, and the person who knows what they are is at the browser, not in the conversation:

scan a device  ──►  new neighbours discovered
      ▲                      │
      │                      ▼
      │            put them ALL on the login form  (netwalk_cred.py request --round N)
      │            each card: credential, or "I don't know", or "not ours"
      │                      │
      └──────  read the answers back (answers) ──┘   log into what you were given

Never drop a discovered host because you do not recognise it. It is tempting to filter an ARP table down to the entries whose OUI looks like infrastructure and quietly skip the rest. Do not: an unrecognised MAC is not evidence that a device is uninteresting, it is evidence that you cannot identify it — which is precisely the question the form exists to ask. This has already gone wrong once in the field: a hypervisor was left off the form because its OUI was not in a lookup table, and it was the single most important host on that VLAN.

If a subnet has more hosts than one page can sensibly carry, put the infrastructure-looking ones on first, then say in the same message exactly how many you left off and on what basis, and offer to add them. Silent filtering and honest triage look identical in the output; only one of them is.

Start netwalk_cred.py serve once at the beginning and leave it running for the whole survey. Each round, push that round's discoveries into the open page with add — the user keeps one tab, keeps one URL, and fills things in at their own pace while the crawl carries on. Never stop and restart the form to add a device: the URL changes and the user loses the page they had open. Cards the user has already answered are marked as such and carry their previous answer; new ones are badged NEW this round. Pass --round N so the header says which pass this is.

Three answers are not credentials and all three are useful:

Read every answer back with answers, which prints IP, port, username, management URL, jump host, tenant and any question you attached — and no secret:

python3 {{TOOLKIT}}/scripts/netwalk_cred.py answers --site acme-hq

Keep going until a round turns up no device the user has not already ruled on. That is the termination condition — not "enough devices", and not "the first ring of neighbours".

The loop

For each device, in this order:

1. Get in. No credential yet → hand off to netwalk-login. Confirm with:

python3 {{TOOLKIT}}/scripts/netwalk_exec.py probe --site <slug> --host <id>

1. Identify the vendor before running anything else. The probe output usually tells you.

Wrong vendor means wrong commands and a pile of syntax errors that look like access problems.

1. Run the discovery pack for that vendor:

``bash python3 {{TOOLKIT}}/scripts/netwalk_exec.py run \ --site acme-hq --host gw01 \ --cmd-file {{TOOLKIT}}/scripts/packs/mikrotik.discovery.txt \ --evidence ~/.netwalk/sites/acme-hq/evidence.jsonl ``

Packs exist for mikrotik, cisco, aruba, hp, fortinet, linux, windows.

Controller-managed estates do not get crawled device by device. UniFi and Omada both know every device they adopted, so read the controller once instead of SSHing into a hundred APs:

``bash python3 {{TOOLKIT}}/scripts/netwalk_unifi.py collect --site acme-hq --host unifi-controller --out unifi.json python3 {{TOOLKIT}}/scripts/netwalk_omada.py info --site acme-hq --host omada-controller python3 {{TOOLKIT}}/scripts/netwalk_omada.py collect --site acme-hq --host omada-controller --out omada.json ``

Run Omada's info first: it hits /api/info, which needs no credential, so it separates "wrong address" from "wrong credential" - the two failures that look identical from the outside. Both adapters take --via user@host when the controller only answers from inside the site. For a vendor with no pack, use the unknown profile (show/display/get/print only) and add commands one at a time with --cmd, checking each with netwalk_exec.py check --vendor ... --cmd .... Some commands in a pack will not exist on a given model — a failed command is normal, not a reason to stop.

For a config export, always add --out <file>. That writes the full text to a 0600 file and prints only a summary — a config is full of PSKs, community strings and password hashes, and anything printed reaches you, the model API and the transcript. Secret-shaped values in printed output are masked as <redacted> as a backstop, but --out is the actual control.

1. Map the output into the scan record, ~/.netwalk/sites/<slug>/scan-<YYYY-MM-DD>.json, against

{{TOOLKIT}}/schema/netwalk-record.schema.json. Per device, capture at minimum:

• identity: hostname, model, serial, OS + version, uptime, role
• every interface: name, description, admin/link state, speed, IPs, VLAN/PVID, error and drop

counters, link-down count (the best cable-fault signal there is), and the MAC/CAM table learned on that port

• the device's own neighbour table (/ip neighbor print detail, show lldp neighbors detail

…). Run it on each device — do not just record what its parent saw about it. Skipping this is how phantom topology gets into diagrams.

• ARP table, DHCP leases, VLANs with tagged/untagged ports, routing table
• for APs: every SSID with its security mode, VLAN, band/channel/width and client count
• for Linux/Windows: running and failed services, listening sockets

1. Hop. For every neighbour not yet visited, resolve its vendor and credential and repeat.

Try the credential that worked on the previous hop first — one account per site is the common case. If it fails, that one device goes to netwalk-login; do not stall the whole crawl. Keep going until the frontier is empty. A neighbour found at hop 4 gets visited at hop 5.

1. Record dead ends honestly. Unreachable, no CLI, credential refused, user said don't:

reachable: false plus a real unreachable_reason, and move on.

Before writing a device off, collect the open questions and send them back through netwalk-login in one batch — the credential form takes --ask, so "which port is the controller on", "is there a jump host", "what is this device on ether16" are all questions the user answers in the browser in one pass. Asking them one at a time in the chat is the slow way, and the answers are not secrets so answers reads them straight back.

Sweeping a range

The crawl finds the managed estate — whatever speaks LLDP, appears in an ARP table, or holds a DHCP lease. It misses the printer nobody remembers, the old server on a static address, the second firewall someone left plugged in. A sweep is the other half of the picture, and on most sites it is where the surprises are.

It cannot run until the owner has authorised the range, by name. That is enforced in code:

python3 {{TOOLKIT}}/scripts/netwalk_sweep.py authorize --site acme-hq \
  --range 10.2.30.0/24 --range 10.2.40.0/24 \
  --authorized-by "Khun Somchai, IT manager, by phone 2026-08-22" \
  --exclude 10.2.30.99          # a box he asked us to leave alone

The name goes in scope.json and comes out again in the report. There is no --force: a range outside the scope is refused, a supernet of an authorised range is refused, public address space needs a second explicit flag, and anything larger than a /16 is refused outright. If you find yourself wanting to get past the gate, the answer is to ask the owner, not to edit the file.

# which addresses answer at all - TCP connect to a few common ports, plus one ping
python3 {{TOOLKIT}}/scripts/netwalk_sweep.py hosts --site acme-hq --range 10.2.30.0/24
 
# what those addresses are listening on - ~68 well-known TCP ports by default
python3 {{TOOLKIT}}/scripts/netwalk_sweep.py ports --site acme-hq --target 10.2.30.99 \
  --profile standard            # or --profile quick, or --ports 22,80,8000-8010
 
# fold the results into the scan record and list what is not in devices[] yet
python3 {{TOOLKIT}}/scripts/netwalk_sweep.py record --site acme-hq \
  --record ~/.netwalk/sites/acme-hq/scan-2026-08-22.json

Three things to know before you read the output: