drpwchen/textbook-to-note/figure-remap

figure-remap — deterministic document-grounding system

This skill calls scripts in your clone of the textbook-to-note repo. At

install time, replace {REPO} below with the absolute path of the clone.

Architecture (read this before implementation details)

figure-remap is a multi-backend deterministic document-grounding system, not a "crop utility." The system has six explicit layers:

┌─ Contract ──────────────────────────────────────────┐
│  figure_remap.extract() → {status, match_quality,   │
│  hard_fail, file, fig_id, reason}                   │
├─ Policy ─────────────────────────────────────────────┤
│  strict = deterministic + ambiguity-intolerant      │
│  L1 / L2 / L3 hard-fail hierarchy                   │
├─ Junk pre-gate ─────────────────────────────────────┤
│  pregate.verdict() on a QC-passed crop:             │
│  chapter-banner geometry / blank → kill, no retry   │
├─ Backend selection ─────────────────────────────────┤
│  Capability-based: _select_backend()                │
│    "geometric"  | "caption_anchor" | [future]       │
├─ Backends ──────────────────────────────────────────┤
│  geometric_match_bbox()  — born-digital rasters     │
│  caption_anchor_bbox()   — scanned / hybrid scans   │
│  [future] layout-aware / table-aware                │
├─ Analysis & cache ──────────────────────────────────┤
│  PageDerived (per-page, sha1-keyed, policy-versioned)│
│  _render_cache/<sha1>_pg<N>_<dpi>dpi.png            │
├─ Debug artifacts ───────────────────────────────────┤
│  <out>.fail.json on every scanned-mode hard_fail    │
└─────────────────────────────────────────────────────┘

Future backends plug in at the Backend layer with their own capability check and a function returning bbox + ambiguity flags — nothing above changes.

==Architectural freeze rule==: do not add per-book heuristics or if scanned/elif born-digital branches into the gate function. New behavior goes into a new backend plus a new capability check in _select_backend.

Backends — when each is used

_select_backend() routes pages by structural capability, not by document type:

The hybrid-scan category is the one that surprises naive backend selection: the PDF library reports multiple "assignable" rasters (the embedded figure photos), but the surrounding OCR-overlay text contaminates the geometric backend's text-bleed QC check. Detecting the page-background raster and routing to caption-anchor is the fix.

Scanned/hybrid backend — caption_anchor

Module: figure_scanned.py. Algorithm:

1. Render the page at 200 dpi to a cache file (keyed by PDF hash + page +

dpi).

1. Walk the page's text-with-position data for Fig X-Y caption text

bounding boxes (with an inline-reference filter — must start at a line beginning, not mid-sentence after a word).

1. Apply Class A OCR normalization (silent, 1-to-1): common OCR

confusions (l/I → 1, S → 5, O → 0) only inside the numeric portion of the fig_id.

1. Build a caption-match object per caption with an alias set.
2. Look up the target fig_id across all caption alias sets:

• 0 matches → L1_no_caption_match hard_fail
• ≥2 matches → L1_target_matches_multiple_captions hard_fail

1. Infer page-global caption direction (caption-below is the layout

default).

1. Compute the figure region: the area opposite the caption, bounded by the

previous/next caption in the same column.

1. Crop the rendered page; run an ink-density trim to tighten the bbox.
2. Scanned QC: whitespace-fill check only (text-bleed is meaningless on

scanned PDFs — the OCR overlay covers the whole page).

1. On hard_fail: write a sibling .fail.json debug artifact with policy

version, captions, columns, backend reasoning.

Known Phase-1 limitations (deferred):

• Class B OCR ambiguity (e.g. 4-28 vs 4-2B) currently fails

L1_no_caption_match for OCR-mangled targets.

• L2 ownership-overlap guard, L3 direction-quality guard not yet built.
• Direction inference is page-global, not per-column.

fail.json schema

Every scanned-mode hard_fail writes a sibling <out>.fail.json debug artifact, with a versioned schema. Stable fields:

Downstream consumers must check schema_version and refuse to parse unrecognized majors.

Performance helper — extract-page for scanned books

When a workflow needs all figures on one scanned page, use the batch CLI subcommand instead of N independent extract calls:

python {REPO}/figures/figure_remap.py extract-page \
  --book "BookID_PartI" \
  --pdf "<path>.pdf" --page N --out-dir "attachments/topic/" \
  --fig-ids "4-1A,4-1B,4-1C,4-1D" \
  --name-template "Fig_{fig_id}_BookShort.png"

==This is NOT a contract addition== — extract-page is a render-cache-aware loop that calls extract() once per fig_id. Each emitted JSON line is a full single-figure contract dict; workflows can substitute N independent extract calls without behavior change — only speed differs.

Entrypoint contract (call THIS, not the gate)

There is ONE sanctioned entrypoint for every figure request — figure_remap.py extract. Workflows MUST go through it and MUST NOT call the internal gate script directly. The entrypoint is the abstraction boundary: it returns a stable minimal contract and hides the implementation (matching method, strict mode, fallback policy, QC ladder), so logging stays uniform and policy is enforced in one place.

python {REPO}/figures/figure_remap.py extract \
  --book "{BookID}" \
  --fig-id "X.Y" \
  --caption "{caption text from textbook md}" \
  --out "attachments/Fig_X-Y_{BookShort}_{topic}.png" \
  --pdf "{path/to/textbook.pdf}" \
  --page N \
  [--source {auto|pdf|existing}]   # default auto (see "Source modes" below)
  [--existing "{book_dir}/figures/Fig_X-Y.{png,jpeg}"] \
  [--no-strict]      # opt OUT of deterministic mode into the fallback ladder (discouraged)

Source modes

--source selects where the figure comes from. Default is auto — callers don't need to think about a pre-extracted fast path; the entrypoint resolves it from the book convention <OUTPUT_DIR>/<book>/figures/Fig_<id>.* automatically.

Contract returned (and ONLY this — never the gate's internal shape):

{ "status": "pass|fail|escalate", "match_quality": "exact|uncertain|failed",
  "hard_fail": true|false, "file": "<path|null>", "fig_id": "<normalized>", "reason": "<text>",
  "qc_degraded": true|false, "qc_skipped": ["<check name>", "..."] }

Those eight keys, no more and no fewer — _validate() in figures/figure_remap.py raises on any extra or missing key.

• qc_degraded / qc_skipped: one of the two BLOCKING checks could not run

(e.g. the source page render was unavailable), so a pass here is weaker than a fully-gated one. Treat a degraded pass as "embed, but eyeball it". A skipped advisory check never sets this — it was never gating anything.

• match_quality: exact = deterministic geometric match · uncertain = a

fallback crop (only with --no-strict) · failed = no crop. ==Branch on this, never on the engine method.== The precise method is logged only in a local QC log, deliberately kept out of the contract.

• status:pass (exit 0) → file is a QC-passed crop; embed it.

==Embed the `file` string verbatim — never the `--out` you asked for, never a name rebuilt from the `Fig_{id}_{Book}` convention.== file is the authoritative path and it legitimately differs from --out: the auto fast path returns the book's pre-extracted image, so a template saying .png can come back .jpeg. A rebuilt name is derived from the real one, so it reads correct in review and only a machine catches it. Prove it: ``bash python {REPO}/figures/figure_embed_lint.py --notes-dir {NOTES_DIR} "<note.md>" `` exit 0 clean · 1 = MISSING (a guessed filename, or an embed written for an extract that actually failed) or CASE MISMATCH (opens on Windows/macOS, breaks on git and Linux) · 3 = notes dir not found = unverifiable, not a pass.

• status:fail (exit 1) → deterministic miss / hard_fail. ==NOT a wrong

figure — a correct refusal.== Fix --page, escalate to vision, or leave a <!-- TODO -->.

• status:fail with reason starting pregate= (exit 1, hard_fail:false)

→ different animal: extraction and QC both succeeded, then the junk pre-gate (figures/pregate.py) judged the crop to be a chapter-title banner or a blank. ==Skip this fig_id entirely — do not retry, do not fix --page, do not leave a retry TODO.== Retrying only re-crops the same banner. Applies to born-digital geometric crops; the scanned caption_anchor path and the existing fast path return no bbox, so the pre-gate abstains there.

• status:escalate (exit 2) → non-strict only; read the page render, then

re-call with --bbox.

Default is strict (deterministic). --no-strict re-enables the legacy-compatibility fallback ladder, which can yield a plausible-but-wrong crop — avoid for note-writing.

⚠️ A pass guarantees the right raster for the fig_id, not that the

figure depicts what you assume. Still read the real caption text before

trusting its content.

Policy version