zby/commonplace/cp-skill-ingest

Ingest source

Preserve a faithful, reusable account of what a source contributes to the installed KB and what its evidence supports.

Ingest one URL-backed primary source into a tracked kb/sources/*.ingest.md report. The reading copy may already exist as a Markdown snapshot under kb/sources/.snapshots/.

Contract

Target: $ARGUMENTS

The direct output is the .ingest.md report in kb/sources/, with the local snapshot's slug. URL snapshotting and connection discovery may write their own local or generated artifacts. The parent may also shorten an unpaired local snapshot's filename under Step 1 without changing its bytes. Do not directly write any other library artifacts.

The parent agent owns target resolution, snapshotting, checksum guards, connection discovery, drafting-worker dispatch, handoff verification, final validation, and user reporting. One primary fresh drafting worker owns the ingest analysis and writes the .ingest.md report; at most one fresh replacement worker may repair that report after a failed handoff. Use only a harness-provided sub-agent or worker mechanism; never launch codex, codex exec, claude, or another agent CLI from the shell. If the harness cannot launch a fresh worker without carrying the parent's conversation history, stop after connection discovery and report that isolated drafting is unavailable. Do not draft locally unless the user explicitly authorizes a local fallback for that run. If authorized, execute the same standalone drafting instruction with the same fixed inputs and report drafting was local, not delegated as a workflow exception.

The drafting worker executes the standalone draft-ingest-report.md instruction. That instruction owns report analysis, writing, and worker-side verification; this skill owns only the surrounding orchestration. For scientific papers, its experimental contribution assessment draws on the worker's pretrained NeurIPS reviewing methodology, interpreted through Commonplace's Popperian approach to problem solving and criticism. The standalone drafting instruction carries this cue into the fresh worker.

The caller may supply an occasion: the question or job that brought the source in. It is the one task-specific input the worker receives beyond the fixed paths. It must be stated before the source is read and must not carry the parent's reading of the source; the worker records it and lets it govern only the report's selection sections. An occasion that is already an exact source-side claim is cp-skill-ground's job once the ingest exists; pass the broader question here, or none.

The canonical empty Quotes block is the following complete Markdown section, including the blank line after its sentence:

## Quotes
 
No source quotes have been retained yet.
 

A re-ingest caller supplies this structured request instead of an ordinary URL-or-file target:

re_ingest_request:
  ingest_path: <exact ingest path>
  snapshot_path: <name-paired snapshot path>
  occasion: <optional; omitted keeps the incumbent's occasion field>

Steps

1. Resolve the target and replacement policy.

• If the target is a re_ingest_request, require exactly one existing

kb/sources/<slug>.ingest.md and its exact name-paired kb/sources/.snapshots/<slug>.md. Reject a different snapshot_path; do not search for a snapshot by checksum. Read the ingest's source and snapshot_sha256, reject the retired source_snapshot and code_revisions fields, and require the snapshot's frontmatter source to equal the ingest's canonical source. Compute lowercase SHA-256 from the exact snapshot bytes.

• Require zero or one incumbent Quotes block; stop before mutation if more

than one exists. The block starts at the exact ## Quotes heading and includes every byte through the byte immediately before the next level-two heading. If the section is absent, use the canonical empty Quotes block; do not infer quotes from any other section. Treat Quotes as empty only when the section is absent or is exactly the canonical empty block. Any other present Quotes block is populated.

• If the snapshot checksum equals the incumbent snapshot_sha256, set

retained_quotes to the extracted block and continue. Do not rewrite, normalize, or reformat it.

• If the checksums differ, stop before connection discovery, backup

creation, worker dispatch, or output mutation. Report both checksums, both paired paths, and the canonical source. An ingest's snapshot_sha256 is immutable because a note marked (snapshot required) may depend on those exact bytes even when Quotes is empty. A changed observation gets a new date-named capture and a new ingest (re-ingest's New observation); re-ingest never changes the incumbent observation.

• Set occasion to the request's value when supplied, else to the

incumbent's frontmatter occasion verbatim, else none.

• Set output_path to the supplied ingest_path, retain whether it existed

at the start of the run, and skip the remaining Step 1 bullets.

• If $ARGUMENTS is empty, list recent

kb/sources/.snapshots/*.md files, then ask which one to ingest.

• For an ordinary arxivb.org/abs/<paper-id> URL, resolve the paper before

searching for an existing ingest. arXivBangers is a discovery site with abstracts and automated editorial commentary; the default ingest target is the full paper it links to. Use https://arxiv.org/abs/<paper-id> as the canonical target, preserving an explicit version suffix and dropping query strings and fragments. Use that URL for duplicate lookup and source metadata, then let cp-skill-snapshot-web fetch the full PDF through its arXiv route. Report this resolution to the user. Do not fall back to the arXivBangers abstract if paper capture fails. An explicit request to analyze the arXivBangers page itself keeps the supplied URL; local-snapshot targets and structured re-ingest requests retain their existing source identity. Never repurpose an existing arXivBangers ingest or change its checksum to represent the full paper.

• For a URL target, first search tracked kb/sources/*.ingest.md files for

an exact frontmatter source match. Several matches are separate observations of one source: list them with their captured dates and stop for the user to name one ingest path, or to request a new observation through re-ingest. If one matches, resolve only its name-paired snapshot. If that file is missing, stop and report snapshot not available on this machine; do not recapture, because a new capture cannot reproduce the pinned bytes. Do not search other snapshots for the incumbent checksum.

• If the target is a paperswithcode.co/paper/ URL, or it is an arXiv paper

and the user explicitly requested code grounding, read and follow the conditional procedure ingest-paper-with-code, resolved from this skill's real location. Use the paper snapshot and code-grounding context it returns. Skip the next two target-input bullets, then resume with the snapshot-frontmatter bullet so the common output, Quotes, and checksum guards still run.

• If no snapshot has been resolved and the target starts with http:// or

https://, invoke cp-skill-snapshot-web on the URL. Parse the Snapshot saved: or Already snapshotted: line from its output; that path is the source snapshot for the next step.

• For a non-URL target, require one Markdown file under

kb/sources/.snapshots/. A directory is not a v1 primary source.

• Read the snapshot frontmatter. Retain source, captured, capture,

optional capture_scope and doi, and flat capture-adapter fields such as status_id, conversation_id, post_count, or api_url. Do not copy snapshot type, description, or tags.

• Compute lowercase SHA-256 from the exact Markdown file bytes after capture

completes. Do not hash a JSON, PDF, image, or other companion.

• Derive kb/sources/<slug>.ingest.md. Before connection discovery, require

the snapshot basename to be at most 63 characters and the derived path's stem (<slug>.ingest) to be at most 70 characters. If either is overlong and no ingest exists at the derived path or was resolved for this source, choose a shorter descriptive slug automatically. Require both the new snapshot path and new ingest path to be unused; on a collision, choose another short slug. Rename only the local Markdown snapshot, verify its SHA-256 is unchanged, update the resolved paths, and continue without asking the user. Do not stop merely because the initial name is overlong. Never rename an existing ingest or its paired snapshot through this recovery; report overlong existing pairs for explicit relocation. If URL resolution already found an ingest, require this derived path to equal it. For every existing output, require the snapshot to be its exact name-paired path, require exact frontmatter source equality, and require its snapshot_sha256 to equal the snapshot checksum. On a mismatch, stop and report the permanent re-ingest.md route; an ordinary target never authorizes a changed observation. Extract retained_quotes exactly as defined above, using the canonical empty block when the incumbent section is absent. For a new output, set retained_quotes to the canonical empty block.

On an ordinary-target checksum mismatch, report one permanent route with the exact ingest path filled in: Read and execute the Commonplace library instruction re-ingest with Target: <path>.

1. Run connection discovery.

Invoke cp-skill-connect on the source snapshot path. Wait for it to finish. For a source snapshot, require the completed report at kb/reports/cache/connect/sources/<snapshot-name>.connect.md. Verify that it exists and is non-empty. The parent does not extract connections, classify the source, or begin value judgments; those belong to the fresh drafting worker.

1. Prepare the drafting handoff.

Resolve these exact inputs without writing a brief file:

• draft_instruction_path: the absolute path of

draft-ingest-report, resolved from this skill's real location, so the worker can open it

• snapshot_path: the Markdown snapshot from Step 1
• connect_report_path: the generated report from Step 2
• output_path: kb/sources/<snapshot-slug>.ingest.md
• snapshot_sha256: the checksum already computed from snapshot_path
• retained_quotes: the complete Quotes block resolved in Step 1
• code_grounding_context: only when returned by the paper-with-code branch;

include its secondary-source commit URLs, claim classifications, pinned citations, evidence boundaries, and execution status; retain paper version, checkout paths, and reviewed commits only in the parent for final reporting

• occasion: the caller's pre-reading question or job, verbatim, or none;

for a re-ingest, the value resolved in Step 1

Require the instruction, snapshot, and connect report to exist and be non-empty. Recompute the snapshot checksum immediately before dispatch and require it to equal snapshot_sha256. Do not pass the parent conversation, any part of an existing ingest other than retained_quotes, the backup path, or the parent's interpretations to the worker.