lev-os/agents/skill-builder

Skill Builder

Skill edits are candidates until relevant behavioral checks pass. Preserve existing work; missing baseline evidence is evaluation work, not authority to delete it. An explicit skills-first update may precede trials, but must remain unqualified.

Contents

• Qualification and selectors
• Source inventory and IR
• Expertise acquisition
• Route the request
• Pointers and invocation
• References and freedom
• Output templates
• Audit and behavior checks
• Absorb into the owner
• Intake and install
• Author a skill
• Extract source material
• Security scan
• Merge skills
• Generate candidates
• Reference index
• Observed rationalizations

Qualification states and selectors

Keep these states separate in every report: structural (the file parses and source obligations map), behavioral (bounded fresh scenarios show the intended judgment), and integration (the target host loads, routes and authorizes it). A structural pass never upgrades the other states. Infer the subject and its unchanged baseline from the candidate's actual changed artifacts and declared outcome, never from the skill running the review; the evaluation-method owner and the subject owner are distinct roles. When skill-builder's own source is among those changed artifacts, its pre-change evaluator (checks, rubrics, fixtures, grader prompts) stays frozen outside the candidate's scope; a changed grader starts a new evaluator generation and never counts as candidate improvement.

The following are skill selectors, not invented runtime APIs:

• --scrape <url|repo> collects exact source snapshots and a coverage manifest for analysis.
• --analyze <source|snapshot> builds per-skill JSON IRs and a graph report without installing or executing the subject.
• --absorb <source> learns into an existing owner; starts with the quick fixture comparison below.
• --audit <skill|folder|all> performs a read-only quality audit.
• --behavior requests bounded fresh behavioral trials within an explicit

budget; it does not install, promote or edit the subject or grader.

• --security enters the existing mandatory security intake scan.

For --audit <target> --security, select the existing security scan. For --audit <target> --behavior, inspect the target and then run the requested bounded trials. If both modifiers are present, run security before behavioral trials within their declared scope and budget. A bare --behavior needs an explicit target or an unambiguous current subject.

Do not compute an aggregate expertise score. Report findings, evidence and remaining gaps by state and by artifact.

Scrape and analyze skill structure

For the AIUX Playground corpus, reuse the source-backed analyzer at /Users/jean-patricksmith/ops/audits/skill-ir-20260930/build_ir.py when that file is available: inspect --help, use --acquire for permitted public GETs, run without that selector for cached analysis, and run --check for integrity. Its catalog-specific acquisition is an implementation reference; other sources follow the same IR obligations below rather than silently using that catalog.

--scrape and --analyze are skill selectors. For skill catalogs, collect each listed entry and its actual package source; keep catalog descriptions separate. For repositories or local folders, pin the revision or file digests. Preserve source attribution and license information, and record inaccessible sources.

1. Collect exact source bytes into the authorized staging location. Resolve

package-local references recursively and inventory bundled scripts without running them. Bound external-link traversal by the chosen source/package scope; record unresolved, missing, excluded and unsafe paths explicitly.

1. Emit one JSON IR per skill: identity and provenance; heading taxonomy and

bullet TOC; triggers and exclusions; rules; ordered workflow and branch routes; inputs and outputs; tools and script dependencies; declared side effects; gates, verification and recovery. Attach source file, line and digest to extracted facts; label inferred relations. Distinguish operative instructions from examples/templates and declared effects from observed runs.

1. Build a typed graph linking skills, resources, steps, routes, tools, effects,

gates and artifacts. Use an available graph library or standard algorithms for components, cycles, shared dependencies and overlap candidates. Explain the edge definitions; similarity is a review cue, not behavioral parity.

1. Validate catalog coverage, source digests, reference containment, IR schema

and graph endpoints. Deliver JSON plus a browsable report with per-skill bullet TOCs, comparisons and explicit coverage gaps. If requested, register metadata-only candidates in skills-db/_todo; keep them inactive and unaudited.

Completion requires every listed entry to have an IR or an explicit source failure, every bundled script to have a static-analysis record, and every reachable reference to have a resolution status. Scraping and analysis do not qualify installation, promotion, execution, or source retirement. Route a later learning request to --absorb and activation to the existing intake checks.

Operational expertise acquisition

When turning a repeated method or source material into a skill, every resulting skill must cover these obligations; they are not a fixed execution sequence:

1. Purpose — name the user decision or outcome and the branch that needs the method.
2. Evidence — collect source passages, observed failures and counterexamples; label supplied, observed, inferred and unknown separately.
3. Judgment — state the distinction that changes the decision, including the boundary where it stops applying.
4. Action — turn the distinction into the smallest ordered action that produces an inspectable artifact or state.
5. Verification — name the command, observation or comparison that can falsify the action's claim.
6. Recovery — define what to preserve, what to retry, and when to stop or escalate after a failed check.

Map each source obligation to its destination before compression, deletion or merging. Keep lossless source mapping in a ledger; unresolved mappings remain open. Use the operationalizing-expertise method's sentence form, “When X, do Y because Z,” and validate against counterexamples. Its corpus, quote-bank, operator-count and triangulation deliverables are for that larger program; do not impose them on a bounded skill edit unless that scope is explicitly chosen.

The acquisition sequence is distinct from those six obligations:

1. Define competence, success conditions and limits from the task and owner evidence; ask the operator only about unresolved consequential choices.
2. Acquire anchored successes, failures and corrections from representative cases and source passages.
3. Extract the fact or distinction that changes the action, including what remains unknown.
4. Challenge it with counterfactuals, missing-evidence cases and boundary exceptions.
5. Encode notice → why → test → bad/good → exception as a trigger, action, invariant, evidence check and recovery path.
6. Test held-out transfer on a fresh case and retain the result as pending until the evidence supports it.

For each encoded rule, make the teaching pattern explicit: notice the signal, state why the distinction matters, name the test, show a bad and a good case, and state the exception or boundary. Use the examples as anchors for judgment, not as a substitute for case evidence.

Routing

steps:
  - id: route
    action: Classify the request and dispatch
    instruction: |
      | Input                          | Workflow         | Jump to     |
      |-------------------------------|------------------|-------------|
      | URL, skills.sh link, skill:// | Intake & Install | intake      |
      | "new skill", "from scratch"   | Author (TDD)     | author_red  |
      | Docs site, GitHub repo, PDF   | Extraction       | extract     |
      | "--audit", audit               | Read-only Audit  | [Audit section](#read-only-audit-and-behavior-checks) |
      | "--security", "is this safe"  | Security Scan    | security    |
      | "--behavior"                   | Behavior Checks  | [Behavior section](#read-only-audit-and-behavior-checks) |
      | "--scrape", "--analyze"       | Source Analysis  | [Scrape and analyze](#scrape-and-analyze-skill-structure) |
      | "--absorb", "learn from this"  | Absorb           | [Absorb](#absorb-learn-and-test-in-the-existing-owner) |
      | "merge these skills"          | Merge            | merge       |
      Ambiguous? Ask: "Are you converting existing material or authoring from scratch?"
    validation: "Primary workflow, target and any explicitly requested additional checks are identified"
    on_failure: "Ask a clarifying question. Do not guess."

Pointer Repair and Invocation

Every context pointer names both its material and the branch or condition that triggers loading it. When a required reference is missed, sharpen that pointer first; inline its operative content only if sharpening fails, except for explicitly requested baked-in rules. When a step ends prematurely, sharpen its completion bound before splitting the sequence. Replace repeated explanatory phrases with a defined, established domain term when that improves retrieval; do not coin opaque slogans. Delete a demonstrated no-op sentence as a whole rather than cosmetically trimming it. A stronger leading word is useful only if observed behavior changes.

Treat always-loaded descriptions as the most expensive context surface: prune their words aggressively and retain only words that change routing or invocation. Separate that context cost from the human cost of remembering explicit entry points. Treat cognitive load as the price of human agency: spend it where human judgment matters and remove it where deterministic routing or existing conventions can carry the decision. Add an independently discoverable skill only when its own trigger or a real caller warrants that cost. Otherwise keep a conditional branch or shared reference under the existing owner. A router reduces the human index burden by naming destinations and when to use them, not by bypassing invocation or authorization policy.

Use the least costly sufficient information tier: keep ordered actions inline, use an in-file reference for on-demand material when that is enough, and disclose external material behind a pointer only when a branch needs it. An in-file reference is a valid hierarchy tier, not an automatic reason to split a skill.