zby/commonplace/reviews-archive

WikiSkill

Evidence basis: WikiSkill paper, captured 2026-09-17; doc-grounded analysis of the described improvement workflow, without implementation or original run artifacts.

WikiSkill separates immutable execution traces, a persistent wiki and the active skill set. Each iteration executes training tasks, compiles successes and failures into wiki patterns and summaries, proposes one skill change, and tests it on validation tasks. The gate accepts only strict score improvements. Rejected skills are rolled back while wiki knowledge and the proposal's diff, score and rejection history survive for later proposals. These are described routes, not independently inspected wiring.

The wiki maintainer receives existing wiki context and sampled traces. The proposer starts with catalog, impact history and task outcomes, then requests particular pages and traces. Inference receives all active skills directly in its system prompt. This separates diagnostic knowledge from operative procedures, but full injection leaves skill retrieval untested. PURPOSE.md preserves a rationale map; its later reading and inference delivery are not established.

The paper reports improved skill evolution and conditional cross-model transfer. Its wiki ablation removes the maintainer together with proposer wiki access, so the contrast concerns that combined mechanism. Validation accepts procedural performance, not the truth of every retained explanation. A simplified case describes revisions informed by prior failures; it does not independently establish that criticism of an explanation caused the measured improvement.

Scope

The exact retained analysis contains the source anchors, route records, mandatory memory and epistemic lenses and all fourteen memory-comparison axes. Reflection and self-improvement are claimed at the described skill-evolution boundary. Criticism-specific learning, faithful dependence on particular recalled content, exact model fixation and runtime enforcement remain unestablished. The paper acknowledges unpruned wiki growth and does not evaluate within-rollout adaptation or very long-horizon tasks. The inference-wiki-access ablation also leaves delivery and authority unspecified.