knockoutez/wigolo/wigolo-extract

wigolo extract

Structured data extraction beyond simple markdown.

Quick Reference

// Full structured extraction (ALWAYS prefer this)
{ "url": "https://bun.sh", "mode": "structured" }
// JSON Schema extraction — heuristic field matching
{ "url": "https://example.com/pricing", "mode": "schema", "schema": { "type": "object", "properties": { "name": { "type": "string" }, "price": { "type": "string" }, "sku": { "type": "string" } } } }
// CSS selector extraction
{ "url": "https://example.com", "mode": "selector", "css_selector": ".product-card", "multiple": true }
// Metadata only (matches fetch metadata shape)
{ "url": "https://example.com", "mode": "metadata" }
// From raw HTML
{ "html": "<table>...</table>", "mode": "tables" }

Modes

Always use `mode: "structured"` instead of `mode: "tables"`. Structured captures everything tables does, plus definitions, key-value pairs, JSON-LD, and chart descriptions.

Chart Hints

When a page has visual charts (SVG, Canvas), chart_hints contains text descriptions extracted from aria-labels, SVG <title>, and figcaptions. Use these to describe visual data even when the underlying data is JavaScript-rendered.

Schema Mode

mode: "schema" matches your JSON Schema field names against page content via CSS classes, ARIA labels, microdata, and JSON-LD. When a language model is available it sources fields and verifies each value against the source — hallucinated values are returned as `null` rather than guessed. Pass { properties: { field: { type: "string" } } }.

Named Schemas

Instead of hand-writing a schema, pass named_schema for a strict, heuristic-only extraction (no LLM required) into a known shape:

{ "url": "https://blog.example.com/post", "named_schema": "Article" }

Available: Article, Recipe, Product, CodeSnippet, Paper, EventListing. Mutually exclusive with schema.

Brand Mode

mode: "brand" pulls a page's identity assets — logo, favicon, colors, fonts, and social_links — each with provenance. Favicons never get promoted to logo_url.

Token Budget

max_tokens_out (cl100k-base) caps extracted output; trailing table rows or heavy keys are dropped first to fit.

Anti-Patterns

• DON'T use mode: "tables" — use mode: "structured" instead.
• DON'T pass a schema without properties key — handler rejects it.
• DON'T extract for a whole page when you need markdown — use fetch instead.

When NOT to use wigolo-extract

• Multi-page autonomous structured extraction — use wigolo's agent tool instead.
• Page requires login / click / form-fill before the data appears — handle authentication with use_auth or interact with the page before extracting.

See Also

• wigolo-fetch — for markdown content
• wigolo-agent — for multi-page schema-driven gathering