knockoutez/wigolo/wigolo-fetch

wigolo fetch

Smart URL fetching: HTTP-first with automatic browser fallback for JS-rendered pages, persistent local cache, optional browser-session auth.

Quick Reference

// Basic fetch
{ "url": "https://react.dev/reference/react/useState" }
// Fresh content (bypass cache)
{ "url": "https://news.ycombinator.com", "force_refresh": true }
// With authentication
{ "url": "https://app.example.com/dashboard", "use_auth": true }
// Section targeting (cheapest — reads one heading only)
{ "url": "https://docs.example.com/api", "section": "Authentication" }
// Compact context for AI
{ "url": "https://docs.example.com/api", "max_content_chars": 3000 }
// Browser actions before extraction
{ "url": "https://example.com", "actions": [{"type": "click", "selector": "#load-more"}, {"type": "wait", "ms": 1000}] }

Parameters

Output

Returns clean markdown plus:

• title, markdown, links, images
• Metadata: og_type, canonical_url, og_image, og_description, keywords (parity with extract metadata mode)
• cached: true/false — repeat fetches are instant

Anti-Patterns

• DON'T fetch a full page when you need one section — use section: "Heading Name".
• DON'T set force_refresh: true by default — defeats the cache.
• DON'T use fetch when you need tables/JSON-LD — use extract instead.

When NOT to use wigolo-fetch

• Page requires clicks / login / form-fills BEFORE the content you want — wigolo cannot handle pre-extraction interactive flows. (use_auth with stored sessions works for already-logged-in pages.)
• Bulk multi-page extraction — use crawl or agent.

See Also

• wigolo-search — when you don't have a URL
• wigolo-extract — when you need structured data, not markdown
• wigolo-crawl — when you need multiple pages from a site