knockoutez/wigolo/wigolo-find-similar

wigolo find_similar

Hybrid semantic discovery: semantic embeddings + keyword search + web search, fused via Reciprocal Rank Fusion (RRF).

Quick Reference

// Find pages similar to a URL
{ "url": "https://docs.astro.build/en/getting-started/" }
// Find pages related to a concept
{ "concept": "JavaScript framework server-side rendering" }
// Scoped to specific domains
{ "url": "https://react.dev/reference/react/use", "include_domains": ["vuejs.org", "svelte.dev"] }
// Cache-only (no web fallback)
{ "url": "https://example.com/page", "include_web": false }

Parameters

Provide either url or concept (not both).

How It Works

1. Embeds the input (URL content or concept text) into a vector.
2. Searches local cache via embedding similarity + keyword matching.
3. Falls back to web search if local hits are sparse.
4. Fuses all signals via 3-way Reciprocal Rank Fusion (RRF).
5. Returns ranked results. Each carries match_signals with the fused_score. Set include_ranking_debug: true to also attach a per-result ranking_debug object exposing the individual source ranks (fts5_rank, embedding_rank, web_rank, rrf_score) so you can audit disagreement between the three ranking sources.

Modes

• `auto` (default) — pick the strategy from available signals.
• `cache` — local hybrid only (keyword + semantic over the cache).
• `web-expansion` — derive key terms and expand via web search.
• `crawl-rank` — 1-hop crawl from the seed URL, embed, and cosine-rank the neighbours.

Cold-Start Signal

When the fused score from local signals is below threshold (env WIGOLO_FIND_SIMILAR_COLD_START_THRESHOLD), the response includes a cold_start string explaining why results came from web search. Pass it verbatim to the user.

Important: Build the Cache First

find_similar works best with a warm cache. Recommended workflow:

// Step 1: crawl to populate cache with embeddings
{ "url": "https://docs.framework.dev", "strategy": "sitemap", "max_pages": 20 }
// Step 2: now find_similar has real semantic signal
{ "url": "https://docs.framework.dev/getting-started" }

Anti-Patterns

• DON'T use find_similar on a fresh install expecting embedding results — crawl first.
• DON'T provide both url and concept — pick one.
• DON'T use when you want web-only results — use search instead.

When NOT to use wigolo-find-similar

• No local cache and no plan to build one — fall back to search with include_domains.

See Also

• wigolo-crawl — build the cache first
• wigolo-search — when you want web results, not cache similarity