banodoco/hivemind/docs

hivemind

A read-only public endpoint exposing the Banodoco knowledge corpus — community knowledge about video/image generation you can't get from official docs: workflow tips, model comparisons, settings tweaks, gotchas, and links to Kijai/Ablejones/community workflows. It layers three things into one searchable feed: messages (raw Discord), resources (articles, transcripts, workflows), and distillations (curated Q&A with cited sources). Distillations make the corpus self-improving — every researched answer you submit becomes permanently searchable.

Quick start

Endpoint: https://ujlwuvkrxlvoswwkerdf.supabase.co/rest/v1
Key (anon publishable, safe to commit):
  apikey: sb_publishable_O38oPBafrBoFrpi_rlWJvA_UJrulFsx

Smoke test (raw search surface — per-token OR on message content; do NOT text-search unified_feed, the UNION scan times out with 57014):

curl -s "https://ujlwuvkrxlvoswwkerdf.supabase.co/rest/v1/message_feed?select=message_id,content,author_name,channel_name,created_at&limit=5&order=created_at.desc&or=(content.ilike.*wan*,content.ilike.*animate*)" \
  -H "apikey: sb_publishable_O38oPBafrBoFrpi_rlWJvA_UJrulFsx"

Surfaces at a glance — pick by what you want

More on each below. The golden rule: match the surface to the question (gotchas in the next section).

The surfaces

unified_feed — the results feed (distillations + resources; NOT text search)

unified_feed is a UNION of three layers with one common shape. Use it kind-scoped for single-row fetches (get_item) of distillations and resources; message get_item uses the indexed message_feed surface. Do NOT use it for text search of any kind (a per-token OR over the UNION scan times out with 57014). For text search use the raw tables (message_feed / external_resources / distillations) or the pack's hivemind.search executor, which does exactly that.

Common columns: kind, source, item_id, title, body, author, context, url, metadata, created_at. Distillations have a lifecycle pending → approved; prefer approved, then pending, then raw items.

Message metadata (the Discord envelope)

unified_feed message rows carry the full Discord envelope in metadata:

Scope + semantics:

• These envelope keys appear only on `kind=message` rows. Resource and

distillation rows carry their own metadata (tags/status) — they do NOT have these keys.

• Deleted messages are filtered out of every surface (view, message_feed,

search, and direct discord_messages reads via RLS) — they never appear.

The two resource signals (attachments + citations)

Attachments → skip the noise, find the resources. A message that actually contains/shows something (image, video, workflow .json, PDF) has a non-empty attachments array; prose-only messages have []. ~188k messages carry attachments. Each element:

{ "filename": "image.png", "url": "https://cdn.discordapp.com/attachments/..." }

filename is always present — infer the type from its extension (image/*, video/*, .json, .pdf). content_type (MIME) is present on only ~3–5% of attachments, so treat it as a bonus, not a guarantee. Use refresh-media-urls for a fresh CDN URL before using one.

`reference_id` → build chains. It is the snowflake of the message this one replies to. Walk it both directions to reconstruct threads of thoughts: read a reply's reference_id, get_item the parent, repeat. A message with a reference_id is a response/correction/elaboration; without one it's a root.

message_feed — raw message content search (the fast text surface)

Each row: message_id, content, author_name, channel_name, channel_id, guild_id, reactions, created_at (deleted already filtered). This is the surface for message content `ilike` — it's index-backed and fast, where unified_feed ilike is not. Scope by channel for speed.

message_filters — structured filtering (schema/036)

A simple, index-backed view over discord_messages with the hot filter columns exposed top-level (not buried in jsonb metadata), so PostgREST filters push to indexes:

GET /rest/v1/message_filters?is_pinned=eq.true&limit=20            # pinned canon
GET /rest/v1/message_filters?channel_name=eq.minimax_h3_resources  # by channel (note: channel_name is NOT unique — use channel_id for exactly one)
GET /rest/v1/message_filters?thread_id=eq.<tid>                    # whole thread
GET /rest/v1/message_filters?reference_id=eq.<id>                  # replies TO a message (chains)
GET /rest/v1/message_filters?attachments=cs.[{"content_type":"video/mp4"}]  # has typed attachment (GIN)
GET /rest/v1/message_filters?attachments=cs.[{"filename":"x.png"}]          # exact file

Columns: message_id, channel_id, guild_id, author_id, thread_id, reference_id, is_pinned, reaction_count, attachments, embeds, content, created_at, channel_name. All filters index-backed (is_pinned ~0.1s, reference_id ~0.1s, attachments containment ~0.1–0.6s). Attachments uses PostgREST's cs operator (jsonb @>). Note: attachments=cs.[{"filename":"wan"}] substring does NOT work — it's exact match; and filtering by content_type only finds attachments the archive typed (~5% of videos, ~3% of images).

get_item — full rows

python3 -m hivemind.executors.get_item.run --kind message|resource|distillation --id <id>
# or: unified_feed?item_id=eq.<id>

Returns the full row + metadata + (for distillations) cites / cited-by.

How to answer a question (the workflow)

1. Distillations first — distillations?status=in.(pending,approved)&or=(question.ilike.*T1*,answer.ilike.*T1*,question.ilike.*T2*,answer.ilike.*T2*) (per-token, not one phrase — a phrase never occurs verbatim and a multi-token OR over unified_feed times out). Hit → relay the answer + its cites.
2. Then the pinned canon — message_filters?is_pinned=eq.true (~72 pins, mostly in wan_resources/ltx_resources; the community's highest-signal artifacts).
3. Then channel-scoped message search — message_feed?channel_name=in.(…)&content=ilike.*term*, starting from the channel map.
4. Verify + deepen — get_item promising ids (pass Discord snowflakes as

decimal strings); walk reply chains (message_filters?reference_id=eq.<id>); pull a whole thread (thread_id=eq.<tid>).

1. Assemble a cited answer — name authors, include Discord/permalink + source links, prefer concrete settings over abstractions.

Paging a big result set — the search response carries has_more, page/pages, and next_offset. When has_more is true, fetch the next page by re-running the same query with --offset <next_offset> — the next offset is given, so do not compute it yourself. total is the ranked pool (~100–300 rows), not the full corpus match count; to dig further than the pool, narrow with --channel/--author/--since/--thread instead of deep offsets.

Ordering — default --sort relevance: score (distinctive tokens in title weigh more than body; approved distillations and parseable workflows float), then recency as tiebreak. Use --sort recent for newest-first.

Let channel flavor guide you — wan_chatter = chat/experience, wan_comfyui = technical/errors, *_gens/*_resources = showcases/files, training_* = training, updates = announcements/essays, daily_summaries = orientation (starts 2024-12-20; before that use topic channels).

Gotchas (the traps)

• `unified_feed` text search times out (HTTP 500 code 57014) for

message-content ilike AND for multi-word / per-token ORs over the UNION (the derived-view scan exceeds the anon role's 3s statement budget). 57014 = statement timeout — NOT bad data. The pack search executor never touches unified_feed: it queries message_feed, external_resources, and distillations directly with per-token predicates and ranks client-side. Use that executor, or the raw tables, or message_filters.

• Do NOT filter on `unified_feed.metadata` jsonb paths — same timeout. Read

the fields from result rows instead, or filter the source columns.

• Rank with `reaction_count`, not `reactions` — reactions is active-only

(~7% of messages); reaction_count is the historical total (~16%) and the reliable popularity signal.

• Snowflakes are strings at the JSON boundary (author_id,

reference_id, thread_id, guild_id). item_id and cite item_id too — never send them as JSON numbers.

• `content_type` on attachments is sparse (~3–5%) — infer type from the

filename extension; cs filters by content_type only find typed ones.

• `channel_name` is not unique — multiple channels can share a name; use

channel_id for exactly one.

Reference

Channel map