veniceai/skills/venice-text-routing

Venice Text-Model Routing

This skill encodes the decision logic for "which Venice text model do I call?" — the routing layer that sits above `venice-chat` (the call surface) and consumes `venice-models` (the discovery API).

Primary use case: a local agent receives a prompt, decides whether the local model can handle it, and — if not — picks the cheapest Venice model that satisfies the privacy / modality / capability requirements.

Snapshot freshness

Before applying the matrix below, read `snapshots/text-routing.json`. If the file is missing, or `snapshot_date` is older than 30 days, run python scripts/refresh_routing.py to regenerate it from GET /models?type=text + GET /models/traits?type=text. Otherwise trust the cached file — do not hit /models on every routing decision.

refresh_routing.py rewrites both `snapshots/text-routing.json` and `routing-matrix.md`. Both endpoints are public, so no key is needed; VENICE_API_KEY / --api-key is optional and tailors the result to that key (a modelPrivacy-restricted key sees a filtered catalog). Run it once on first install, then ~monthly (CI nightly is also fine).

In the snapshot, each model's privacy label is e2ee when supportsE2EE is true, tee when only supportsTeeAttestation is true, and otherwise the model's own private / anonymized value (every TEE/E2EE model is private in /models, so treat tee and e2ee as private too); tier is bucketed by input price only; beta_access mirrors model_spec.beta (only listed to beta-access keys, so it is always false in the shipped keyless snapshot) and beta_status mirrors model_spec.betaModel (callable, but may change or disappear).

Per model the snapshot carries only: the 13 capability flags shown in the matrix, max_images, available_context_tokens, max_completion_tokens, input/output price per 1M, beta_access, beta_status, offline, region_restrictions and deprecation_date. Filters that need anything else (uncensored, supportsLogProbs, reasoningEffortOptions, maxVideos, cache_input / extended pricing, replacementModelId) have to read GET /models?type=text.

When to load this skill

• Picking a Venice text model from a prompt at runtime.
• Building a local-first agent that escalates to Venice for hard prompts.
• Deciding privacy tier (anonymized vs private vs TEE vs E2EE).
• Choosing between a trait shortcut (default_reasoning, most_intelligent, …) and a hand-filtered candidate.

For the chat call surface itself, see `venice-chat`. For raw discovery of every model field, see `venice-models`.

Privacy tier ladder

Pick the least restrictive tier that satisfies the request — restricting tier shrinks the candidate pool and often raises cost.

model_spec.privacy has only two values, private and anonymized. TEE and E2EE are capability flags on top of private.

Today every listed TEE model is also E2EE-capable and uses an e2ee-* ID (e.g. e2ee-glm-5-3-p, e2ee-kimi-k3-p, e2ee-qwen3-8-27b). TEE and E2EE use the same model ID: send E2EE headers for E2EE, omit them (or set venice_parameters.enable_e2ee: false) for TEE-only. Legacy tee-* IDs are unlisted aliases of e2ee-* models — don't route on the prefix; use the capability flags.

E2EE trade-offs: on E2EE requests Venice injects no web search, scraping, character, or Venice system prompt, and file parts return 400; the E2EE guide also requires stream: true and lists function calling as unsupported (supportsFunctionCalling describes the model, not E2EE mode). Several E2EE models also have small limits (e.g. e2ee-qwen-2-5-7b-p: 32K context, 4,096 output tokens). E2EE is not supported on /responses; TEE-only is (with enable_e2ee: false).

API keys can carry a modelPrivacy restriction: PRIVATE_TEXT (text, embedding and decision models must be Private, TEE, or E2EE) or PRIVATE_ONLY (every model). Such a key's /models list is already filtered; a disallowed model returns 403 — filter to privacy: "private" for such keys.

Sources: docs.venice.ai/overview/privacy, docs.venice.ai/guides/features/tee-e2ee-models.

Verifying a TEE claim

Two endpoints let you check that inference really ran inside an enclave. Both are GET, unauthenticated on purpose (attestation evidence has to be verifiable by any party without credentials), and rate limited to 10 requests per minute per IP (429 beyond that). Neither is a path in the published OpenAPI spec; they are live, and the supportsTeeAttestation capability description in GET /models points at them.

Both return 400 if the model exists but is not TEE-attested, and 404 if the model ID is unknown. The attestation endpoint returns 502 (with verified: false) when verification fails; both return 502 when the TEE provider is unavailable. Chat responses from TEE models carry X-Venice-TEE: true and X-Venice-TEE-Provider.

Verify the chain of trust in this order: fetch the attestation to get the signing public key and hardware type, confirm the recovered signer matches the attestation signing address, then verify the signature over the exact signed text the signature endpoint returned. Treat the request and response hashes as provider-reported values unless you can recompute them yourself from a documented canonical format.

Capability filters

Map prompt requirement → model_spec field (full list in the model_spec.capabilities section of `venice-models`). Sending image / audio / video parts, tools / tool_choice, a non-text response_format, or logprobs to a model without the matching flag returns 400 before inference; some other features (e.g. enable_x_search) are silently ignored instead.

Cost tiers

These buckets are this skill's own convention (the same boundaries refresh_routing.py uses), keyed on model_spec.pricing.input.usd per 1M tokens. Upper bounds are exclusive (a model at exactly $1.00 is in M), and the snapshot's tier_boundaries_usd_per_1m_input gives each bucket's exclusive upper bound. Pick the smallest bucket that hosts a model satisfying your filters.

Output prices vary widely within a bucket (from ~1.5× to ~10× input), so tie-break on output price. Authoritative per-model pricing lives in model_spec.pricing (input and output rates are mirrored as pricing_per_1m in the snapshot) — never hard-code dollar figures from this prose.

Routing decision tree

Walk top-down. Stop at the first rule that applies.

1. Local-first check
   - Prompt is ≤ ~500 tokens, no special-capability requirement,
     no privacy escalation, no tool calls expected
     → handle on the local model. Do not call Venice.
 
2. Privacy gate
   - User flagged "private" OR prompt contains regulated data (PHI, secrets, legal),
     OR the API key has modelPrivacy PRIVATE_TEXT / PRIVATE_ONLY:
       require model_spec.privacy === "private"   (includes every TEE/E2EE model;
       in the snapshot accept privacy "private", "tee" or "e2ee").
   - User flagged "E2EE" / "must be encrypted end to end":
       require capabilities.supportsE2EE. Use /chat/completions only.
   - User flagged "TEE" / "verifiable inference":
       require capabilities.supportsTeeAttestation.
   - Otherwise: any tier acceptable (still prefer "private" when otherwise tied).