superdesigndev/treg/web-access

Web access

One treg key reads the web: free search and page fetch first, and the rest of the catalog behind them for the pages and platforms a free fetch can't reach. Nothing runs on this machine: no browser, no cookies, no burner accounts. treg calls the vendor and, on a routed job, tries the next one when a vendor comes back empty or blocked.

Runs on treg, through its MCP tools or the treg CLI; either is enough. Over MCP, each command below maps onto a tool: treg catalog search is catalog_search, treg catalog get is catalog_get, treg call is call. With neither, treg is not set up: https://treg.to/llms.txt says how. Install only if the human asked you to; otherwise tell them how and stop.

Pick the job

Know the URL?             → read it (free first)
Need to find pages?       → search (free) → read the 3–5 best results
A platform object?        → never scrape its HTML: use the platform's job
  (post, profile, video,     structured fields, fewer tokens, survives redesigns
   product, review, place)
A whole site?             → map it → read only the pages that matter
Need typed fields?        → structured extract with a JSON schema or an instruction
Need it recent or dated?  → news search, not web search

Free first

Web search and page reads are free through TinyFish:

• treg call treg.web.search --data '{"q":"…"}' --header 'X-Treg-Route-Prefer: tinyfish'
• treg call treg.web.extract --data '{"url":"…"}' --header 'X-Treg-Route-Prefer: tinyfish'

Both are routed: when TinyFish comes back empty or blocked, treg tries the next vendor (Crawl4AI, AnyAPI, …). You pay the vendor that answered plus any vendor before it that bills a miss (treg catalog get says how many do); X-Treg-Route-Max-Cost caps the walk. To read up to ten URLs in one free call, call tinyfish.web.fetch directly with {"urls":[…]}; it does not fall back.

Platform data (social, stores, reviews, maps) is not free: cents or fractions of a cent a call. Say so before a batch.

Jobs

Search the catalog for the words in the second column. The vendors are examples of who answers today; take whatever the catalog ranks best now.

Finding the endpoint

The catalog changes every week, so never trust a remembered endpoint id.

1. treg catalog search "<the job in plain words>", not a vendor name.
2. Prefer a routed treg.* endpoint when there is one: the team's own keys first, then vendors in

order, and most misses are free.

1. treg catalog get <id>: the price, the vendors behind a routed job, and each one's measured

hit rate (it found something), not just works (it returned 200), and speed.

1. One small call first. Read _treg.served_by, _treg.tried and the reported charge, then scale.
2. Nothing fits: treg catalog request "<what is missing>".

Steering a routed job

Options ride as headers, never in the body:

Batches: always send a Max-Cost.

Habits that save money and tokens

• Read pages as markdown, and cut navigation and footers before they reach the context.
• A platform job returns fields; a scraped page of the same thing costs several times the tokens.
• Search, then read only the results you need. Never read a whole site to answer one question.
• Keep the source URL next to every fact you hand back.

Recipes

• Research a company: web search → read its site, pricing and docs pages → news search for the last

90 days → its LinkedIn company profile.

• Read a docs site into context: map the site with q set to the topic → read the top pages.
• Watch a page for changes: read it, keep a hash of the text, read it again next run and report the

diff. Schedule it with the user's agent (Claude Code /schedule or /loop, cron for others).

• Social listening: search posts on X, Reddit and LinkedIn → read the comments on the best ones →

summarise by theme with links.

• Compare a product across stores: search it on each store → product detail for the matches → one

table of price, rating and review count.

• Collect reviews: app store and Google Play for apps, Trustpilot and Glassdoor for companies, Google

Maps for places → quote the patterns, with counts.

• Videos into notes: YouTube search → video detail and comments → summarise.

Check the setup

treg --version && treg balance
treg call treg.web.search --data '{"q":"treg web access"}' --header 'X-Treg-Route-Prefer: tinyfish'

Over MCP: balance, then call with treg.web.search and {"q": "treg web access"}.

A result with _treg.served_by and no charge means free search works. Something wrong: treg feedback "<what happened>" reaches the team.

Lines not to cross

Public data only. Never log in as the user, ask for their cookies or get round a paywall. Don't bulk collect personal data about private individuals. When a vendor refuses a page, take the fallback or tell the user; don't hammer it.