hedgehog-finance/hedgehog-skills/web-fetch

Web Fetch

Portable CLI parameters

When a documented CLI accepts a parameter object, use the same rule on every Agent and operating system; existing positional file inputs remain positional:

1. When every business value is a non-empty, single-line string | finite number | boolean, pass it as a named argument (--key value or --key=value). Names are case-sensitive and are not normalized.
2. When any value is an object, array, null, multiline text, a numeric/boolean-looking string that must remain a string, or contains difficult quoting, write the complete parameter object as UTF-8 JSON and pass the file option documented by this Skill.
3. Agent-created parameter files must have a unique basename matching tmp-<skill-name>-<unique-id>.json, must not use the reserved .hedgehog/ directory, and must be removed after the call when no longer needed. UTF-8 BOM is accepted.
4. Do not inline nested JSON or combine flat arguments with a JSON/file payload. Create JSON with the Agent's file-writing capability, not echo, a shell heredoc, or PowerShell string assembly.

POSIX/Git Bash form: node '<script>' --key 'single-line value' or node '<script>' <file-option> '<workspace>/tmp-<skill-name>-<id>.json'.

PowerShell form: node "<script>" --key "single-line value" or node "<script>" <file-option> "<workspace>\\tmp-<skill-name>-<id>.json".

On Windows, use PowerShell or a verified Git for Windows Bash; cmd.exe is unsupported. Keep each command on one physical line. The process runs with the current Agent user's permissions and that Agent's native sandbox; HogAgent marks its Windows shell as UNSANDBOXED.

Fetch a web page URL and extract its main content into clean Markdown format.

Runtime

• Node.js: >=18

Dependencies

Install the packages declared in this Skill's package.json before first use:

npm install --prefix "<skill_path>"

Replace <skill_path> with the directory containing this SKILL.md. Run the command again if node_modules is absent, after reinstalling/updating the Skill, or when Node reports Cannot find package / Cannot find module.

Usage

node <skill_path>/cli.mjs --url "<url>" [--max-length N] [--output save --dir <sessionTaskDir>]

Where <skill_path> is the actual installed path of this skill.

Parameters

Output Strategy (Token Efficiency)

• When --output save --dir <sessionTaskDir> is used: full content saved to sessionTaskDir/data-<datetime>-<N>.md, only an 800-token preview + file path printed to stdout
• When content exceeds 1600 tokens (even without --output save), auto-saves if --dir is provided
• After save, access data via: read(path, offset, limit) or bash("head -20 <file>")
• Prohibited: full read-back of a saved file into context

Examples

# Basic fetch
node <skill_path>/cli.mjs --url "https://example.com/article"
 
# Fetch with length limit
node <skill_path>/cli.mjs --url "https://example.com/article" --max-length 4000
 
# Fetch and save to task directory (recommended for large pages)
node <skill_path>/cli.mjs --url "https://example.com/long-article" --output save --dir <sessionTaskDir>

Output Format

Direct Output (default)

# [Article Title]
 
[Extracted Markdown content...]

Save Mode Output

[WebFetch Saved] <filepath>
URL: <url>
Title: <title>
Size: <chars> chars
 
Preview:
[First ~800 tokens of content...]
 
Hint: read("<filepath>", offset, limit) to view full content

Constraints

• Fetch timeout: 30 seconds
• Non-HTML responses returned as plain text (max 10000 chars)
• Invalid or unreachable URLs return an error message
• Content is extracted via Readability algorithm; pages without article structure fall back to full body conversion
• Response bodies are limited to 10 MiB, including streaming and non-streaming runtimes. Redirect targets remain HTTP(S) URLs without embedded credentials.
• Saved files use exclusive creation with collision-safe names, so concurrent calls never overwrite an existing result.

落盘来源与目录

Gateway 托管运行使用明确的 SessionTaskDir；Development 的 --dir 使用正式项目 data 目录，--artifact-root 使用项目根。缺少运行目录时先报告上下文缺口，不回退到 workspace。独立 CLI 保留明确指定输出目录的用法。

落盘调用增加 --artifact-root <SessionTaskDir或项目根>。它只决定来源注释归属，不改变 --dir、--out 的基准。脚本按实际文件内容写脱敏来源注释；不要手工编辑 .hedgehog。注释失败保留已下载数据并提示，不重复请求接口。旧调用省略该参数仍可落盘，但不会登记来源。