groktopus/groktocrawl/spa-vs-article-scraping

Claude Code and agent skill by groktopus/groktocrawl

Trust tier 3
  • community
Source check pending

Checking that the SKILL.md file is still in the repository.

Diagnostic pattern: when groktocrawl scrape fails on a SPA index page, individual server-rendered article pages may still work. Test one article before committing to browser extraction.

Install

npx vskill@latest install groktopus/groktocrawl/spa-vs-article-scraping

SKILL.md

SPA Index vs. Server-Rendered Article Scraping

The Pattern

When groktocrawl scrape fails on a site's index page (React SPA shell, < 100 chars), individual article pages on the same domain may still be server-rendered and scrape successfully. Always test scrape on one individual article page before committing to browser-based extraction.

Diagnostic Workflow

  1. Try groktocrawl scrape <index-page> — if < 100 chars, index is client-rendered
  2. Check for a sitemap: curl -sL <domain>/sitemap.xml
  3. Test scrape on one article: groktocrawl scrape <article-url>
  4. If article returns rich markdown (> 500 chars), batch-scrape all articles

Worked Example: YC Library (June 2026)

  • Index: ycombinator.com/library → empty (SPA shell)
  • Sitemap: ycombinator.com/library/sitemap.xml → 464 article URLs
  • Article test: ycombinator.com/library/4D-slug → full server-rendered markdown
  • Batch: 464 articles × ~7s each = ~15 min total extraction
  • Savings: ~50+ minutes vs browser-based extraction

Security

  • Security scanPassed, 90 of 100Pattern checks on the skill's files.
  • Model reviewPassed

See the full security report

Quality evaluation

  • Task evaluationNot runNo evaluation yet. Run one in Skill Studio.

See evaluation results

Source

Popularity

Repository stars
89
Repository forks
14
7-day trend score
0

Stars and forks are for the whole GitHub repository, not this skill alone. Not refreshed yet.

Works with

Any agent vskill installs to: 56 targets in all. These 39 have a page on this site.

  • Claude Code
  • Cursor
  • GitHub Copilot
  • Windsurf
  • Codex
  • Gemini CLI
  • Cline
  • Amp
  • Roo Code
  • Goose
  • Continue
  • Kilo Code
  • Junie
  • Trae
  • OpenHands
  • Qwen Code
  • AI Agent
  • Tabnine Chat
  • Sourcegraph Cody
  • Amazon Q
  • Firebase Genkit
  • Sweep
  • Devin
  • Phind
  • Mentat
  • Claude Desktop
  • Aider
  • GPT Pilot
  • Plandex
  • bolt.new
  • v0
  • Replit Agent
  • Anse
  • TypingMind
  • BetterChatGPT
  • LibreChat
  • Warp AI
  • Pieces
  • Fig AI
Embed badge
vskill VERIFIED badge
[![vskill VERIFIED](https://verified-skill.com/api/v1/skills/groktopus/groktocrawl/spa-vs-article-scraping/badge)](https://verified-skill.com/skills/groktopus/groktocrawl/spa-vs-article-scraping)

Something wrong with this skill or its page? Tell us.