nirholas/xactions/twitter-scraping

Twitter Scraping

Auth Setup

All scrapers require an auth_token cookie from x.com (DevTools -> Application -> Cookies -> copy auth_token value).

Node.js Scraper API

npm install xactions -- all functions from src/scrapers/index.js.

import { createBrowser, createPage, loginWithCookie, scrapeProfile } from 'xactions';
const browser = await createBrowser();
const page = await createPage(browser);
await loginWithCookie(page, AUTH_TOKEN);
const profile = await scrapeProfile(page, 'nichxbt');
await browser.close();

Browser Console Scripts

Standalone IIFEs -- paste into DevTools console on x.com. No dependencies.

Interactive browser toolbox (recommended for console scraping)

For paste-in-console scraping, prefer scripts/twitter/scraper-toolbox.js: an on-page control panel (start/pause/stop, live filters, user exclusions, keyword rules) that captures X's own GraphQL responses, so counts are exact integers and long posts are untruncated. Exports JSON/CSV/Markdown/TXT/HTML and copies to clipboard. Works on any timeline: profile, search, list, likes, bookmarks, home. Docs: scripts/twitter/README-scraper-toolbox.md.

MCP Scraping Tools

All scrapers available via the MCP server for AI agents:

Notes

• Always call browser.close() when done with Node.js scrapers
• Browser scripts stop on page navigation -- stay on the page while running
• All scripts support JSON and CSV export
• Increase limit parameter for deeper scraping (default varies by function)
• Rate limiting is built in -- avoid reducing delays below defaults