--- name: geonode description: | Geonode gives AI agents and apps web scraping and proxy infrastructure: extract, batch, crawl, map, and search via Scraper API and MCP, plus residential and ISP proxies. This skill routes coding agents to MCP first, REST when integrating into app code, and existing Geonode docs for setup. --- # Geonode Geonode helps agents extract clean Markdown or HTML from webpages, process many URLs in batch, crawl and map sites, run search jobs, and route traffic through residential or ISP proxies when needed. Human documentation (source of truth for setup details): - Web Data / Scraper API: https://docs.geonode.com/docs/scraper-api - MCP guides: https://docs.geonode.com/docs/scraper-api/guides/mcp/00_overview - Proxies: https://docs.geonode.com/docs/proxies ## What this skill is for Use this skill when: - The agent needs live web content during a session (prefer MCP) - The agent is adding Geonode Scraper API calls to application code (REST) - The user asks how to connect Cursor, Claude, VS Code, or similar to Geonode Do **not** invent a Geonode CLI. Today the agent surfaces are **MCP** and **REST**. ## Get credentials Every Scraper API and MCP request needs a Geonode API key. 1. Human signs in at https://app.geonode.com/ 2. Open **API Keys** 3. Create or copy a key 4. Store it securely (env var / MCP client config). Never commit API keys. Quick start for humans: https://docs.geonode.com/docs/scraper-api/quick-start Auth header for REST and recommended MCP config: ```http X-Api-Key: YOUR_API_KEY ``` Some MCP clients cannot set custom headers. In those cases, pass `api_key` as a tool argument. Prefer `X-Api-Key` when the client supports it. ## Choose your path | Need | Path | | --- | --- | | Live scrape/crawl/batch/search inside this agent session | Path A — MCP | | Ship Geonode calls inside the user's product code | Path B — REST | | Configure a specific IDE/client for a human | Path C — MCP client setup docs | | Only need proxy IPs / sticky sessions / geo routing | Path D — Proxies docs | --- ## Path A: MCP (default for live agent work) MCP endpoint: ```text https://scraper.geonode.io/mcp ``` Configure the MCP client with that URL and the human's API key (`X-Api-Key`). Prerequisites and tools: https://docs.geonode.com/docs/scraper-api/guides/mcp/01_before-you-start ### Task routing (MCP tools) | User intent | Prefer tool | | --- | --- | | Extract one page | `extract` | | Check / list extraction jobs | `job`, `jobs` | | Process many URLs | `batch`, then `batch_status` / `cancel_batch` | | Crawl a site | `crawl`, then `crawl_status` / `cancel_crawl` | | Usage / success metrics | `statistics` | Describe the task in plain language; the assistant should pick the tool. If MCP is not configured in this environment, use Path B (REST) or ask the human to complete Path C for their client. --- ## Path B: REST (integrate into app code) Base URL: ```text https://scraper.geonode.io ``` Send `X-Api-Key` on every request. ### Product routing | Need | Product / guide | | --- | --- | | One URL → Markdown/HTML | [Extraction](https://docs.geonode.com/docs/scraper-api/guides/extraction/01_understanding_extraction) | | Many URLs in one job | [Batch](https://docs.geonode.com/docs/scraper-api/guides/batch/00_understanding_batch) | | Discover + extract a site | [Crawl](https://docs.geonode.com/docs/scraper-api/guides/crawl/00_overview) | | Discover URLs under a base URL | [Map](https://docs.geonode.com/docs/scraper-api/guides/map/00_understanding_map) | | Search query → results | [Search](https://docs.geonode.com/docs/scraper-api/guides/search/01_search_overview) | API reference entry: https://docs.geonode.com/docs/scraper-api/v1 When writing integration code: 1. Read the matching product guide and endpoint docs 2. Put `GEONODE_API_KEY` (or project equivalent) in env — never hardcode 3. Smoke-test one successful call before expanding the feature --- ## Path C: MCP client setup (humans) Point the human at the guide for their client: - Overview: https://docs.geonode.com/docs/scraper-api/guides/mcp/00_overview - Before you start: https://docs.geonode.com/docs/scraper-api/guides/mcp/01_before-you-start - Cursor: https://docs.geonode.com/docs/scraper-api/guides/mcp/04_cursor - Claude Code: https://docs.geonode.com/docs/scraper-api/guides/mcp/02_claude-code - Visual Studio Code: https://docs.geonode.com/docs/scraper-api/guides/mcp/08_visual-studio-code - Codex: https://docs.geonode.com/docs/scraper-api/guides/mcp/09_codex - Devin AI: https://docs.geonode.com/docs/scraper-api/guides/mcp/07_devin-ai - Docker MCP: https://docs.geonode.com/docs/scraper-api/guides/mcp/05_docker-mcp --- ## Path D: Proxies (separate product) Use Proxies docs when the task is proxy configuration, sticky sessions, or geo targeting — not page extraction. - Proxies overview: https://docs.geonode.com/docs/proxies - Residential: https://docs.geonode.com/docs/proxies/guides/residential-proxies/overview - Unlimited Residential: https://docs.geonode.com/docs/proxies/guides/unlimited-residential-proxies/00_unlimited_residential_proxies - ISP: https://docs.geonode.com/docs/proxies/guides/isp-proxies/isp-proxies For scraping page content, prefer Scraper API / MCP (Paths A–B). Proxies can still be used underneath Scraper API geo-targeting when the API supports it. --- ## Defaults for agents 1. Prefer **MCP** for live web work in-session 2. Prefer **REST** when shipping durable product code 3. Link humans to existing docs instead of inventing new setup steps 4. Never commit or log API keys 5. On auth failures, send the human to create/copy a key in the dashboard