`-as-2501` | Specify region or ASN (state and city cannot be combined) | | **Protocol** | HTTP / HTTPS | Default port 9000 | | **Session Type** | — | Rotate or Sticky sessions | | **Output Format** | — | Customize response format |
`-as-2501` | Specify region or ASN (state and city cannot be combined) | | **Protocol** | HTTP / HTTPS | Default port 9000 | | **Session Type** | — | Rotate or Sticky sessions | | **Output Format** | — | Customize response format |
Request-Based
Pay based on successful page extractions and choose the request capacity that fits your workload.
View Request-Based PricingUnlimited
Get unlimited requests with a fixed monthly price based on concurrent threads.
View Unlimited Pricing
### Choose Request-Based if
* Your volume is small or unpredictable.
* You want to pay based on request usage.
* You do not need high concurrency.
* You are testing or starting a project.
### Choose Unlimited if
* Your volume is consistently high.
* You need many extractions to run in parallel.
* You want predictable monthly pricing.
* You regularly process large scraping workloads.
## You Can Change Plans as Your Workload Grows
Your initial choice does not need to be permanent.
You can start with Request-Based pricing while evaluating your workload. If your extraction volume grows or you need more parallel processing, you can move to an Unlimited plan.
Similarly, if your workload does not require high concurrency, a request-based plan may be more appropriate.
The best plan is the one that matches your actual workload rather than simply choosing the plan with the highest limits.
## Related Guides
- Request-Based Pricing
- Unlimited Scraper API Pricing
- Extract a Single URL
- Extract Multiple URLs
local"] end subgraph products["Product pages"] direction LR C["3. Batch PDPs
POST /v1/batch"] --> D["4. Parse
local"] D --> E["5. Merge"] E --> F["amazon_products.json"] end discovery --> C classDef api fill:#e8f5f3,stroke:#054D4D,color:#033333 classDef local fill:#f5f5f5,stroke:#666,color:#222 classDef result fill:#fff8e6,stroke:#b8860b,color:#333 class A,C api class B,D,E local class F result `} /> | API | Why we use it here | | -------------------------------------------------------------------------- | ------------------------------------------- | | [Extract](/docs/scraper-api/guides/extraction/01_understanding_extraction) | One known SERP URL; JS + residential + wait | | [Batch](/docs/scraper-api/guides/batch/00_understanding_batch) | Many `/dp/` URLs with the same settings | | Local parse | ASINs and product fields | **Geonode Search is not used.** You already know `amazon.com`. **Map is not used.** You are not inventorying a sitemap; you start from one search URL. **Crawl is not used.** You do not want an unbounded walk of Amazon. Compare with other real-world guides: | Guide | Difference | | ------------------------------------------------------------------------------------------------- | ---------------------------------------------- | | [Retail category price catalog](/docs/scraper-api/real-world/build_retail_category_price_catalog) | Sitemap + PLP pagination on a lighter retailer | | [Docs knowledge corpus](/docs/scraper-api/real-world/build_docs_knowledge_corpus) | Crawl public docs | | [Basic Auth headers](/docs/scraper-api/real-world/build_authenticated_basic_auth) | Authenticated Extract, not marketplace SERP | *** ## Before you start You need: * A Geonode API key * Python 3.9+, `requests`, `python-dotenv` ```bash export GEONODE_SCRAPER_API_KEY="YOUR_API_KEY" ``` Optional: ```bash export AMAZON_SEARCH_URL="https://www.amazon.com/s?k=logitech+mx+master" export AMAZON_MAX_ASINS="10" ``` *** ## Step 1: Extract the search results page ### What we need HTML/Markdown of one Amazon SERP so we can collect ASINs. ### What we will do `POST /v1/extract` with: * `render_js: true` * `proxy: { "country": "US", "type": "residential" }` * `wait_config` with `wait_until: "networkidle"` (Amazon is a heavy SPA) ### What you should expect A large Markdown/HTML blob with `/dp/{ASIN}` links, or a soft block / empty shell. Save raw output either way so you can debug. ```python import os import requests api_key = os.environ["GEONODE_SCRAPER_API_KEY"] url = os.environ.get( "AMAZON_SEARCH_URL", "https://www.amazon.com/s?k=logitech+mx+master", ) response = requests.post( "https://scraper.geonode.io/v1/extract", headers={ "X-Api-Key": api_key, "Content-Type": "application/json", }, json={ "url": url, "formats": ["markdown", "html"], "render_js": True, "processing_mode": "sync", "proxy": {"country": "US", "type": "residential"}, "wait_config": { "wait_until": "networkidle", "wait_timeout": 20000, }, }, ) # Save markdown / html for ASIN parsing ```
Basic Auth"] B --> C["3. Batch
same header"] end subgraph output["Output"] direction LR D["4. Parse"] --> E["5. Merge"] E --> F["auth_corpus.json"] end auth_flow --> D classDef api fill:#e8f5f3,stroke:#054D4D,color:#033333 classDef local fill:#f5f5f5,stroke:#666,color:#222 classDef result fill:#fff8e6,stroke:#b8860b,color:#333 class A,B,C api class D,E local class F result `} /> | API | Why we use it here | | ---------------------------------------------------------------------------------- | --------------------------------------- | | [Extract](/docs/scraper-api/guides/extraction/01_understanding_extraction) | Clear before/after on one URL | | [Batch](/docs/scraper-api/guides/batch/00_understanding_batch) | Same Basic header on several URLs | | [Custom headers](/docs/scraper-api/guides/making-requests/07_using_custom_headers) | `Authorization` is a target-site header | Compare with other real-world guides (all **public**, no auth): | Guide | Auth | | ------------------------------------------------------------------------------------------------- | -------------------------- | | [B2B agency lead list](/docs/scraper-api/real-world/build_b2b_agency_lead_list) | None | | [Retail category price catalog](/docs/scraper-api/real-world/build_retail_category_price_catalog) | None | | [Docs knowledge corpus](/docs/scraper-api/real-world/build_docs_knowledge_corpus) | None | | **This guide** | **`Authorization: Basic`** | *** ## Before you start * A Geonode API key * Python 3.9+, `requests` ```bash export GEONODE_SCRAPER_API_KEY="YOUR_API_KEY" ```
/v1/map"] C -->|"JS grids → 0 links"| D["3. Extract
/v1/extract + JS"] end subgraph enrich["Enrich"] direction LR subgraph profiles["Profiles"] direction TB E["4. Batch
/v1/batch"] --> F["5. Parse
local"] end subgraph contacts["Contacts"] direction TB G["6. Batch sites
/v1/batch"] --> H["7. Parse
emails / phones"] end end subgraph output["Output"] direction LR I["8. Merge"] --> J["companies_master.json"] end discover --> enrich F --> G F --> I H --> I classDef api fill:#e8f5f3,stroke:#054D4D,color:#033333 classDef local fill:#f5f5f5,stroke:#666,color:#222 classDef result fill:#fff8e6,stroke:#b8860b,color:#333 class A,C,D,E,G api class F,H,I local class B,J result `} /> | API | Why we use it here | | -------------------------------------------------------------------------- | ------------------------------------------------- | | [Search](/docs/scraper-api/guides/search/01_search_overview) | You do not know the best listing URL yet | | [Map](/docs/scraper-api/guides/map/00_understanding_map) | Fast URL inventory check: often fails on JS grids | | [Extract](/docs/scraper-api/guides/extraction/01_understanding_extraction) | Render JS listing pages and collect profile links | | [Batch](/docs/scraper-api/guides/batch/00_understanding_batch) | Many known URLs (profiles, then homepages) | | Local parsing | Turn HTML into structured fields, then merge | **Crawl is not used.** After Extract you already know the profile URLs. After profile parse you already know the websites. You do not need a deep multi-page walk of one domain. *** ## Before you start You need: * A Geonode API key * Python 3.9 or later * The `requests` package ```bash export GEONODE_SCRAPER_API_KEY="YOUR_API_KEY" pip install requests ``` On Windows PowerShell: ```powershell $env:GEONODE_SCRAPER_API_KEY="YOUR_API_KEY" pip install requests ```
local"] --> E["3. Merge"] E --> F["docs_corpus.json"] end crawl_job --> D classDef api fill:#e8f5f3,stroke:#054D4D,color:#033333 classDef local fill:#f5f5f5,stroke:#666,color:#222 classDef result fill:#fff8e6,stroke:#b8860b,color:#333 class A,B api class D,E local class C,F result `} /> | API | Why we use it here | | ------------------------------------------------------ | ----------------------------------------------------------- | | [Crawl](/docs/scraper-api/guides/crawl/01_first-crawl) | One seed, unknown linked tree, content + discovery together | | Local parsing | Turn Markdown into indexable fields | **Search is not used.** You already know the docs URL. **Map is not required.** Map would only return URLs; you would still Batch Extract. Crawl returns content. **Batch is not required.** You do not have a URL list until Crawl finishes, and by then pages are already extracted. Compare with other real-world guides: | Guide | Why Crawl was skipped | | ------------------------------------------------------------------------------------------------- | ---------------------------------------------------- | | [B2B agency lead list](/docs/scraper-api/real-world/build_b2b_agency_lead_list) | Directory + pagination gave profile URLs; then Batch | | [Retail category price catalog](/docs/scraper-api/real-world/build_retail_category_price_catalog) | Sitemap + PLP Extract gave product URLs; then Batch | *** ## Before you start You need: * A Geonode API key * Python 3.9 or later * The `requests` package ```bash export GEONODE_SCRAPER_API_KEY="YOUR_API_KEY" pip install requests ``` On Windows PowerShell: ```powershell $env:GEONODE_SCRAPER_API_KEY="YOUR_API_KEY" pip install requests ```
/v1/map"] C --> D["Category PLPs /c/..."] end subgraph products["Collect products"] direction LR E["3. Extract PLP
/v1/extract + JS + page=N"] --> F["Product URLs"] F --> G["4. Batch PDPs
/v1/batch"] end subgraph output["Output"] direction LR H["5. Parse HTML
local"] --> I["6. Merge"] I --> J["products_master.json"] end discover --> products G --> H classDef api fill:#e8f5f3,stroke:#054D4D,color:#033333 classDef local fill:#f5f5f5,stroke:#666,color:#222 classDef result fill:#fff8e6,stroke:#b8860b,color:#333 class A,C,E,G api class H,I local class B,D,F,J result `} /> | API | Why we use it here | | -------------------------------------------------------------------------- | --------------------------------------------------------------------- | | [Map](/docs/scraper-api/guides/map/00_understanding_map) | Fast URL inventory from sitemap XML and taxonomy | | [Extract](/docs/scraper-api/guides/extraction/01_understanding_extraction) | Render a JS category grid and parse product links; one call = one URL | | [Batch](/docs/scraper-api/guides/batch/00_understanding_batch) | Many known PDP URLs in one job | | Local parsing | Turn HTML into price/SKU fields, then merge | **Search is not used.** You already know the retailer. **Crawl is not required for this demo.** After Extract you already have product URLs. Crawl is the right tool later if you want a BFS walk of one category with a page `limit` instead of looping `?page=`. *** ## Before you start You need: * A Geonode API key * Python 3.9 or later * The `requests` package ```bash export GEONODE_SCRAPER_API_KEY="YOUR_API_KEY" pip install requests ``` On Windows PowerShell: ```powershell $env:GEONODE_SCRAPER_API_KEY="YOUR_API_KEY" pip install requests ```
- Output Formats
- JavaScript Rendering
- Waiting for Dynamic Content
- Processing Modes
- Proxy and Geo-Targeting
- Using Custom Headers
- Understanding Map
- Map Workflows
- Map Best Practices
- Understanding Map
- Map Workflows
- Map Best Practices
job\_id or status\_url to monitor progress and retrieve results.