Geonode
API ReferenceGuidesScraper APIChange Log
Scraper apiGeneratedMap

Discover all URLs under a base URL

Returns the list of URLs found under the given base URL by combining sitemap parsing with HTML link extraction from the seed page. The optional search parameter filters the discovered URLs by case-insensitive substring match — it does NOT query a search engine.

POST
/v1/map

Authorization

X-Api-Key<token>

In: header

Request Body

urlstring

Base URL to discover links from

Formaturi
Length1 <= length
search?string|null
include_subdomains?boolean

Probe common sibling subdomains (docs, blog, help, support) for their own sitemaps

Defaulttrue
ignore_query_parameters?boolean

Strip query parameters when normalizing discovered URLs

Defaulttrue

Response Body

application/json

application/json

application/json

application/json

application/json

application/json

application/json

application/json

application/json

curl -X POST "https://scraper.geonode.io/v1/map" \  -H "Content-Type: application/json" \  -d '{    "url": "http://example.com"  }'
{
  "success": true,
  "links": [
    {
      "url": "string",
      "source": "sitemap"
    }
  ],
  "warning": "string"
}
{
  "code": "string",
  "message": "string",
  "correlation_id": "string",
  "retryable": true
}
{
  "code": "string",
  "message": "string",
  "correlation_id": "string",
  "retryable": true
}
{
  "code": "string",
  "message": "string",
  "correlation_id": "string",
  "retryable": true
}
{
  "code": "string",
  "message": "string",
  "correlation_id": "string",
  "retryable": true
}
{
  "detail": [
    {
      "loc": [
        "string"
      ],
      "msg": "string",
      "type": "string"
    }
  ]
}
{
  "code": "string",
  "message": "string",
  "correlation_id": "string",
  "retryable": true
}
{
  "code": "string",
  "message": "string",
  "correlation_id": "string",
  "retryable": true
}
{
  "code": "string",
  "message": "string",
  "correlation_id": "string",
  "retryable": true
}