Geonode
Getting StartedProxiesWeb DataChange Log
Scraper apiGeneratedMap

Discover all URLs under a base URL

POST
/v1/map

Returns the list of URLs found under the given base URL by combining sitemap parsing with HTML link extraction from the seed page. By default discovery is scoped to the exact seed host (a leading www. is treated as equivalent) and, when the URL includes a path, to URLs under that path; sibling subdomains and outbound links are excluded. Set include_subdomains=true to widen the scope to the whole registrable domain and probe common subdomains (docs, blog, help, support). The optional search parameter filters the discovered URLs by case-insensitive substring match — it does NOT query a search engine.

Authorization

ApiKeyAuth
X-Api-Key<token>

In: header

Request Body

application/json

TypeScript Definitions

Use the request body type in TypeScript.

Response Body

application/json

application/json

application/json

application/json

application/json

application/json

application/json

application/json

application/json

curl -X POST "https://example.com/v1/map" \  -H "Content-Type: application/json" \  -d '{    "url": "http://example.com"  }'
{
  "success": true,
  "links": [
    {
      "url": "string",
      "source": "sitemap"
    }
  ],
  "metadata": {
    "url": "string",
    "duration_ms": 0,
    "links_count": 0
  },
  "warning": "string"
}
{
  "code": "string",
  "message": "string",
  "correlation_id": "string",
  "retryable": true
}
{
  "code": "string",
  "message": "string",
  "correlation_id": "string",
  "retryable": true
}
{
  "code": "string",
  "message": "string",
  "correlation_id": "string",
  "retryable": true
}
{
  "code": "string",
  "message": "string",
  "correlation_id": "string",
  "retryable": true
}
{
  "detail": [
    {
      "loc": [
        "string"
      ],
      "msg": "string",
      "type": "string"
    }
  ]
}
{
  "code": "string",
  "message": "string",
  "correlation_id": "string",
  "retryable": true
}
{
  "code": "string",
  "message": "string",
  "correlation_id": "string",
  "retryable": true
}
{
  "code": "string",
  "message": "string",
  "correlation_id": "string",
  "retryable": true
}