Your First Map

In this guide, you'll use the Map endpoint to discover URLs under a website.

By the end of this guide, you'll know how to:

Send your first Map request.
Understand the request body.
Read the response.
Work with the discovered URLs.

Before You Begin

Before using the Map API, make sure you have:

A valid Geonode API key.
The Scraper API base URL.
A website URL to map.

The Map API discovers URLs by combining sitemap parsing with HTML link extraction from the provided website.

Step 1 — Send a Map Request

Send a POST request to the Map endpoint.

POST /v1/map

The minimum request only requires a website URL.

{
  "url": "https://geonode.com"
}

Step 2 — Understand the Request

The Map endpoint accepts the following request fields.

Field	Required	Description
`url`	Yes	Base URL to discover links from.
`search`	No	Filters discovered URLs using a case-insensitive substring match.
`include_subdomains`	No	Includes common sibling subdomains during discovery. Default: `true`.
`ignore_query_parameters`	No	Removes query parameters when normalizing discovered URLs. Default: `true`.

For example:

{
  "url": "https://geonode.com",
  "include_subdomains": true,
  "ignore_query_parameters": true
}

Step 3 — Review the Response

A successful request returns a response similar to:

{
  "success": true,
  "links": [
    {
      "url": "https://geonode.com/docs",
      "source": "html"
    },
    {
      "url": "https://geonode.com/blog",
      "source": "sitemap"
    }
  ]
}

The response contains three fields.

Field	Description
`success`	Indicates whether the request completed successfully.
`links`	List of discovered URLs.
`warning`	Optional non-fatal advisory message.

Understanding Discovered Links

Each discovered URL contains two values.

Field	Description
`url`	The discovered page URL.
`source`	Where the URL was discovered from (`html` or `sitemap`).

Filtering Results

Use the optional search field to return only URLs containing specific text.

For example:

{
  "url": "https://geonode.com",
  "search": "docs"
}

The search parameter performs a case-insensitive substring match against the discovered URLs. It does not query a search engine.

Working with the Results

After receiving the response, you can:

Review the discovered website structure.
Select specific pages for extraction.
Pass URLs to the Crawl API.
Build your own processing workflow.

Complete Workflow

Next Steps

Now that you've created your first Map request, continue to Configuring Map Requests to learn how to customize URL discovery using the available request parameters.

Your First Map

On this page