Map Best Practices

The Map API is often the first step in a scraping workflow. Following a few best practices can help you discover relevant URLs, reduce unnecessary processing, and build more efficient applications.

Start with the Base Domain

Whenever possible, begin mapping from the root of the website.

For example:

https://example.com

instead of:

https://example.com/docs/getting-started

Starting from the base domain gives the Map API a broader view of the website and increases the chances of discovering all relevant pages.

Use search to Reduce Results

If you're only interested in a specific section of a website, use the search parameter.

For example:

{
  "url": "https://geonode.com",
  "search": "docs"
}

This reduces the number of returned URLs and makes it easier to work with the results.

Typical search values include:

• docs
• blog
• api
• support

Configure Subdomain Discovery Appropriately

The include_subdomains option controls whether supported subdomains are included during URL discovery.

Enable it when your content is distributed across multiple subdomains, such as:

docs.example.com
blog.example.com
support.example.com

Disable it if you only want to discover pages from the primary domain.

Decide How to Handle Query Parameters

Many websites generate multiple URLs that differ only by query parameters.

For example:

/products?page=1
/products?page=2
/products?sort=newest

When ignore_query_parameters is enabled, these URLs are normalized during discovery.

Disable this option only if the query parameters represent unique content that should be treated as separate pages.

Review Results Before Processing

The Map API discovers URLs—it does not extract page content.

Review the discovered URLs before deciding what to process next.

This approach helps reduce unnecessary extraction and crawling.

Reuse Existing Mapping Jobs

If you have already mapped a website, retrieve the existing job instead of creating another mapping request.

Reusing existing jobs helps avoid duplicate requests and allows your application to continue working with previously discovered URLs.

Handle Empty Results Gracefully

A successful request may return an empty links array.

For example:

{
  "success": true,
  "links": [],
  "warning": "No results found. If you targeted a sub-path, try mapping the base domain for broader coverage."
}

This does not indicate that the request failed.

Instead, consider:

• Mapping the base domain instead of a sub-path.
• Using a less restrictive search value.
• Verifying that the target website contains discoverable pages.

Build a Complete Mapping Workflow

The Map API works best as the first stage of a larger scraping pipeline.

This workflow allows you to discover URLs first and then process only the pages that are relevant to your application.

Best Practices Checklist

Before deploying your application, make sure you:

• Start mapping from the base domain whenever possible.
• Use search to narrow large result sets.
• Enable subdomain discovery only when needed.
• Configure query parameter handling based on your use case.
• Review discovered URLs before extraction or crawling.
• Reuse existing mapping jobs whenever possible.
• Handle empty results as a valid response.
• Build your workflow around URL discovery before content extraction.

Next Steps

You now understand how to use the Map API effectively.

Continue to the Map API Reference for detailed endpoint documentation, request schemas, response fields, and additional examples.