Configuring Batch Requests

Every batch request starts with a list of URLs.

You can further customize how those URLs are processed by configuring output formats, JavaScript rendering, proxy settings, custom headers, and other optional request parameters.

About the examples

All examples in this guide create a new batch job.

Regardless of which request options you use, the Batch endpoint immediately returns a job ID and begins processing the accepted URLs in the background.

Request Body Overview

The following request fields are available when creating a batch job.

Field	Required	Description
`urls`	✅	List of URLs to process.
`ignore_invalid_urls`	No	Continue processing even if some URLs are invalid.
`formats`	No	Choose the output format for extracted content.
`render_js`	No	Render JavaScript before extraction.
`processing_mode`	No	Configure how URLs are processed.
`proxy`	No	Route requests through a proxy.
`headers`	No	Include custom HTTP headers.
`wait_config`	No	Wait for dynamic content before extraction.

All of these options are optional except urls.

URLs

Every batch request must include one or more valid URLs.

{
  "urls": [
    "https://geonode.com",
    "https://docs.geonode.com"
  ]
}

Each URL is processed independently as part of the same batch job.

Ignoring Invalid URLs

By default, every URL must be valid.

If you want Geonode to continue processing valid URLs while skipping invalid ones, enable ignore_invalid_urls.

{
  "urls": [
    "https://geonode.com",
    "https://docs.geonode.com",
    "not-a-valid-url"
  ],
  "ignore_invalid_urls": true
}

For a complete example of how invalid URLs are handled, see Your First Batch.

Output Formats

Choose one or more output formats for the extracted content.

{
  "urls": [
    "https://geonode.com"
  ],
  "formats": [
    "markdown",
    "html"
  ]
}

Learn more in Output Formats.

JavaScript Rendering

Enable JavaScript rendering for websites that load content dynamically.

{
  "urls": [
    "https://geonode.com"
  ],
  "render_js": true
}

Learn more in JavaScript Rendering.

Processing Mode

Control how the batch request is processed.

{
  "urls": [
    "https://geonode.com"
  ],
  "processing_mode": "parallel"
}

Learn more in Processing Modes.

Proxy Configuration

Use a proxy when accessing geo-restricted or protected websites.

{
  "urls": [
    "https://geonode.com"
  ],
  "proxy": {
    "...": "..."
  }
}

Learn more in Proxy & Geo-Targeting.

Custom Headers

Include custom HTTP headers with every request.

{
  "urls": [
    "https://geonode.com"
  ],
  "headers": {
    "Authorization": "Bearer YOUR_TOKEN"
  }
}

Learn more in Using Custom Headers.

Waiting for Dynamic Content

Delay extraction until specific content becomes available.

{
  "urls": [
    "https://geonode.com"
  ],
  "wait_config": {
    "...": "..."
  }
}

Learn more in Waiting for Dynamic Content.

Complete Example

The following example combines several request options.

{
  "urls": [
    "https://geonode.com",
    "https://docs.geonode.com"
  ],
  "formats": [
    "markdown"
  ],
  "render_js": true,
  "ignore_invalid_urls": true
}

The response immediately returns a queued batch job.

{
  "job_id": "e9379def-8986-4624-86f1-f49c35afe711",
  "status": "queued",
  "status_url": "/v1/batch/e9379def-8986-4624-86f1-f49c35afe711",
  "accepted_urls": 2,
  "invalid_urls": []
}

Next Steps

Continue to Batch Results to learn how to monitor batch jobs, retrieve completed results, and understand the different job states.

Configuring Batch Requests

On this page