Monitoring Batch Jobs

Batch jobs are processed asynchronously.

After creating a batch job, use the returned job_id to monitor its progress and retrieve the extraction results.

Retrieve a Batch Job

Use the following endpoint to retrieve the latest status and results for a batch job.

GET /v1/batch/{job_id}

Request

curl -X GET "https://scraper.geonode.io/v1/batch/YOUR_JOB_ID" \
  -H "X-Api-Key: YOUR_API_KEY"

Replace YOUR_JOB_ID with the job ID returned when the batch was created.

Example Response

{
  "job_id": "d54da3de-7afc-4127-a5ee-5f7e0378a3e6",
  "status": "completed",
  "created_at": "2026-07-05T16:14:22.721Z",
  "completed_at": "2026-07-05T16:14:25.318Z",
  "total_urls": 3,
  "completed_urls": 2,
  "failed_urls": 1,
  "pending_urls": 0,
  "cancelled_urls": 0,
  "token_summary": {
    "tokens_charged_total": 2,
    "tokens_reserved": 0
  },
  "results": [
    {
      "input_index": 0,
      "url": "https://geonode.com",
      "status": "completed"
    },
    {
      "input_index": 1,
      "url": "https://docs.geonode.com",
      "status": "completed"
    },
    {
      "input_index": 2,
      "url": "https://this-domain-does-not-exist-123456789.com",
      "status": "failed"
    }
  ]
}

Job Information

The top-level response describes the overall batch job.

Processing Statistics

These fields summarize the progress of the batch.

Token Usage

The response also includes a summary of token usage.

Individual Results

Each submitted URL appears in the results array.

Each result contains information about a single URL.

Job Status

Each URL has its own processing status.

Partial Failures

A batch job can complete successfully even if some URLs fail.

For example, a batch containing three URLs may produce:

• 2 completed URLs
• 1 failed URL
• Overall batch status: completed

Failed URLs include additional error information.

{
  "url": "https://this-domain-does-not-exist-123456789.com",
  "status": "failed",
  "error_code": "INTERNAL_ERROR",
  "error_message": "An unexpected error occurred on the server"
}

This allows you to retry only the failed URLs instead of rerunning the entire batch.

Metadata

Each completed result includes processing metadata.

This information can help monitor performance and troubleshoot extraction issues.

Download All Results as One Zip

Once a batch has finished, you can download every result in a single zip file instead of paging through the results.

GET /v1/batch/{job_id}/download

Request

curl -X GET "https://scraper.geonode.io/v1/batch/YOUR_JOB_ID/download" \
  -H "X-Api-Key: YOUR_API_KEY"

Example Response

{
  "job_id": "07c70a3b-0a74-4153-be5e-6ba0297ede12",
  "download_url": "https://...",
  "expires_at": "2026-11-10T11:28:33Z"
}

Open download_url to download the file batch-{job_id}.zip. The link works for one hour, until expires_at. Call the endpoint again to get a new link to the same archive. This request does not consume tokens.

Responses

What Is in the Zip

• One file per successful URL for each requested format, for example 0007-example.com-docs-intro.md and 0007-example.com-docs-intro.html. The number at the start is the position of the URL in the batch, so the files keep the order in which the URLs were submitted.
• A manifest.json file with the totals for the batch and a skipped list of every URL that has no file.

{
  "job_id": "07c70a3b-0a74-4153-be5e-6ba0297ede12",
  "status": "completed",
  "formats": ["markdown", "html"],
  "generated_at": "2026-11-10T10:28:14+00:00",
  "files": 4,
  "total_urls": 3,
  "completed_urls": 2,
  "failed_urls": 1,
  "cancelled_urls": 0,
  "invalid_urls": 0,
  "skipped": [
    {
      "input_index": 2,
      "url": "https://this-domain-does-not-exist-123456789.com",
      "status": "failed",
      "error_code": "INTERNAL_ERROR"
    }
  ]
}

Best Practices

When working with batch jobs:

• Save the returned job_id so you can retrieve the results later.
• Wait until the job status is completed before using the extracted data.
• Review the processing statistics to quickly identify failed URLs.
• Retry only failed URLs instead of rerunning the entire batch.
• Download the results as one zip within 7 days of the batch finishing.
• Monitor token usage when processing large batches.

Next Steps

Now that you know how to monitor batch jobs and retrieve their results, continue to Batch Workflows to learn common patterns for processing large collections of URLs.