Batch Limits
Before creating a batch job, it's important to understand the limits and validation rules enforced by the API.
Following these guidelines helps prevent validation errors and ensures your requests are accepted successfully.
Batch Request Limits
The following limits apply when creating a batch job.
| Constraint | Details |
|---|---|
| Required field | The urls field is required. Omitting it returns a 422 Unprocessable Entity error. |
| Minimum URLs | A batch request must contain at least 1 URL. An empty array ([]) is not allowed. |
| Maximum URLs | A single batch request can contain up to 1000 URLs. |
| Duplicate URLs | Duplicate URLs are accepted and processed as separate requests. |
| Processing | Batch requests are processed asynchronously. A successful request returns a job_id instead of the extraction results. |
Required URLs Field
The urls field must always be included in the request body.
Invalid Request
{}Response
{
"detail": [
{
"type": "missing",
"loc": [
"body",
"urls"
],
"msg": "Field required"
}
]
}Empty URL List
Providing an empty array is also considered an invalid request.
Invalid Request
{
"urls": []
}Response
{
"detail": [
{
"type": "too_short",
"loc": [
"body",
"urls"
],
"msg": "List should have at least 1 item after validation, not 0"
}
]
}Maximum Batch Size
A single batch request can contain up to 1000 URLs.
If you need to process more than 1000 URLs, split them into multiple batch requests.
Duplicate URLs
Duplicate URLs are allowed.
Each URL in the request is treated as a separate extraction request, even if the same URL appears multiple times.
For example:
{
"urls": [
"https://example.com",
"https://example.com",
"https://example.com"
]
}All three URLs are accepted and processed independently.
Asynchronous Processing
Creating a batch job does not immediately return the extracted content.
Instead, the API returns a job_id that can be used to monitor the job until it completes.
{
"job_id": "9e60e068-b6ac-408d-aa6e-f120f41faf0d",
"status": "queued",
"status_url": "/v1/batch/9e60e068-b6ac-408d-aa6e-f120f41faf0d",
"accepted_urls": 3,
"invalid_urls": []
}Batch processing is asynchronous. After creating a batch job, use the returned job_id to check the job status and retrieve the results. The complete workflow is covered in the next guide.
Next Steps
Continue to Batch Processing Workflows to learn how to create a batch job, monitor its progress, and retrieve the final results.