Batch Workflows

A batch extraction is an asynchronous workflow. Instead of waiting for every URL to finish processing, the API immediately creates a job and returns a unique job ID.

That job ID becomes the center of the workflow. You use it to monitor progress, retrieve results, or cancel the job if necessary.

Complete Batch Workflow

The following diagram shows how the Batch API endpoints work together.

Every batch job follows this lifecycle, from creation to completion.

Typical Workflow

Most applications follow these steps when working with batch extraction.

Step 1 — Create a Batch

Start by submitting one or more URLs.

POST /v1/batch

The API immediately returns:

• job_id
• status
• status_url
• accepted_urls

At this point, the extraction has been queued and continues in the background.

Step 2 — Store the Job ID

Save the returned job_id.

You'll need this value for every subsequent operation, including:

• Monitoring progress
• Viewing results
• Cancelling the batch

Step 3 — Monitor Progress

Use the Job Status endpoint until the batch finishes.

GET /v1/batch/{job_id}

A batch can move through the following states:

queued
processing
completed
failed
cancelled

Most applications poll this endpoint every few seconds until processing finishes.

Step 4 — Process the Results

Once the job reaches the completed state, the response contains:

• Successfully processed URLs
• Failed URLs
• Output for each completed extraction

Your application can now download, store, or process the extracted content.

Finding Previous Jobs

Sometimes an application loses the job ID due to a restart or network interruption.

Instead of creating another batch, retrieve the existing one.

GET /v1/batch/jobs

This prevents duplicate processing and allows your application to continue from where it left off.

Cancelling a Batch

You can cancel a running batch if it is no longer needed.

Common reasons include:

• Incorrect URLs
• Wrong proxy configuration
• Incorrect request settings
• User cancelled the operation

DELETE /v1/batch/{job_id}

After cancellation:

• No new URLs are scheduled for processing.
• URLs that are already running may finish.
• Remaining URLs are skipped.

End-to-End Example

The complete workflow usually looks like this.

Common Workflow Patterns

Standard Batch Processing

This is the most common workflow for asynchronous processing.

Recovering After an Application Restart

This allows applications to recover without creating duplicate jobs.

Cancelling an Active Batch

Best Practices

• Store the returned job_id immediately after creating a batch.
• Poll the Job Status endpoint instead of submitting duplicate batch requests.
• Use the List Jobs endpoint to recover lost job IDs.
• Cancel jobs that are no longer required instead of letting them continue processing.
• Wait until the batch reaches the completed state before processing the final results.
• Handle every possible job status (queued, processing, completed, failed, and cancelled) in your application.

Next Steps

You now understand the complete lifecycle of a batch job.

Continue to the Batch API Reference for detailed request parameters, response fields, and endpoint-specific examples.