Cancelling Crawl Jobs

If you no longer need a crawl job to continue processing, you can cancel it using its job ID.

Cancelling a crawl job stops scheduling new crawl pages while allowing any pages already being processed to finish.

Cancel a Crawl Job

Cancel an existing crawl job.

DELETE /v1/crawl/{job_id}

If the cancellation request is accepted, the API returns information about the crawl job, including its current status, a URL for checking job status, and the number of pages that are still being processed.

Example:

{
  "job_id": "453bd7d7-5355-4d6d-a38e-d9e7eb218c3f",
  "status": "queued",
  "status_url": "string",
  "in_flight_pages": 0
}

Response Fields

What Happens After Cancellation?

After the cancellation request is accepted, you can continue monitoring the crawl job by retrieving its latest status using the status_url or the job ID.

Unable to Cancel a Job

If a crawl job cannot be cancelled in its current state, the API returns a 409 Conflict response.

For example, attempting to cancel a job that has already completed returns:

{
  "code": "INVALID_STATE",
  "message": "Crawl job 9293f045-0523-409e-a4e9-874abc663a9e is already completed and cannot be cancelled",
  "retryable": false
}

Error Fields

Possible Responses