Search Jobs

Every Search request returns a unique job_id. You can use this ID to retrieve the details of a completed search job, or use the Search Jobs endpoint to find previous searches.

Search Job Workflow

The following diagram shows how the Search Job endpoints work together.

List Search Jobs

Use the Search Jobs endpoint to list search jobs for the authenticated user.

GET /v1/search/jobs

The endpoint supports optional filtering and pagination.

Example Request

curl "YOUR_SCRAPER_API_URL/v1/search/jobs" \
  -H "X-Api-Key: YOUR_API_KEY"

Example Response

The following is a real response from the Search API:

{
  "jobs": [
    {
      "job_id": "a478199c-54d2-4884-ba8d-d6d7568614e9",
      "query": "web scraping best practice",
      "status": "completed",
      "results_count": 10,
      "duration_ms": 1800,
      "block_reason": null,
      "error_code": null,
      "created_at": "2026-08-16T17:44:07.447657Z",
      "completed_at": "2026-08-16T17:44:09.673985Z"
    },
    {
      "job_id": "ed9f8595-be2a-4d3e-873b-1a1c52620a0e",
      "query": "https://scraper.geonode.io",
      "status": "completed",
      "results_count": 10,
      "duration_ms": 5077,
      "block_reason": null,
      "error_code": null,
      "created_at": "2026-08-16T17:43:29.525681Z",
      "completed_at": "2026-08-16T17:43:35.053054Z"
    }
  ],
  "page": 1,
  "page_size": 10,
  "page_count": 1
}

The response contains:

Field	Description
`jobs`	Search jobs on the current page.
`page`	Current page number.
`page_size`	Number of jobs returned per page.
`page_count`	Total number of pages available.

These fields are defined by the SearchJobsResponse schema.

Search Job Fields

Each item in the jobs array is a SearchListItemResponse.

The available fields are:

Field	Description
`job_id`	Unique job identifier.
`query`	Search query that was submitted.
`status`	Job status.
`results_count`	Number of results returned.
`duration_ms`	Execution time in milliseconds.
`block_reason`	Upstream block classification for failed jobs.
`error_code`	Machine-readable error code for failed jobs.
`created_at`	Time when the search job was created.
`completed_at`	Time when the search job finished.

The OpenAPI defines job_id, query, status, and created_at as required fields. The remaining fields can be absent or nullable according to the schema.

Filter Search Jobs

You can filter the jobs returned by GET /v1/search/jobs.

The available filters are:

Parameter	Description
`query`	Filter by search query using a partial match.
`status`	Filter by job status.
`start_date`	Filter jobs created on or after this date.
`end_date`	Filter jobs created on or before this date.

For example, to filter jobs by query:

curl "YOUR_SCRAPER_API_URL/v1/search/jobs?query=web%20scraping" \
  -H "X-Api-Key: YOUR_API_KEY"

To filter by status:

curl "YOUR_SCRAPER_API_URL/v1/search/jobs?status=completed" \
  -H "X-Api-Key: YOUR_API_KEY"

The OpenAPI defines these four filters for the Search Jobs endpoint.

Paginate Search Jobs

The Search Jobs endpoint supports pagination using:

page
page_size

page specifies the page number and defaults to 1.

page_size specifies the number of results per page. It must be between 1 and 100 and defaults to 10.

For example:

curl "YOUR_SCRAPER_API_URL/v1/search/jobs?page=1&page_size=20" \
  -H "X-Api-Key: YOUR_API_KEY"

The response includes page, page_size, and page_count so you can determine the current page and the total number of available pages.

Maximum Page Size

The page_size parameter accepts values from 1 through 100.

Retrieve Search Job Details

Once you have a job_id, use the Search Job endpoint to retrieve the full details and results for a completed search job.

GET /v1/search/{job_id}

For example:

curl "YOUR_SCRAPER_API_URL/v1/search/a478199c-54d2-4884-ba8d-d6d7568614e9" \
  -H "X-Api-Key: YOUR_API_KEY"

Example Response

The following is a real response for the search job used in the examples above:

{
  "job_id": "a478199c-54d2-4884-ba8d-d6d7568614e9",
  "query": "web scraping best practice",
  "locale": null,
  "page": 1,
  "safe": "off",
  "time_range": null,
  "status": "completed",
  "results": [
    {
      "position": 1,
      "title": "Web Scraping Best Practices in 2026",
      "url": "https://www.scrapingbee.com/blog/web-scraping-best-practices/",
      "snippet": "Web scraping is the automated process of retrieving data from websites and transforming raw HTML or other web data into structured formats for analysis or use. Whether you are working on a small web scraping project or managing large-scale data collection activities, choosing the right web scraping tool and following best practices is essential. In this article, I'll walk you through the best ...",
      "thumbnail": null,
      "displayed_url": "www.scrapingbee.com",
      "source_host": "www.scrapingbee.com"
    },
    {
      "position": 2,
      "title": "10 Best Sample Websites for Web Scraping Practice in 2026",
      "url": "https://thunderbit.com/blog/best-web-scraping-test-sites",
      "snippet": "Practice web scraping on the best sample sites for all skill levels. Thunderbit helps automate extraction, handle complex sites, and streamline data workflows.",
      "thumbnail": null,
      "displayed_url": "thunderbit.com",
      "source_host": "thunderbit.com"
    },
    {
      "position": 3,
      "title": "11 Web Scraping Best Practices for Reliable Data Collection",
      "url": "https://scrapfly.io/blog/posts/web-scraping-best-practices",
      "snippet": "11 web scraping best practices for 2026: robots.txt, rate limiting, hidden APIs, proxy rotation, retries, validation, and monitoring, with working code.",
      "thumbnail": null,
      "displayed_url": "scrapfly.io",
      "source_host": "scrapfly.io"
    }
  ],
  "results_count": 10,
  "suggestions": [],
  "spelling_correction": null,
  "attempts": 2,
  "final_url": "http://searxng:8080/search?q=web+scraping+best+practice&format=json&engines=duckduckgo&pageno=1&safesearch=0",
  "duration_ms": 1800,
  "tokens_charged": 1,
  "block_reason": null,
  "error_code": null,
  "error_message": null,
  "created_at": "2026-08-16T17:44:07.447657Z",
  "completed_at": "2026-08-16T17:44:09.673985Z"
}

The GET /v1/search/{job_id} endpoint is documented as retrieving the full details and results for a completed search job.

Search Job Details

The detailed response contains the search configuration, status, results, and execution information.

Field	Description
`job_id`	Unique job identifier.
`query`	Search query that was submitted.
`locale`	Locale requested for the search.
`page`	Result page that was requested.
`safe`	Safe-search level requested.
`time_range`	Time range filter, if requested.
`status`	Job status.
`results`	Search results.
`results_count`	Number of results returned.
`suggestions`	Query suggestions returned by the engine.
`spelling_correction`	Spelling correction applied to the query.
`attempts`	Number of upstream attempts made.
`final_url`	Final upstream URL that produced the results.
`duration_ms`	Execution time in milliseconds.
`tokens_charged`	Tokens charged for the job.
`block_reason`	Upstream block classification for failed jobs.
`error_code`	Machine-readable error code.
`error_message`	Human-readable error description.
`created_at`	Time when the search job was created.
`completed_at`	Time when the search job finished.

These fields are defined by SearchJobDetailResponse in the OpenAPI.

Job Status

Search jobs use the JobStatus schema.

The available statuses are:

queued
processing
completed
failed
cancelled

The Search Jobs response uses this status field for each listed search job.

Complete Search Job Workflow

What's Next?

You now know how to list Search jobs, filter and paginate the job list, and retrieve the details of a completed search job.

Continue with Search Errors to learn about the Search API error responses.

Search Jobs

On this page