Geonode Logo light

Retries

retries: object;

What's This?

The retries option controls how many times the Scraper will attempt to fetch data from a website if the initial attempt fails. This ensures that temporary issues don't prevent successful data retrieval.

Properties

  • useRetries: A setting that, when true, allows the Scraper to make additional attempts if the first one fails. By default, this is set to true.

  • maxRetries: Specifies the maximum number of retry attempts the Scraper API should make if a request fails. Default is 2.

How Does it Work?

If you set retries= {useRetries: true, maxRetries: 3}, the Scraper will try to fetch the data up to three times if the initial attempt doesn't succeed.

When Should I Use This?

Activate the retries option to increase the reliability of your scraping tasks, especially when dealing with websites that might occasionally have issues or slow response times.

Methods

setRetries(boolean, number)

Use this to control the number of retries.

Configuration Name: retries 

Initializing Scraper

const GeonodeScraperApi = require('geonode-scraper-api');
const scraper = new GeonodeScraperApi('<Your_username>', '<Your_password>');

Using Method

scraper.setRetries(true, 3);
scraper.scrape('https://example.com/');

Using Configuration Object

const config = { retries: { useRetries: true, maxRetries: 3 } };
scraper.scrape('https://example.com/', config);