Geonode Logo light

Mode

mode: string | 'load';

What's This?

The mode option tells the scraper when to consider its job done. Websites can load content in various ways, and this option helps the scraper know when it has captured everything you need.

How Does it Work?

  • mode=’SPA’: Use this for websites that load their content all at once, often after a slight delay. These are called Single Page Applications.

  • mode=’longPolling’: Some websites keep checking for new data in the background. This mode tells the scraper to wait for such activities.

  • mode=’domLoaded’:The scraper finishes its job when the main structure of the website (the DOM when DOMContentLoaded event is fired ) is ready, even if some parts are still loading.

  • mode=’documentLoaded’: The scraper wraps up as soon as it gets the main content of the website, without waiting for everything else.

  • Default (mode=’load’): The scraper waits until the entire page, including images and other resources, is fully loaded.

When Should I Use This?

Most of the time, the default mode will do the trick. But if you notice missing content or if the scraper seems to finish too quickly, you might want to try a different mode that matches the website's loading style.

Methods

setMode(string) 

Configuration Name: mode

Initializing Scraper

const GeonodeScraperApi = require('geonode-scraper-api');
const scraper = new GeonodeScraperApi('<Your_username>', '<Your_password>');

Using Method

scraper.setMode('SPA');
scraper.scrape('https://example.com/');

Using Configuration Object

const config = { mode: 'SPA' };
scraper.scrape('https://example.com/', config);