Selenium

This guide will help you integrate the Geonode API with Selenium to manage proxies effectively while running headless browsers. Selenium is widely used for web automation, and using Geonode proxies ensures secure and anonymous browsing.

We will be using Python for this guide.

​

Prerequisites

• Python installed on your system
• Geonode API credentials (username and password)
• ChromeDriver installed (compatible with your Chrome version)

​

Step 1: Set Up a Virtual Environment

Creating a virtual environment helps isolate dependencies and avoid conflicts.

# Install virtualenv if not already installed
pip install virtualenv

# Create a virtual environment
python -m venv .venv

# Activate the virtual environment

## On Windows
.venv\Scripts\activate

## On macOS/Linux
source .venv/bin/activate

​

Step 2: Install Required Libraries

pip install selenium python-dotenv selenium-wire

• Selenium: For browser automation
• python-dotenv: For managing environment variables
• selenium-wire: To handle proxy authentication, as Selenium doesn’t provide it natively

​

Step 3: Get Your Geonode API Credentials

1. Log in to your Geonode dashboard.
2. Navigate to Services > Proxies.
3. Copy your username and password for proxy authentication.

OR

Follow this guide to Access Proxy Credentials to get the credentials

​

Step 4: Configure Geonode Proxy Endpoint

With the help of the Endpoint Generator, you can easily generate the proxy with specific configurations such as:

• Target country
• Port
• Session persistence
• And many more

Refer to the guide How to Use the Endpoint Generator to generate your endpoints.

​

Step 5: Get Your Geonode API Credentials

Scroll down to the API Code Generator section:

Copy your configured endpoint credentials: username, password, and GEONODE_DNS.

​

Step 6: Setting up environment variables

1. Create a .env file in your project directory.
2. Add your credentials:

GEONODE_USERNAME=your_geonode_username
GEONODE_PASSWORD=your_geonode_password
GEONODE_HOST=proxy.geonode.io
GEONODE_PORT=9000
GEONODE_DNS=your_geonode_dns

​

Step 7: Configure Geonode Proxy with Selenium

There are many different format of endpoints. In this guide, we will follow HTTP proxy with authentication

​

Using HTTP Proxy with Authentication

HTTP proxy syntax looks like thus

http://username:password@server:port

here;

http://geonode_demouser-type-residential:demopass@proxy.geonode.io:9000

​

Code Implementation

​

I. Import

import os
from dotenv import load_dotenv
from seleniumwire import webdriver  
from selenium.webdriver.chrome.options import Options

​

ii. Load environment variables

load_dotenv()

proxy_host = os.getenv('GEONODE_HOST')
proxy_port = os.getenv('GEONODE_PORT')
username = os.getenv('GEONODE_USERNAME')
password = os.getenv('GEONODE_PASSWORD')
GEONODE_DNS = os.getenv('GEONODE_PROXY')

​

iii. Configure Proxy options

proxy_options = {
    'proxy': {
        'http': f'http://{username}:{password}@{proxy_host}:{proxy_port}',
        'https': f'https://{username}:{password}@{proxy_host}:{proxy_port}',
    }
}

​

iv. Configure Browser Options

chrome_options = Options()
chrome_options.add_argument('--disable-gpu')
chrome_options.add_argument('--start-maximized')
chrome_options.add_argument('--ignore-certificate-errors')

​

v. Initialize Browser

browser = webdriver.Chrome(seleniumwire_options=proxy_options, options=chrome_options)

​

vi. Open IP address website to check

urlToGet = "https://ip-api.com/"
browser.get(urlToGet)

Output:

​

vii. Keep the browser open

input("Press Enter to close the browser...")

​

viii. Quit the browser

browser.quit()

​

xi. Full Code

import os
from dotenv import load_dotenv
from seleniumwire import webdriver  
from selenium.webdriver.chrome.options import Options


load_dotenv()

proxy_host = os.getenv('GEONODE_HOST')
proxy_port = os.getenv('GEONODE_PORT')
username = os.getenv('GEONODE_USERNAME')
password = os.getenv('GEONODE_PASSWORD')
GEONODE_DNS = os.getenv('GEONODE_PROXY')

proxy_options = {
    'proxy': {
        'http': f'http://{username}:{password}@{proxy_host}:{proxy_port}',
        'https': f'https://{username}:{password}@{proxy_host}:{proxy_port}',
    }
}



chrome_options = Options()
chrome_options.add_argument('--disable-gpu')
chrome_options.add_argument('--start-maximized')
chrome_options.add_argument('--ignore-certificate-errors')

browser = webdriver.Chrome(seleniumwire_options=proxy_options, options=chrome_options)


urlToGet = "https://ip-api.com/"
browser.get(urlToGet)


input("Press Enter to close the browser...")

browser.quit()

​

x. Folder Stucture

Project Root
├── .venv/
├── .env
└── app.py

​

Rotating Proxy

• Rotating Proxy: Automatically rotates without additional configuration.
• Sticky Proxy: Rotates based on the rotation interval set during endpoint configuration.

​

Key Points

• Replace placeholders like your_geonode_username, your_geonode_password, and proxy.geonode.com:9000 with your actual Geonode credentials.
• Use https://httpbin.org/ip to verify the IP address.
• Ensure ChromeDriver matches your Chrome version.

​

Use Cases of Integrating Geonode with Selenium

Integrating Geonode with Selenium can be beneficial for a wide range of applications, including:

1. Web Scraping: Collect data from websites while maintaining anonymity to avoid IP bans.
2. Ad Verification: Test and verify advertisements across different geographies to ensure proper delivery.
3. Price Monitoring: Track pricing changes on e-commerce platforms without getting blocked.
4. SEO Monitoring: Monitor search engine results and competitor websites without affecting personalized search results.
5. Market Research: Gather data from various sources to analyze trends and competitor performance.
6. Social Media Automation: Manage multiple social media accounts while avoiding detection.
7. Fraud Detection: Simulate real-world traffic for security testing and fraud detection systems.

Explore more use cases here https://geonode.com/use-cases

​

Troubleshooting Tips

• Timeout Errors: Check if the proxy is active or switch to another proxy.
• Authentication Issues: Double-check your Geonode API credentials.
• Incompatible ChromeDriver: Ensure ChromeDriver matches your Chrome version.

​

FAQs

This guide helps you seamlessly integrate Geonode API with Selenium for secure, anonymous, and efficient web automation.