How to use ModHeader in Selenium WebDriver

Table of Content

• Introduction
• Installation
• Usage
• API
• FAQ

Introduction

modheader/modheader_selenium is our repo for using ModHeader in WebDriver. We created special builds for ModHeader that expose some URLs as APIs to manipulate ModHeader. These APIs are not available in the regular ModHeader. With these builds, you can setup WebDriver to configure ModHeader programmatically.

Installation

To use this in NodeJS for Chrome, install the chrome-modheader package:

npm install chrome-modheaderCopy

To use this in NodeJS for Firefox, install the firefox-modheader package:

npm install firefox-modheaderCopy

For other programming languages, you can download the prepackaged extensions below and load them into WebDriver as needed.

Download for Chrome

Download for Firefox

Usage

Take a look at the examples directory for more detailed examples

Selenium Webdriver with Chrome:

const { getExtension, getAddHeaderUrl } = require('chrome-modheader');
const options = new chrome.Options().addExtensions(getExtension());
const driver = await new Builder().forBrowser('chrome').setChromeOptions(options).build();
await driver.get(getAddHeaderUrl('HeaderName', 'HeaderValue'));
await driver.wait(until.titleIs('Done'), 5000);Copy

Selenium Webdriver with Firefox:

const { getExtension, getAddHeaderUrl } = require('firefox-modheader');

const options = new firefox.Options();
options.addExtensions(getExtension());
const driver = await new Builder().forBrowser('firefox').setFirefoxOptions(options).build();
await driver.get(getAddHeaderUrl('HeaderName', 'HeaderValue'));
await driver.wait(until.titleIs('Done'), 5000);Copy

Webdriver.io:

Modify wdio.conf.js file

const chromeModheader = require('chrome-modheader');
const firefoxModheader = require('firefox-modheader');

exports.config = {
  capabilities: [
    {
      browserName: 'chrome',
      'goog:chromeOptions': {
        extensions: [chromeModheader.getEncodedExtension()]
      }
    },
    {
      browserName: 'firefox'
    }
  ],
  services: [
    ['selenium-standalone', { logPath: 'logs', installArgs: { drivers }, args: { drivers } }],
    [
      'firefox-profile',
      {
        extensions: [firefoxModheader.getExtension()]
      }
    ]
  ],

  before: function (capabilities, specs) {
    browser.url(chromeModheader.getAddHeaderUrl('accept-encoding', ''));
  }
};Copy

Java Selenium

For Chrome

Path currentRelativePath = Paths.get("chrome-modheader/modheader.crx");
ChromeOptions options = new ChromeOptions();
options.addExtensions(new File(currentRelativePath.toAbsolutePath().toString()));
ChromeDriver driver = new ChromeDriver(options);
driver.get("https://webdriver.modheader.com/add?test=ModHeader%20Test");
new WebDriverWait(driver, Duration.ofSeconds(500)).until(ExpectedConditions.titleIs("Done"));Copy

For Firefox

FirefoxDriver driver = new FirefoxDriver();
driver.installExtension(Paths.get("firefox-modheader/modheader.xpi"));
driver.get("https://webdriver.modheader.com/add?test=ModHeader%20Test");
new WebDriverWait(driver, Duration.ofSeconds(500)).until(ExpectedConditions.titleIs("Done"));Copy

Python Selenium

For Chrome

options = webdriver.ChromeOptions()
options.add_extension('chrome-modheader/modheader.crx')
driver = webdriver.Chrome(options=options, service=Service(ChromeDriverManager().install()))
driver.get("https://webdriver.modheader.com/add?test=ModHeader%20Test")
WebDriverWait(driver, 5).until(expected_conditions.title_is("Done"))Copy

For Firefox

driver = webdriver.Firefox(service=Service(GeckoDriverManager().install()))
driver.install_addon('firefox-modheader/modheader.xpi')
driver.get("https://webdriver.modheader.com/add?test=ModHeader%20Test")
WebDriverWait(driver, 5).until(expected_conditions.title_is("Done"))Copy

API

All APIs are URL-based. Please make sure to URL encode your name and value properly.

Note that the webdriver.modheader.com URLs only work when the extensions are properly loaded. Older versions of the extensions use webdriver.bewisse.com and bewisse.com. These will continue to work in the newer version.

If you are using npm, you can also use the getAddHeaderUrl(), getAddHeadersUrl() and getClearHeadersUrl() functions to craft these URLs. Be sure to do driver.get(), and be mindful that these will change the URL of the WebDriver.

Add request header:

https://webdriver.modheader.com/add?{name1}={value1}&{name2}={value2}&...Copy

e.g., https://webdriver.modheader.com/add?Test=1

Node API equivalent:

function getAddHeaderUrl(name, value) {}
function getAddHeadersUrl({ name: value }) {}Copy

Construct the URL above using getAddHeaderUrl('Test', '1') or getAddHeadersUrl({ Test: '1' })

Clear all modified request headers:

https://webdriver.modheader.com/clearCopy

Node API equivalent:

function getClearHeadersUrl() {}Copy

Load custom profile:

https://webdriver.modheader.com/load?profile={exported_profile_in_json}Copy

exported_profile_in_json can be obtained from the regular ModHeader extension using ... -> Export Profile.

Node API equivalent:

function getLoadProfileUrl(exported_profile) {}Copy

FAQ

• Does this work in headless mode?
  • Yes, but be sure to use --headless=new, not --headless.
  • NOTE: As of Chrome 109, you will need to use --headless=new. Older versions of Chrome should use --headless=chrome.
• I am stuck on the webdriver.modheader.com page. What is wrong?
  • Most likely, this is because the ModHeader selenium extension is not installed correctly. Please make sure to download the latest ModHeader selenium extension at the top of this page (Do NOT use the ModHeader extension from Chrome Web Store).
  • To debug further, please pause the WebDriver execution, and go to chrome://extensions in the WebDriver Chrome instance. Make sure ModHeader selenium extension is installed. If not, that means that is something wrong with the WebDriver setup. If it is installed, enable "Developer mode" (top right corner), then click on ModHeader's "background page" link. This should open up the "Console". Send the errors to support@modheader.com. This will help us identify the root cause.
  • Note: The webdriver.modheader.com subdomain maps to a static page hosted on S3. It is used as a hint for ModHeader selenium extension to redirect to the extension internal page. We do not log any requests to this page.
• ModHeader sometimes fail to modify request headers in WebDriver, why?
  • When you navigate to webdriver.modheader.com, Please add wait until the page's title is "Done" before proceeding. This will make sure that the modification has been correctly stored in ModHeader.