Selenium (Chrome) and BrowserMob doesn't work for https

Question

I have been trying to integrate BrowserMob to my selenium tests. It works fine with website that work on http, but with https websites the browsers stop working and the HAR file doesn't contain any requests.

When navigating to a https site I get this error on the browser.

"There is something wrong with the proxy server or the address is incorrect."

Here is my code.

    public class Browsermob {

  BrowserMobProxy proxy = new BrowserMobProxyServer();

  @Test
  public void browsermobtest() {


    proxy.start(9091);

    // get the Selenium proxy object
    Proxy seleniumProxy = ClientUtil.createSeleniumProxy(proxy);


    // configure it as a desired capability
    DesiredCapabilities capabilities = new DesiredCapabilities();
    capabilities.setCapability(CapabilityType.PROXY, seleniumProxy);
    System.setProperty("webdriver.chrome.driver", "C:/Users/Madis/Documents/chromedriver.exe");

    WebDriver driver = new ChromeDriver(capabilities);

    // enable more detailed HAR capture, if desired (see CaptureType for the complete list)
    proxy.enableHarCaptureTypes(CaptureType.REQUEST_CONTENT, CaptureType.RESPONSE_CONTENT);

    // create a new HAR with the label "google.com"
    proxy.newHar("http://www.google.com/");

    // open google.com
    driver.get("https://www.google.ee/#gfe_rd=cr");
    driver.findElement(By.cssSelector("#gb_70")).click();



  }

  @AfterMethod
  public void Afterthetest() {

    // get the HAR data
    Har har = proxy.getHar();

    File harFile = new File("C:/Users/Madis/Documents/har.har");
    try {
      har.writeTo(harFile);
    } catch (IOException e) {
      e.printStackTrace();
    }

  }
}

Answer 1

You don't need to specify the sslProxy on the Selenium Proxy object. ClientUtil.createSeleniumProxy does this for you, and in most simple cases it chooses a suitable default value (using InetAddress.getLocalHost(); if that's working for HTTP, it will work for HTTPS as well).

A few things to keep in mind:

1. You'll receive SSL warnings in the browser unless you either tell the browser to ignore cert errors (on Chrome, use the --ignore-certificate-errors command-line flag), or install the BMP CA in the browser's trust store (for Chrome on Windows, you must install it in the Windows trust store).
2. Depending on your version of Chrome and OS, you may need to specify an alternate user-data-dir using a command line option. For example, --user-data-dir=/tmp/insecurechrome.
3. BMP has its own source of trusted certificates (Java trust store + a recent list from Mozilla), so if you're trying to connect to internal websites with certificates issued by a private CA, you need to tell BMP to either trust the private CA or skip certificate validation using .setTrustAllServers(true).
4. The proxy must be started using .start(...) before calling createSeleniumProxy().

Combining all these things, your code would look something like this:

BrowserMobProxy proxy = new BrowserMobProxyServer();
proxy.setTrustAllServers(true);
proxy.start(9091);

// get the Selenium proxy object
Proxy seleniumProxy = ClientUtil.createSeleniumProxy(proxy);
// NOTE: there is no call to .setSslProxy() here

// configure it as a desired capability
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(CapabilityType.PROXY, seleniumProxy);
System.setProperty("webdriver.chrome.driver", "C:/Users/Madis/Documents/chromedriver.exe");

ChromeOptions options = new ChromeOptions();
options.addArgument("--ignore-certificate-errors");
// replace 'somedirectory' with a suitable temp dir on your filesystem
options.addArgument("--user-data-dir=somedirectory");
capabilities.setCapability(ChromeOptions.CAPABILITY, options);

WebDriver driver = new ChromeDriver(capabilities);

// [...]