After searching for many hours I am starting to think this is impossible.
I need to run Chrome through selenium using different authenticated (not public) proxy's for each run.
PROXY_IP = "<some IP address>"
UID = "<the user id>"
PWD = "<the password">
options = webdriver.ChromeOptions()
options.add_argument("--proxy-server=%s:%s@%s" % (UID,PWD,PROXY_IP))
driver = webdriver.Chrome(executable_path=".\\driver\\chromedriver.exe",
chrome_options=options)
driver.get("<site URL>")
Chrome will fire-up and display the error:
This webpage is not available
ERR_NO_SUPPORTED_PROXIES
If I use a public proxy requiring no authentication like this...
PROXY_IP = "<public proxy IP address>"
options = webdriver.ChromeOptions()
options.add_argument("--proxy-server=%s" % PROXY_IP)
driver = webdriver.Chrome(executable_path=".\\driver\\chromedriver.exe",
chrome_options=options)
driver.get("<site URL>")
...it runs just fine and displays the site while using the proxy.
I also tried a variant with http://
in front of the user ID:
options.add_argument("--proxy-server=http://%s:%s@%s" % (UID,PWD,PROXY_IP))
The fact that I have searched far and wide and haven't found a solution leads me to believe none might exist.
I did find this but I can't make sense out of it:
selenium chromedriver authentication proxy
Not sure what browswermob-proxy
is or is supposed to do or how to implement and test in Python. I hate piling up band-aid solutions unless they are absolutely necessary.
To use proxies with auth in python selenium you can use seleniumwire.
Fistly, install it with pip install selenium-wire
Then import webdriver from seleniumwire instead selenium
from seleniumwire import webdriver
options = {
'proxy': {
'http': 'http://username:password@host:port',
'https': 'https://username:password@host:port',
'no_proxy': 'localhost,127.0.0.1' # excludes
}
}
browser = webdriver.Chrome(path_to_driver, seleniumwire_options=options)
Now you can use your browser instance exact the same way as selenium: browser.get('https://api.ipify.org')
and so on...