How can i set proxy with authentication in selenium chrome web driver using python

Question

I have following script to visit a web page using python selenium Chrome driver.

from selenium import webdriver
USERNAME = 'usename'
PASSWORD = 'pass'
proxies = ["xxx.xxx.xxx.xxx"]
proxy_tpl ='{0}:{1}'
proxy = proxy_tpl.format(proxies[0],'xx') 
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--proxy-server=%s' % proxy)
chrome = webdriver.Chrome(chrome_options=chrome_options)
chrome.get("http://{0}:{1}@whatismyipaddress.com".format(USERNAME, PASSWORD))
driver.close()

Chrome still asking username and password when i try to run script. How can i authenticate proxy server from script ?

https://stackoverflow.com/a/75954249/20443541 does support that, as well dynamically — kaliiiiiiiii, Jun 28 '23 at 16:18

score 24 · Answer 1 · answered Jun 20 '15 at 12:04

24

Inspired by this this solution in php, i wrote a equivalent in python:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options
import zipfile

manifest_json = """
{
    "version": "1.0.0",
    "manifest_version": 2,
    "name": "Chrome Proxy",
    "permissions": [
        "proxy",
        "tabs",
        "unlimitedStorage",
        "storage",
        "<all_urls>",
        "webRequest",
        "webRequestBlocking"
    ],
    "background": {
        "scripts": ["background.js"]
    },
    "minimum_chrome_version":"22.0.0"
}
"""

background_js = """
var config = {
        mode: "fixed_servers",
        rules: {
          singleProxy: {
            scheme: "http",
            host: "XXX.XXX.XXX.XXX",
            port: parseInt(XXXX)
          },
          bypassList: ["foobar.com"]
        }
      };

chrome.proxy.settings.set({value: config, scope: "regular"}, function() {});

function callbackFn(details) {
    return {
        authCredentials: {
            username: "XXXXXXXXX",
            password: "XXXXXXXXX"
        }
    };
}

chrome.webRequest.onAuthRequired.addListener(
            callbackFn,
            {urls: ["<all_urls>"]},
            ['blocking']
);
"""


pluginfile = 'proxy_auth_plugin.zip'

with zipfile.ZipFile(pluginfile, 'w') as zp:
    zp.writestr("manifest.json", manifest_json)
    zp.writestr("background.js", background_js)

co = Options()
co.add_argument("--start-maximized")
co.add_extension(pluginfile)


driver = webdriver.Chrome("path/to/chromedriver",  chrome_options=co)
driver.get("http://www.google.com.br")

In background_js string, replace the XXX with your information.

answered Jun 20 '15 at 12:04

user3286105

493
1
3
8

Hi, Thanks for this code. it worked. I am stuck at another place. I am trying to login myself using OAuth2.0 authentication and i am doing this on my localhost so I set localhost in my redirect_uri. But due to this proxy browser cannot access localhost. Can you suggest me sth to access the same. – Abdul Khalid Jun 17 '16 at 10:45
1

i want to ask if {urls: ["< all_urls >"]}, needs any modification or works as it is. this is not clear to me. – b10n1k Feb 17 '17 at 15:09
@AbdulKhalid Did you try to put localhost explicitly in permissions? – user3286105 Feb 17 '17 at 17:02
@user3286105 what urls does it expect anyway? – b10n1k Feb 17 '17 at 19:19
1

Note that this solution doesn't work if you start chrome in Incognito mode. – user666 Feb 23 '17 at 00:12
Work well... the only solution there is. – Brana May 03 '17 at 03:13
This works but I can't use `co.add_argument('headless')` I get `selenium.common.exceptions.WebDriverException: Message: unknown error: failed to wait for extension background page to load: chrome-extension` – hayj Nov 20 '17 at 14:24
@hayj exactly. I want to start it in headless mode. How can I achieve it? – Sagun Shrestha Mar 26 '19 at 15:13
@SagunShrestha You can take a look at: https://intoli.com/blog/running-selenium-with-headless-chrome/ – user3286105 Apr 05 '19 at 20:02

How can i set proxy with authentication in selenium chrome web driver using python

1 Answers1