Whitelist domains Selenium / Firefox can connect to

Question

I am using Selenium webdriver with firefox. I am wondering if there is a setting i can change such that it to only requesting resources from certain domains. (Specifically i want it only to request content which is on the same domain as the webpage itself).

My current set up, written in Python, is:

 from selenium import webdriver
 firefox_profile = webdriver.FirefoxProfile()
 ## Here, I change various default setting in Firefox, and install a couple of monitoring extensions
 driver = webdriver.Firefox(firefox_profile)
 driver.get(web_address)

What i want to do, is if i specify the web address wwww.domain.com, then to only load content served by domain.com, and not e.g. all the tracking content hosted by other domains that would typically be requested. Hoping could be achieved by a change to the profile settings in firefox, or via an extension.

Note - there is a similar question (without an answer) - Restricting Selenium/Webdriver/HtmlUnit to a certain domain - but it is four years old, and i think Selenium has evolved a lot since then.

Possible duplicate of http://stackoverflow.com/a/38168865/5409601 — Rafayet Ullah, Jul 03 '16 at 10:17

score 5 · Accepted Answer · answered Jul 24 '15 at 22:55

With thanks to Vicky, (who's approach of using Proxy settings i followed - although directly from Selenium), the code below will change the proxy settings in firefox such that it will not connect to a domain except that on the white-list.

I suspect several setting changes are unnecessary and can be omitted for most purposes. Code in Python.

    from selenium import webdriver
    firefox_profile = webdriver.FirefoxProfile()

    ## replace desired_domain.com below with whitelisted domain. Separate domains by comma.
    firefox_profile.set_preference("network.proxy.no_proxies_on","localhost,127.0.0.1,desired_domain.com")      

    firefox_profile.set_preference("network.proxy.backup.ftp","0.0.0.0")
    firefox_profile.set_preference("network.proxy.backup.ftp_port",1)
    firefox_profile.set_preference("network.proxy.backup.socks","0.0.0.0")
    firefox_profile.set_preference("network.proxy.backup.socks_port",1)
    firefox_profile.set_preference("network.proxy.backup.ssl","0.0.0.0")
    firefox_profile.set_preference("network.proxy.backup.ssl_port",1)
    firefox_profile.set_preference("network.proxy.ftp","0.0.0.0")
    firefox_profile.set_preference("network.proxy.ftp_port",1)
    firefox_profile.set_preference("network.proxy.http","0.0.0.0")
    firefox_profile.set_preference("network.proxy.http_port",1)
    firefox_profile.set_preference("network.proxy.socks","0.0.0.0")
    firefox_profile.set_preference("network.proxy.socks_port",1)
    firefox_profile.set_preference("network.proxy.ssl","0.0.0.0")
    firefox_profile.set_preference("network.proxy.ssl_port",1)
    firefox_profile.set_preference("network.proxy.type",1)
    firefox_profile.set_preference("network.proxy.share_proxy_settings",True)       

    driver = webdriver.Firefox(firefox_profile)
    driver.get(web_address_desired)

score 2 · Answer 2 · edited Jul 22 '15 at 18:42

I think it is still impossible in selenium.But you can still achieve this by using proxies like browsermob. Webdriver integrates well with browsermob proxy.

Sample pseudeocode in java

    //LittleProxy-powered 2.1.0 release

    LegacyProxyServer server = new BrowserMobProxyServer();

    server.start(0);

    // Blacklist websites
    server.blacklistRequests("https?://.*\\.blocksite\\.com/.*", 410);//these sites will be blocked

    /// get the Selenium proxy object
    Proxy proxy = ClientUtil.createSeleniumProxy(server);

    // configure it as a desired capability
    DesiredCapabilities capabilities = new DesiredCapabilities();

    capabilities.setCapability(CapabilityType.PROXY, proxy);

    // initialize the driver with the capabilities   ;
    Webdriver driver = new FirefoxDriver(capabilities);

Hope this helps you.Kindly get back if you need any further help

Thanks for pointing in right direction - my code, which works by changing firefox's poxy settings directly (i.e. without need for browsermob) is below — kyrenia, Jul 24 '15 at 22:59

Whitelist domains Selenium / Firefox can connect to

2 Answers2