0

I am doing web-scraping with selenium/geckodriver on windows machine. My create_driver function defines a specific Firefox profile:

def create_driver()
    firefox_profile = webdriver.FirefoxProfile()                        # Disable CSS
    firefox_profile.set_preference('permissions.default.stylesheet', 2) # Disable images
    firefox_profile.set_preference('permissions.default.image', 2)      # Disable Flash
    driver = webdriver.Firefox(firefox_profile=firefox_profile)         # Create driver
    return driver

driver = create_driver()

However, for every run Firefox creates a new, 400 KB profile file in C:/User/Appdata/Local/temp (tmpaddon-xxxxxx, see picture below):

This obviously creates significant overhead that I need to get rid of. I know that profiles can be created manually using Firefox Profile Manager. Those can then be used with selenium like explained here. However, I did not figure out yet how to create & populate those profile files using python.

Q1: How to save and reuse one Firefox profile with specs that have been defined in python to disk?

Q2: How to prevent Firefox from auto-saving a profile once every driver instance runs?

Firefox profile files

Eugene S
  • 6,709
  • 8
  • 57
  • 91
sudonym
  • 3,788
  • 4
  • 36
  • 61

1 Answers1

3

Something like this should work:

profile = webdriver.FirefoxProfile('/path/profile.default')
driver = webdriver.Firefox(profile)
Eugene S
  • 6,709
  • 8
  • 57
  • 91
  • which leavers the question on how to edit the '/path/profile.default' file – sudonym Dec 13 '17 at 05:37
  • Why can't you use the Firefox Profile Manager? – Eugene S Dec 13 '17 at 05:39
  • 1
    my scraper is written in python - therefore the process of creating and storing a profile should ideally be accomplished in python as well - what I just did is to let firefox create it in the temp folder, then rename and relocate it - this works. However, the temp files mentioned above are still created for every iteration – sudonym Dec 13 '17 at 06:27
  • @sudonym Perhaps I didn't understand the situation but I thought that if you call the driver providing an existing, manually generated profile(like I suggested) instead of creating a clean Webdriver instance(like in your question), it will not create an additional profile file and just reuse the one you provided instead. Isn't that the case? – Eugene S Dec 13 '17 at 07:08
  • According to my current understanding/observation this is not the case unfortunately. See here: https://stackoverflow.com/a/40527704/6060083 - The thing is that this is an IO/Disc operation which slows things down - not ideal. If you have suggestions on how I should edit the title of this question in order for people to better undertstand, feel free. Sorry for my non-precision here – sudonym Dec 13 '17 at 07:14