15

I'm using Selenium with Python to test a web app; however, Firefox will periodically go in the "Not Responding" state. While Firefox is in this state the script will hang until I close it.

I tried:

br = webdriver.Firefox()
br.set_page_load_timeout(10)

and

br = webdriver.Firefox()
br.implicitly_wait(10)

I fear my issue won't be solved by a wait or timeout method since the page is Ajax. The .click() or .submit() method sends a POST to answer a question and the response is a dictionary which via JavaScript updates the page with a new question. (This is what I understand from using the Network tab in Google Chrome Developer Tools Ctrl+Shift+I)

The question is:

How do I catch the Firefox Not Responding situation in Selenium?

I wish to catch this situation so I can close the browser and start a new one.

Edited after ExperimentsWithCode's answer:
Here is a snippet of my code to illustrate my problem.

def answer_questions(self):
    q = QuestionParser(self.br.page_source)
    q.add_to_db()
    qid = q.get_qid()
    # My answer
    my_answer_id = q.get_my_answer()
    self.br.find_element_by_id(my_answer_id).click()
    # Submit answer
    self.br.find_element_by_xpath('//*[contains(@id, "submit_btn_")]').click()
    try:
        find_id = "submit_btn_" + str(qid)
        element = WebDriverWait(self.br,10).until(EC.presence_of_element_located((By.ID, find_id)))
    except:
        print 'I caught you'
    self.rnd(mu=7, s=2) # a method to sleep for a random time

I call the answer_questions() method in a loop. This code runs fine until Firefox crashes then it will hang until I close it. I read the docs and fear I've exhausted everything. The problem is not loading a page or element. The problem is that I don't know how to detect when Firefox crashes. WebDriverWait seemed promising but it did not throw a TimeoutException when Firefox crashed.

Thank you for your time.

I devised a remedy to my problem. Here is a snippet what I did: 

import os, socket
from threading import Timer
def force_timeout(self):
    os.system('taskkill /im firefox.exe /f /t')
    # for Windows
    # /im firefox.exe is to select firefox
    # /f is to forcefully kill
    # /t is to kill all child processes

for i in range(0, 1000):
    try:
        t = Timer(60, self.force_timeout)
        # run force_timeout from new thread after 60 seconds
        t.start() # start timer
        self.answer_questions()
        # if self.answer_questions() finishes before 60s, cancel timer
        t.cancel() # cancel timer
    except socket.error:
        self.new_browser() # create a new browser
        self.login()       # login
        self.go_to_questions() # Go to questions page so the loop can continue
        # Code to recover from crash here

If the answer_questions() method does not finish in 60 seconds the force_timeout() will run from a new thread and forcefully kill firefox.exe. Once Firefox is terminated the code will throw a socket.error error. Catch this error then do what you need to do to recover from a browser crashing. I find the code to be a bit dirty but it's working exactly as i need it to.
For future reference: OS - Win8, Selenium Version - 2.40.0, Firefox Version - 27.0.1

Windows Taskkill
Python threading.Timer

Delicious
  • 972
  • 12
  • 20
  • 1
    The problem is that while the page don't load completely the browser or your code is awaiting for the response. – Victor Sigler Mar 12 '14 at 13:35
  • 1
    @Vkt0rS. This is not my case. I literally mean Firefox is Not Responding i.e. I have to go in task manager to forcefully kill Firefox. I updated my question with a remedy I used to solve my problem since I concluded Selenium can't handle a browser in the Not Responding state. – Delicious Mar 16 '14 at 01:52
  • Very nice remedy! Thanks. You could post it as an answer though. – Amir Ali Akbari Apr 08 '14 at 10:25
  • Wouldn't you need to say `t.cancel()` in `except socket.error` branch? Otherwise your timer may kill firefox even as you're trying to restart it. – max Jul 22 '15 at 15:11
  • @max `t.cancel()` has to be in try. If Firefox stops responding the timer will hit 60 seconds and kill FireFox. If FireFox doesn't crash we have to cancel the timer i.e. `t.cancel()`. If the timer hits 60 seconds the `force_timeout` method will execute and kill FireFox throwing a `socket.error` error. – Delicious Aug 06 '15 at 04:36
  • Same here. Change to Google Chrome solve my problem. – Kevin Dec 18 '15 at 13:33

3 Answers3

1

The Webdriver docs have a section on things like this here.

From the docs, you could try an Explicit Wait:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait # available since 2.4.0
from selenium.webdriver.support import expected_conditions as EC # available since 2.26.0

br = webdriver.Firefox()
br.get("http://somedomain/url_that_delays_loading")
try:
    element = WebDriverWait(br, 10).until(EC.presence_of_element_located((By.ID, "_element_You_Are_Need_To_Load")))
finally:
    br.quit()

This will time out if the element cannot be found in 10 seconds, or return the element within that window.

ExperimentsWithCode
  • 1,174
  • 4
  • 14
  • 30
  • 1
    Thanks, I tried your suggestion; however, it did not throw a `TimeoutException` when Firefox crashed. The script simply idled until I closed it. I updated my question to reflect this. – Delicious Mar 12 '14 at 17:21
  • Could it be possible that element had loaded while the rest of the page did not? Can you try having it search for an ID that does not exist on that page and see if it gets a TimeoutException? – ExperimentsWithCode Mar 13 '14 at 18:14
  • Yes, I tried searching for an ID that does not exist e.g. in the code in my question I did this: `find_id = "1submit_btn_" + str(qid)` the extra `1` in front of submit doesn't exist. The code throws an error as expected. I mean Firefox literally crashes. I have to go in task manager to forcibly close it. I was hoping Selenium had a way to catch this and forcibly close a non-responding browser. – Delicious Mar 14 '14 at 18:53
  • 1
    Hmm... I do not believe selenium is able to handle a crash. This question goes into how you can kill the firefox process after n seconds, but its in Java. http://stackoverflow.com/questions/19991019/selenium-force-close-firefox – ExperimentsWithCode Mar 14 '14 at 19:18
  • 2
    I was trying to avoid having to manually code a function to forcefully close Firefox. I devised a remedy to my problem. It works perfectly but it's a bit "dirty". I updated my question with my remedy. – Delicious Mar 15 '14 at 23:13
0

For those of you on *nix, 

os.system("pkill -P %d firefox" % os.getpid())

works nicely to kill the Firefox children of your current process.

pkill man page

JMax
  • 26,109
  • 12
  • 69
  • 88
PlasmaSauna
  • 235
  • 1
  • 5
0

I don't know how to explain the often unpredictable unresponsiveness of Selenium, but I can say that quitting or closing the driver (driver.quit() or driver.close() ) doesn't help, because the object and the process that controls the browser is still alive.

I recommend deleting the object (del driver) and re-creating a new driver.

Oluwaseun Cardoso
  • 9
  • 1