Random "Element is no longer attached to the DOM" StaleElementReferenceException

Question

I'm hoping it's just me, but Selenium Webdriver seems like a complete nightmare. The Chrome webdriver is currently unusable, and the other drivers are quite unreliable, or so it seems. I am battling many problems, but here is one.

Randomly, my tests will fail with a

"org.openqa.selenium.StaleElementReferenceException: Element is no longer attached 
to the DOM    
System info: os.name: 'Windows 7', os.arch: 'amd64',
 os.version: '6.1', java.version: '1.6.0_23'"

I'm using webdriver versions 2.0b3. I have seen this happen with FF and IE drivers. The only way I can prevent this is to add an actual call to Thread.sleep before the exception occurs. That is a poor workaround though, so I'm hoping someone can point out an error on my part that will make this all better.

Answer 1

Yes, if you're having problems with StaleElementReferenceExceptions it's because there is a race condition. Consider the following scenario:

WebElement element = driver.findElement(By.id("foo"));
// DOM changes - page is refreshed, or element is removed and re-added
element.click();

Now at the point where you're clicking the element, the element reference is no longer valid. It's close to impossible for WebDriver to make a good guess about all the cases where this might happen - so it throws up its hands and gives control to you, who as the test/app author should know exactly what may or may not happen. What you want to do is explicitly wait until the DOM is in a state where you know things won't change. For example, using a WebDriverWait to wait for a specific element to exist:

// times out after 5 seconds
WebDriverWait wait = new WebDriverWait(driver, 5);
    
// while the following loop runs, the DOM changes - 
// page is refreshed, or element is removed and re-added
wait.until(presenceOfElementLocated(By.id("container-element")));        

// now we're good - let's click the element
driver.findElement(By.id("foo")).click();

The presenceOfElementLocated() method would look something like this:

private static Function<WebDriver,WebElement> presenceOfElementLocated(final By locator) {
    return new Function<WebDriver, WebElement>() {
        @Override
        public WebElement apply(WebDriver driver) {
            return driver.findElement(locator);
        }
    };
}

You're quite right about the current Chrome driver being quite unstable, and you'll be happy to hear that the Selenium trunk has a rewritten Chrome driver, where most of the implementation was done by the Chromium developers as part of their tree.

PS. Alternatively, instead of waiting explicitly like in the example above, you can enable implicit waits - this way WebDriver will always loop up until the specified timeout waiting for the element to become present:

driver.manage().timeouts().implicitlyWait(10, TimeUnit.SECONDS)

In my experience though, explicitly waiting is always more reliable.

Answer 2

I have been able to use a method like this with some success:

WebElement getStaleElemById(String id) {
    try {
        return driver.findElement(By.id(id));
    } catch (StaleElementReferenceException e) {
        System.out.println("Attempting to recover from StaleElementReferenceException ...");
        return getStaleElemById(id);
    }
}

Yes, it just keeps polling the element until it's no longer considered stale (fresh?). Doesn't really get to the root of the problem, but I've found that the WebDriver can be rather picky about throwing this exception -- sometimes I get it, and sometimes I don't. Or it could be that the DOM really is changing.

So I don't quite agree with the answer above that this necessarily indicates a poorly-written test. I've got it on fresh pages which I have not interacted with in any way. I think there is some flakiness in either how the DOM is represented, or in what WebDriver considers to be stale.

Answer 3

I get this error sometimes when AJAX updates are midway. Capybara appears to be pretty smart about waiting for DOM changes (see Why wait_until was removed from Capybara ), but the default wait time of 2 seconds was simply not enough in my case. Changed in _spec_helper.rb_ with e.g.

Capybara.default_max_wait_time = 5

Answer 4

I was facing the same problem today and made up a wrapper class, which checks before every method if the element reference is still valid. My solution to retrive the element is pretty simple so i thought i'd just share it.

private void setElementLocator()
{
    this.locatorVariable = "selenium_" + DateTimeMethods.GetTime().ToString();
    ((IJavaScriptExecutor)this.driver).ExecuteScript(locatorVariable + " = arguments[0];", this.element);
}

private void RetrieveElement()
{
    this.element = (IWebElement)((IJavaScriptExecutor)this.driver).ExecuteScript("return " + locatorVariable);
}

You see i "locate" or rather save the element in a global js variable and retrieve the element if needed. If the page gets reloaded this reference will not work anymore. But as long as only changes are made to doom the reference stays. And that should do the job in most cases.

Also it avoids re-searching the element.

John

Answer 5

I had the same problem and mine was caused by an old selenium version. I cannot update to a newer version due to development environment. The problem is caused by HTMLUnitWebElement.switchFocusToThisIfNeeded(). When you navigate to a new page it might happen that the element you clicked on the old page is the oldActiveElement (see below). Selenium tries to get context from the old element and fails. That's why they built a try catch in future releases.

Code from selenium-htmlunit-driver version < 2.23.0:

private void switchFocusToThisIfNeeded() {
    HtmlUnitWebElement oldActiveElement =
        ((HtmlUnitWebElement)parent.switchTo().activeElement());

    boolean jsEnabled = parent.isJavascriptEnabled();
    boolean oldActiveEqualsCurrent = oldActiveElement.equals(this);
    boolean isBody = oldActiveElement.getTagName().toLowerCase().equals("body");
    if (jsEnabled &&
        !oldActiveEqualsCurrent &&
        !isBody) {
      oldActiveElement.element.blur();
      element.focus();
    }
}

Code from selenium-htmlunit-driver version >= 2.23.0:

private void switchFocusToThisIfNeeded() {
    HtmlUnitWebElement oldActiveElement =
        ((HtmlUnitWebElement)parent.switchTo().activeElement());

    boolean jsEnabled = parent.isJavascriptEnabled();
    boolean oldActiveEqualsCurrent = oldActiveElement.equals(this);
    try {
        boolean isBody = oldActiveElement.getTagName().toLowerCase().equals("body");
        if (jsEnabled &&
            !oldActiveEqualsCurrent &&
            !isBody) {
        oldActiveElement.element.blur();
        }
    } catch (StaleElementReferenceException ex) {
      // old element has gone, do nothing
    }
    element.focus();
}

Without updating to 2.23.0 or newer you can just give any element on the page focus. I just used element.click() for example.

Answer 6

Just happened to me when trying to send_keys to a search input box - that has autoupdate depending on what you type in. As mentioned by Eero, this can happen if your element does some Ajax updated while you are typing in your text inside the input element. The solution is to send one character at a time and search again for the input element. (Ex. in ruby shown below)

def send_keys_eachchar(webdriver, elem_locator, text_to_send)
  text_to_send.each_char do |char|
    input_elem = webdriver.find_element(elem_locator)
    input_elem.send_keys(char)
  end
end

Answer 7

To add to @jarib's answer, I have made several extension methods which help eliminate the race condition.

Here is my setup:

I have a class Called "Driver.cs". It contains a static class full of extension methods for the driver and other useful static functions.

For elements I commonly need to retrieve, I create an extension method like the following:

public static IWebElement SpecificElementToGet(this IWebDriver driver) {
    return driver.FindElement(By.SomeSelector("SelectorText"));
}

This allows you to retrieve that element from any test class with the code:

driver.SpecificElementToGet();

Now, if this results in a StaleElementReferenceException, I have the following static method in my driver class:

public static void WaitForDisplayed(Func<IWebElement> getWebElement, int timeOut)
{
    for (int second = 0; ; second++)
    {
        if (second >= timeOut) Assert.Fail("timeout");
        try
        {
            if (getWebElement().Displayed) break;
        }
        catch (Exception)
        { }
        Thread.Sleep(1000);
    }
}

This function's first parameter is any function which returns an IWebElement object. The second parameter is a timeout in seconds (the code for the timeout was copied from the Selenium IDE for FireFox). The code can be used to avoid the stale element exception the following way:

MyTestDriver.WaitForDisplayed(driver.SpecificElementToGet,5);

The above code will call driver.SpecificElementToGet().Displayed until driver.SpecificElementToGet() throws no exceptions and .Displayed evaluates to true and 5 seconds have not passed. After 5 seconds, the test will fail.

On the flip side, to wait for an element to not be present, you can use the following function the same way:

public static void WaitForNotPresent(Func<IWebElement> getWebElement, int timeOut) {
    for (int second = 0;; second++) {
        if (second >= timeOut) Assert.Fail("timeout");
            try
            {
                if (!getWebElement().Displayed) break;
            }
            catch (ElementNotVisibleException) { break; }
            catch (NoSuchElementException) { break; }
            catch (StaleElementReferenceException) { break; }
            catch (Exception)
            { }
            Thread.Sleep(1000);
        }
}

Answer 8

I think I found convenient approach to handle StaleElementReferenceException. Usually you have to write wrappers for every WebElement method to retry actions, which is frustrating and wastes lots of time.

Adding this code

webDriverWait.until((webDriver1) -> (((JavascriptExecutor) webDriver).executeScript("return document.readyState").equals("complete")));

if ((Boolean) ((JavascriptExecutor) webDriver).executeScript("return window.jQuery != undefined")) {
    webDriverWait.until((webDriver1) -> (((JavascriptExecutor) webDriver).executeScript("return jQuery.active == 0")));
}

before every WebElement action can increase stability of your tests but you still can get StaleElementReferenceException from time to time.

So this is what I came up with (using AspectJ):

package path.to.your.aspects;

import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;
import org.aspectj.lang.reflect.MethodSignature;
import org.openqa.selenium.JavascriptExecutor;
import org.openqa.selenium.StaleElementReferenceException;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.remote.RemoteWebElement;
import org.openqa.selenium.support.pagefactory.DefaultElementLocator;
import org.openqa.selenium.support.pagefactory.internal.LocatingElementHandler;
import org.openqa.selenium.support.ui.WebDriverWait;

import java.lang.reflect.Field;
import java.lang.reflect.Method;
import java.lang.reflect.Proxy;

@Aspect
public class WebElementAspect {
    private static final Logger LOG = LogManager.getLogger(WebElementAspect.class);
    /**
     * Get your WebDriver instance from some kind of manager
     */
    private WebDriver webDriver = DriverManager.getWebDriver();
    private WebDriverWait webDriverWait = new WebDriverWait(webDriver, 10);

    /**
     * This will intercept execution of all methods from WebElement interface
     */
    @Pointcut("execution(* org.openqa.selenium.WebElement.*(..))")
    public void webElementMethods() {}

    /**
     * @Around annotation means that you can insert additional logic
     * before and after execution of the method
     */
    @Around("webElementMethods()")
    public Object webElementHandler(ProceedingJoinPoint joinPoint) throws Throwable {
        /**
         * Waiting until JavaScript and jQuery complete their stuff
         */
        waitUntilPageIsLoaded();

        /**
         * Getting WebElement instance, method, arguments
         */
        WebElement webElement = (WebElement) joinPoint.getThis();
        Method method = ((MethodSignature) joinPoint.getSignature()).getMethod();
        Object[] args = joinPoint.getArgs();

        /**
         * Do some logging if you feel like it
         */
        String methodName = method.getName();

        if (methodName.contains("click")) {
            LOG.info("Clicking on " + getBy(webElement));
        } else if (methodName.contains("select")) {
            LOG.info("Selecting from " + getBy(webElement));
        } else if (methodName.contains("sendKeys")) {
            LOG.info("Entering " + args[0].toString() + " into " + getBy(webElement));
        }

        try {
            /**
             * Executing WebElement method
             */
            return joinPoint.proceed();
        } catch (StaleElementReferenceException ex) {
            LOG.debug("Intercepted StaleElementReferenceException");

            /**
             * Refreshing WebElement
             * You can use implementation from this blog
             * http://www.sahajamit.com/post/mystery-of-stale-element-reference-exception/
             * but remove staleness check in the beginning (if(!isElementStale(elem))), because we already caught exception
             * and it will result in an endless loop
             */
            webElement = StaleElementUtil.refreshElement(webElement);

            /**
             * Executing method once again on the refreshed WebElement and returning result
             */
            return method.invoke(webElement, args);
        }
    }

    private void waitUntilPageIsLoaded() {
        webDriverWait.until((webDriver1) -> (((JavascriptExecutor) webDriver).executeScript("return document.readyState").equals("complete")));

        if ((Boolean) ((JavascriptExecutor) webDriver).executeScript("return window.jQuery != undefined")) {
            webDriverWait.until((webDriver1) -> (((JavascriptExecutor) webDriver).executeScript("return jQuery.active == 0")));
        }
    }

    private static String getBy(WebElement webElement) {
        try {
            if (webElement instanceof RemoteWebElement) {
                try {
                    Field foundBy = webElement.getClass().getDeclaredField("foundBy");
                    foundBy.setAccessible(true);
                    return (String) foundBy.get(webElement);
                } catch (NoSuchFieldException e) {
                    e.printStackTrace();
                }
            } else {
                LocatingElementHandler handler = (LocatingElementHandler) Proxy.getInvocationHandler(webElement);

                Field locatorField = handler.getClass().getDeclaredField("locator");
                locatorField.setAccessible(true);

                DefaultElementLocator locator = (DefaultElementLocator) locatorField.get(handler);

                Field byField = locator.getClass().getDeclaredField("by");
                byField.setAccessible(true);

                return byField.get(locator).toString();
            }
        } catch (IllegalAccessException | NoSuchFieldException e) {
            e.printStackTrace();
        }

        return null;
    }
}

To enable this aspect create file src\main\resources\META-INF\aop-ajc.xml and write

<aspectj>
    <aspects>
        <aspect name="path.to.your.aspects.WebElementAspect"/>
    </aspects>
</aspectj>

Add this to your pom.xml

<properties>
    <aspectj.version>1.9.1</aspectj.version>
</properties>

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <version>2.22.0</version>
            <configuration>
                <argLine>
                    -javaagent:"${settings.localRepository}/org/aspectj/aspectjweaver/${aspectj.version}/aspectjweaver-${aspectj.version}.jar"
                </argLine>
            </configuration>
            <dependencies>
                <dependency>
                    <groupId>org.aspectj</groupId>
                    <artifactId>aspectjweaver</artifactId>
                    <version>${aspectj.version}</version>
                </dependency>
            </dependencies>
        </plugin>
</build>

And thats all. Hope it helps.

Answer 9

You can solve this by using explicit wait so that you don't have to use hard wait.

If you fetching all the elements with one property and iterating through it using for each loop you can use wait inside the loop like this,

List<WebElement> elements = driver.findElements("Object property");
for(WebElement element:elements)
{
    new WebDriverWait(driver,10).until(ExpectedConditions.presenceOfAllElementsLocatedBy("Object property"));
    element.click();//or any other action
}

or for single element you can use below code,

new WebDriverWait(driver,10).until(ExpectedConditions.presenceOfAllElementsLocatedBy("Your object property"));
driver.findElement("Your object property").click();//or anyother action 

Answer 10

In Java 8 you can use very simple method for that:

private Object retryUntilAttached(Supplier<Object> callable) {
    try {
        return callable.get();
    } catch (StaleElementReferenceException e) {
        log.warn("\tTrying once again");
        return retryUntilAttached(callable);
    }
}

Answer 11

FirefoxDriver _driver = new FirefoxDriver();

// create webdriverwait
WebDriverWait wait = new WebDriverWait(_driver, TimeSpan.FromSeconds(10));

// create flag/checker
bool result = false;

// wait for the element.
IWebElement elem = wait.Until(x => x.FindElement(By.Id("Element_ID")));

do
{
    try
    {
        // let the driver look for the element again.
        elem = _driver.FindElement(By.Id("Element_ID"));

        // do your actions.
        elem.SendKeys("text");

        // it will throw an exception if the element is not in the dom or not
        // found but if it didn't, our result will be changed to true.
        result = !result;
    }
    catch (Exception) { }
} while (result != true); // this will continue to look for the element until
                          // it ends throwing exception.