Detect if element wasn't found after some time using Selenium Webdriver

Question

I'm connecting to a website multiple times every few seconds and sometimes the website just enters a HUGE loading phase where it takes like 5 minutes to stop loading. When it finally stops, it throws an expected ElementNotFoundException which i then catch and restart the process from the last point. However, i want to be able to detect this loading phase and handle it as soon as possible instead of waiting 5 minutes for it to end. I tried using a FluentWait implementation and a WebDriverWait one but none works so far. Here are the two implementations i tried :

FluentWait :

private static WebElement fluentWait(final By locator, WebDriver driver) {
    Wait<WebDriver> wait = new FluentWait<>(driver)
            .withTimeout(10, TimeUnit.SECONDS)
            .pollingEvery(2, TimeUnit.SECONDS);
            //.ignoring(NoSuchElementException.class);

    return wait.until((Function<WebDriver, WebElement>) driver1 -> {
        assert driver1 != null;
        return driver1.findElement(locator);
    });
    }

WebDriverWait:

new WebDriverWait(driver,5).until(ExpectedConditions.presenceOfElementLocated(new By.ByCssSelector(someSelector)));

P.S: I'm using Selenium Webdriver and specifically ChromeDriver(). Here's an image of how the website looks when it enters that huge loading phase ->

Could be something against ddos attacks, the website might not be responding because you're sending too many requests at once — martijn p, Nov 04 '19 at 14:37
@martijnp i tried a way to circumvent that and i have updated my answer. Would you mind taking a look? — Stelios Papamichail, Nov 06 '19 at 23:49

score 1 · Accepted Answer · answered Nov 07 '19 at 09:59

1

This is not something to worry about. You can check the below links and I hope this will help.

Selenium -- How to wait until page is completely loaded

Wait for page load in Selenium

answered Nov 07 '19 at 09:59

Vaibhav_Sharma

546
1
9
22

What do you mean by it's not something to worry about? I don't want to wait until the page is loaded (like i state in my question) , instead i want to detect if it isn't loaded after just a few secs and then do something in that case – Stelios Papamichail Nov 07 '19 at 10:35
1

An element that triggers page load waits until the DOM gets loaded before returning control to the driver. For example - if we submit a form by clicking the Submit button then the next statement after the submit button click operation will be attempted by web driver only after the page gets loaded completely. So, in most cases, we don't even have to wait for the page to load. Although try to use wait.until(ExpectedConditions.elementToBeClickable(lastElementToLoad)); as well – Vaibhav_Sharma Nov 07 '19 at 10:42
Wow i had no idea, so basically no matter what i try to do, the driver won't check if an element is present until the website has finished loading? – Stelios Papamichail Nov 07 '19 at 10:44
1

That's not the case the thing is selenium Webdriver doesn't provide any inherent support for wait till page load implementation. But we can make use of Explicit Waits to achieve the desired outcome. For this, we need to identify the element on the webpage that's the last one to load or become available for interaction. Now, we need to use the explicit wait with the appropriate Expected condition like "ElementToBeClickable" for page load implementation. Thus, making sure that the driver waits till all the web elements get loaded successfully before attempting to interact with them. – Vaibhav_Sharma Nov 07 '19 at 10:53

score 0 · Answer 2 · answered Nov 07 '19 at 08:15

Based on your question I'm gathering you want to have the code wait for a specific amount of time and timeout if execution hasn't been finished before then?

If you want to do this, I think the best method would be to implement a simple class that uses an executor to wait for a result. Executors support timeouts and are able to return a value OR throw an exception if execution takes too long.

For this I wrote this simple class:

public class TimedWait<T> {
    public T run(Callable<T> run, long timeoutMs) throws TimeoutException, ExecutionException, InterruptedException {
        ExecutorService executor = Executors.newSingleThreadExecutor();
        return executor.invokeAny(Collections.singletonList(run), timeoutMs, TimeUnit.MILLISECONDS);
    }
}

This will use the timeout I mentioned and will run a provided Callable with a specified timeout, after which an exception is thrown.

An example of how it could function is as followed (I did do some quick and dirty exception handling for example purposes, you might want to clean that up a bit to avoid try catch overuse)

public void run() {
    try {
        String result = new TimedWait<String>().run(() -> {
            Thread.sleep(500);
            return "Some Result";
        }, 1000L);
        System.out.println(result);

        String timeOut = new TimedWait<String>().run(() -> {
            Thread.sleep(1500);
            return "This will time out";
        }, 1000L);
        System.out.println(timeOut);

    } catch (Exception e) {
        e.printStackTrace();
    }
}

Since in this example the delay for the first output is smaller than the timeout, the result will be printed. For the second output, the delay is bigger than the timeout resulting in an exception.

Output being:

Some Result
java.util.concurrent.TimeoutException

Detect if element wasn't found after some time using Selenium Webdriver

2 Answers2