Selenium How to Continue when Not Found

Question

I have this code:

import selenium.webdriver as webdriver
import time
import requests
from selenium.webdriver.chrome.options import Options
from usp.tree import sitemap_tree_for_homepage
import os
from selenium.webdriver.common.by import By
from urllib.parse import urlparse
from urllib.parse import parse_qs

options = Options()
options.add_argument('--allow-running-insecure-content')
options.add_argument('--ignore-certificate-errors')

driver = webdriver.Chrome()

sitemap = 'https://adesivimoto.eu/sitemap.xml'
next_pages = []

tree = sitemap_tree_for_homepage(sitemap)
for page in tree.all_pages():
    driver.get(page.url)
    print("visito "+page.url)
    time.sleep(2)
    
    if driver.find_element(By.CLASS_NAME, "next"):
        next_page = driver.find_element(By.CLASS_NAME, "next").get_attribute('href')
        next_pages.append(next_page)

for next_page in next_pages:
    print(next_page)
    
os.system("rundll32.exe powrprof.dll,SetSuspendState 0,1,0")

Where the code says next_page = driver.find_element(By.CLASS_NAME, "next"), if the element is not available, there will be an error. I tried to work around this using if, but was not successful. How can I make is so that, if the element is not available, the code just goes to the next iteration of the loop?

score 1 · Answer 1 · answered Feb 10 '23 at 12:27

1

Wrap up the line of code with a try-except{} block inducing WebDriverWait for the visibility_of_element_located() and catching TimeoutException as follows:

try:
    next_page =  WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.CLASS_NAME, "next"))).get_attribute('href')
    next_pages.append(next_page)
except TimeoutException:
    pass

answered Feb 10 '23 at 12:27

undetected Selenium

183,867
41
278
352

Is there really not a canonical duplicate for this? – Karl Knechtel Feb 10 '23 at 12:59
@KarlKnechtel I didn't find a handy example of `except TimeoutException: pass`. Else I would have marked as duplicate. – undetected Selenium Feb 10 '23 at 13:12

score 0 · Accepted Answer · edited Feb 10 '23 at 13:47

0

try: # try getting the element
    next_page = driver.find_element(By.CLASS_NAME, "next")
    # code, if element gets found
    # ....
except:
    pass # not found ==> do nothing

If that's not what you need, feel free to leave a comment and I'll edit my answer.

edited Feb 10 '23 at 13:47

undetected Selenium

183,867
41
278
352

answered Feb 10 '23 at 12:25

kaliiiiiiiii

925
1
2
21

Ehh catching raw exception :) – undetected Selenium Feb 10 '23 at 12:32
1

Well yes that's true, In optimal case, you wouldn't make the except statement that broad, and also wait for the element to get loaded:) Your provided answer is more complete in this case. – kaliiiiiiiii Feb 10 '23 at 12:37
still luck seems to be with your side ;) – undetected Selenium Feb 10 '23 at 13:31
Well:) Edited my answer with the improvements you suggested then. – kaliiiiiiiii Feb 10 '23 at 13:39
Well :) no, please don't undermine your own creativity. Let your answer be in the purest form you constructed. It's OP's wish which answer he/she finds (most) helpful. – undetected Selenium Feb 10 '23 at 13:47
Let's discuss in the [Selenium](https://chat.stackoverflow.com/rooms/223360/selenium) room. – undetected Selenium Feb 10 '23 at 13:49
It's a bad practice to catch all exceptions, by not specifying the exceptions you want to catch. While OP may have accepted this answer, please don't propagate bad practices by leaving this answer as is. OP will learn something as well as future readers. – JeffC Feb 10 '23 at 17:02

Selenium How to Continue when Not Found

2 Answers2