0

I have a project scraping data from tandfonline which I have institutional access to. To get the data from each article I need to click the issue button each time with selenium library, which has a unique class or id. I tried this code:

driver = webdriver.Chrome()
driver.get(url)
driver.implicitly_wait(3)
my_element = driver.find_element(By.ID,"vnvpp20-18").click()

but got an error:enter image description here

Traceback (most recent call last):
  File "C:\Users\97254\PycharmProjects\pythonProject\venv\Final_assignment.py", line 49, in <module>
    my_element = driver.find_element(By.ID,"vnvpp20-18").click()
  File "C:\Users\97254\PycharmProjects\pythonProject\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 740, in find_element
    return self.execute(Command.FIND_ELEMENT, {"using": by, "value": value})["value"]
  File "C:\Users\97254\PycharmProjects\pythonProject\venv\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 346, in execute
    self.error_handler.check_response(response)
  File "C:\Users\97254\PycharmProjects\pythonProject\venv\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 245, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"css selector","selector":"[id="vnvpp20-18"]"}
  (Session info: chrome=114.0.5735.199); For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors#no-such-element-exception
Stacktrace:
Backtrace:
    GetHandleVerifier [0x0039A813+48355]
    (No symbol) [0x0032C4B1]
    (No symbol) [0x00235358]
    (No symbol) [0x002609A5]
    (No symbol) [0x00260B3B]
    (No symbol) [0x0028E232]
    (No symbol) [0x0027A784]
    (No symbol) [0x0028C922]
    (No symbol) [0x0027A536]
    (No symbol) [0x002582DC]
    (No symbol) [0x002593DD]
    GetHandleVerifier [0x005FAABD+2539405]
    GetHandleVerifier [0x0063A78F+2800735]
    GetHandleVerifier [0x0063456C+2775612]
    GetHandleVerifier [0x004251E0+616112]
    (No symbol) [0x00335F8C]
    (No symbol) [0x00332328]
    (No symbol) [0x0033240B]
    (No symbol) [0x00324FF7]
    BaseThreadInitThunk [0x75E57D59+25]
    RtlInitializeExceptionChain [0x7752B74B+107]
    RtlClearBits [0x7752B6CF+191]

which id, class name, or tag on the page from these shown in the screenshot name should be in the selenium library I use and how should I build the iteration to click an issue each time after I finish grabbing the data I need from each article? Thanks

Ajeet Verma
  • 2,938
  • 3
  • 13
  • 24
Ido Kobi
  • 1
  • 1

2 Answers2

0

Given the HTML:

html

To click on the desired element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following locator strategies:

  • Using CSS_SELECTOR:

    WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "ul.list-of-issues > li.vol_li > button.volume_link#vol_18_2023"))).click()

  • Using XPATH:

    WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//ul[@class='list-of-issues']/li[@class='vol_li']/button[@class='volume_link' and @id='vol_18_2023']"))).click()

  • Note: You have to add the following imports :

    from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

References

You can find a couple of relevant discussions on NoSuchElementException in:

undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
0

First of all, you did not write the id correctly. The id is vol_18_2023 as you can see, you wrote the data-id.

In general, try to narrow the search as much as possible. You could do something like this:

driver.find_elements(By.TAG, 'button').find_element(By.ID, 'vol_18_2023').click()

Successfully!

ylj
  • 77
  • 11