Error when trying to to get html element using selenium

Question

Im trying to get a HTML element from twitter.com search bar. Using selenium I'm using the find_element_by_xpath method.

Code

search_input = driver.find_element_by_xpath('//input[@aria-label="Search query"]')

Error

Traceback (most recent call last):
  File "C:\Users\jorda\OneDrive - Limerick Institute Of Technology\College\Semester 6\Data Analytics\Data Scraping\twit'ter.py", line 25, in <module>
    search_input = driver.find_element_by_xpath('//input[@data-testid="SearchBox_Search_Input""]')
  File "C:\Users\jorda\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 394, in find_element_by_xpath
    return self.find_element(by=By.XPATH, value=xpath)
  File "C:\Users\jorda\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 978, in find_element
    'value': value})['value']
  File "C:\Users\jorda\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\webdriver.py", line 321, in execute
    self.error_handler.check_response(response)
  File "C:\Users\jorda\AppData\Local\Programs\Python\Python37\lib\site-packages\selenium\webdriver\remote\errorhandler.py", line 242, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.InvalidSelectorException: Message: invalid selector: Unable to locate an element with the xpath expression //input[@data-testid="SearchBox_Search_Input""] because of the following error:
SyntaxError: Failed to execute 'evaluate' on 'Document': The string '//input[@data-testid="SearchBox_Search_Input""]' is not a valid XPath expression.
  (Session info: chrome=86.0.4240.198)

The element I'm trying to get is supposed to be aria-label="Search query". Any help is appreciated.

If you read the error, it says : `"is not a valid XPath expression."` — Mohsen_Fatemi, Nov 16 '20 at 04:33
seei im just following a video and the code is the exact same. I don't see what the error is — Madmitten, Nov 16 '20 at 04:34
Im following this video : https://www.youtube.com/watch?v=3KaffTIZ5II&list=WL&index=73&t=162s&ab_channel=IzzyAnalytics about 5:50 in — Madmitten, Nov 16 '20 at 04:37
`//*[@id="react-root"]/div/div/div[2]/main/div/div/div/div[2]/div/div[2]/div/div/div/div[1]/div/div/div/form/div[1]/div/div/div[2]/input` — Mohsen_Fatemi, Nov 16 '20 at 04:48
It seems the home page makes the input tag disappear if it's not fullscreen. — Arundeep Chohan, Nov 16 '20 at 05:05
The string '//input[@data-testid="SearchBox_Search_Input""]' is not a valid XPath expression . Your xpath is not valid because you have 2 double quote after SearchBox_Search_Input — Millie Anne Volante, Nov 16 '20 at 06:41
Info provided in your question is misleading: the predicate in code `[@aria-label="Search query"]` while predicate in exception log is `[@data-testid="SearchBox_Search_Input""]`. Update your question with exact data — JaSON, Nov 16 '20 at 08:41

score 1 · Accepted Answer · answered Nov 16 '20 at 14:44

This error message...

selenium.common.exceptions.InvalidSelectorException: Message: invalid selector: Unable to locate an element with the xpath expression //input[@data-testid="SearchBox_Search_Input""] because of the following error:
SyntaxError: Failed to execute 'evaluate' on 'Document': The string '//input[@data-testid="SearchBox_Search_Input""]' is not a valid XPath expression.

...implies that the xpath which you have used wasn't a valid xpath expression.

The error was observed for the xpath as:

//input[@data-testid="SearchBox_Search_Input""]

is different from the xpath you have provided as code trials.

However the xpath with in your code trial is a valid xpath and won't raise this error.

Ideally, to click on the element you need to induce WebDriverWait for the element_to_be_clickable() and you can use either of the following Locator Strategies:

Using CSS_SELECTOR:

search_input = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "input[aria-label='Search query']")))

Using XPATH:

search_input = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//input[@aria-label='Search query']")))

Note: You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

References

You can find a couple of relevant detailed discussions in:

Error when trying to to get html element using selenium

1 Answers1

References