Python/Selenium: How to grab value from the HTML?

Question

I am trying to grab some values from the following code example but am unable to get a successful answer.

Using the US Patent and Trademark Office's website with a random trademark entered in this link. I want to grab the Registration Date which has a value of May 12, 2015

    <div class="double table">
        <div class="row">
               <div class="key">US Serial Number:</div>
               <div class="value">85931937</div>
               <div class="key">Application Filing Date:</div>
               <div class="value">May 14, 2013</div>
        </div>
       
        <div class="row">
               <div class="key">US Registration Number:</div>
               <div class="value">4735834</div>
               <div class="key">Registration Date:</div>
               <div class="value">May 12, 2015</div>
        </div>

Notice this would be the fourth time the class name value is being called in the code example and is nested within other div classes.

This is what I have tried so far:

values = browser.find_elements(By.CLASS_NAME, 'value')
print(values[3])

but values returns an empty list []

Please advise on what I am doing wrong, thank you in advance.

score 1 · Accepted Answer · answered Jan 14 '22 at 06:32

To print the text May 12, 2015 you can use either of the following Locator Strategies:

Using xpath and text attribute:

print(driver.find_element(By.XPATH, "//div[@class='key' and text()='Registration Date:']//following-sibling::div[1]").text)

Ideally you need to induce WebDriverWait for the visibility_of_element_located() and you can use either of the following Locator Strategies:

Using XPATH and get_attribute("innerHTML"):

print(WebDriverWait(driver, 20).until(EC.visibility_of_element_located((By.XPATH, "//div[@class='key' and text()='Registration Date:']//following-sibling::div[1]"))).get_attribute("innerHTML"))

Note : You have to add the following imports :

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

You can find a relevant discussion in How to retrieve the text of a WebElement using Selenium - Python

References

Link to useful documentation:

get_attribute() method Gets the given attribute or property of the element.
text attribute returns The text of the element.
Difference between text and innerHTML using Selenium

XPATH and getattribute("innerHTML") did the trick, thank you! Solved. — badenduser, Jan 14 '22 at 06:47

Python/Selenium: How to grab value from the HTML?

1 Answers1

References