I am trying to scrape 100.0% from style attribute:
<div class="w-full mt-1 bg-white rounded-lg shadow">
<div class="py-1 bg-purple-900 rounded-lg" style="width: 100.0%"></div>
</div>
The page source response does not go this deep into the page, I have tried:
def ScrapePercents():
URL = "https://citystrides.com/cities/26013/search_striders?page=1"
page = requests.get(URL)
soup = BeautifulSoup(page.content, "html.parser")
results = soup.find("div", class_="flex flex-wrap space-y-4")
percents = results.find_all("div", class_="py-1 bg-purple-900 rounded-lg")
pl = []
for percent in percents:
cleantext = percent['style'].lstrip('width: ')
percent_neat = (cleantext.strip('%'))
percent_float = float(percent_neat)
pl.append(percent_float)
print("pl as it appends ", pl)
return pl
And
def Selenium():
print("shell for selenium")
pl = "shell for selenium percents"
driver = webdriver.Chrome("chromedriver.exe")
driver.get("https://citystrides.com/cities/26013/search_striders")
content = driver.page_source
soup = BeautifulSoup(content)
results = soup.find("div", class_="flex flex-wrap space-y-4")
return soup
The second code is set just to see whether the content includes the nested div. I'm struggling to work out how to get the nested div.
