0

So, I'm trying to pull a temperature value from www.theweathernetwork.com and when I inspect the element I'm getting the following structure: 

<div class="mainTemp-wrap">
    <p class="temperature">
    14
    <span>
    °C
    </span>
    </p>
    <p class="mcity-feels-like">Feels like 
    <span>13</span>
    </p>
</div>

But when I run the following code, I get nothing returned: 

import requests
from bs4 import BeautifulSoup

url = "https://www.theweathernetwork.com/us/weather/illinois/chicago"
r = requests.get(url)
soup = BeautifulSoup(r.content)

g_data = soup.findAll("p", {"class": "temperature"})

print g_data
AnabolicHippo
  • 45
  • 4
  • It looks like for the `url` provided it should be `soup.findAll("p", {"class": "temp"})` – Cory Shay Sep 28 '16 at 17:18
  • The content is added dynamically so there is no p tag with the class temperature in what you get back – Padraic Cunningham Sep 28 '16 at 17:19
  • @CoryShay I was going to post an answer with the same thing, but the classes he is looking for are dynamically generated. Selenium will more than likely be the solution – Wondercricket Sep 28 '16 at 17:20
  • @Wondercricket Taking a look further into things, when posting out the full contents of soup, though it's difficult to read to begin with, that the temperature values are there either. I'll take a look into Selenium. – AnabolicHippo Sep 28 '16 at 17:22
  • @Wondercricket Just noticed that. AnabolicHippo, it does appear there is a call to get JSON information from their servers with a lot of the values but the keys are obfuscated to a point and Selenium would probably be your best bet. – Cory Shay Sep 28 '16 at 17:35

0 Answers0