2

I made a script in python and selenium that makes a search on youtube. When it's completely loaded, I'm only able to fetch all titles from the results. Is there any line of code I can integrate in order to fetch date publishing too?

This is my code:

def youTube():
    term = 'bitcoin'
    tit = []
    
    d = webdriver.Firefox()
    d.get('https://www.youtube.com/results?search_query='+term+'&sp=CAISAhAB')
    sleep(3)
    
    d.find_element_by_xpath("//*[contains(text(), 'Accetto')]").click()
    sleep(2)
    
    scrollHeight = d.execute_script("return window.scrollMaxY")
    print(scrollHeight)
    scrolled_pages = 0
    # while we have not reached the max scrollHeight
    while d.execute_script("return window.pageYOffset") < 3000:
        d.execute_script("window.scrollByPages(1)")
        scrolled_pages += 1
        sleep(0.2)
    for my_elem in WebDriverWait(d, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//yt-formatted-string[@class='style-scope ytd-video-renderer' and @aria-label]"))):
        tit.append(my_elem.text)
undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
giacomomaraglino
  • 177
  • 2
  • 14

1 Answers1

2

To retrieve the date/time when video has been released on Youtube using Python and Selenium you can use the following Locator Strategy:

  • Code Block:

    driver.get("https://www.youtube.com/results?search_query=%27+term+%27&sp=CAISAhAB")
print([my_elem.get_attribute("innerHTML") for my_elem in WebDriverWait(driver, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[@id='metadata-line' and @class='style-scope ytd-video-meta-block']////following::span[2][contains(., 'hours') or contains(., 'day')]")))])

  • Console Output:

    ['2 hours ago', '2 hours ago', '3 hours ago', 'Streamed 3 hours ago', 'Streamed 3 hours ago', '5 hours ago', 'Streamed 6 hours ago', '6 hours ago', '6 hours ago', '7 hours ago', '8 hours ago', 'Streamed 8 hours ago', '9 hours ago', 'Streamed 10 hours ago', '11 hours ago']
undetected Selenium
  • 183,867
  • 41
  • 278
  • 352
  • Seems correct but If I try to execute I got this error: dates = WebDriverWait(d, 20).until(EC.visibility_of_all_elements_located((By.XPATH, "//div[@id='metadata-line' and @class='style-scope ytd-video-meta-block']////following::span[2][contains(., 'hours') or contains(., 'day')]"))) File "C:\Users\Surface\AppData\Local\Programs\Python\Python38-32\lib\site-packages\selenium\webdriver\support\wait.py", line 80, in until raise TimeoutException(message, screen, stacktrace) selenium.common.exceptions.TimeoutException: Message: – giacomomaraglino Jan 03 '22 at 18:15