0

I'm using puppeteer to scrape some data off of a website, but all of my selections for a certain element are always undefined.

    const tempFunction = await page.evaluate(() => {
        let a = document.querySelectorAll(".flex.flex-wrap.w-100.flex-grow-0.flex-shrink-0.ph2.pr0-xl.pl4-xl.mt0-xl.mt3")
        let container = document.querySelector(".flex.flex-wrap.w-100.flex-grow-0.flex-shrink-0.ph2.pr0-xl.pl4-xl.mt0-xl.mt3")
        let b = container.getElementsByClassName("mb1 ph1 pa0-xl bb b--near-white w-33")
        return b
    })

For some reason this code always returns undefined, but similar code works fine.

    const checkData = await page.evaluate(() =>{
        let tempArray = []
        let element = document.querySelectorAll('.weather-block')
        
        tempArray.push(element[0].innerText)  
        return tempArray
    })

Even when trying to use specific selectors or id's, I only get undefined. Not sure where to go from here.

Vastagon
  • 13
  • 2
  • What's the website, please? Every one is unique so there are many reasons why an element might not be selectable (the site detecting and blocking your bot, A/B tests, iframes, shadow DOMs, failing to wait for the element, incorrect selector/path, JS hiding the element, ...). The first example won't work in any case because you can't return DOM nodes--only serializable data like text. – ggorlen Jul 16 '22 at 14:29
  • It had to do with the DOM nodes and the data. I didn't know that I couldn't do that. – Vastagon Jul 16 '22 at 15:46
  • See [Puppeteer page.evaluate querySelectorAll return empty objects](https://stackoverflow.com/questions/46377955/puppeteer-page-evaluate-queryselectorall-return-empty-objects). The second code block should work though, assuming the selector works, but it could be written more succinctly with `return document.querySelector(".weather-block").innerText` – ggorlen Jul 16 '22 at 15:47

0 Answers0