Extract meta description field with goquery

Question

I'm using the goquery package to extract pieces of information from a webpage. Please see my code below. The outcome when after running the function is:

Description field: text/html; charset=iso-8859-15
Description field: width=device-width
Description field: THIS IS THE TEXT I WANT TO EXTRACT

I'm almost there, however I only want to get the meta field where the name == 'description'. Unfortunately I can't figure out how to add this extra condition to my code.

func ExampleScrapeDescription() {
    htmlCode :=
        `<!doctype html>
<html lang="NL">
    <head>
        <meta http-equiv="content-type" content="text/html; charset=iso-8859-15">
        <meta name="viewport" content="width=device-width">
        <meta name="description" content="THIS IS THE TEXT I WANT TO EXTRACT">
        <title>page title</title>
    </head>
    <body class="fixedHeader">
        page body
    </body>
</html>`

    doc, err := goquery.NewDocumentFromReader(strings.NewReader((htmlCode)))
    if err != nil {
        log.Fatal(err)
    }

    doc.Find("meta").Each(func(i int, s *goquery.Selection) {
        description, _ := s.Attr("content")
        fmt.Printf("Description field: %s\n", description)
    })
}

icza · Accepted Answer · 2015-05-27T07:01:57.850

11

Just examine the value of the name attribute whether it matches "description":

doc.Find("meta").Each(func(i int, s *goquery.Selection) {
    if name, _ := s.Attr("name"); name == "description" {
        description, _ := s.Attr("content")
        fmt.Printf("Description field: %s\n", description)
    }
})

You may want to compare the value of the name attribute in a case insensitive manner, for that you can use strings.EqualFold():

if name, _ := s.Attr("name"); strings.EqualFold(name, "description") {
    // proceed to extract and use the content of description
}

edited May 27 '15 at 07:01

answered May 27 '15 at 06:47

icza

389,944
63
907
827

Thanks, you helped me a lot! – Rogier Lommers May 28 '15 at 19:32

score 3 · Answer 2 · answered Oct 06 '21 at 20:23

3

    attr, _ := doc.Find("meta[name='description']").Attr("content")

answered Oct 06 '21 at 20:23

Suleyman Yilmaz

41
1
2

While this code may answer the question, it would be more easier for others to understand if you added an explanation of how it works to your answer. – Ryan M Oct 09 '21 at 02:07

Extract meta description field with goquery

2 Answers2