How can i get the html of the page?

Question

Here is the code I use to perform a web request. I'm getting all of the HTML except for the comments section in the URL.

HttpWebRequest req = (HttpWebRequest) HttpWebRequest.Create(
  "http://u-handbag.typepad.com/uhandblog/2013/11/choosing-bag-fabrics.html#comment-6a00d8341c574653ef019b022fc96f970d"
);
StreamReader reader = new StreamReader(req.GetResponse().GetResponseStream());
htl = reader.ReadToEnd();

Can anyone explain why?

I don't really know to be honest, but wouldn't the javascript modify the output. Maybe that's why you have different data. — SchautDollar, Dec 12 '13 at 05:32

score 1 · Answer 1 · answered Dec 12 '13 at 05:36

1

Use this chunk of code. Variable result should have the html code.

 System.Net.WebClient webClient = new System.Net.WebClient();
 string result = webClient.DownloadString(URL);

answered Dec 12 '13 at 05:36

chris_techno25

2,401
5
20
32

i Do this but failed ... – user3089799 Dec 12 '13 at 06:01
I tried the code I posted. It works in my machine. I'm getting everything including the comments section. Make sure your URL really points to something. Please check your URL. – chris_techno25 Dec 12 '13 at 06:24
dear every thing correct i think the comments section of the url loading form ajax or may be jquery and my request get the html before ajaxx/jquery request ....is any way to get the html after ajax/jquery request ??? – user3089799 Dec 12 '13 at 06:33
I'm not sure if I got your problem. Anyways, I checked the URL and the comments section had links to another URL, which means yeah it does use Ajax to load the comments. If you want to get the html source of the comments, manipulating the original URL like searching for the comment URLs and getting the source might be a good idea. I actually did something like this before so I know it works. I'm just not sure if this is what you want. – chris_techno25 Dec 12 '13 at 06:54
yeah exactly thats my problem how to solve it ? – user3089799 Dec 12 '13 at 07:47
If you want to get the html source of the comments, manipulating the original URL like searching for the comment URLs and getting the source might be a good idea. – chris_techno25 Dec 12 '13 at 07:51
i dnt get what you say ??? – user3089799 Dec 12 '13 at 08:57

Thilina H · Answer 2 · 2013-12-12T15:20:04.053

Getting HTML code from a website page. You can use code like this.

string urlAddress = "http://u-handbag.typepad.com/uhandblog/2013/11/choosing-bag-fabrics.html#comment-6a00d8341c574653ef019b022fc96f970d";

HttpWebRequest request = (HttpWebRequest)WebRequest.Create(urlAddress);
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
if (response.StatusCode == HttpStatusCode.OK)
{
  Stream receiveStream = response.GetResponseStream();
  StreamReader readStream = null;
  if (response.CharacterSet == null)
    readStream = new StreamReader(receiveStream);
  else
    readStream = new StreamReader(receiveStream, Encoding.GetEncoding(response.CharacterSet));
  string data = readStream.ReadToEnd();
  response.Close();
  readStream.Close();
}

or better to use WebClient

        using System.Net;
        using (WebClient client = new WebClient())
        {
            string htmlCode = client.DownloadString("http://u-handbag.typepad.com/uhandblog/2013/11/choosing-bag-fabrics.html#comment-6a00d8341c574653ef019b022fc96f970d");
        }

Dear i apply this code but failed to get (comments section(in the above url ) ).......... — user3089799, Dec 12 '13 at 06:01
i am unable to get html the comments section in the url..please help me..? — user3089799, Dec 12 '13 at 06:34

How can i get the html of the page?

2 Answers2