0

I have a list of URLs, and the meaning of this is that I am checking our websites if anyone is down / offline we would get a notification and that works except some of the URLs crash at this line HttpWebResponse httpRes = (HttpWebResponse)httpReq.GetResponse(); But the rest is working just fine? can anyone tell me what I'm doing wrong? I've tried URLs with HTTPS, HTTP and even with only www...

 public void CheckUrl()//List Of URLs
    {
        List<string> urls = new List<string>() {
        "https//:www.example.com/something1/buy",
        "https//:www.example.com/something2/buy",
        "https//:www.example.com/something3/buy",
        "https//:www.example.com/something4/buy",
        };

        //walks through all the URL:s
        foreach (var url in urls)
        {
            //Creating URL Request
            HttpWebRequest httpReq = (HttpWebRequest)WebRequest.Create(url);
            httpReq.AllowAutoRedirect = false;
            try
            {
                WebClient client = new WebClient();
                string downloadString = client.DownloadString(url);


                //Trying to find a response
                HttpWebResponse httpRes = (HttpWebResponse)httpReq.GetResponse();
                if ((httpRes.StatusCode != HttpStatusCode.OK || httpRes.StatusCode != HttpStatusCode.Found) && downloadString.Contains("404 -") || downloadString.Contains("Server Error"))
                {
                    // Code for NotFound resources goes here.
                    SendVerificationLinkEmail(httpRes.StatusCode.ToString(), url);
                    foreach (var number in Numbers)
                    {
                        SendSms(url, number);
                    }

                }
                //Close the response.
                httpRes.Close();
            }
            catch(Exception e)
            {

                //sending only to admin to check it out first
                SendExeptionUrl(url);
                foreach (var number in Numbers)
                {
                    SendSms(url, number);
                }
            }

        }
        Application.Exit();
    }
leiit
  • 97
  • 1
  • 8
  • 1
    What error do you get when it crashes? Did you try browsing the url directly in the browser? Is it working fine in browser? – Chetan Jan 30 '18 at 13:29
  • Bad Request means you didn't send the request correctly containing the information the site is expecting, and in the right format. If these are URLs designed to be viewed in a browser, they may have specific requirements designed to frustrate access from automated tools. If you're trying to check whether your sites are accessible, aren't there better methods, e.g. monitoring tools on the servers, looking for things in logs etc? – ADyson Jan 30 '18 at 13:29
  • Hello, yes the all of the links work correctly but like I said some of them just crashes, the ERROR message is `500 internal server error `, I got over 70 links to different landing pages under same domain/host, so why do the others work if I'm sending the wrong request, I'm also checking if site has a small error or bad content, but i need the request to do that – leiit Jan 30 '18 at 13:35
  • 500 error indicates the there is an issue at the server end. You can not figure out that issue from the client. – Chetan Jan 30 '18 at 13:39
  • Possibly caused by SSL. Check this: https://stackoverflow.com/questions/560804/how-do-i-use-webrequest-to-access-an-ssl-encrypted-site-using-https – LocEngineer Jan 30 '18 at 13:47
  • @ChetanRanpariya Not necessarily true, depends what's in the Response body. – Mark C. Jan 30 '18 at 13:56
  • When I say issue at the server end I mean the server is not able to process the request due to unknown reason. If server can handle the issue the it will return proper details in the response but then there is no point returning 500. https://www.w3.org/Protocols/HTTP/HTRESP.html – Chetan Jan 30 '18 at 14:01
  • @LocEngineer No, not really because I've tried HTTP and HTTPS and without... – leiit Jan 30 '18 at 14:14
  • Then try with adding useragent and referrer as in this accepted answer: https://stackoverflow.com/questions/702079/why-does-my-httpwebrequest-return-400-bad-request – LocEngineer Jan 30 '18 at 14:39
  • @LocEngineer well, unfortunately, that did not the trick either, I feel that this is so wired, btw if anyone would like to try link is [ https:// www.courlux.com/se/flexifront/x5 ] this is the one who crashes all the time. – leiit Jan 30 '18 at 14:59
  • "why do the others work if I'm sending the wrong request"...because every request to a different URL is different and is processed differently by the server, according to whatever code runs at that endpoint. Each endpoint could set different requirements about what should be in the request, including headers etc. HTTP requests are quite complicated. But if you're convinced the URLs and requests are correctly formed for the environment in question then, you need to look at the server logs to try and find out what causes the 500 responses. – ADyson Jan 30 '18 at 15:58
  • Also in your title you say you get Bad Request, which is returned as a HTTP 400 status, but in the comments you say it's 500 Internal Server Error. You should clarify in your question and title. – ADyson Jan 30 '18 at 16:03
  • Must be something with the site itself. I have tested this with various sites and it is working. Only this specific site throws an error 500. So no bad request but a server error. Firefox console also shows some errors upon loading the site in browser, so I am not specifically surprised. – LocEngineer Jan 30 '18 at 16:54
  • @LocEngineer Hmm, any idea how to solve this then? – leiit Jan 31 '18 at 07:47
  • Do you own the sites? Are you the admin? Only then can you do _anything_ at all. The error is server-side, there is absolutely _nothing_ you can do client-side. – LocEngineer Jan 31 '18 at 10:25
  • Yes I do, and yes I am an admin, how do you mean by server-side? What can I do to solve it? @LocEngineer – leiit Jan 31 '18 at 12:19
  • Check https://stackoverflow.com/questions/2640526/detailed-500-error-message-asp-iis-7-5 and https://stackoverflow.com/questions/5385714/deploying-website-500-internal-server-error to get more information on why the site returns a 500. – LocEngineer Jan 31 '18 at 12:27
  • I 've checked and they are talking about other problem, my problem is that I use Web Forms and getting a null when sending a request, when those guys talk about MVC problems and why they have crash on their sites wich o don't have. – leiit Feb 02 '18 at 08:54

0 Answers0