Using a self-signed certificate with .NET's HttpWebRequest/Response

Question

I'm trying to connect to an API that uses a self-signed SSL certificate. I'm doing so using .NET's HttpWebRequest and HttpWebResponse objects. And I'm getting an exception that:

The underlying connection was closed: Could not establish trust relationship for the SSL/TLS secure channel.

I understand what this means. And I understand why .NET feels it should warn me and close the connection. But in this case, I'd like to just connect to the API anyway, man-in-the-middle attacks be damned.

So, how do I go about adding an exception for this self-signed certificate? Or is the approach to tell HttpWebRequest/Response not to validate the certificate at all? How would I do that?

Answer 1

Turns out, if you just want to disable certificate validation altogether, you can change the ServerCertificateValidationCallback on the ServicePointManager, like so:

ServicePointManager.ServerCertificateValidationCallback = delegate { return true; };

This will validate all certificates (including invalid, expired or self-signed ones).

Answer 2

@Domster: that works, but you might want to enforce a bit of security by checking if the certificate hash matches what you expect. So an expanded version looks a bit like this (based on some live code we're using):

static readonly byte[] apiCertHash = { 0xZZ, 0xYY, ....};

/// <summary>
/// Somewhere in your application's startup/init sequence...
/// </summary>
void InitPhase()
{
    // Override automatic validation of SSL server certificates.
    ServicePointManager.ServerCertificateValidationCallback =
           ValidateServerCertficate;
}

/// <summary>
/// Validates the SSL server certificate.
/// </summary>
/// <param name="sender">An object that contains state information for this
/// validation.</param>
/// <param name="cert">The certificate used to authenticate the remote party.</param>
/// <param name="chain">The chain of certificate authorities associated with the
/// remote certificate.</param>
/// <param name="sslPolicyErrors">One or more errors associated with the remote
/// certificate.</param>
/// <returns>Returns a boolean value that determines whether the specified
/// certificate is accepted for authentication; true to accept or false to
/// reject.</returns>
private static bool ValidateServerCertficate(
        object sender,
        X509Certificate cert,
        X509Chain chain,
        SslPolicyErrors sslPolicyErrors)
{
    if (sslPolicyErrors == SslPolicyErrors.None)
    {
        // Good certificate.
        return true;
    }

    log.DebugFormat("SSL certificate error: {0}", sslPolicyErrors);

    bool certMatch = false; // Assume failure
    byte[] certHash = cert.GetCertHash();
    if (certHash.Length == apiCertHash.Length)
    {
        certMatch = true; // Now assume success.
        for (int idx = 0; idx < certHash.Length; idx++)
        {
            if (certHash[idx] != apiCertHash[idx])
            {
                certMatch = false; // No match
                break;
            }
        }
    }

    // Return true => allow unauthenticated server,
    //        false => disallow unauthenticated server.
    return certMatch;
}

Answer 3

Note, that in .NET 4.5 you can override SSL validation per HttpWebRequest itself (and not via global delegate which affects all requests):

http://msdn.microsoft.com/en-us/library/system.net.httpwebrequest.servercertificatevalidationcallback.aspx

HttpWebRequest request = (HttpWebRequest)HttpWebRequest.Create(uri);
request.ServerCertificateValidationCallback = delegate { return true; };

Answer 4

Add the self signed cert to the Local Computer Trusted Root Certification Authorities

You can import the cert by running the MMC as Administrator.

How to: View Certificates with the MMC Snap-in

Answer 5

The scope of the validation callback used in Domster's answer can be limited to a specific request using the sender parameter on the ServerCertificateValidationCallback delegate. The following simple scope class uses this technique to temporarily wire up a validation callback that only executes for a given request object.

public class ServerCertificateValidationScope : IDisposable
{
    private readonly RemoteCertificateValidationCallback _callback;

    public ServerCertificateValidationScope(object request,
        RemoteCertificateValidationCallback callback)
    {
        var previous = ServicePointManager.ServerCertificateValidationCallback;
        _callback = (sender, certificate, chain, errors) =>
            {
                if (sender == request)
                {
                    return callback(sender, certificate, chain, errors);
                }
                if (previous != null)
                {
                    return previous(sender, certificate, chain, errors);
                }
                return errors == SslPolicyErrors.None;
            };
        ServicePointManager.ServerCertificateValidationCallback += _callback;
    }

    public void Dispose()
    {
        ServicePointManager.ServerCertificateValidationCallback -= _callback;
    }
}

The above class can be used to ignore all certificate errors for a specific request as follows:

var request = WebRequest.Create(uri);
using (new ServerCertificateValidationScope(request, delegate { return true; }))
{
    request.GetResponse();
}

Answer 6

Just building on answer from devstuff to include subject and issuer...comments welcome...

public class SelfSignedCertificateValidator
{
    private class CertificateAttributes
    {
        public string Subject { get; private set; }
        public string Issuer { get; private set; }
        public string Thumbprint { get; private set; }

        public CertificateAttributes(string subject, string issuer, string thumbprint)
        {
            Subject = subject;
            Issuer = issuer;                
            Thumbprint = thumbprint.Trim(
                new char[] { '\u200e', '\u200f' } // strip any lrt and rlt markers from copy/paste
                ); 
        }

        public bool IsMatch(X509Certificate cert)
        {
            bool subjectMatches = Subject.Replace(" ", "").Equals(cert.Subject.Replace(" ", ""), StringComparison.InvariantCulture);
            bool issuerMatches = Issuer.Replace(" ", "").Equals(cert.Issuer.Replace(" ", ""), StringComparison.InvariantCulture);
            bool thumbprintMatches = Thumbprint == String.Join(" ", cert.GetCertHash().Select(h => h.ToString("x2")));
            return subjectMatches && issuerMatches && thumbprintMatches; 
        }
    }

    private readonly List<CertificateAttributes> __knownSelfSignedCertificates = new List<CertificateAttributes> {
        new CertificateAttributes(  // can paste values from "view cert" dialog
            "CN = subject.company.int", 
            "CN = issuer.company.int", 
            "f6 23 16 3d 5a d8 e5 1e 13 58 85 0a 34 9f d6 d3 c8 23 a8 f4") 
    };       

    private static bool __createdSingleton = false;

    public SelfSignedCertificateValidator()
    {
        lock (this)
        {
            if (__createdSingleton)
                throw new Exception("Only a single instance can be instanciated.");

            // Hook in validation of SSL server certificates.  
            ServicePointManager.ServerCertificateValidationCallback += ValidateServerCertficate;

            __createdSingleton = true;
        }
    }

    /// <summary>
    /// Validates the SSL server certificate.
    /// </summary>
    /// <param name="sender">An object that contains state information for this
    /// validation.</param>
    /// <param name="cert">The certificate used to authenticate the remote party.</param>
    /// <param name="chain">The chain of certificate authorities associated with the
    /// remote certificate.</param>
    /// <param name="sslPolicyErrors">One or more errors associated with the remote
    /// certificate.</param>
    /// <returns>Returns a boolean value that determines whether the specified
    /// certificate is accepted for authentication; true to accept or false to
    /// reject.</returns>
    private bool ValidateServerCertficate(
        object sender,
        X509Certificate cert,
        X509Chain chain,
        SslPolicyErrors sslPolicyErrors)
    {
        if (sslPolicyErrors == SslPolicyErrors.None)
            return true;   // Good certificate.

        Dbg.WriteLine("SSL certificate error: {0}", sslPolicyErrors);
        return __knownSelfSignedCertificates.Any(c => c.IsMatch(cert));            
    }
}

Answer 7

To add as a possible help to someone else... If you want it to prompt the user to install the self-signed cert, you can use this code (modified from above).

Does not require admin rights, installs to the local users trusted profiles:

    private static bool ValidateServerCertficate(
        object sender,
        X509Certificate cert,
        X509Chain chain,
        SslPolicyErrors sslPolicyErrors)
    {
        if (sslPolicyErrors == SslPolicyErrors.None)
        {
            // Good certificate.
            return true;
        }

        Common.Helpers.Logger.Log.Error(string.Format("SSL certificate error: {0}", sslPolicyErrors));
        try
        {
            using (X509Store store = new X509Store(StoreName.My, StoreLocation.CurrentUser))
            {
                store.Open(OpenFlags.ReadWrite);
                store.Add(new X509Certificate2(cert));
                store.Close();
            }
            return true;
        }
        catch (Exception ex)
        {
            Common.Helpers.Logger.Log.Error(string.Format("SSL certificate add Error: {0}", ex.Message));
        }

        return false;
    }

This seems to work well for our application, and if the user presses no, the communication will not work.

Update: 2015-12-11 - Changed StoreName.Root to StoreName.My - My will install into the local users store, instead of Root. Root on some systems will not work, even if you "run as administrator"

Answer 8

First of all - I apologize, because I have used the solution that was described by @devstuff. However, I have found some ways to improve it.

• adding self-signed certificates handling
• comparison by the Raw data of certificates
• actual certificate authority validation
• some additional comments and improvements

Here is my modification:

private static X509Certificate2 caCertificate2 = null;

/// <summary>
/// Validates the SSL server certificate.
/// </summary>
/// <param name="sender">An object that contains state information for this validation.</param>
/// <param name="cert">The certificate used to authenticate the remote party.</param>
/// <param name="chain">The chain of certificate authorities associated with the remote certificate.</param>
/// <param name="sslPolicyErrors">One or more errors associated with the remote certificate.</param>
/// <returns>Returns a boolean value that determines whether the specified certificate is accepted for authentication; true to accept or false to reject.</returns>
private static bool ValidateServerCertficate(
        object sender,
        X509Certificate cert,
        X509Chain chain,
        SslPolicyErrors sslPolicyErrors)
{
    if (sslPolicyErrors == SslPolicyErrors.None)
    {
        // Good certificate.
        return true;
    }

    // If the following line is not added, then for the self-signed cert an error will be (not tested with let's encrypt!):
    // "A certificate chain processed, but terminated in a root certificate which is not trusted by the trust provider. (UntrustedRoot)"
    chain.ChainPolicy.VerificationFlags = X509VerificationFlags.AllowUnknownCertificateAuthority;

    // convert old-style cert to new-style cert
    var returnedServerCert2 = new X509Certificate2(cert);

    // This part is very important. Adding known root here. It doesn't have to be in the computer store at all. Neither do certificates.
    chain.ChainPolicy.ExtraStore.Add(caCertificate2);

    // 1. Checks if ff the certs are OK (not expired/revoked/etc) 
    // 2. X509VerificationFlags.AllowUnknownCertificateAuthority will make sure that untrusted certs are OK
    // 3. IMPORTANT: here, if the chain contains the wrong CA - the validation will fail, as the chain is wrong!
    bool isChainValid = chain.Build(returnedServerCert2);
    if (!isChainValid)
    {
        string[] errors = chain.ChainStatus
            .Select(x => String.Format("{0} ({1})", x.StatusInformation.Trim(), x.Status))
            .ToArray();

        string certificateErrorsString = "Unknown errors.";

        if (errors != null && errors.Length > 0)
        {
            certificateErrorsString = String.Join(", ", errors);
        }

        Log.Error("Trust chain did not complete to the known authority anchor. Errors: " + certificateErrorsString);
        return false;
    }

    // This piece makes sure it actually matches your known root
    bool isValid = chain.ChainElements
        .Cast<X509ChainElement>()
        .Any(x => x.Certificate.RawData.SequenceEqual(caCertificate2.GetRawCertData()));

    if (!isValid)
    {
        Log.Error("Trust chain did not complete to the known authority anchor. Thumbprints did not match.");
    }

    return isValid;
}

setting certificates:

caCertificate2 = new X509Certificate2("auth/ca.crt", "");
var clientCertificate2 = new X509Certificate2("auth/client.pfx", "");

passing delegate method

ServerCertificateValidationCallback(ValidateServerCertficate)

client.pfx is generated with KEY and CERT as such:

openssl pkcs12 -export -in client.crt -inkey client.key -out client.pfx

Answer 9

One thing to keep in mind is that having the ServicePointManager.ServerCertificateValidationCallback does not seem to mean that the CRL check and servername validation are not done, it only provides a means to override their result. So your service might still take a while to get a CRL, you'll only know afterwards that it failed some checks.

Answer 10

I was running into the same problem as the OP where the web request would throw that exact exception. I had everything setup correctly I thought, the certificate was installed, I could locate it in the machine store just fine and attach it to the web request, and I had disabled the verification of certificates on the request context.

It turned out that I was running under my user account, and that the certificate was installed to the machine store. This caused the web request to throw this exception. To solve the problem I had to either be running as administrator or install the certificate to the user store and read it from there.

It would seem that C# is able to find the certificate in the machine store even though it can't be used with a web request, and that this results in the OP's exception being thrown once the web request is issued.