SMTPS: OpenSSL - SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol s23_clnt.c:787:

Question

I'm using OpenSSL in order to encrypt some emails, that a piece of hardware sends. But, whenever I try to call SSL_connect(), I get : SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol

After sending "EHLO" and "STARTTLS" I call the following function:

SSL_CTX *ctx = NULL;
SSL *ssl = NULL;

    void CreateTLSSession(int sockfd)
    {
        printf("///////////////creating TLS Session/////////////////////\n");
        SSL_library_init();
        SSL_load_error_strings();
        OpenSSL_add_all_algorithms();
        ctx = SSL_CTX_new(SSLv23_client_method());
        if (ctx == NULL)
        {
            printf("failed to initialize context\n");
            return;
        }
        SSL_CTX_set_options(ctx, SSL_OP_NO_SSLv2);
        ssl = SSL_new(ctx);
        if (!SSL_set_fd(ssl, sockfd))
        {
            printf("failed to bind to socket fd\n");
            return;
        }
        if (SSL_connect(ssl) < 1)
        {
            ERR_print_errors_fp(stdout);
            fflush(stdout);
            printf("SSL_connect failed\n");
            return;
        }
    }

I've tried connecting to :

smtp.live.com : 587 --> SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol s23_clnt.c:787:
smtp.live.com : 25 --> SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol s23_clnt.c:787:
smtp.gmail.com : 587 --> SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol s23_clnt.c:787:
smtp.gmail.com : 465 --> no response from server at all!
smtp.gmail.com : 25 --> SSL routines:SSL23_GET_SERVER_HELLO:unknown
protocol s23_clnt.c:787:

I've tried different ports, since some similar questions on this SO suggested, that such error is usually related to using the wrong port for SSL.

Am I missing something here?

UPDATE:

All other methods (i.e. TLSv1_1_method(), SSLv3_method() ...) lead to SSL3_GET_RECORD:wrong version number

UPDATE:

I was able to observe the following on wireshark:

"EHLO"

"at your service"

"STARTTLS"

"Ready to starttls"

-->now I call the above function

unreadable request (encrypted)

unreadable reply (encrypted)

--> ERROR

possible duplicate of ["SSL23\_GET\_SERVER\_HELLO:unknown protocol" Error Trying to Reach Outlook smtp Server](http://stackoverflow.com/questions/22361863/ssl23-get-server-hellounknown-protocol-error-trying-to-reach-outlook-smtp-ser) — jww, Mar 17 '14 at 16:03
@noloader The Link does not really answer my question. The answer simply suggests starting the communication unencrypted and then encrypt the data once STARTTLS has been issued. But I already send my "EHLO" and "STARTTLS" before encrypting, as I have already mentioned. — H_squared, Mar 18 '14 at 10:34

score 3 · Answer 1 · answered Mar 17 '14 at 13:16

3

SMTP servers on ports 587 and 25 are usually plain text and will switch to TLS only after the initial SMTP dialog and a STARTTLS command from the client. And trying SSL_connect on the plain text socket will fail.

answered Mar 17 '14 at 13:16

Steffen Ullrich

114,247
10
131
172

Well I start by "EHLO" then "STARTTLS" (so far NOT encrypted), then I call the above mentioned function. From here on, the communication should be encrypted! – H_squared Mar 18 '14 at 10:37
You have to check the responses from the server too, e.g. the SSL_connect should only be startet after you got the complete response to starttls from the server and the response was success. – Steffen Ullrich Mar 18 '14 at 10:40
Exactly. I call SSL_connect after I get "ready to STARTTLS" from the server. Also check the update in my question. – H_squared Mar 18 '14 at 10:44

score 1 · Answer 2 · answered Mar 18 '14 at 17:50

Another way to solve this problem may be to run your C program under Scott Gifford's sslclient (see http://www.superscript.com/ucspi-ssl/sslclient.html). sslclient will spawn your program and open an tcp connection to the server, and pipe your program's stdout to the server, and pipe output from the server to your program's stdin. He has a patched version for TLS that will start the connection off in plain text, then once the two sides have agreed on STARTTLS, your program can signal to sslcient to enable SSL encryption on the connection by writing a command to a file descriptor for this purpose. The nice thing about doing it this way is that you can let sslclient to all the heavy lifting as far as setting up the sockets and ssl, etc., and you can focus on the core function of your program.

score 0 · Accepted Answer · answered Apr 07 '14 at 05:52

0

The underlying socked was non-blocking. The problem was solved, by using select and waiting till the TLS handshake completes.

answered Apr 07 '14 at 05:52

H_squared

1,251
2
15
32

SMTPS: OpenSSL - SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol s23_clnt.c:787:

3 Answers3

Linked