24

Context

I have a personal server that I use for the web. I sometimes need to SSH/SFTP to it.

Disclamer: I have very little experience with nginx internals.

Problem

This morning, I figured out that the free wifi in a well-know cafe chain was blocking SSH (actually, they are blocking anything that's not on 80/443). But when I need SSH, I need it, so I looked for ways to share SSH and HTTPS on the same port.

What I looked at

I have looked at a few possible solutions that can run on port 443:

  • SSHL: a SSH/OpenVPN/HTTPS multiplexer;
  • OpenVPN: a VPN solution has a built-in multiplexer for OpenVPN and HTTPS;
  • HAProxy: a webserver/load balancer can also multiplex everything.

All of these seem pretty straight-forward but I don't really like the fact of adding layers and complexity and possibly slowing things down just in the unlikely event that I need to SSH on 443.

Putting nginx into the mix

I know that nginx already supports raw TCP streams handling. So I was wondering if I could use that on port 443 too directly in nginx. The idea being that nginx could choose to use the http module if it recognizes HTTP(S) or stream for everything else.

Questions

In that context, I have two questions:

  • Is nginx even capable of doing such a distinction? (I am not even sure I would be able to listen on port 443 in both the http and the stream block at the same time.)
  • If so, would there be any blatant performance issue with that setup? (I am thinking about transfer speed with SFTP for instance, not really SSH per se.)
JohnW
  • 377

9 Answers9

23

Since nginx version 1.15.2 added new variable $ssl_preread_protocol. And in official blog added post about how to use this variable for multiplexing HTTPS and SSH on the same port https://www.nginx.com/blog/running-non-ssl-protocols-over-ssl-port-nginx-1-15-2/

Example of configuring SSH(by default) and HTTPS:

stream {
    upstream ssh {
        server 192.0.2.1:22;
    }

    upstream web {
        server 192.0.2.2:443;
    }

    map $ssl_preread_protocol $upstream {
        default ssh;
        "TLSv1.2" web;
    }

    # SSH and SSL on the same port
    server {
        listen 443;

        proxy_pass $upstream;
        ssl_preread on;
    }
}
sattellite
  • 366
11

I have a working configuration tunneling ssh over tls on port 443, using the nginx stream module. I also do XMPP over TLS and normal HTTP on the same port. I do the mutliplexing via ALPN.

(You need nginx > 1.13.10 to use the ssl_preread module with alpn http://nginx.org/en/docs/stream/ngx_stream_ssl_preread_module.html#ssl_preread)

My configuration uses the docker version of nginx, but it should also work without docker.

stream {
    # check ALPN for xmpp client or server and redirect to local ssl termination endpoints
    map $ssl_preread_alpn_protocols $ssl_multiplexer {
        "xmpp-client"     127.0.0.1:5422;
        "xmpp-server"     127.0.0.1:5469;
        "identifyssh"     127.0.0.1:8822;
        default           127.0.0.1:8443;
    }

    server {
        listen 443;
        ssl_preread on;
        proxy_pass $ssl_multiplexer;
        proxy_protocol on;
        set_real_ip_from  172.18.0.0/32;
    }

    # ssl termination for c2s connections
    server {
        listen 5422 ssl proxy_protocol;
        # ... <- tls keys and options here
        proxy_ssl off;
        proxy_pass ejabberd:5222;
    }

    # ssl termination for s2s connections
    server {
        listen 5469 ssl proxy_protocol;
        # ... <- tls keys and options here
        proxy_ssl off;
        proxy_pass ejabberd:5269;
    }

    # ssl termination for ssh connections
    server {
        listen 8822 ssl proxy_protocol;
        # ... <- tls keys and options here
        proxy_ssl off;
        proxy_pass yourserver:22;
    }
}

If you want to use the XMPP stuff you have to add some SRV records to point to your servers 443 port, see https://xmpp.org/extensions/xep-0368.html

If you want to connect to your ssh server you have to wrap your ssh session in a ssl session that sends the ALPN string you defined in your stream config. I used "identifyssh" in the example above. You can use anything, but try to not collide with the official defined names: https://www.iana.org/assignments/tls-extensiontype-values/tls-extensiontype-values.xhtml#alpn-protocol-ids

To start the ssh session from your client to your prepared server use:

ssh you@yourserver -o "ProxyCommand openssl s_client -alpn identifyssh -ign_eof -connect yourserver:443"

And you should be connected.

I should also note, that I use the proxy_protocol to keep the clients headers and IP-address while passing to my backends.

Your normal http server configured in the http {} section should take care of this:

server {
  listen 8443 ssl proxy_protocol;
  # ...
}

The best thing is, that you don't need tools like sslh, stunnel, proxytunnel or others to make this work. You only need a newer nginx and openssl. Hope this helps somebody. It would helped me digging into that stuff.

tux
  • 111
5

Or, you could proxytunnel ssh through your nginx with HTTP/S connection

Take a look: proxytunnel

mtsdev
  • 151
  • 1
2

This question is slightly related to another one I've answered a while ago:

https://stackoverflow.com/questions/34741571/nginx-tcp-forwarding-based-on-hostname/34958192#34958192

Yes, it is technically possible to differentiate between ssh and https traffic, and route the connection appropriately; however, nginx currently doesn't have such support, to my knowledge.

However, what you could do is simply run sshd directly on the https port in addition to the ssh one (/usr/sbin/sshd -p 22 -p 443), and/or use the firewall and/or port knocking in order to differentiate where connections to port 443 get routed to.

Community
  • 1
cnst
  • 2,615
2

You could setup an iptables rule to forward connections from your well known cafe on port 443 to port 22. Alternatively bounce the traffic off a port relay elsewhere on the internet, changing the pirt number.

Just trying to solve the problem with nginx isn't going to work.

symcbean
  • 432
2

There exists unmaintained patch for nginx to support protocol multiplexing for SSH and HTTPS: https://github.com/shawnl/nginx-ssh

However it is not maintained anymore, non-nginx project which supports what you are looking for is: https://github.com/yrutschle/sslh

I suggest you use SSLH infront of your nginx and ssh server.

Ladislav Zitka
  • 121
1

As far as I know, currently nginx doesn't support it.

In SSH-2, client will send a hello message to server:

When the connection has been established, both sides MUST send an identification string. This identification string MUST be

 SSH-protoversion-softwareversion SP comments CR LF

In TLS 1.2, clients need to send first:

  Client                                                Server

ClientHello                   -------->
                                                   ServerHello
                                            [ChangeCipherSpec]
                                <--------             Finished
  [ChangeCipherSpec]
  Finished                      -------->
  Application Data              <------->     Application Data

Clients can use this information for implementation.

Community
  • 1
Gea-Suan Lin
  • 161
1

Since Nginx Version 1.9.0,NGINX support ngx_stream_core_module module, it should be enabled with the --with-stream. When stream module is enable they are possible to ssh protocol tcp proxy

stream {
upstream ssh {
    server 192.168.1.12:22;
}
    server {
    listen        12345;
    proxy_pass    ssh;

} }

https://www.nginx.com/resources/admin-guide/tcp-load-balancing/

Hendi Fauzi
  • 11
  • 1
0

Maybe you can use nginScript to implement your own multiplexer? See here for an introduction: https://www.nginx.com/blog/introduction-nginscript/

Thraidh
  • 131