Unknown limit for Websocket connections

Question

I have a strange problem with my WebSocket. I can reach only 381 sockets connected. This is always 381, I don't know where is this number from. I'm using Ubuntu 17.10.

I'm using multiple browsers for testing. 382nd and later requests from browser have no response. example -> http://prntscr.com/j7blpq

Hardware info:

• 2xcpu -> Intel Core i7

• 4GB RAM

• 20GB HD

I'm using nginx to hold connections. This is my nginx.conf:

user www-data;
worker_rlimit_nofile 8192;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
    worker_connections 768;
    multi_accept on;
}

http {

    ##
    # Basic Settings
    ##

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    types_hash_max_size 2048;
    # server_tokens off;

    # server_names_hash_bucket_size 64;
    # server_name_in_redirect off;

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    ##
    # SSL Settings
    ##

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
    ssl_prefer_server_ciphers on;

    ##
    # Logging Settings
    ##

    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    ##
    # Gzip Settings
    ##

    gzip on;
    gzip_disable "msie6";

    # gzip_vary on;
    # gzip_proxied any;
    # gzip_comp_level 6;
    # gzip_buffers 16 8k;
    # gzip_http_version 1.1;
    # gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

    ##
    # Virtual Host Configs
    ##

    keepalive_requests 100000;

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}

This is my PHP WebSocket:

<?php  

class ws_server{
  var $master;
  var $sockets = array(); //create an array of socket objects 
  var $users   = array(); //create an array of users objects to handle discussions with users
  var $debug   = false;

  function ascii_banner() //just for old-skool fun...
  {
    $banner="               _    ____             _        _   \n";
    $banner.=" __      _____| |__/ ___|  ___   ___| | _____| |_\n ";
    $banner.="\ \ /\ / / _ \ '_ \___ \ / _ \ / __| |/ / _ \ __|\n";
    $banner.="  \ V  V /  __/ |_) |__) | (_) | (__|   <  __/ |_ \n";
    $banner.="   \_/\_/ \___|_.__/____/ \___/ \___|_|\_\___|\__|\n";
    return $banner;

  }

  function __construct($address,$port){

      $chatHandler = new ChatHandler();
      $null = NULL;

      error_reporting(E_ALL);
      set_time_limit(0);
      ob_implicit_flush();

      $this->master=socket_create(AF_INET, SOCK_STREAM, SOL_TCP)     or die("socket_create() failed");
      socket_set_option($this->master, SOL_SOCKET, SO_REUSEADDR, 1)  or die("socket_option() failed");
      socket_bind($this->master, $address, $port)                    or die("socket_bind() failed");
      socket_listen($this->master,44444)                              or die("socket_listen() failed");
      $this->sockets[] = $this->master;

      $chatHandler->say($this->ascii_banner() );
      $chatHandler->say("PHP WebSocket Server running....");
      $chatHandler->say("Server Started : ".date('Y-m-d H:i:s'));
      $chatHandler->say("Listening on   : ".$address." port ".$port);
      $chatHandler->say("Master socket  : ".$this->master."\n");
      $chatHandler->say(".... awaiting connections ...");

      $clientSocketArray = array($this->master);
      while (true) {
          echo 'log-1';

          $newSocketArray = $clientSocketArray;
          echo 'count socket new array'.count($newSocketArray);
          #socket_select($newSocketArray, $null, $null, null);
          if (false === socket_select($newSocketArray, $null, $null, null)) {
              echo "socket_select() failed, reason: " .
              socket_strerror(socket_last_error()) . "\n";
          }
          echo 'log 0';

          if (in_array($this->master, $newSocketArray)) {
          echo 'log1';
              $newSocket = socket_accept($this->master);
              echo 'log2';
              $clientSocketArray[] = $newSocket;

              $chatHandler->connect($newSocket);
              echo 'log3';

              $user = $chatHandler->getuserbysocket($newSocket);
              echo 'log4';

              $bytes = @socket_recv($newSocket,$buffer,4096,0);
              echo 'log5';

              $chatHandler->doHandshake($user,$buffer,$address,$port);
              echo 'log6';

              $newSocketIndex = array_search($this->master, $newSocketArray);
              unset($newSocketArray[$newSocketIndex]);
              echo 'log7';
          }

          foreach ($newSocketArray as $newSocketArrayResource) {
              echo 'log 7-8';
              $bytes = @socket_recv($newSocketArrayResource,$buffer,4096,0);
              echo 'log8';
              if($bytes==0){ 
                echo 'log9';
                $chatHandler->disconnect($newSocketArrayResource); 
                echo 'log10';
                $newSocketIndex = array_search($newSocketArrayResource, $clientSocketArray);
                echo 'log11';
                unset($clientSocketArray[$newSocketIndex]);     

              } else {
                $chatHandler->process($user,$chatHandler->frame_decode($buffer) ); 
                echo 'log12';
              }
          }
      }
      echo 'log13';
      socket_close($this->master);
    }
  }

I created logs to see if this is a connection problem or PHP, but everything looks fine, socket_select() is listening, no errors occurs but user count stops on 381 users connected again. Every user loop reach log-7 and ends on log-1 right before socket_select() which means that socket is listening.

Some of related server settings:

• file-max = 396850;
• ulimit = 4096;
• net.ipv4.netfilter.ip_conntrack_max = 10000;

I follow this instruction: Increasing the maximum number of tcp/ip connections in linux

Answer 1

In your nginx.conf you have worker_connections 768; this will allow 768 connections, if you you connect only via websocket to your server without reverse proxy.

If you have a ngnix configured as reverse proxy, you can have only 384 open ws connections because 768/2 = 384.

May some connection come from your local machine where you check your ws so this adds one or two more connections.

Reference: https://serverfault.com/questions/787919/optimal-value-for-nginx-worker-connections