Set Queue Limit via jetty.xml in Jetty 9 Maven Plugin

Question

The thread pool in Jetty by default is implemented with an unbounded queue once the thread pool fills up. I'd like to put a limit on the size of the queue. There is a constructor for BlockingArrayQueue that takes a maxCapacity value but I can see no way to call it using jetty.xml. As of Jetty 9, there is no setter for the threadpool in org.eclipse.jetty.server.Server, I can only get a reference to the thread pool that has already been instantiated and mutate it (see this answer). And the setter for the queue field in QueuedThreadPool throws an UnsupportedOperationException, saying to use constructor injection. But this is impossible if I can only mutate the thread pool, not set a new one on the server instance. Attempting to define the thread pool as a constructor arg yields the following warning:

2014-09-22 13:15:13.688:WARN:oejx.XmlConfiguration:main: Ignored arg: | 200501000| 6000| false|

This is with the Jetty Maven Plugin v9.2.2.v20140723. Here is the configuration in my pom.xml:

    <configuration>
      <jettyXml>${basedir}/jetty.xml</jettyXml>
      <stopKey>x</stopKey>
      <stopPort>7999</stopPort>
      <requestLog implementation="org.eclipse.jetty.server.NCSARequestLog">
        <append>true</append>
      </requestLog>
      <webApp>
        <war>${basedir}/target/app</war>
        <contextPath>/app</contextPath>
      </webApp>
      <scanTargets>
        <scanTarget>${basedir}/src/main/webapp/WEB-INF/</scanTarget>
      </scanTargets>
      <reload>manual</reload>
    </configuration>

Answer 1

Update: The Server constructor is how you configure the thread pool, unfortunately, you cannot configure the Server constructor from within jetty-maven-plugin. That is out of scope for the jetty-maven-plugin.

---

The ThreadPool in Jetty 9 is now setup in the Constructor of the Server instance.

Using XML, you can reconfigure it, it's even documented in the jetty.xml itself.

Feel free to change it, just be aware that bound thread pools are known to cause problems if the limit is too small.

Here's a handy napkin formula.

Max Threads = (((number of cpu cores) * 4) * (connector count)) + (max concurrent requests)

In practice, most values under 400 are going to cause you problems on even mildly busy servers. Don't take this statement as meaning 400 is a good starting point, that would be grossly improper of you, you need to test, and monitor, and keep adjusting until you find a happy value for your server. (Don't forget to test for load spikes, and what happens when your databases fail)

Thinking that you need to set an upper bounds for performance is a form of premature optimization.

Its true that the QueuedThreadPool that Jetty uses is unbound, but it also cleans itself up and removes threads from the pool over time, allowing the server to handle sudden loads and back off when the load subsides.

If its memory or other resources you are concerned about, know that the default Jetty installation runs fine on even Android 2.3 (Gingerbread) (note: Jetty 7 w/QTP), Android 4.4 (Jetty 9), and Raspberry Pi (Jetty 7 thru 9).

Lastly, how to set your accept queue size.

Configure your ServerConnector (usually found in the etc/jetty-http.xml)

<Set name="acceptQueueSize">40</Set>

The default value is 0, and relates to the backlog parameter in the ServerSocketChannel.bind(SocketAddress,int) call at the lowest levels.

Answer 2

We faced a similar issue where in we wanted the Server to reject the requests if all threads are busy instead of queuing them indefinitely.

I think this is a genuine ask, following is the approach.

Since its not possible to override the default pool size or type in the Jetty Server. We wrote a custom ServerConnector, where in you can set the queue size or your executor for that matter.

jetty.addServerCustomizers((Server server) -> {
    LinkedBlockingQueue<Runnable> queue = new LinkedBlockingQueue< (maxQueueSize);
    ExecutorService exceutorService =
        new ThreadPoolExecutor(minThreads, maxThreads, 1,
            TimeUnit.HOURS, queue);

    // extract host and port from existing connector...
    String host = "0.0.0.0";
    int port = 1900;
    for (Connector c : server.getConnectors()) {
        if (c instanceof ServerConnector) {
            host = ((ServerConnector) c).getHost();
            port = ((ServerConnector) c).getPort();
        }
    }


    ServerConnector connector = new ServerConnector(server, exceutorService, null, null, -1, -1, new HttpConnectionFactory());
    connector.setHost(host);
    connector.setPort(port);
    server.setConnectors(new Connector[] { connector });
}

Cons

1) Since Jetty expects a Queued threadpool, when we tried the same it used to give random results where server startup failed in-spite of correct configurations. The behavior was not predictable.

2) Since ExecutorService does not implement Threadpool the metrics collection plugin for Jetty might not work properly.

Will keep this answer updated when we hit production.