Jetty Connectors and Protocols

Jetty provides two WebSocket implementations: one based on the Java WebSocket APIs defined by JSR 356, provided by module websocket-javax, and one based on Jetty specific WebSocket APIs, provided by module websocket-jetty. The Jetty websocket module enables both implementations, but each implementation can be enabled independently.

To enable WebSocket support, you also need to decide what version of the HTTP protocol you want WebSocket to be initiated from, and whether you want secure HTTP.

For example, to enable clear-text WebSocket from HTTP/1.1, use the following command (issued from within the $JETTY_BASE directory):

To enable secure WebSocket from HTTP/2, use the following command (issued from within the $JETTY_BASE directory):

When enabling secure protocols you need a valid KeyStore (read this section to create your own KeyStore). As a quick example, you can enable the test-keystore module, that creates on-the-fly a KeyStore containing a self-signed certificate:

To enable WebSocket on both HTTP/1.1 and HTTP/2, both clear-text and secure, use the following command (issued from within the $JETTY_BASE directory):

Enabling the WebSocket Jetty modules comes with a startup cost because Jetty must perform two steps:

WebSocket support is by default enabled for all web applications.

For a specific web application, you can disable step 2 for Java WebSocket support (i.e. when the websocket-javax module is enabled) by setting the context attribute org.eclipse.jetty.websocket.javax to false:

Furthermore, for a specific web application, you can disable step 1 (and therefore also step 2) as described in the annotations processing section.

Web applications may need to use a WebSocket client to communicate with third party WebSocket services.

If the web application uses the Java WebSocket APIs, the WebSocket client APIs are provided by the Servlet Container and are available to the web application by enabling the WebSocket server APIs, and therefore you must enable the websocket-javax Jetty module.

However, the Java WebSocket Client APIs are quite limited (for example, they do not support secure WebSocket). For this reason, web applications may want to use the Jetty WebSocket Client APIs.

When using the Jetty WebSocket Client APIs, web applications should include the required jars and their dependencies in the WEB-INF/lib directory of the *.war file. Alternatively, when deploying your web applications in Jetty, you can enable the websocket-jetty-client Jetty module to allow web applications to use the Jetty WebSocket Client APIs provided by Jetty, without the need to include jars and their dependencies in the *.war file.

This section explains how to configure Jetty to serve your WordPress site.

The prerequisites are:

Then, the secure HTTP and/or the secure HTTP/2 Jetty modules should be enabled to allow browsers to connect to Jetty.

Lastly, enable the fcgi-proxy module to provide FastCGI support (to convert HTTP requests from browsers to FastCGI for php-fpm and vice versa), and the core-deploy module to deploy your WordPress web application as a Jetty context XML file.

For example:

Now you can deploy a Jetty context XML file that represents your WordPress web application.

Use the following file as example, copy it as $JETTY_BASE/webapps/wordpress.xml and customize it as necessary:

Now you can start Jetty and navigate to http://localhost:8080 with your browser to enjoy WordPress:

Secure protocols have a slightly more complicated configuration since they require to configure a KeyStore. Refer to the KeyStore section for more information about how to create and manage a KeyStore.

For simple cases, you only need to configure the KeyStore path and KeyStore password as explained in this section.

For more advanced configuration you may want to configure the TLS protocol versions, or the ciphers to include/exclude, etc. The correct way of doing this is to create a custom Jetty XML file and reference it in $JETTY_BASE/start.d/ssl.ini:

The ssl.ini file above only shows the lines that are not commented out (you can leave the lines that are commented unmodified for future reference).

You want to create the $JETTY_BASE/etc/tls-config.xml with the following template content:

The tls-config.xml file references the sslContextFactory component (created by the ssl Jetty module) that configures the KeyStore and TLS parameters, so that you can now call its APIs via XML, and you will have full flexibility for any advanced configuration you want (see below for few examples).

Refer to the SslContextFactory.Server javadocs for the list of methods that you can call through the Jetty XML file.

When you create a certificate, you must specify for how many days it is valid.

The typical validity is 90 days, and while this period may seem short, it has two advantages:

To renew a certificate, you must go through the same steps you followed to create the certificate the first time, and then you can reload the KeyStore without the need to stop Jetty.

Jetty can be configured to monitor the directory of the KeyStore file, and reload the SslContextFactory component if the KeyStore file changed.

This feature can be enabled by activating the ssl-reload Jetty module:

For more information about the configuration of the ssl-reload Jetty module, see this section.

If not explicitly configured, the TLS implementation is provided by the JDK you are using at runtime.

OpenJDK’s vendors may replace the default TLS provider with their own, but you can also explicitly configure an alternative TLS provider.

The standard TLS provider from OpenJDK is implemented in Java (no native code), and its performance is not optimal, both in CPU usage and memory usage.

A faster alternative, implemented natively, is Google’s Conscrypt, which is built on BoringSSL, which is Google’s fork of OpenSSL.

To use Conscrypt as the TLS provider just enable the conscrypt Jetty module:

Server Name Indication (SNI) is a TLS extension that clients send to indicate what domain they want to connect to during the initial TLS handshake.

Modern TLS clients (e.g. browsers) always send the SNI extension; however, older TLS clients may not send the SNI extension.

Being able to handle the SNI is important when you have virtual hosts and a KeyStore with multiple certificates, one for each domain.

For example, you may have deployed over a secure connector two web applications, both at context path /, one at virtual host one.com and one at virtual host two.net. The KeyStore contains two certificates, one for one.com and one for two.net.

There are three ssl module properties that control the SNI behavior on the server: one that works at the TLS level, and two that works at the HTTP level.

The property that works at the TLS level is:

When jetty.sslContext.sniRequired=true, clients that don’t send a valid SNI receive a TLS failure, and their attempt to connect to the server fails. The details of this failure may not be reported and could be difficult to figure out that the failure is related to an invalid SNI.

For this reason, other two properties are defined at the HTTP level, so that clients can received an HTTP 400 response with more details about what went wrong while trying to connect to the server:

When jetty.ssl.sniRequired=true, the SNI is matched against the certificate sent to the client, and only if there is a match the request is accepted.

When the request is accepted, there could be an additional check controlled by the following property:

In the normal case with the default server configuration, for a TLS clients that sends SNI, and then sends an HTTP request with the correct Host header, Jetty will pick the correct certificate from the KeyStore based on the SNI received from the client, and accept the request.

Accepting the request does not mean that the request is responded with an HTTP 200 OK, but just that the request passed successfully the SNI checks and will be processed by the server. If the request URI is for a resource that does not exist, the response will likely be a 404 Not Found.

You may modify the default values of the SNI properties if you want stricter control over old/broken TLS clients or bad HTTP requests.

The Forwarded HTTP header is added by the intermediary with information about the client and the client request, for example:

In the example above, the intermediary added the Forwarded header specifying that the client remote address is 2.36.72.144:21216 and that the request was made with the https scheme.

Let’s assume you have already configured Jetty with the HTTP/1.1 protocol with the following command (issued from within the $JETTY_BASE directory):

Support for the Forwarded HTTP header (and its predecessor X-Forwarded-* headers) is enabled with the http-forwarded Jetty module:

With the http-forwarded Jetty module enabled, Jetty interprets the Forwarded header and makes its information available to web applications via the standard Servlet APIs.

For further information about configuring the http-forwarded Jetty module, see this section.

The Proxy Protocol is the de facto standard, introduced by HAProxy, to communicate client information to backend servers via the TCP connection, rather than via HTTP headers.

The information about the client connection is sent as a small data frame on each newly established connection. This mechanism is therefore independent of any protocol, so it can be used for TLS, HTTP/1.1, HTTP/2, etc.

Support for the proxy protocol can be enabled for the clear-text connector or for the secure connector (or both).

Let’s assume you have already configured Jetty with the HTTP/1.1 clear-text protocol with the following command (issued from within the $JETTY_BASE directory):

To enable proxy protocol support for the clear-text connector, enable the proxy-protocol Jetty module:

Starting Jetty yields:

Note how in the example above the list of protocols for the clear-text connector is first proxy and then http/1.1. For every new TCP connection, Jetty first interprets the proxy protocol bytes with the client information; after this initial proxy protocol processing, Jetty interprets the incoming bytes as HTTP/1.1 bytes.

Enabling proxy protocol support for the secure connector is similar.

Let’s assume you have already configured Jetty with the HTTP/1.1 secure protocol and the test KeyStore with the following command (issued from within the $JETTY_BASE directory):

Enable the proxy-protocol-ssl Jetty module with the following command (issued from within the $JETTY_BASE directory):

Starting Jetty yields:

Note how in the example above the list of protocols for the secure connector is first proxy, then ssl and then http/1.1.

HAProxy is an open source solution that offers load balancing and proxying for TCP and HTTP based application, and can be used as a replacement for Apache or Nginx when these are used as reverse proxies.

The deployment proposed here has HAProxy playing the role that Apache and Nginx usually do: to perform the TLS offloading (that is, decrypt incoming bytes and encrypt outgoing bytes) and then forwarding the now clear-text traffic to a backend Jetty server, speaking either HTTP/1.1 or HTTP/2. Since HAProxy’s TLS offloading is based on OpenSSL, it is much more efficient than the Java implementation shipped with OpenJDK.

After you have installed HAProxy on your system, you want to configure it so that it can perform TLS offloading.

HAProxy will need a single file containing the X509 certificates and the private key, all in PEM format, with the following order:

Refer to the section about KeyStores for more information about generating the required certificates and private key.

Now you can create the HAProxy configuration file (in Linux it’s typically /etc/haproxy/haproxy.cfg). This is a minimal configuration:

On the Jetty side, you need to enable the following modules:

You need to specify the host (127.0.0.1) and port (8282) you have configured in HAProxy when you start Jetty:

With this configuration for HAProxy and Jetty, browsers supporting HTTP/2 will connect to HAProxy, which will decrypt the traffic and send it to Jetty. Likewise, HTTP/1.1 clients will connect to HAProxy, which will decrypt the traffic and send it to Jetty.

The Jetty connector, configured with the http2c and the http modules is able to distinguish whether the incoming bytes are HTTP/2 or HTTP/1.1 and will handle the request accordingly.

The response is relayed back to HAProxy, which will encrypt it and send it back to the remote client.

This configuration offers you efficient TLS offloading, HTTP/2 support and transparent fallback to HTTP/1.1 for clients that don’t support HTTP/1.1.