Jetty Server

The Jetty Server object is the central component that links protocol connectors to web applications.

The Server component is defined by the server Jetty module, that in turn depends on other Jetty modules that provide key functionalities, in particular:

Logging

You can configure two types of logging in Jetty: server logging and request logging.

  • Server logging refers to the console output produced by Jetty itself.

  • Request logging refers to the information that Jetty can capture about incoming HTTP requests and responses.

Server Logging

Jetty uses the SLF4J API for its logging. SLF4J is a generic abstraction layer that is supported by many different logging frameworks (or SLF4J bindings).

Jetty provides a default binding via the jetty-slf4j-impl Maven artifact, but you can plug in the SLF4J binding of your choice provided by other logging frameworks.

Jetty’s server logging is enabled by default with the logging Jetty module. You typically won’t have to enable the logging module directly, since it is a transitive dependency of the server module, and thus pulled in by many of the most commonly used modules.

The logging module is a virtual module and its default implementation is provided by the logging-jetty Jetty module, which uses the Jetty SLF4J binding.

Default Configuration

Jetty’s default SLF4J binding uses an appender (org.eclipse.jetty.logging.StdErrAppender) to format a logging message with metadata (like a timestamp) before sending it to System.err. The default StdErrAppender format is:

<datetime>:<level>:<logger name>:<thread name>:<message>

where <datetime> is a timestamp with the format yyyy-MM-dd HH:mm:ss.SSS.

You can configure the appender via a file named jetty-logging.properties, which must be found in the server class-path. When you enable the logging-jetty module — either directly or by transitive dependency, as in the following example — Jetty automatically generates a jetty-logging.properties file in $JETTY_BASE/resources/:

$ java -jar $JETTY_HOME/start.jar --add-modules=http
INFO  : mkdir ${jetty.base}/start.d
INFO  : server          transitively enabled, ini template available with --add-modules=server
INFO  : logging-jetty   transitively enabled
INFO  : http            initialized in ${jetty.base}/start.d/http.ini
INFO  : resources       transitively enabled
INFO  : threadpool      transitively enabled, ini template available with --add-modules=threadpool
INFO  : logging/slf4j   dynamic dependency of logging-jetty
INFO  : bytebufferpool  transitively enabled, ini template available with --add-modules=bytebufferpool
INFO  : mkdir ${jetty.base}/resources
INFO  : copy ${jetty.home}/modules/logging/jetty/resources/jetty-logging.properties to ${jetty.base}/resources/jetty-logging.properties
INFO  : Base directory was modified

You can specify the following configuration options in jetty-logging.properties:

org.eclipse.jetty.LEVEL=<logging level>

Sets the logging level for the logger tree based at org.eclipse.jetty. You can specify any of the usual SLF4J logging levels — TRACE, DEBUG, INFO (default), WARN and ERROR — plus two additional levels: ALL (an alias for TRACE) and OFF (disables logging entirely). You can also configure a default logging level for specific loggers, or arbitrary logger trees:

  • com.example.MyComponent.LEVEL=DEBUG (sets logging level of logger com.example.MyComponent to DEBUG)

  • com.example.LEVEL=DEBUG (sets logging level of tree com.example.* to DEBUG)

com.example.STACKS=<boolean>

Specifies whether to hide stack traces for some arbitrary logger tree com.example.*. The exception type and message are logged normally; only stack traces are hidden. Default value is false

org.eclipse.jetty.logging.appender.NAME_CONDENSE=<boolean>

Specifies whether to condense logger names, so that for example org.eclipse.jetty.util.QueuedThreadPool becomes oeju.QueuedThreadPool. Default value is true.

org.eclipse.jetty.logging.appender.MESSAGE_ALIGN=<integer>

Specifies the column at which the logging <message> should be printed. The value 0 specifies no alignment. Default value is 0.

org.eclipse.jetty.logging.appender.MESSAGE_ESCAPE=<boolean>

Specifies whether to escape ISO control characters such as \r or \n present in the message. Character \r is replaced with < and character \n is replaced with |; all other ISO control characters are replaced with ?. Default value is false.

org.eclipse.jetty.logging.appender.ZONE_ID=<timezone id>

Specifies the timezone ID (such as PST, or America/Los_Angeles or GMT-8:00) for the <datetime> part of the logging line. The empty string specifies the UTC timezone. Default value is the local timezone.

When using the Jetty SLF4J binding, the logging levels can be dynamically changed via JMX, see the troubleshooting section for more information.

Capturing Logs to a Rolling File

Logging to System.err may be fine at development time, but you will typically want to capture logs on disk for later inspection, or if you don’t have a terminal access (for example, if you started Jetty as a service).

The console-capture Jetty module allows you to capture what is written to System.out and System.err and write it to a log file. By default, console-capture logs to a file in the $JETTY_BASE/logs/ directory.

See the console-capture module documentation for details on configuring how logs are written to the log directory.

The console-capture Jetty module should be used only in conjunction with the logging-jetty module, as other SLF4J bindings such as LogBack or Log4j2 have their own, more sophisticated, rolling file appenders.

Custom Configuration

You can use a different SLF4J binding if you are more familiar with other logging libraries, or if you need custom logging appenders. There are a number of out-of-the-box Jetty modules that you can use:

  • logging-logback, to use the LogBack binding

  • logging-log4j2, to use the Log4j2 binding

  • logging-log4j1, to use the Log4j1 binding (note that Log4j 1.x is end-of-life)

  • logging-jul, to use the java.util.logging binding

  • logging-noop, to use the SLF4J no-operation binding (discards all logging)

Logging with LogBack

To enable the logging-logback module, run:

$ java -jar $JETTY_HOME/start.jar --add-modules=logging-logback,http

Since LogBack is released under a license that is different from Jetty’s, you will be prompted to accept the LogBack license. Once you accept the LogBack license, your $JETTY_BASE directory will have the following structure.

$JETTY_BASE
├── lib
│   └── logging
│       ├── logback-classic-<version>.jar
│       └── logback-core-<version>.jar
├── resources
│   └── logback.xml
└── start.d
    ├── http.ini
    └── logging-logback.ini

Jetty downloaded the required LogBack *.jar files, and created a $JETTY_BASE/resources/logback.xml file for configuring your LogBack logging. Please refer to the LogBack configuration manual for more information about how to configure LogBack.

Logging with Log4j2

To enable the logging-log4j2 module, run:

$ java -jar $JETTY_HOME/start.jar --add-modules=logging-log4j2,http

After accepting the Log4j2 license, you will have the following directory structure:

$JETTY_BASE
├── lib
│   └── logging
│       ├── log4j-api-<version>.jar
│       ├── log4j-core-<version>.jar
│       └── log4j-slf4j2-impl-<version>.jar
├── resources
│   └── log4j2.xml
└── start.d
    ├── http.ini
    └── logging-log4j2.ini

Jetty downloaded the required Log4j2 *.jar files, and created a $JETTY_BASE/resources/log4j2.xml file that you can configure to customize your Log4j2 logging.

Please refer to the Log4j2 configuration manual for more information about how to configure Log4j2.

Bridging Logging to SLF4J

When you use libraries that provide the features you need (for example, JDBC drivers), it may be possible that those libraries use a different logging framework than SLF4J.

SLF4J provides bridges for legacy logging APIs that allows you to bridge logging from one of these legacy logging frameworks to SLF4J. Once the logging is bridged to SLF4J, you can use Jetty’s default configuration or a custom configuration so that your logging is centralized in one place.

Jetty provides the logging-jul-capture module for bridging from java.util.logging to SLF4J.

The modules logging-jcl-capture and logging-log4j1-capture similarly provide bridges from Jakarta Commons Logging (JCL) and Apache Log4j, respectively; however, these modules are obsolete and should not be used anymore.

Bridging from java.util.logging

For libraries that use java.util.logging as their logging framework, you can enable Jetty’s logging-jul-capture module:

$ java -jar $JETTY_HOME/start.jar --add-modules=logging-jul-capture

The logging-jul-capture module implies --exec and therefore spawns a second JVM because it needs to provide the system property java.util.logging.config.file (so that java.util.logging can read the configuration from the specified file), and because it needs to make available on the System ClassLoader the class org.slf4j.bridge.SLF4JBridgeHandler.

For example, a library that uses java.util.logging as its logging library is the Postgresql JDBC driver. With the logging-jul-capture Jetty module, the logging follows this diagram:

Diagram

Note how Jetty logs directly to SLF4J, while the Postgresql JDBC driver logs to SLF4J through the SLF4JBridgeHandler. They both arrive to the SLF4J binding, in this case the Jetty SLF4J binding (but could be any other SLF4J binding such as LogBack).

Request Logging

HTTP requests and responses can be logged to provide data that can be later analyzed with other tools, that can provide information such as the most frequently accessed request URIs, the response status codes, the request/response content lengths, geographical information about the clients, etc.

Request logging is enabled by enabling the requestlog Jetty module. In the example below, both the http Jetty module and the requestlog module are enabled, so that you can make HTTP requests to the server and have them logged:

$ cd $JETTY_BASE
$ java -jar $JETTY_HOME/start.jar --add-modules=http,requestlog

The $JETTY_BASE directory looks like this:

$JETTY_BASE
├── logs
├── resources
│   └── jetty-logging.properties
└── start.d
    ├── http.ini
    └── requestlog.ini

The $JETTY_BASE/start.d/requestlog.ini file is the Jetty module configuration file that allows you to configure the requestlog module, see this section for more details.

By default the requestlog Jetty module produces the $JETTY_BASE/logs/yyyy_MM_dd.request.log, where the pattern yyyy_MM_dd is replaced with the current date, for example 2020_01_31.

The format of the request log lines is the result of a format string that uses formatting symbols to log relevant request/response data.

The default format is the NCSA Format extended with referrer data and user-agent data. A typical log line looks like this:

192.168.0.100 - - [31/Jan/2020:20:30:40 +0000] "GET / HTTP/1.1" 200 6789 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36"

The line above (that uses fake values) shows 192.168.0.100 for the client IP address, a hard-coded - for the identity, - for the authenticated user name, [31/Jan/2020:20:30:40 +0000] for the date and time with timezone, "GET / HTTP/1.1" for the HTTP request line, 200 for the HTTP response status code, 6789 for the HTTP response content length, "-" for the referrer and "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/97.0.4692.71 Safari/537.36" for the user-agent.

The format string can be customized as described in this section. Request log files are rolled every day, and retained for customizable number of days, by default 90 days.

When Jetty is behind a load balancer, you want to log the remote client IP address, not the load balancer IP address. Refer to this section to configure the load balancer and Jetty to retain the remote client IP address information.

Thread Pooling

Jetty uses thread pooling to efficiently execute tasks that provide Jetty functionalities.

Like any other component, the Jetty thread pool is configured and enabled via the threadpool Jetty module, that is transitively enabled by the server Jetty module which, in turn, is transitively enabled by a protocol module such as the http Jetty module:

$ java -jar $JETTY_HOME/start.jar --add-modules=http

The command above gives you the default configuration for the thread pool.

If you want to explicitly configure the thread pool, it is enough to explicitly specify the threadpool module:

$ java -jar $JETTY_HOME/start.jar --add-modules=threadpool,http

After the command above, the $JETTY_BASE directory looks like this:

$JETTY_BASE
├── resources
│   └── jetty-logging.properties
└── start.d
    ├── http.ini
    └── threadpool.ini

Now you can customize the threadpool.ini file to explicitly configure the thread pool.

Virtual Threads Support

Virtual threads have been introduced as a preview feature in Java 19 and Java 20, and have become an official feature since Java 21.

The threadpool-virtual-preview Jetty module provides support for virtual threads in Java 19 and Java 20, and it is mutually exclusive with the threadpool Jetty module.

The threadpool-virtual Jetty module provides support for virtual threads in Java 21 or later, and it is mutually exclusive with the threadpool Jetty module.

If you have already enabled the threadpool Jetty module, it is sufficient to remove it by removing the $JETTY_BASE/start.d/threadpool.ini file.

When using Java 21 or later, you can enable the threadpool-virtual module:

$ java -jar $JETTY_HOME/start.jar --add-modules=threadpool-virtual,http

After the command above, the $JETTY_BASE directory looks like this:

$JETTY_BASE
├── resources
│   └── jetty-logging.properties
└── start.d
    ├── http.ini
    └── threadpool-virtual.ini

Now you can customize the threadpool-virtual.ini file to explicitly configure the thread pool and the virtual threads and then start Jetty:

2024-06-21 15:33:15.012:INFO :oe.jetty:main: Virtual threads enabled. Using virtual threads for application tasks.
2024-06-21 15:33:15.447:INFO :oejs.Server:main: jetty-12.0.11-SNAPSHOT; built: 2024-06-21T15:12:19.556Z; git: 8a0c4da385682b6620a97d005826658e2c8aaf60; jvm 22.0.1+8
2024-06-21 15:33:15.498:INFO :oejs.AbstractConnector:main: Started ServerConnector@1e800aaa{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
2024-06-21 15:33:15.524:INFO :oejs.Server:main: Started oejs.Server@6adbc9d{STARTING}[12.0.11-SNAPSHOT,sto=5000] @1823ms