Server Threading Architecture

As explained in the general Jetty threading architecture, web applications can specify the InvocationType of the Jetty implementation tasks that invoke web application code.

There are two main Jetty implementation tasks that invoke web application code:

  • The task that invokes the Handler tree, detailed here.

  • The task that invokes the Request demand callback, detailed here.

Handler Tree InvocationType

The InvocationType of the task that invokes the Handler tree is determined by the structure of the Handler tree, and defines the InvocationType with which HTTP request are processed.

As explained in this section, the Handler tree starts at the Server instance.

In a simple Handler tree, the Server wraps a ContextHandler instance, that wraps an application code Handler, for example:

class ShopHandler extends Handler.Abstract
{
    public ShopHandler()
    {
        // Specifies that the handle() implementation is non-blocking.
        super(InvocationType.NON_BLOCKING); (1)
    }

    @Override
    public boolean handle(Request request, Response response, Callback callback)
    {
        // Implement the shop application.
        callback.succeeded();
        return true;
    }
}

Server server = new Server();

ContextHandler contextHandler = new ContextHandler("/shop");
server.setHandler(contextHandler);

contextHandler.setHandler(new ShopHandler());
1 Specifies that the ShopHandler.handle(...) implementation is NON_BLOCKING.

The correspondent structure is the following:

Server
└── ContextHandler /shop
    └── ShopHandler

Both Server and ContextHandler extend from Handler.Wrapper, which reports the same InvocationType as its child Handler. ShopHandler specifies its own InvocationType to be NON_BLOCKING, so the whole Handler tree is NON_BLOCKING.

A more complex Handler tree is one that uses ContextHandlerCollection, for example:

Server
└── ContextHandlerCollection
    ├── ContextHandler /shop
    │   └── ShopHandler
    └── ContextHandler /api
        └── RESTHandler

Another more complex Handler tree is one that uses PathMappingsHandler, for example:

Server
└── ContextHandler /shop
    └── PathMappingsHandler
        ├── CatalogHandler  /catalog/*
        └── CheckoutHandler /checkout/*

Both ContextHandlerCollection and PathMappingsHandler extend from Handler.Container that calculates the InvocationType combining the InvocationTypes of all the children Handlers (see Invocable.combine(InvocationType, InvocationType)).

When all the children Handlers are NON_BLOCKING, then the Handler.Container is also NON_BLOCKING. Otherwise, if one of the children is BLOCKING, then the Handler.Container is also BLOCKING.

WebSocketUpgradeHandler InvocationType

WebSocketUpgradeHandler is a special case of Handler.Wrapper because it combines the InvocationType of the child Handler (that processes HTTP requests) with the InvocationType of the ServerWebSocketContainer (that processes WebSocket messages).

WebSocket applications should set the InvocationType of the ServerWebSocketContainer to match whether WebSocket EndPoints have been implemented in a blocking or non-blocking way:

Server server = new Server();

ContextHandler contextHandler = new ContextHandler("/ws");
server.setHandler(contextHandler);

// The InvocationType for both WebSocket EndPoints and HTTP requests.
Invocable.InvocationType invocationType = Invocable.InvocationType.NON_BLOCKING;

WebSocketUpgradeHandler wsHandler = WebSocketUpgradeHandler.from(server, contextHandler,
    wsContainer -> wsContainer.setInvocationType(invocationType)); (1)

contextHandler.setHandler(wsHandler);

wsHandler.setHandler(new Handler.Abstract(invocationType) (2)
{
    @Override
    public boolean handle(Request request, Response response, Callback callback)
    {
        // Handle HTTP requests here.
        callback.succeeded();
        return true;
    }
});
1 Setting the InvocationType for the ServerWebSocketContainer.
2 Setting the InvocationType for the leaf HTTP Handler.

In the example above, the Handler tree is the following:

Server
└── ContextHandler /ws
    └── WebSocketUpgradeHandler
        └── Handler.Abstract

HTTP requests that flow through the WebSocketUpgradeHandler are influenced by the InvocationType of the WebSocketUpgradeHandler, even if WebSocketUpgradeHandler just forwards them to its child Handler.

Request Demand InvocationType

The InvocationType of the task that invokes the Request demand callback is determined from the demand callback object passed to Request.demand(Runnable), and defines the InvocationType with which the demand callback is invoked.

Rather than passing a simple Runnable or a lambda expression (for which the InvocationType is assumed to be BLOCKING), you can pass an Invocable.Task, that implements both Invocable (to specify the InvocationType) and Runnable (to specify the demand callback).

For example:

class NonBlockingHandler extends Handler.Abstract
{
    public NonBlockingHandler()
    {
        super(InvocationType.NON_BLOCKING); (1)
    }

    @Override
    public boolean handle(Request request, Response response, Callback callback) throws Exception
    {
        // Read the request content in non-blocking way.
        request.demand(new Invocable.Task.Abstract(InvocationType.NON_BLOCKING) (2)
        {
            @Override
            public void run()
            {
                while (true)
                {
                    Content.Chunk chunk = request.read();

                    if (chunk == null)
                    {
                        request.demand(this); (2)
                        return;
                    }

                    if (Content.Chunk.isFailure(chunk))
                    {
                        callback.failed(chunk.getFailure());
                        return;
                    }

                    // Process the Chunk in non-blocking way.
                    processNonBlocking(chunk);

                    chunk.release();

                    if (chunk.isLast())
                    {
                        callback.succeeded();
                        return;
                    }
                }
            }
        });
        return true;
    }
}
1 Specifies the Handler's InvocationType for HTTP request processing.
2 Specifies the demand callback to be an Invocable.Task with InvocationType NON_BLOCKING.