Faster Websites Using Asynchronous Processing With Tomcat 8

A New York Times article reports that people “will visit a website less often if it is slower than a close competitor by more than 250 milliseconds.” Depending on the specific task, a user may be willing to wait more or less. But in general, an improvement in a site’s response time could mean the difference between someone staying and someone leaving.

There are plenty of performance enhancements that can be tweaked and modified on a Tomcat server to enhance performance, but one major improvement for Tomcat 8 is that it supports non-blocking IO out of the box with use of a non-blocking IO (NIO) HTTP connector. This allows a server to be able to continue accepting new requests while still processing previous requests.

Synchronous Example

Let’s take a look at an example of a controller in Spring:

@RequestMapping("sync") public @ResponseBody String sync() {

// Simulate two seconds worth of work try { Thread.sleep(2000); } catch (InterruptedException e) { e.printStackTrace(); } return "sync"; }

To simply imitate a process that takes a while to respond, I made the thread sleep for two seconds. During slow hours, this controller might not be an issue. However during high volume, this controller will cause the server to freeze.

To test the performance of this controller, I created a JMeter test with 2,000 threads sending 10 consecutive requests each for a total of 20,000 requests without pauses. This test produced these abysmal results for the synchronous controller:

Average Response Time:       18.9 seconds

Minimum response time:               2 seconds

Maximum response time:        24.0 seconds

Throughput:                            98.3 requests/second

Optimally, each response should only take approximately two seconds. So how can we change this? It turns out it only takes a few simple changes.

Asynchronous Implementation

First, we need to add support for asynchronous processing. Add the following line to the web.xml inside the servlet tag:

true

Second, we need to change the Controller to return a Callable for Spring. Our example controller now looks like this:

@RequestMapping(value = "async") public @ResponseBody Callable async() {

// Simulate two seconds worth of work in another thread return new Callable() { public String call() throws Exception { try { Thread.sleep(2000); } catch (InterruptedException e) { e.printStackTrace(); } return"async"; } }; }

So how do these changes affect our previous results? Let’s run JMeter again. With the same test as before, we now get these results:

Average response Time:        2.7 seconds

Minimum response time:          2.3 seconds

Maximum response time:        6.5 seconds

Throughput:                            571.8 requests/second

These results show a decrease in response time of approximately 700% on average and more than five times higher throughput. The following graph compares the results of synchronous and asynchronous processing. For my server, a task that took even 100 milliseconds started to show improvements with the NIO connector.

Although this specific example is simple, there are plenty of scenarios where the use of an asynchronous task can be used to increase performance, such as external calls or long running calculations. Setting up a project to utilize Tomcat 8’s NIO is simple enough, and it might just save you that 250 milliseconds.

Reach out to us at findoutmore@credera.com if you'd like to learn more.

• Asynchronous Processing
• Tomcat 8