Real Story Group. Make Better Technology Decisions.

Kas Thomas

Benchmarking Java application servers

21-Feb-2008

Tags: Portals and Content Integration, Web Content and Experience Management, Industry Standards

Ever wondered how the major Java EE application servers compare with regard to basic session-handling performance? Jonathan Campbell at Java-net has published some interesting numbers. Campbell ran the jRealBench benchmarks against JBoss, Glassfish, WebLogic, and WebSphere Community Edition, to test how many sessions each could create per second.

The results (drum roll, please) show WebSphere CE to be the winner, although not by a huge margin. What's perhaps surprising is that BEA WebLogic finished last, by a decisive margin. (WebSphere CE 2.0 processed 50 percent more hits per second than WebLogic 10.) A Java 1.5 JVM was used, presumably Sun's. It would be interesting to see the tests repeated using BEA's own vaunted JRockit JVM, which might give WebLogic a more favorable showing. (Then again, maybe not. It could further reinforce the existing differences.)

As with all benchmarks, the jRealKit numbers should be viewed with caution. There's a vast difference between repeating a simple operation thousands of times under lab conditions, and handling real-world user requests under real-world conditions. Garbage collection and caching issues tend to moot benchmark numbers pretty quickly. Nevertheless, the test results posted by Campbell (like election exit-poll results) make for interesting reading.

I bring this up to you because in our experience, Web CMS and (especially) Enterprise Portal customers employing Java-based solutions tend to underestimate the the degree to which the underlying appserver can affect performance, and the extent of lower-level tuning required to get the system to work speedily enough. Plan accordingly, and consider your own benchmarks.