jSuneido - Server

I may have been a little premature when I said "writing a multi-threaded socket server seems a lot simpler in Java than in C++".

The examples I looked at that were so simple were for thread-per-connection. This is a reasonable model for low volume or for short connections (i.e. one request per connection).

But when you have persistent, intermittently active connections as Suneido does, then thread-per-connection doesn't scale very well. There are several problems - one is that each thread consumes resources so if you have a lot of inactive threads waiting for intermittent requests you tie up unnecessary resources. Another problem is that a thread that's been inactive will no longer be in the cpu caches and may even have been swapped out to disk, especially when resources are tight. This makes it slow to respond and can lead to thrashing.

One of the alternatives is the Reactor pattern. I was planning to use the ACE reactor implementation for the C++ version. The reactor model uses a smaller pool of threads that carry out requests, not tied to specific connections.

I could write my own reactor in Java, but it's tricky. The basic idea is simple, but getting the multi-threading right and getting good performance aren't so simple.

I may use Grizzly - a Java implementation of the reactor pattern, but it doesn't appear to support the Leader/Follower pattern.

The "obvious" approach to implementing a reactor is to have one "dispatcher" thread that listens for requests and a pool of worker threads to handle them. But this "forces" at least one context switch per request, not so good for "short" requests. With the Leader/Follower pattern there is no dispatcher - threads take turns accepting a request and then handling it. This avoids the dispatching context switch. If the operating system supports it, all the non-busy threads can simultaneously "wait" on all the connections and the OS will give an incoming request to one of them. If the OS doesn't support this, then one of the non-busy threads is chosen as the "Leader" and accepts the next request, and then another thread is promoted to "Leader".

Leader/Follower may not be that important for Suneido since most requests are large enough that the context switch overhead is probably insignificant.

There is also the Apache MINA network application framework. It seems a little "heavier" than what I need.