Peter Lawrey at IT Days
Of Java and low latency
Little’s Law defines concurrency as:
To increase throughput we can either:
- increase server resources (scaling vertically or horizontally)
- decrease latency (improve performance)
While enterprise applications are usually designed for scaling, trading systems focus on lowering latencies.
As surprising as it may sound, most trading systems, I’ve heard of, are actually written in Java. Java automatic memory management is a double-edged sword, since it trades simplicity for flexibility.
A low latency system worse nightmare is a stop-of-the-world process, like a garbage collector major collection. So in order to avoid such situations, you must go off-heap.
Java doesn’t directly offer off-heap memory management so you have to resort to unorthodox methods, like interacting with the sun.misc.Unsafe class.
As it’s name clearly suggests, you shouldn’t be using this class in the very first place. This class allows you to bypass many JVM safety mechanisms, so you can:
- allocate off-heap memory
- define classes without an actual ClassLoader
- reassign memory data, even constants
This class is not meant to be used by most Java professionals, being mostly useful for spoon bending libraries such as:
For low latency trading systems, you cannot use any off-the-shelf messaging system. To reach 20 millions per seconds you need a high performance queuing solution.
Java Chronicle is a much better alternative to using memory-mapped files with the risky sun.misc.Unsafe API. In short, Java Chronicle allows a producer process to write data to a shared memory location, only to be consumed by some other process.
So, Java Chronicle is more like a memory-mapped file utility than an actual messaging system. So if you plan on connecting two separate applications running in the same box, then you should definitely give a try to this low latency library.
While you should always strive for the least possible latency, an enterprise application requires a totally different queuing solution. An enterprise system is built of several applications that need to be inter-connected and these applications might reside on different nodes.
So an enterprise messaging system must be designed for networking in the very first place. Compared to trading systems latencies (microseconds), network latencies are much higher (milliseconds).
An enterprise queue must deal with both multiple producers as well as multiple consumers. Some systems use brokers, to acknowledge consumed messages (e.g. JMS), while other systems delegate message reading offsets to the consumer-side (Apache Kafka).
Apache Kafka is a publish-subscribe messaging solution based on a distributed commit log. So while Java Chronicle increases throughput by reducing latency, Kafka uses distributed scaling instead.