21

Let's imagine an hypothetical HFT system in Java, requiring (very) low-latency, with lots of short-lived small objects somewhat due to immutability (Scala?), thousands of connections per second, and an obscene number of messages passing around in an event-driven architecture (akka and amqp?).

For the experts out there, what would (hypothetically) be the best tuning for JVM 7? What type of code would make it happy? Would Scala and Akka be ready for this kind of systems?

Note: There has been some similar questions, like this one, but I've yet to find one covering Scala (which has its own idiosyncratic footprint in the JVM).

Community
  • 1
  • 1
Hugo Sereno Ferreira
  • 8,886
  • 6
  • 43
  • 89
  • 3
    Also the question arises if the JVM is the right choice? Maybe C++ would offer more predictable latency. – usr Mar 30 '12 at 23:18
  • I've heard about Scala used to produce C code doing actual HFT, but I cannot recall any details. As the 1-3 sec mentioned in the linked question is too much for HFT, I don't think it is a good idea to write HFT software on JVM. – Rafał Rawicki Mar 31 '12 at 10:17
  • 1
    The question is way to general. – moodywoody Apr 01 '12 at 03:29
  • I just don't understand why people try low-latency in a garbage-collecting language. Why isn't C more commonly used? –  Jun 23 '12 at 21:21
  • From my experience I can say that Scala is not good for high-performance with low latency, because of amount of magic going under the hood. Akka, also, is not the fastest thing, it is much slower than Disruptor. – Stas Aug 21 '12 at 10:55
  • 2
    You may have thousands of messages going in & out in a second, but you certainly won't have thousands of connections in an HFT system. – Ted Graham Feb 08 '13 at 21:39

3 Answers3

37

It is possible to achieve very good performance in Java. However the question needs to be more specific to provide a credible answer. Your main sources of latency will come from follow non-exhaustive list:

  1. How much garbage you create and the work of the GC to collect and promote it. Immutable designs in my experience do not fit well with low-latency. GC tuning needs to be a big focus.

  2. Warm up the JVM so that classes are loaded and the JIT has had time to do its work.

  3. Design your algorithms to be O(1) or at least O(log2 n), and have performance tests that assert this.

  4. Your design needs to be lock-free and follow the "Single Writer Principle".

  5. A significant effort needs to be put into understanding the whole stack and showing mechanical sympathy in its use.

  6. Design your algorithms and data structures to be cache friendly. Cache misses these days are the biggest cost. This is closely related to process affinity which if not set up correctly can result and significant cache pollution. This will involve sympathy for the OS and even some JNI code in some cases.

  7. Ensure you have sufficient cores so that any thread that needs to run has a core available without having to wait.

I recently blogged about a case study of such an exercise.

Martin Thompson
  • 371
  • 2
  • 2
26

On my laptop the average latency of ping messages between Akka 2.3.7 actors is ~300ns and it is much less than the latency expected due to GC pauses on JVMs.

Code (incl. JVM options) & test results for Akka and other actors on Intel Core i7-2640M here.

P.S. You can find lots of principles and tips for low-latency computing on Dmitry Vyukov's site and in Martin Thompson's blog.

Andriy Plokhotnyuk
  • 7,705
  • 2
  • 42
  • 66
  • 2
    FYI, the improvement is now resolved and part of the JDK 7 newer releases. – Sanjay T. Sharma Sep 01 '13 at 09:25
  • Note: average latency is really just an inverse of the throughput. You want to know the distribution of latencies based on when the messages should have been sent instead of when they were sent. i.e. you want to avoid co-ordinate omission. http://www.azulsystems.com/sites/default/files/images/HowNotToMeasureLatency_LLSummit_NYC_12Nov2013.pdf – Peter Lawrey Mar 06 '14 at 11:18
  • I removed reference to throughput test on typesafe blog to avoid misinterpretation that 1/latency = throughput. – Andriy Plokhotnyuk Mar 10 '14 at 15:01
11

You may find that use of a ring buffer for message passing will surpass what can be done with Akka. The main ring buffer implementation that people use on the JVM for financial applications is one called Disruptor which is carefully tuned for efficiency (power of two size), for the JVM (no GC, no locks) and for modern CPUs (no false sharing of cache lines).

Here is an intro presentation from a Scala point of view http://scala-phase.org/talks/jamie-allen-sdisruptor/index.html#1 and there are links on the last slide to the original LMAX stuff.

Michael Dillon
  • 31,190
  • 5
  • 66
  • 104