I’ll be giving some talks over the next few months: 8 Oct: Auckland Software Craftsmanship - 6 Years of test automation. 16 Oct: Auckland JVM Group - Stuff I learned about performance. 5 Nov: QCon San Francisco 2014 - Stuff I learned about performance.

I’ve decided that I’m a bit bored of the whole putting a beta tag on various versions of the Disruptor so I’ve decide to send forth Disruptor 3.0.0 into the world. The big challenges of this release were to clean up the code and come up with a better algorithm for handling multiple producers. If I was lucky, make it even faster. I went down a couple of dark alleys initially with this release, but have come up again for air with a version that is less different to the 2.

I’m happy to announce that I will speaking at Tech Mesh in December. I’ll be speaking about the Disruptor from two perspectives, firstly looking briefly back at some of the history and motivations behind the Disruptor. Then spending some time explaining at the challenges of building high performance concurrent systems (like the Disruptor) and delving into how the JVM and hardware could change to support the development of these systems.

Last week I gave a talk on non-blocking concurrency at JAX London. Here are the slides:

During a recent conversation on the LJC mailing list around a proposal for adding a library to the JDK that would add support for handling integer overflow a question arose. Would the JVM be able to optimise this code to make efficient use of the hardware support for overflow detection if this functionality was implemented as a library. I made the comment that this is problem is probably one best solved using intrinsics, but in the course of writing an explanation I thought it would be better explained in a blog post, so here goes…

I’ve be working sporadically on the next major revision of the Disruptor, but still making steady progress. I’ve merged my experimental branch into the main line and I’m working on ensure comprehensive test coverage and re-implement the Disruptor DSL. As a matter of course I’ve been running performance tests to ensure that we don’t regress performance. While I’ve not been focusing on performance, just some refactoring and simplification I got a nice surprise. The new version is over twice as fast for the 1P1C simple test case; approximately 85M ops/sec versus 35M ops/sec.

The Disruptor version 2.10 has been released. It is available from the Google Code download page and has been submitted to Maven central. Changes Remove deprecated timeout methods. Added OSGI metadata to jar file. Removed PaddedAtomicLong and use Sequence in all places. Fix various generics warnings. Change Sequence implementation to work around IBM JDK bug and improve performance by ~10%. Add a remainingCapacity() call to the Sequencer class.

Last month the guys from the Distributed Podcast interviewed me about LMAX and the Disruptor. Many thanks to Jonathan and Rinat, I had a great time.

Along with Martin Thompson, I’m speaking about non-blocking concurrency at QCon London in March.

My slides from JAX London - Beginner’s Guide to Hardcore Concurrency: Beginners guide-concurrency View more presentations from Michael Barker Video: LJC@Playfish, JAX London My slides from Devoxx - Disruptor Tools In Action: Disruptor tools in action View more presentations from Michael Barker Video: Devoxx (Payment required)

Within a project I’ve been working on I’ve had the need to simulate the capabilities of Linux’s /proc/cpuinfo on Mac OS. Specifically I needed to build a topology of the CPUs on a given system. I.e. I need to map the operating system’s processors to hardware threads, then build a picture of which cores and sockets those threads reside. For example my Mac looks something like: CPU0 (Thread 0) ---+ |---> Core 0 ---+ CPU1 (Thread 1) ---+ | | ----> Socket 0 CPU2 (Thread 2) ---+ | |---> Core 1 ---+ CPU3 (Thread 3) ---+ While this sounds very simple, it’s actually fraught with a number of little niggles.

A few weeks ago one of the users of the Disruptor posted some worrying benchmarks: ThreePublisherToOneProcessorSequencedThroughputTest run 0: BlockingQueue=645,161 Disruptor=1,772 ops/sec run 1: BlockingQueue=1,250,000 Disruptor=20,000,000 ops/sec run 2: BlockingQueue=1,250,000 Disruptor=56 ops/sec It appears under heavy contention with fewer available cores than busy threads the Disruptor can perform terribly. After a bit of investigation I managed to isolate the problem. One of the most complex parts of the Disruptor is the multi-threaded claim strategy. It is the only place in the Disruptor where - out of necessity - we break the single-writer principal.

Firstly an apology. On my previous blog, I mentioned that a string splitting algorithm implemented in Scala had a complexity of O(n2)O(n^2). One commenter mentioned that they did not understand how I came to that calculation. I though I should revisit my guess work and actually do a more thorough analysis. What I found was interesting, I had overstated the complexity, in reality it was O(n.log2(n))O(n.log_{2}(n)). I’ve included my working here. Complexity of the Scala Implementation

Introducing ConcurrentHashTrie While spending some time looking at Clojure’s concurrency model, I did a little bit of research into their persistent collection implementation that uses Bagwell’s Hash Array Mapped Tries. After a little bit of thought it occurred to me that you could apply the persistent collection model to an implementation of ConcurrentMap. The simple solution would be to maintain a reference to a persistent map within a single AtomicReference and apply a compareAndSet (CAS) operation each time an update to the map was performed.

Being someone who’s interested in Open Source and very focused on performance, I’ve been looking at the Java & the JVM and wondering about what I would most like to see added. IMHO one of the fundamental missing pieces is support for complex stack allocated types. This is not a new idea. C# calls them structs. In C, C++ and Go stack & heap allocation is orthogonal to the definition of the type. An implementation for Java is described by John Rose (some time ago) on his blog.

The Disruptor, the design pattern at the core of the financial exchange I’m helping build at LMAX has been open sourced. If you’ve seen the presentation that Martin Thompson and I gave at QCon or the one I did for the LJC @Skillsmatter, this is the ring-buffer based code that we’ve been banging on about for a while. What is the Disruptor? At its simplest, it’s an alternative to a queue, but can also can be thought of as a simple actor-like concurrency framework.

InfoQ have published the video of Martin and my talk from QCon. A couple of quick apologies, I haven’t done many videoed talks and I completely forgot the repeat the questions during the Q&A session. Martin and I were alternating between each of the slides, so watching the video might make you a little bit sea-sick.

Slides from the talk that Martin Thompson and I gave at QCon San Francisco are now available on-line.