Posts

Data Access Object (DAO) is a commonly used pattern to persist domain objects into a database. The most common form of a DAO pattern is a class that contains CRUD methods for a particular domain entity type. Assumes that I have a domain entity class “Account”: package com.thinkinginobjects.domainobject; public class Account { private String userName; private String firstName; private String lastName; private String email; private int age; public boolean hasUseName(String desiredUserName) { return this.userName.equals(desiredUserName); } public boolean ageBetween(int minAge, int maxAge) { return age >= minAge && age <= maxAge; } } Follow the common DAO approach, I create a DAO interface:

Farnam Street Blog holds up a coding competition as evidence that: If you want to make your computer programmers and engineers more effective give them “privacy, personal space, control over their physical environments, and freedom from interruption.” The only trouble is, the coding competition is fatally flawed as a measure of normal developer productivity. It’s setup such that developers work on their own: Each participant was also assigned a partner from the same company.

Sarah Goff-Dupont has a post on the Atlassian blog about how Atlassian is using feature branches and still doing continuous integration: In the not-so-recent past, continuous integration in the context of story branching was considered so impractical as to be outright incompatible. Who has time to manually configure a CI scheme for dozens of active branches that will only live a couple of days? Nobody –that’s who. And story branch-loving teams would frequently encounter *ahem* “surprises” when merging work onto the main code line –“master”, in Git-speak.

I’ve be working sporadically on the next major revision of the Disruptor, but still making steady progress. I’ve merged my experimental branch into the main line and I’m working on ensure comprehensive test coverage and re-implement the Disruptor DSL. As a matter of course I’ve been running performance tests to ensure that we don’t regress performance. While I’ve not been focusing on performance, just some refactoring and simplification I got a nice surprise. The new version is over twice as fast for the 1P1C simple test case; approximately 85M ops/sec versus 35M ops/sec.

Recently, I got myself a VStarCam IP camera, model H6837WI, relatively cheap for what you get - a H264 capable, wireless/wired IP camera with two way audio, SD card recording and few other nice features. The software provided with the camera is Windows only, which is a system I don’t use very often at home ;) …so I started with exploring the camera’s web interface, by default run on port 81. It turned out that the quicktime plugin didn’t seem to work in any browser and other than getting the direct H264 stream, this was the only way to get live video feed without the proprietary software.

Tinola was off the air for a couple of days due to a double fault. By which I mean I lost the primary and backup systems in the same powercut. Arguably they should have been in different datacentres. Nonetheless its unusual to get that unlucky, and now I’m considering how to move forward on the hosting front. The new server is smaller than the old one, and after restoring from backup, getting Zimbra up and running required some manual tweaks.

While I was waiting for my Raspberry Pi to arrive, I decided to put my Nokia N900 for a better use and create a Gentoo chroot on the SD card to give the hardened ARM toolchain a go. For the record, I found these two links to be particularly useful when working on the chroot. N900 is not the fastest arm board out there, but it was the only ARM board I had at hand…Anyway, creating Gentoo chroot on N900 is quite simple actually.

There’s a huge amount of complaining and problem solving for various dependency management solutions (particularly maven, but ivy and friends certainly aren’t immune). Problems range from having optional dependencies unnecessarily included to having duplicate classes or class conflicts and the solutions tend to be complex and involve a lot of trade offs. All these problems stem from breaking the golden rule of dependency management: Own your own repository – Sutton’s golden rule of dependency management

I have finally got my hands on the awesome Raspberry Pi board with a vicious plan of running a hardened Gentoo on it of course ;] But before that could happen, I had to get a decent SD card for it, which turned out to be not that obvious. There’s a wiki page with a list of SD cards that should and shouldn’t work with your Raspberry. There’s also a discussion thread on the Raspberry Pi forum about performance of various cards, which is vital to the overall performance of the system.

Having configuration differences between development and production is largely unavoidable, but it’s important to keep the number of differences to a minimum to avoid unexpected bugs appearing in production that don’t occur in development mode. More than that though, it’s important to minimise configuration. Configuration is Code Often things are put in configuration files so that they are easy to change later, often without a full redeployment of the software, but that also implies it can be changed without going through the full testing that a release would usually be subject to.

Good news! The Firefox and Thunderbird ebuilds in the portage tree disable JIT by default, using the two configuration options I’ve posted about before. Instead of using the pax_kernel USE flag, they incorporate the jit flag, which is by default disabled on the hardened profile. So, to make the long story short - if you have selected the hardened profile, your Firefox and Thunderbird will work without use of RWX memory pages and with correctly enforced mprotect() restrictions…by default!

It is hard to let things go sometimes. Porting Linux to the VAX architecture taught me a lot about the detail of how Linux works, as well as how computer hardware and operating systems interact. It both removed the air of mystery about the kernel, and opened my eyes to the complexity of the GCC compiler and toolchain. The VAX architecture is simple and easy to understand for a beginner - so much so, that when I started there were several good computer science text books which used it as a teaching system/example.

Most systems have multiple configuration profiles – one for production, one for development and often other profiles for staging, testing etc. Minimising differences between these configurations is critical but there are inevitably some things that just have to be different. This then leaves the question, what should the default settings be? There are three main options: Default to production values Default to development values Refuse to start unless an environment has been explicitly chosen My preference is to default to development values.

There is a lot of discussion at the moment about the Oracle v Google trial today, mostly centring around how impossible it is for the jury to understand the very technical concepts involved in the case. As Daring Fireball puts it: How could a randomly-selected jury possibly decide this? No knock intended against the jurors themselves — and it sounds like they’re doing their best to make an informed decision. But there’s a difference between a jury of your citizen peers and a jury of your technical peers.

I thus think (= hope) that it’s a mistake to extrapolate from today’s crappy input systems on tablets to a future of tablet-based couch potatoes still watching Hollywood crap. We’re one innovation away from lowering the creativity hurdle on tablets. Maybe it’ll be a truly responsive keyboard. Or something that translates sub-vocalizations into text (because I’m too embarrassed to dictate into my table while in public places). Or, well, something. via Joho the Blog » Will tablets always make us non-social consumers?

The Disruptor version 2.10 has been released. It is available from the Google Code download page and has been submitted to Maven central. Changes Remove deprecated timeout methods. Added OSGI metadata to jar file. Removed PaddedAtomicLong and use Sequence in all places. Fix various generics warnings. Change Sequence implementation to work around IBM JDK bug and improve performance by ~10%. Add a remainingCapacity() call to the Sequencer class.

We all know by now that continuous integration is part of good software development – check in regularly and have a suite of automated tests run to confirm that everything is working as expected. If a test fails, jump on it quickly and get the build back to green. Simple right? But what happens when something goes wrong in a way that can’t be fixed quickly? For example, the build server has a hardware fault or runs out of disk space.

Last month the guys from the Distributed Podcast interviewed me about LMAX and the Disruptor. Many thanks to Jonathan and Rinat, I had a great time.

It’s been a while and Firefox has moved from version 5 to version 10.0.1, now that’s a pace! ;) But the important bits are…enforcing MPROTECT has never been easier…well, almost. ;) Thanks to this attachment in this bug, the latest version of Firefox compiles fine on hardened profiles (or simply on grsec kernels). In order to enable MPROTECT restrictions, edit the ebuild and at the top add pax_kernel flag to IUSE so it reads like this:

Along with Martin Thompson, I’m speaking about non-blocking concurrency at QCon London in March.

My slides from JAX London - Beginner’s Guide to Hardcore Concurrency: Beginners guide-concurrency View more presentations from Michael Barker Video: LJC@Playfish, JAX London My slides from Devoxx - Disruptor Tools In Action: Disruptor tools in action View more presentations from Michael Barker Video: Devoxx (Payment required)

Some aspects of linux have the reputation of being hard. Traffic control via queueing disciplines for bandwidth management for example. Even the title is enough to strike fear into the heart of a seasoned system admin. Which is a pity really, as the things outlined in chapter 9 of the lartc are very useful in practise. The problem is the documentation is very descriptive - which is good once you know roughly what you’re doing - but which has quite a steep learning curve if you don’t.

My company launched their new website recently. When we launched before Christmas we encountered a reoccurring problem that was more difficult than most to diagnose. The problem itself is very specific to our site so I doubt the exact details will help many people, but maybe the troubleshooting steps involved will prove interesting to someone. I’m not particularly proud of the time it took to track down nor our exact thought process (hardly blowing my own horn with this post) but here we go anyway.

We had to clean up some left over cgroups after another set of experiments with LXC.The guys doing it encountered problems, as the logic is the opposite of what you expect from your experience on a normal unix filesystem. Specifically the problem happens when you have a nested cgroup - for example /cgroup/foo/bar/ The problem happens because this is a virtual filesystem with its own rules. If you create a cgroup (in this case we’ll just mount the net_cls cgroup because its smaller)

Within a project I’ve been working on I’ve had the need to simulate the capabilities of Linux’s /proc/cpuinfo on Mac OS. Specifically I needed to build a topology of the CPUs on a given system. I.e. I need to map the operating system’s processors to hardware threads, then build a picture of which cores and sockets those threads reside. For example my Mac looks something like: CPU0 (Thread 0) ---+ |---> Core 0 ---+ CPU1 (Thread 1) ---+ | | ----> Socket 0 CPU2 (Thread 2) ---+ | |---> Core 1 ---+ CPU3 (Thread 3) ---+ While this sounds very simple, it’s actually fraught with a number of little niggles.

A few weeks ago one of the users of the Disruptor posted some worrying benchmarks: ThreePublisherToOneProcessorSequencedThroughputTest run 0: BlockingQueue=645,161 Disruptor=1,772 ops/sec run 1: BlockingQueue=1,250,000 Disruptor=20,000,000 ops/sec run 2: BlockingQueue=1,250,000 Disruptor=56 ops/sec It appears under heavy contention with fewer available cores than busy threads the Disruptor can perform terribly. After a bit of investigation I managed to isolate the problem. One of the most complex parts of the Disruptor is the multi-threaded claim strategy. It is the only place in the Disruptor where - out of necessity - we break the single-writer principal.

Firstly an apology. On my previous blog, I mentioned that a string splitting algorithm implemented in Scala had a complexity of O(n2)O(n^2). One commenter mentioned that they did not understand how I came to that calculation. I though I should revisit my guess work and actually do a more thorough analysis. What I found was interesting, I had overstated the complexity, in reality it was O(n.log2(n))O(n.log_{2}(n)). I’ve included my working here. Complexity of the Scala Implementation

Introducing ConcurrentHashTrie While spending some time looking at Clojure’s concurrency model, I did a little bit of research into their persistent collection implementation that uses Bagwell’s Hash Array Mapped Tries. After a little bit of thought it occurred to me that you could apply the persistent collection model to an implementation of ConcurrentMap. The simple solution would be to maintain a reference to a persistent map within a single AtomicReference and apply a compareAndSet (CAS) operation each time an update to the map was performed.

While setting up rhoda’s blog on tinola I came across a couple of hurdles converting from one site to multiple sites. The pivotx manual helps with a section on running multiple websites. But not everything works that smoothly. So, here’s what I needed to do to make it work. Firstly, make the “sites” directories for the current and new blog(s) # go to the pivotx subdirectory of the root of the pivotx installation cd /var/www/blog/pivotx # make the sites directories mkdir -p sites/blog.

A few months ago, my colleague Mike Barker tackled a word split algorithm (see here) and proved that taking the parallel approach may not always be right. In some case, a simple 20 lines single threaded “naive” solution can out-perform your hard thought complex multi-threaded monster. Still - the problem is intrigued me because on the face of it - This is an ideal problem for a parallel algorithm. It is a classic “Data parallelism” problem borrowed from Guy Steele’s examples of