How should one use Disruptor (Disruptor Pattern) to build real-world message systems?

Question

As the RingBuffer up-front allocates objects of a given type, how can you use a single ring buffer to process messages of various different types?

You can't create new object instances to insert into the ringBuffer and that would defeat the purpose of up-front allocation.

So you could have 3 messages in an async messaging pattern:

1. NewOrderRequest
2. NewOrderCreated
3. NewOrderRejected

So my question is how are you meant to use the Disruptor pattern for real-world messageing systems?

Thanks

Links: http://code.google.com/p/disruptor-net/wiki/CodeExamples

http://code.google.com/p/disruptor-net

http://code.google.com/p/disruptor

Answer 1

One approach (our most common pattern) is to store the message in its marshalled form, i.e. as a byte array. For incoming requests e.g. Fix messages, binary message, are quickly pulled of the network and placed in the ring buffer. The unmarshalling and dispatch of different types of messages are handled by EventProcessors (Consumers) on that ring buffer. For outbound requests, the message is serialised into the preallocated byte array that forms the entry in the ring buffer.

If you are using some fixed size byte array as the preallocated entry, some additional logic is required to handle overflow for larger messages. I.e. pick a reasonable default size and if it is exceeded allocate a temporary array that is bigger. Then discard it when the entry is reused or consumed (depending on your use case) reverting back to the original preallocated byte array.

If you have different consumers for different message types you could quickly identify if your consumer is interested in the specific message either by knowing an offset into the byte array that carries the type information or by passing a discriminator value through on the entry.

Also there is no rule against creating object instances and passing references (we do this in a couple of places too). You do lose the benefits of object preallocation, however one of the design goals of the disruptor was to allow the user the choice of the most appropriate form of storage.