I’m looking for a distributed system messaging package (something that provides reliable and ordered messaging like: spread). Does anyone know of anything I should look in to?
Distributed concepts are still new to me, so maybe I am missing something, but a message broker like RabbitMQ is not quite the same, right? From what I can gather it handles the underlying message distribution, but not guarantees like reliability or message ordering. Please correct me if I’m wrong 
You mean like NSQ which is written in go?
11.5K stars on github: https://github.com/nsqio/nsq
Features: http://nsq.io/overview/features_and_guarantees.html
Here’s a post on how Segment scaled NSQ to 750 Billion Messages: https://segment.com/blog/scaling-nsq/
I actually did look into NSQ and it seems like it may be my best option, but from the documentation:
You cannot rely on the order of messages being delivered to consumers.
Similar to message delivery semantics, this is the result of requeues, the combination of in-memory and on disk storage, and the fact that each nsqd node shares nothing.
With Spread you can guarantee a total order of messages on all nodes. Unfortunately, I have yet to find anything in Go that can provide this level of ordering.
Do you really need guaranteed ordering of messages? How would you even realize in a distrusted environment?
Guaranteeing ordering essentially means to only have one message at a time delivered, sequentially, regardless of the number of your workers.
Consider Worker A processing message 1 and worker B Processing message 2. A dies during processing the message and can’t ACK, so the message distributor needs to resend 1 somewhen, but 2 has also been delivered and possibly ACK’d, that’s a dilemma when you want guaranteed ordering.
This is why I think guaranteed ordering is a bad idea in distributed systems.
Sometimes I do understand the requirements of a local ordering though (a user can’t get deleted before he has been created) and in the legacy system I’m watching over it has been solved by requeueing messages that don’t make sense yet until they actually make sense or they get to old.
There are cases that ordering is very important. For example, PAXOS relies on total ordering. At the very least, causal dependencies can exist between messages which require ordering their delivery.
It may be a silly example, but consider two processes sending messages (P1 and P2) and a third observing (P3). P1 broadcasts a message m1: “P2, your house burned down”, then m2: “April Fools!”. P2 delivers m1 and m2 then responds with m3: “hahaha so funny!”. If causal dependencies are violated, P3 can see m1, m3 then m2 (which would be very confusing for P3). P3 can also make a decision based on this ordering, but m3 was broadcast based on seeing both m1 and m2.
Without ordering messages there is no way to satisfy causal dependencies.
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.