Fixing the World (2011)(zguide.zeromq.org)
zguide.zeromq.org
Fixing the World (2011)
http://zguide.zeromq.org/page:all#Fixing-the-World
28 comments
The ZeroMQ Guide [1] (originally by Pieter Hintjens, now maintained by the community) is undoubtedly one of the best pieces of technical writing I've ever encountered - like another commenter posted, it's worth reading even if you will never use ZeroMQ.
[1]: http://zguide.zeromq.org
[1]: http://zguide.zeromq.org
> originally by Pieter Hintjens
Some context (because it's been 3 years already!) :
Cancer diagnosis: https://news.ycombinator.com/item?id=11520888
Legacy: https://news.ycombinator.com/item?id=12650682
More: https://hn.algolia.com/?q=Pieter+Hintjens
Some context (because it's been 3 years already!) :
Cancer diagnosis: https://news.ycombinator.com/item?id=11520888
Legacy: https://news.ycombinator.com/item?id=12650682
More: https://hn.algolia.com/?q=Pieter+Hintjens
Wow, I disagree -- I find this incredibly difficult to wade through. It's full of marketing fluff and pointless tangents. I gave up after the first several pages.
I'm not sure I agree. It reminded me of certain "great" textbooks that sometimes get recommended by experts, that aren't actually that great for someone learning something for the first time. They get the reputation as being "great" because they are great for people that already have a good basic grasp of the material: They are precise and pithy, and they contain insights that are deep and useful, but hard to appreciate if this is your first encounter with the material. (Kleppner & Kolenkow would be a good example of a freshman physics textbook in this category.) It seems to me that maybe the ZeroMQ Guide is in this category. Which, of course, is a noble and important category, but maybe not the most important category if you're just starting out on a topic.
I'd like to disagree. The first time I read the guide was after my first programming internship during college and didn't really have a background in distributed computing or message queues. I only knew how to read and write code good enough for a CRUD based web app. I understood sockets though.
I found this book a great introduction to MQ and distributed computing in general. I didn't read cover to cover but I remember reading around 2/3rds of the book and didn't find it super hard to follow even though I did have to reread some sections. Also I must mention I was implementing a toy ZeroMQ project while I was reading the book.
I found this book a great introduction to MQ and distributed computing in general. I didn't read cover to cover but I remember reading around 2/3rds of the book and didn't find it super hard to follow even though I did have to reread some sections. Also I must mention I was implementing a toy ZeroMQ project while I was reading the book.
That's fair - on the other hand if you already understand the principles of messaging (maybe from Hohpe, or some other introduction to pi calculus), it's a great piece of writing about an otherwise dry topic - a C library.
I love the way you expressed this.
I'll say that docs are delightful to read and beautifully explain the domain. I would recommend going through them even if you never use the ZMQ itself.
I remember reading the ZeroMQ docs when I was first learning about concurrency. Those plus Joe Armstrong's Programming Erlang book were really fun and eye opening.
I never used ZeroMQ much but it taught me a lot of the language around distributed and high-concurrency systems and problems that come up.
I never used ZeroMQ much but it taught me a lot of the language around distributed and high-concurrency systems and problems that come up.
Do you have an opinion on which to read first?
Not OP but if you're on the edge, try reading the first few chapters of the ZeroMQ guide. It does not make a lot of assumptions about the reader's background in computing.
I read the docs a couple of years ago but never got to actually use ZMQ for anything. Is it still worth considering for new projects?
Yes, nothing else comes close. The nanomsg author seems to have abandoned nanomsg.
Typically software infrastructure like 0MQ, ext2fs, Valgrind, HotSpot, and gzip takes about a decade to get good, remains optimal for a decade or two, and remains relevant for half a century after that. A couple of years is not a long time.
Typically software infrastructure like 0MQ, ext2fs, Valgrind, HotSpot, and gzip takes about a decade to get good, remains optimal for a decade or two, and remains relevant for half a century after that. A couple of years is not a long time.
Great points! I'd like to add that RabbitMQ is an easier option if it has enough number of features that'd cover your needs. So, it might be better to checkout ready solutions before diving into a MQ framework.
Yes, they aren't nearly as similar as the names suggest. Like PostgreSQL vs. ext4fs.
> it's not something that you will be able to understand and use right away.
I think it took me an hour or three to get some benchmarking programs running when I first encountered it, but that was mostly because I was using the documentation for the wrong major version. So I don't think it's such a big hassle. I definitely continued to learn important things about it for months, though.
It's definitely not a message queuing system like RabbitMQ, though. It's something much simpler and more general.
I think it took me an hour or three to get some benchmarking programs running when I first encountered it, but that was mostly because I was using the documentation for the wrong major version. So I don't think it's such a big hassle. I definitely continued to learn important things about it for months, though.
It's definitely not a message queuing system like RabbitMQ, though. It's something much simpler and more general.
We use 0MQ in our products at work. Once we got it set up correctly, it just worked! It makes distributed systems really easy to use. I can’t say enough about it.
We use ProtoBuf for the “formatting” of all the data the goes through 0MQ layer. They pair together nicely
This lets us pair together say a processor built in C on one machine to a test GUI built in Matlab on another machine without significant effort.
We use ProtoBuf for the “formatting” of all the data the goes through 0MQ layer. They pair together nicely
This lets us pair together say a processor built in C on one machine to a test GUI built in Matlab on another machine without significant effort.
That section focuses on ease of use, compared to raw TCP, HTTP, and so on, but I'm rather unsure whether it extends (or is any notable) for arbitrary projects (that is, not counting few for which it is suited particularly well), beyond experimental and basic ones.
I used ZMQ in production 6+ years ago, and there were hard-to-reproduce bugs somewhere. Not sure if I managed to find all their sources before rewriting, but ZMQ was one of the suspects (with a reported bug that would have explained what I have observed, IIRC), certainly less stable than plain POSIX sockets, and making it harder to localize the issue. There is plenty of open issues in the core library still [0]. It's a rather common problem with dependencies and abstractions, not ZMQ-specific, but it adds quite a bit of complexity; if it isn't necessary (or at least very useful), I think it's a good idea to avoid. For the past few years I use pipes and UNIX domain sockets for IPC between successors of those programs, and it works well.
Though I keep seeing programmers/software using both TCP and POSIX sockets wrong, making all kinds of wrong assumptions about their operation (and then there are libraries intended to simplify it, but complete with bugs too!); I think that's where HTTP abuse helps by providing a basic API (via relatively polished HTTP libraries) for basic use cases. Probably ZMQ (and other MQs) are indeed useful for that, and for some complex scenarios that one would have to implement from scratch otherwise, but I don't think it's fair to present/recommend as strictly better sockets.
It also talks about distributed P2P, which I find exciting in general, but the example it provides further in the guide only demonstrates automated reconnects and message queuing (which is potentially useful, but not a major challenge).
[0] https://github.com/zeromq/libzmq/issues
I used ZMQ in production 6+ years ago, and there were hard-to-reproduce bugs somewhere. Not sure if I managed to find all their sources before rewriting, but ZMQ was one of the suspects (with a reported bug that would have explained what I have observed, IIRC), certainly less stable than plain POSIX sockets, and making it harder to localize the issue. There is plenty of open issues in the core library still [0]. It's a rather common problem with dependencies and abstractions, not ZMQ-specific, but it adds quite a bit of complexity; if it isn't necessary (or at least very useful), I think it's a good idea to avoid. For the past few years I use pipes and UNIX domain sockets for IPC between successors of those programs, and it works well.
Though I keep seeing programmers/software using both TCP and POSIX sockets wrong, making all kinds of wrong assumptions about their operation (and then there are libraries intended to simplify it, but complete with bugs too!); I think that's where HTTP abuse helps by providing a basic API (via relatively polished HTTP libraries) for basic use cases. Probably ZMQ (and other MQs) are indeed useful for that, and for some complex scenarios that one would have to implement from scratch otherwise, but I don't think it's fair to present/recommend as strictly better sockets.
It also talks about distributed P2P, which I find exciting in general, but the example it provides further in the guide only demonstrates automated reconnects and message queuing (which is potentially useful, but not a major challenge).
[0] https://github.com/zeromq/libzmq/issues
I remember a couple of years ago trying the examples on and off. It wasn't until the beginning of this year that I dedicated time (few weeks) to it and started to grok what ZeroMQ was for me and how I could do blazing fast RPC with it. I created a RPC (async/sync) library for myself in Rust, Java and Javascript that uses ZMQ under the hood. While I don't plan to open source it anytime soon, I do recommend taking some time to dedicate to play around with it and try it in your own projects.
[deleted]
Interesting, iMatix went out of business?
This article is long winded, very hyperbolic and incredibly self righteous. Take a look and tell me I'm wrong.
That was my PI's response to my own similar exhortations when I tried to sell him on using ZMQ in the laboratory context for coordinating control and data passing for time series data gathering scientific experiment equipment, so I did a proof of concept which quickly won him over and ZMQ is still there, years on, and works without complaint. Some bombast for such a minimal and powerful lib is justified. https://github.com/zplab/rpc-scope
I'm with you, but this is a really poor way to engage with people.
If you decide to use ZeroMQ be prepared to spend many hours reading the docs and designing the architecture you desire with the building blocks ZMQ provides. Also be prepared to build out wrappers for your developers to use so they have a simplified API to deal with.