Websocket Shootout: Clojure, C++, Elixir, Go, Node.js, and Ruby(hashrocket.com)
hashrocket.com
Websocket Shootout: Clojure, C++, Elixir, Go, Node.js, and Ruby
https://hashrocket.com/blog/posts/websocket-shootout
39 comments
All interesting points. Even without them, Phoenix does great in this benchmark.
My takeaways from this article were:
1) Phoenix and Rails are clear winners on API.
2) Rails performance is unacceptable, which makes its nice API irrelevant.
3) All others are good enough that if I wanted to choose them for other reasons, this article wouldn't stop me.
My takeaways from this article were:
1) Phoenix and Rails are clear winners on API.
2) Rails performance is unacceptable, which makes its nice API irrelevant.
3) All others are good enough that if I wanted to choose them for other reasons, this article wouldn't stop me.
QQ .. this might be a naive question so please excuse my ignorance ... why did you not go with vanilla websockets as the protocol? I am a big fan of Elixir and would love to use it and Phoenix for work .. coworkers just get the heebeejeebees when I admit the wire protocol is specifically for Phoenix. Would love to understand this better. Btw .. loved the book!!
[deleted]
Some discussion on the elixir subreddit a few days ago: https://www.reddit.com/r/elixir/comments/50nvhg/websocket_sh...
Of note is that of these, I think only the elixir and ruby (?) solutions are distributed. Additionally, Phoenix channels are doing a lot of work that the other solutions aren't.
Of note is that of these, I think only the elixir and ruby (?) solutions are distributed. Additionally, Phoenix channels are doing a lot of work that the other solutions aren't.
Should have used https://github.com/uWebSockets/uWebSockets for Node.js, if performance was the objective.
There's a lot that could be improved about the Node one.
The library and the the lack of clustering (so you can compare apples to apples) are the main two.
Honestly? Running at 39% of the connections on one core is pretty impressive. Especially with its low memory footprint.
The library and the the lack of clustering (so you can compare apples to apples) are the main two.
Honestly? Running at 39% of the connections on one core is pretty impressive. Especially with its low memory footprint.
There are of course other advantages to a Node implementation (first class simple objects to/from JSON being high among them). Far less of a disconnect to the front end is huge. Also, if you need to have the reliability/scale of more than one server anyway, then some of that ground can be overcome with clustering and/or multiple instances.
Though clustering would have meant the need for another control channel.. that said, I'd be surprised if Redis + Node(clustered) didn't perform in with the leaders of the pack.
Though clustering would have meant the need for another control channel.. that said, I'd be surprised if Redis + Node(clustered) didn't perform in with the leaders of the pack.
Agreed on all points.
With clustering -as that has overhead - I'd expect it to perform at least as fast as Elixir or Go.
Of course you can also just use multiple cheaper single core servers too! I've seen that used a lot for certain applications - kinda overkill here tho.
With clustering -as that has overhead - I'd expect it to perform at least as fast as Elixir or Go.
Of course you can also just use multiple cheaper single core servers too! I've seen that used a lot for certain applications - kinda overkill here tho.
I'm not a node.js dev so I might be wrong here, but I imagine clustering would complicate the broadcast, no? I was under the impression that communication between the nodes of the cluster can be a pain.
In a way yes, you'd need a control channel to centralize the broadcast to the other processes. In more common use cases, Redis is used in conjunction with node for these types of things. There are relays for redis with sockjs, socket.io and others to make the setup/configuration a bit easier to work with.
For sure, it wouldn't be as short, but it wasn't a LoC benchmark so...
Nice article. Not trying to start a subjective debate on languages/ technologies, but it did seem odd they would include a language like ruby but not python, c#, or java / scala.
I work with the author, the languages included came down purely to expertise available. Additional implementations have been submitted for Rust, Haskell and Ruby's event machine. A host of pull requests have been submitted to tweak the existing implementations as well.
It would be awesome to see implementations in python, c# and java. There are a number of ways to do each, I assume.
It would be awesome to see implementations in python, c# and java. There are a number of ways to do each, I assume.
I mostly thought it was odd that of all of the JVM languages they included it was Clojure. Ruby sort of makes since given that Rails recently included ActionCable.
Yeah, not having Scala or Java but having Clojure was strange to me. I would be curious to see how a CLR language (C# or F# mainly) fares relative to the JVM.
Java would have been a bit longer than Clojure, but performance and memory usage would likely have been better.
Additionally, I would like to see Rust, Haskell, and .NET.
http://bitemyapp.com//posts/2016-09-03-websocket-shootout-ha...
I happened to come across this in /r/haskell today. The results are very impressive.
I happened to come across this in /r/haskell today. The results are very impressive.
Careful, there was a mistake in that Haskell code and possible bug causing it to drop a lot of messages.
With C++ one can go far further than that. It's the matter of time spent on the solution. With all that other languages there is a performance ceiling that can't be breached.
Is single threaded node was 39% I wonder how a clustered node app would compare.
Plus with an efficient lib as well. They certainly didn't use a very performant one.
+1. They needed a cluster benchmark to make this a more fair test.
Odd there were no mention of Java.
Both high speed and simple to develop :
http://docs.spring.io/spring/docs/current/spring-framework-r...
https://spring.io/guides/gs/messaging-stomp-websocket/
Both high speed and simple to develop :
http://docs.spring.io/spring/docs/current/spring-framework-r...
https://spring.io/guides/gs/messaging-stomp-websocket/
Using the cluster module for nodejs will make it almost the fastest and when we add activities like reading from a database nodejs asynchronous pattern beats everybody
Not likely. Node will have to send out to an external source to pass messages between the cores of the cluster. Go and Elixir would do that fairly naturally without the external latency. Elixir and Phoenix would do the same over multiple machines with multiple cores without the need for a database or Redis to handle routing.
Both also make it pretty natural to async everything. If Node beat either in a distributed benchmark it would be a shock to all involved.
Both also make it pretty natural to async everything. If Node beat either in a distributed benchmark it would be a shock to all involved.
It is just more natural too since all of io is inherently async.
ahem, apart from C++
Citation needed. Not that i object to the fact that c++ is usually faster :)
That was the citation :-)
Seriously though, Node is written in C++ and is a single threaded interpreted / Jitted / Garbage Collected Virtual Machine which has to jump through trampolines and layers of abstraction to call system services.
While it might be very efficient at what it does, it isn't as close to the machine as a native C++ application.
One asynchronous WebSocket server written in C++ can run rings around one Node WebSocket server.
If the premise is that to mitigate this we can set up an entire cluster of node servers that will beat the C++ server, it follows that we can set up an entire cluster of C++ servers that would be even faster.
C++ is faster in every regard. There is no "usually", about it.
If we are talking about some metric like programmer productivity, sure sometimes it takes more effort to write code in C++ to achieve what may take a single line in Node.
Seriously though, Node is written in C++ and is a single threaded interpreted / Jitted / Garbage Collected Virtual Machine which has to jump through trampolines and layers of abstraction to call system services.
While it might be very efficient at what it does, it isn't as close to the machine as a native C++ application.
One asynchronous WebSocket server written in C++ can run rings around one Node WebSocket server.
If the premise is that to mitigate this we can set up an entire cluster of node servers that will beat the C++ server, it follows that we can set up an entire cluster of C++ servers that would be even faster.
C++ is faster in every regard. There is no "usually", about it.
If we are talking about some metric like programmer productivity, sure sometimes it takes more effort to write code in C++ to achieve what may take a single line in Node.
Ok let me rephrase: The speed benefit of C++ not always matters. At Google scale, sure. For most intents and purposes, probably not.
Otherwise, totally agreed.
Otherwise, totally agreed.
I use node quite a lot, mostly for prototyping, demos and where I just plain don't have the time or need to use something else. I have no problem with node as an application development platform.
Right tool for the right job is fine
The trick is to minimize aggregate kWh of energy use per unit task. Sometimes that means ordering a take out, and occasionally baking a pie.
A lot of the time I'm in the position of buildng upstream tools and platforms for people downstream, so whatever I do has a direct impact on their performance. The way I look at it is it is my duty to make them look good.
I do take that a little far though on occasion. Shipping is a feature too of course. Heh.
Right tool for the right job is fine
The trick is to minimize aggregate kWh of energy use per unit task. Sometimes that means ordering a take out, and occasionally baking a pie.
A lot of the time I'm in the position of buildng upstream tools and platforms for people downstream, so whatever I do has a direct impact on their performance. The way I look at it is it is my duty to make them look good.
I do take that a little far though on occasion. Shipping is a feature too of course. Heh.
C/C++/Rust and other low-level languages should be able to perform faster than a Node implementation... Although a less-simple example would mitigate a lot of those differences, as many interfaces would need to interact with other systems, and with any kind of redundancy (multiple servers) coordination becomes an issue as well.
In the end, if you're within 10% is probably moot, and at 39% on a single instance for node, that's very impressive and once you add multiple system, multiple instances per system aren't any harder and node would probably make up most of the difference.
In the end, if you're within 10% is probably moot, and at 39% on a single instance for node, that's very impressive and once you add multiple system, multiple instances per system aren't any harder and node would probably make up most of the difference.
[deleted]
- Phoenix Channels is a higher-level abstraction over raw WS. We spawn isolated, concurrent "channels" on the underlying WebSocket connection. We monitor these channels so that clients can detect errors and recover automatically without dropping the entire connection. This contributes to overhead in both memory and throughput, which should be highlighted with how Phoenix faired in the runs.
- Phoenix channels runs on a distributed pubsub system. None of the other contestants had a distribution story, so their broadcasts are only node-local implementations, where ours is distributed out of the box
Phoenix faired quite well in these runs, considering we are comparing a robust feature set vs raw ws/pubsub implementations.