JXcore – A Node.js Distribution with Multi-threading(flippinawesome.org)
flippinawesome.org
JXcore – A Node.js Distribution with Multi-threading
http://flippinawesome.org/2014/03/03/jxcore-a-node-js-distribution-with-multi-threading/
42 comments
"This thing is dead on arrival. ... it's not open source, ..."
The open source argument holds except in situations where there is a compelling advantage/value. Companies that are dedicated to open source will not be customers, but may not have been paying customers anyway.
MapR is a good example of a closed source technology doing well in the largely open source Hadoop ecosystem. Many companies will not touch MapR, however there are enough companies that really need the features / capabilities / value delivered by MapR that they are willing to pay for it.
Closer to home, there were people who thought Meteor would not take off b/c it dropped npm in favor of a new (open source) package manager. Yet, Meteor provides enough compelling value that it's growing rapidly even with a new package manager.
That said, this product will need to deliver compelling value to a customer segment that's willing to pay, while accepting the fact that many in the node community will not use it. If there are enough of these customers then the company will do fine. However, that's a tradeoff they'll have to evaluate.
As a total aside, it's my opinion that the open source requirement is selectively applied. Mac is clearly not open source, yet BSD is freely available. Yet people choose to use a closed source OS b/c it provides value to them. Same thing with editors and just about everything else. To those who are open source up and down their stack, then kudos for the consistency. For the rest of us, we should at least be honest and own up to the fact that open source is not an absolute requirement, but a selective requirement that's arbitrarily applied. (flame away :)
The open source argument holds except in situations where there is a compelling advantage/value. Companies that are dedicated to open source will not be customers, but may not have been paying customers anyway.
MapR is a good example of a closed source technology doing well in the largely open source Hadoop ecosystem. Many companies will not touch MapR, however there are enough companies that really need the features / capabilities / value delivered by MapR that they are willing to pay for it.
Closer to home, there were people who thought Meteor would not take off b/c it dropped npm in favor of a new (open source) package manager. Yet, Meteor provides enough compelling value that it's growing rapidly even with a new package manager.
That said, this product will need to deliver compelling value to a customer segment that's willing to pay, while accepting the fact that many in the node community will not use it. If there are enough of these customers then the company will do fine. However, that's a tradeoff they'll have to evaluate.
As a total aside, it's my opinion that the open source requirement is selectively applied. Mac is clearly not open source, yet BSD is freely available. Yet people choose to use a closed source OS b/c it provides value to them. Same thing with editors and just about everything else. To those who are open source up and down their stack, then kudos for the consistency. For the rest of us, we should at least be honest and own up to the fact that open source is not an absolute requirement, but a selective requirement that's arbitrarily applied. (flame away :)
I would like to remind some things here;
> Maybe you don't have a case but there are many scenarios can benefit from multithreading including web hosting.
> The goal is to have a zero compatibility issue and we are almost there with the next beta.
> I believe that the node developers would enjoy benefiting from load aware instance monitored processes instead a trivial multi processing. BTW, still you can combine multithreading with multiple processes in case you want to keep process is active during v8 is GC'ing on one of the threads.
> Maybe you don't have a case but there are many scenarios can benefit from multithreading including web hosting.
> The goal is to have a zero compatibility issue and we are almost there with the next beta.
> I believe that the node developers would enjoy benefiting from load aware instance monitored processes instead a trivial multi processing. BTW, still you can combine multithreading with multiple processes in case you want to keep process is active during v8 is GC'ing on one of the threads.
[deleted]
"I spend most of my waking hours writing all sorts of crazy things in node, and I still can't think of any scenario in which using this makes sense."
> You are not alone. remember node.js team members were doing the same in the past but couldn't finish it.
Obviously there are many scenarios this could help.
> You are not alone. remember node.js team members were doing the same in the past but couldn't finish it.
Obviously there are many scenarios this could help.
[deleted]
Am I correct in understanding that the main benefit is that JXCore allows you to run x number of tasks in parallel where x is the number of CPUs?
The problem is solved by running multiple node processes, which is standard deployment for node (i.e. if a machine has 8 cores then 8 node processes are started).
One issue with running multiple threads is that many developers use fail fast as a best practice when building node applications. In other words, uncaught exceptions cause the node process to fail, die and restart. So, it's perfectly acceptable practice to write your application to be written to accept failure as a given (similar to how Netflix uses Simian Army).
That said, how well is each thread isolated from everything else. Does an uncaught exception kill just the thread and its state, or does it kill the process? More specifically, when is the main process killed and what is contained within the thread?
The problem is solved by running multiple node processes, which is standard deployment for node (i.e. if a machine has 8 cores then 8 node processes are started).
One issue with running multiple threads is that many developers use fail fast as a best practice when building node applications. In other words, uncaught exceptions cause the node process to fail, die and restart. So, it's perfectly acceptable practice to write your application to be written to accept failure as a given (similar to how Netflix uses Simian Army).
That said, how well is each thread isolated from everything else. Does an uncaught exception kill just the thread and its state, or does it kill the process? More specifically, when is the main process killed and what is contained within the thread?
It's actually pretty silly to write node.js apps in a crash-only manner. Since they often handle thousands of connections concurrently, a failure in one client's processing is pretty horrendous if it brings down a whole server (even if the server thread gets immediately restarted). This project doesn't try to solve that either though.
In the tradeoff between a short post and a long elaboration, I erred on the side of being too terse.
Node applications can be split between stateful servers (ex. chat) and stateless (ex. API for mobile clients).
It's the stateless servers where some developers write fault tolerant / fail fast / fast restart applications. Doing so in a stateful server would be counter productive.
Also, this does not mean to imply that developers are writing sloppy code that fails constantly or that they fail to implement proper error handling. What I was stating is that unhandled exceptions are unexpected, but when they do occur they indicate something is seriously wrong. Importantly, this puts the application's state into an unknown state, which is difficult to recover from. In such situations, a robust approach is to let Node fail, restart fast, and have clean state.
The reasons this is a sound approach are:
1. If the failure is due to a memory leak, then the graphs will highlight said leak clearly
2. The unhandled exception indicates that something is very wrong. A server restart is easily seen in the logs and is a warning that deeper inspection is necessary
3. Recovering state after an unhandled exception is difficult. In a stateless server its better to just restart from a clean state. This assumes the engineers wrote the application to work from a clean state (i.e. after a restart there is no need to recreate state)
4. A fault tolerant architecture is good practice as disks can fail, CPUs can fail, network connections can fail, etc. In a cluster failure is expected and applications are architected to continue operation in the face of failure
Node applications can be split between stateful servers (ex. chat) and stateless (ex. API for mobile clients).
It's the stateless servers where some developers write fault tolerant / fail fast / fast restart applications. Doing so in a stateful server would be counter productive.
Also, this does not mean to imply that developers are writing sloppy code that fails constantly or that they fail to implement proper error handling. What I was stating is that unhandled exceptions are unexpected, but when they do occur they indicate something is seriously wrong. Importantly, this puts the application's state into an unknown state, which is difficult to recover from. In such situations, a robust approach is to let Node fail, restart fast, and have clean state.
The reasons this is a sound approach are:
1. If the failure is due to a memory leak, then the graphs will highlight said leak clearly
2. The unhandled exception indicates that something is very wrong. A server restart is easily seen in the logs and is a warning that deeper inspection is necessary
3. Recovering state after an unhandled exception is difficult. In a stateless server its better to just restart from a clean state. This assumes the engineers wrote the application to work from a clean state (i.e. after a restart there is no need to recreate state)
4. A fault tolerant architecture is good practice as disks can fail, CPUs can fail, network connections can fail, etc. In a cluster failure is expected and applications are architected to continue operation in the face of failure
I actually think I understood you, but I'm saying that in that case where statelessness should be an advantage, node.js is actually a much less fault tolerant environment when you compare it to most other web application servers. Most other web app servers offer
(1) request isolation (so most failures in one request can't break other requests) and
(2) a way to catch all exceptions/errors in a single request (and domains don't accomplish this, unless you know what to expect errors from, or wrap everything).
Since node.js doesn't offer those features, it's not even as fault tolerant as PHP was 15 years ago. I'm a huge fan of node.js, but one of the hardest things to do on a large application with a large number of users is to keep an instance of the server from restarting and dropping all the other in-progress requests. If you write your node.js code to be crash-only (like one might do with erlang) your clients are going to have a terrible time.
(1) request isolation (so most failures in one request can't break other requests) and
(2) a way to catch all exceptions/errors in a single request (and domains don't accomplish this, unless you know what to expect errors from, or wrap everything).
Since node.js doesn't offer those features, it's not even as fault tolerant as PHP was 15 years ago. I'm a huge fan of node.js, but one of the hardest things to do on a large application with a large number of users is to keep an instance of the server from restarting and dropping all the other in-progress requests. If you write your node.js code to be crash-only (like one might do with erlang) your clients are going to have a terrible time.
We had the problem you're describing for awhile, but have since figured out how to avoid processes going down and interrupting other reqs. Essentially, you attach a global domain, and when that domain catches an error you stop accepting new connections (obviously you have to be load-balancing between procs) and start a countdown. Some reasonable amount of time later (I think we wait 30 seconds?) you assume that any in-progress request is done and restart the process. We've found this to be very successful.
We do this too, attaching req and res objects to a domain, as well as databases and other network related objects (like smtp clients, etc). This is a huge improvement, but I'm still seeing occasional uncaught error events in our logs on a very large codebase and only in production. Some of them are just ECONNRESET events with no details given, so their origins are REALLY hard to track down. Have you got some magic for catching everything without explicitly having to find all objects that could be emitting? I'd love to hear it if so...
As soon as possible during startup, create a domain and enter it. Because entered domains form a stack, this will be a fallback if an error occurs at a place that isn't covered by any other domains.
Thanks for the comments.
>JXCore allows you to run x number of tasks in parallel where x is the number of CPUs?
You may have a 8 cores but configure JXcore to use 64 threads.
> The problem is solved by running multiple node processes, which is standard deployment for node
You can still run multiple node process but 2 threads per each. This will improve the responsiveness of each process by balancing the load exactly on the native side. That means, if anything happens on one of the V8 threads (GC etc). the other one will be handling the load.
> That said, how well is each thread isolated from everything else.
Totally isolated. We already started to update native c,c++ modules for isolated multithreading.
>Does an uncaught exception kill just the thread and its state, or does it kill the process?
On this very beta release, it throws into main thread (when it's uncaught by thread) but coming beta (internal monitoring is implemented) will be optionally resetting the sub thread itself.
You may simply consider each thread as a separate node.js host.
You may have a 8 cores but configure JXcore to use 64 threads.
> The problem is solved by running multiple node processes, which is standard deployment for node
You can still run multiple node process but 2 threads per each. This will improve the responsiveness of each process by balancing the load exactly on the native side. That means, if anything happens on one of the V8 threads (GC etc). the other one will be handling the load.
> That said, how well is each thread isolated from everything else.
Totally isolated. We already started to update native c,c++ modules for isolated multithreading.
>Does an uncaught exception kill just the thread and its state, or does it kill the process?
On this very beta release, it throws into main thread (when it's uncaught by thread) but coming beta (internal monitoring is implemented) will be optionally resetting the sub thread itself.
You may simply consider each thread as a separate node.js host.
Thanks for the detailed reply.
A small diagram on the home page showing x cores * y threads would be interesting / helpful. Such a diagram would also highlight that this adds to the multi-process approach (i.e. I don't have to give up my multi-process approach, but get to add multiple threads to it).
Anyway, the solution sounds interesting.
A small diagram on the home page showing x cores * y threads would be interesting / helpful. Such a diagram would also highlight that this adds to the multi-process approach (i.e. I don't have to give up my multi-process approach, but get to add multiple threads to it).
Anyway, the solution sounds interesting.
Did anything ever come of the conversation concerning a mutually-agreeable comparison of the performance of NodeJX and Vert.x [1]?
[1] https://twitter.com/obastemur/status/425638466433474562
[1] https://twitter.com/obastemur/status/425638466433474562
I fail to see the advantage that this would give over node-cluster for scaling, as that can scale across multiple processes as well, while still using the regular NodeJS.
Either I am just paranoid or something seems wrong. The oldest blog entry is "January 10, 2014" and a whois says it was resisted "05-feb-2014". They mention have Nubisa Inc. copyright, which doesn't seem to be nubis.com (whois again).
Other than that they have huge claims about optimizations that the whole node.js community didn't come up with yet and there are just binaries available.
Other than that they have huge claims about optimizations that the whole node.js community didn't come up with yet and there are just binaries available.
Haha yeah something is fishy alright. The part where they reimplemented JS in llvm and it's already 25% faster than v8? Just a side project. No big deal.
I agree - there's no way anyone is going to beat V8 at its own game in a "side project" that was done over a weekend. However, the benchmarks they are showing are primitive types only; it's not hard to believe that perhaps they managed to optimize the hell out of that in order to produce a better result. For now. On an incomplete engine. Seems highly, highly unlikely that the full engine will be anywhere near able to compete with V8.
The multithreaded performance optimizations are really interesting, however - but I'd be much more interested in this making it into mainline Node, rather than using this crazy closed-source fork. You're not going to find me using anything closed-source on my servers if I can help it, least of all something like Node which is hackable to its core.
If the performance improvements really mean something, and mean something outside of contrived for() and fib() loops, I hope the Node core teams have a serious talk with these guys about a merge. Node's cluster module is definitely not the final solution for properly utilizing a server's full capacity, and the team knows it; that module has been at level 1 (experimental) for a long time.
The multithreaded performance optimizations are really interesting, however - but I'd be much more interested in this making it into mainline Node, rather than using this crazy closed-source fork. You're not going to find me using anything closed-source on my servers if I can help it, least of all something like Node which is hackable to its core.
If the performance improvements really mean something, and mean something outside of contrived for() and fib() loops, I hope the Node core teams have a serious talk with these guys about a merge. Node's cluster module is definitely not the final solution for properly utilizing a server's full capacity, and the team knows it; that module has been at level 1 (experimental) for a long time.
I don't think a Node fork,even if it is closed-source, is such a bad idea. Look at the rivalry between NPM & Nodejitsu on one side and Joyent & StrongLoop on the other. They are all for-profit companies and pushing things to their advantage. I am not sure if this is all beneficial to the project. I have a project that I was going to use Node but after seeing last week's well publicized incident, I am not so sure anymore. IMHO an alternative independent distribution might just be a good thing after all.
Clearly, you don't know what you are talking about. LLVM can beat V8. As far as I understand, he put LLVM frontend intogether to see the results and it's not the part of the project yet.
I'm not doubting that it can beat V8, in a mature project with very significant programmer time behind it. But I find it very hard to believe that one weekend of fiddling with an LLVM javascript engine is enough to produce results better than V8, which has had the benefit of years of work from some of the smartest minds in the industry.
At what part I said that Its all done over a weekend?
from the post "..Last weekend I could finish the prototype and measure the initial performance of the solution.."
If you still understand that it's all finished over a weekend. No way.. I don't think it is easy to develop a LLVM frontend with those features over a weekend.
If you could have some details on LLVM, you wouldn't accuse me on something you miss read.
Because;
There is no such an LLVM JS engine. LLVM has its own IR and that prototype turns the JS codes into a wrapped LLVM IR.. As a result the final code functions at native level, so it is already fast.
For this reason, 20% or more performance difference because of LLVM actually shows how V8 is fast! It doesn't show the prototype is over a weekend or anything else..
from the post "..Last weekend I could finish the prototype and measure the initial performance of the solution.."
If you still understand that it's all finished over a weekend. No way.. I don't think it is easy to develop a LLVM frontend with those features over a weekend.
If you could have some details on LLVM, you wouldn't accuse me on something you miss read.
Because;
There is no such an LLVM JS engine. LLVM has its own IR and that prototype turns the JS codes into a wrapped LLVM IR.. As a result the final code functions at native level, so it is already fast.
For this reason, 20% or more performance difference because of LLVM actually shows how V8 is fast! It doesn't show the prototype is over a weekend or anything else..
The initial name was nodejx but we changed it to jxcore. (not nubis.com / it's nubisa.com)
...meanwhile, everyone writing npm packages assumes a single thread.
It doesn't change anything for javascript modules. On the other hand, we already embedded most popular c,c++ modules internally (with multithreading in mind) (coming with next beta)
Besides, making a native c,c++ module multithread compatible will be mostly changing couple of lines of code. We were working on a wrapper and it will be part of beta 2 also.
That's quite interesting. I'd love to have a look at the source! I don't think I'm going to deploy it to production anytime soon though, as I have no idea how secure this is.
Here is where you can download the executable if you want to try it (I found it weird that no link was given in the post): http://jxcore.com/downloads/
Here is where you can download the executable if you want to try it (I found it weird that no link was given in the post): http://jxcore.com/downloads/
Great point. I added a link to the project page at the beginning of the article and a link to the downloads at the end.
Great job guys!
But I fail to understand what's the difference between JXcore and native Node clusters. Can you develop this a little bit more?
But I fail to understand what's the difference between JXcore and native Node clusters. Can you develop this a little bit more?
In case people aren't aware there is webworker-threads although this doesn't really solve the same problem.
This looks really cool, but 99% of the time I can get by with something simpler like PM2 (https://github.com/Unitech/pm2) and continue business as usual.
Well, I'm just not going to use it.
I don't know about you guys, but I can't rely on some proprietary blob, if not for the security concerns, because doesn't even have an ARM binary, or even an IA32 binary for RedHat derivatives.
I don't know about you guys, but I can't rely on some proprietary blob, if not for the security concerns, because doesn't even have an ARM binary, or even an IA32 binary for RedHat derivatives.
So it's basically a closed source and commercial fork of nodejs ?
it's not a commercial fork. "For now" it's closed source.
So it will be open-source at some point?
I thought node's non-blocking I/O implied that threads were unnecessary?
What's the core mechanism that permits node's implicit concurrency?
What's the core mechanism that permits node's implicit concurrency?
It is, but that also means a Node application will not use all four cores of a quadcore efficiently by default. I think the usual approach in that case is to just rev up four node processes with a load balancer of sorts in front of it.
If you want an OSS alternative, you can check out http://nodyn.io/
not you need the jvm to run that.And frankly nodyn doesnt run anything yet.
Some performance comparisons would be useful... especially since the point of the project is to increase performance.
Threads in JavaScript?
Nope.
Nope.
The touted benefit is marginally increased parallel performance, but even if you buy this (I don't), the best case scenario is this buys you a slightly decreased server bill for the price of closedness, lock-in, and compatibility headaches. If your code shards horizontally like this then you can already trivially parallelize and shard across servers with first-class node core clustering.
I spend most of my waking hours writing all sorts of crazy things in node, and I still can't think of any scenario in which using this makes sense.