What's all this fuss about Erlang? (2007)(pragprog.com)
pragprog.com
What's all this fuss about Erlang? (2007)
https://pragprog.com/articles/erlang/
192 comments
That's true, but when I read that sentence I place the emphasis on should. I see it as aspirational.
A better way to phrase this would perhaps would be to say that perfectly preemptive scheduler should be able to keep all cores hot. Blocking of one Erlang process should not halt the program's progress.
As Erlang requires message passing overhead by design, it will never see perfectly linear scaling behaviour. Still, its scheduler, its GC model and M:N threading implementation are extremely robust and can be relied upon to write extremely reliable networked services.
A better way to phrase this would perhaps would be to say that perfectly preemptive scheduler should be able to keep all cores hot. Blocking of one Erlang process should not halt the program's progress.
As Erlang requires message passing overhead by design, it will never see perfectly linear scaling behaviour. Still, its scheduler, its GC model and M:N threading implementation are extremely robust and can be relied upon to write extremely reliable networked services.
I also interpret it this way.
If you recall the famous Wide Finder benchmark for parallel programming by Tim Bray, back in 2007 as well [1], those were essentially the results.
Languages like Erlang or JoCaml (Ocaml plus joint calculus), whose models are inherently parallel yield simple and fast solutions.
Other languages too, but they require a lot of expert optimization and ugly code.
[1] https://www.tbray.org/ongoing/When/200x/2007/09/20/Wide-Find...
If you recall the famous Wide Finder benchmark for parallel programming by Tim Bray, back in 2007 as well [1], those were essentially the results.
Languages like Erlang or JoCaml (Ocaml plus joint calculus), whose models are inherently parallel yield simple and fast solutions.
Other languages too, but they require a lot of expert optimization and ugly code.
[1] https://www.tbray.org/ongoing/When/200x/2007/09/20/Wide-Find...
> That's true, but when I read that sentence I place the emphasis on should. I see it as aspirational.
Despite all the arguing I'm taking part in, this is probably the true and reasonable explanation. I read 'should' as 'if it doesn't there's a problem'.
Despite all the arguing I'm taking part in, this is probably the true and reasonable explanation. I read 'should' as 'if it doesn't there's a problem'.
Yes, indeed, don't forget his very next line is "Is this true? Almost. It’s early still, but we’re optimistic (Wildly optimistic; I haven’t seen this kind of optimism for the last 20 years!)."
> As Erlang requires message passing overhead by design
That's Erlang's main weakness. For parallel code, structurally shared data is immensely useful for processes running in the same address space. With Erlang, you have to give that up.
OTOH, not having this possibility makes distributing processes to other nodes a lot simpler.
That's Erlang's main weakness. For parallel code, structurally shared data is immensely useful for processes running in the same address space. With Erlang, you have to give that up.
OTOH, not having this possibility makes distributing processes to other nodes a lot simpler.
> If you have one of those
And how would you have one of those? Erlang encourages you to design and think about programs in a certain way, that typical C, Java or Python does not. I've seen Armstrong in one of his talks to actually emphasize the importance of learning distributed systems if you wanted Erlang to be useful for you. That's way beyond "embarrassingly parallel" point of view, Amdahl's law has no use here.
And how would you have one of those? Erlang encourages you to design and think about programs in a certain way, that typical C, Java or Python does not. I've seen Armstrong in one of his talks to actually emphasize the importance of learning distributed systems if you wanted Erlang to be useful for you. That's way beyond "embarrassingly parallel" point of view, Amdahl's law has no use here.
Obviously said "program" must be a server that services independent requests. I mean, not obviously, but if you work in telecom, web, etc. that's pretty much all programs you see.
I get that, but as I say we already knew how to scale those linearly in most programming languages. Erlang brought nothing new there.
It does have advances in things like reloading and supervision.
It does have advances in things like reloading and supervision.
Erlang was invented 30 years ago to solve problems that were not easily solvable by the back then prevalent programming languages. The claim that Erlang hasn’t brought anything new to the table couldn’t be further from the truth. Designing concurrent, fault-tolerant programs was not a problem solved in a satisfying way by the programming languages and technologies that were mainstream back then, which is exactly why Ericsson pursued the path of inventing their own programming language. I suggest everyone to read Joe Armstrong’s paper “Making reliable distributed systems in the presence of software errors”. It’s easy to read and mind expanding even if you do not intend to use Erlang. It can also save one from public embarrassment when one claims ridiculous things as parent and many other remarks in the other comments.
I acknowledged the areas where Erlang did bring advancements such as fault tolerance.
I read that paper, more than once, and his thesis, when I was doing my own PhD on parallelism, so I know the area and related work fairly well.
I don't think I'm embarrassing myself when I say that it's silly to claim that an arbitrary Erlang program should achieve linear scalability. But we don't need to debate with vague claims to originality - as I say Amdahl has done the maths and shows that it is only possible for a program that is embarrassingly parallel.
I read that paper, more than once, and his thesis, when I was doing my own PhD on parallelism, so I know the area and related work fairly well.
I don't think I'm embarrassing myself when I say that it's silly to claim that an arbitrary Erlang program should achieve linear scalability. But we don't need to debate with vague claims to originality - as I say Amdahl has done the maths and shows that it is only possible for a program that is embarrassingly parallel.
Concurrency is not parallelism. You conflate parallelism with concurrency in your comments both here and above. I should also mention that for a big part of its life the Erlang virtual machine has had no support for parallelism. Erlang was not invented to solve parallelism so your hypothesis is downright wrong. Parallelism was just a consequence of the concurrency which was not exploited by the Erlang virtual machine until a much later point. And yes, if you read the paper I have referred to in my previous comment you will happily find out exactly the reasons for why Erlang was invented and what sort of problems it has solved. Also, to clarify, I did not claim that you were wrong about Amdahl’s law; you are however completely wrong in your claim that Erlang has brought nothing new other than supervisors and hot code reloading to the table.
I think OP specifically talked about this assertion, "Your Erlang program should just run N times faster on an N core processor." This assertion is about parallelism, not concurrency.
I don't know what to say apart from that I disagree with how your describing my argument back to me.
Funny how often that's the only reasonable response. Nitpick: s/your/you're/. I do my part to stop language drift.
I searched many times the comparison of Erlang BEAM and JVM regarding performance but couldn't find much. May be it is not done, or maybe it is not publicly available. At my work some folks keep pushing RabbitMQ over ActiveMQ for no other good reason than it is in Erlang so we will get much better performance. I know it is not true but could not find web links to support that.
BEAM is slower than the JVM for sequential code but it doesn't have a block the world GC and it has preemptive multi processing. It'll have worse steady state performance but a lot of the worst cases that Java can have are heavily mitigated.
It's also realllly good at routing; the control logic is a somewhat slow more-or-less interpreted language speed but then the act of routing bytes around runs at native speeds (as long as you keep things in binaries and don't convert them to "strings"), with the Erlang framework making sure by the nature of how it works that there's no contention or anything getting in the way because everything's immutable.
It's sort of like how people talk about how NumPy can be so useful; Python is itself one of the slowest languages (especially since when it comes to NumPy we can't necessarily count on PyPy, though that's coming along) but it doesn't matter because you just use the slow language to script up the computation you want to do, which then goes and runs in highly-optimized assembler implementations. You can look at Erlang being very similar when being used in its native domain for networking, using a slower language to decide how to route bytes around.
So it's quite possible for competent Erlang to beat out competent Java for such a task. Java optimized to within an inch of its life can certainly beat Erlang, but it's going to be a much uglier code base to get there.
It's sort of like how people talk about how NumPy can be so useful; Python is itself one of the slowest languages (especially since when it comes to NumPy we can't necessarily count on PyPy, though that's coming along) but it doesn't matter because you just use the slow language to script up the computation you want to do, which then goes and runs in highly-optimized assembler implementations. You can look at Erlang being very similar when being used in its native domain for networking, using a slower language to decide how to route bytes around.
So it's quite possible for competent Erlang to beat out competent Java for such a task. Java optimized to within an inch of its life can certainly beat Erlang, but it's going to be a much uglier code base to get there.
You are making the error of comparing BEAM with a specific JVM implementation.
Not all available JVMs on the market have "a block the world GC" and the way multi-processing is done is implementation defined.
Not all available JVMs on the market have "a block the world GC" and the way multi-processing is done is implementation defined.
I could be wrong but I was under the impression that the four major JVM gc's all had a stop the world phase and while multi-processing is implementation specific I'm not aware of a robust pre-emptive multiprocessing model built for the JVM. If you have robust examples that are contrary to this I'd be interested in hearing about them.
Azul doesn't. It's not a differnt GC for the hotspot VM, it's a separate VM altogether
"Trivial" to write parallel programs in C?
You should stop doing whatever it is that you're doing, because you are in the top 0.00000000001% of programmers, in the Universe, and I want to work on whatever multi-billion dollar startup you undoubtedly have going on.
You should stop doing whatever it is that you're doing, because you are in the top 0.00000000001% of programmers, in the Universe, and I want to work on whatever multi-billion dollar startup you undoubtedly have going on.
I said it is trivial to parallelise a C program that implements an embarrassingly parallel algorithm. Embarrassingly parallel means no resources need to be shared, and there needs to be no communication, so no need for locks or messages, even in C. That's why I say it'd be trivial to parallelise it. Just add a pthread_start. Of course almost no applications are embarrassingly parallel.
And then you also need to implement mailboxes to receive new data, supervisors to watch the threads and revive them when they fail, initialization methods to get back to the original state, etc.
You can trivialize any issue if you ignore all the facts, but it doesn't make your argument have any correctness.
You can trivialize any issue if you ignore all the facts, but it doesn't make your argument have any correctness.
I'm really surprised at the arguing in this thread.
Embarrassingly parallel algorithms are trivial in pretty much every language. That is why is embarrassing.
>And then you also need to implement mailboxes to receive new data
If you did, it is not embarrassingly parallel.
You know, a huge amount of parallel work is done in C/C++. It was the language used for multiple parallel algorithms courses in my university. Being able to do basic MPI type stuff in C does not make you a top programmer. And not being able to do an embarrassingly parallel algorithm in C would ring alarm bells.
Embarrassingly parallel algorithms are trivial in pretty much every language. That is why is embarrassing.
>And then you also need to implement mailboxes to receive new data
If you did, it is not embarrassingly parallel.
You know, a huge amount of parallel work is done in C/C++. It was the language used for multiple parallel algorithms courses in my university. Being able to do basic MPI type stuff in C does not make you a top programmer. And not being able to do an embarrassingly parallel algorithm in C would ring alarm bells.
People's obsession with language choice often cloud their judgement.
Parallel programs really are not that hard to write. The hard part is figuring out how to make a problem into embarrassing parallel. Once that's figured out, the coding is simple. In this case, the language choice won't help.
> Parallel programs really are not that hard to write.
Yes... but!
> The hard part is figuring out how to make a problem into embarrassing parallel
The other hard part is the debugging - especially when you fall short in the "embarrassingly" department.
It's not until C11 that the language itself had any notion of multithreading without reaching out to a separate standard - POSIX - and various nonstandard compiler extensions required to implement it all (e.g. for memory barriers to constrain the reordering of code)
I've further found it distressingly frequent that in porting code from one platform to another that basic primitives are plainly broken to the point where I can write unit tests to catch the issue - usually leveraging e.g. ARM hardware and it's weaker memory ordering semantics, a topic which few fully understand.
For example, a "lock free" circular queue one might use to implement a task system to roll your embarassingly parallel programs atop of. Which were indeed lock free - and volatile free - and atomic free - and memory barrier free - and used in parallel from multiple threads...
Even the experts occasionally forget a mutex lock or something. I play catchup with additional static analysis annotations when new compilers expose new options to catch this kind of thing, but...
Yes... but!
> The hard part is figuring out how to make a problem into embarrassing parallel
The other hard part is the debugging - especially when you fall short in the "embarrassingly" department.
It's not until C11 that the language itself had any notion of multithreading without reaching out to a separate standard - POSIX - and various nonstandard compiler extensions required to implement it all (e.g. for memory barriers to constrain the reordering of code)
I've further found it distressingly frequent that in porting code from one platform to another that basic primitives are plainly broken to the point where I can write unit tests to catch the issue - usually leveraging e.g. ARM hardware and it's weaker memory ordering semantics, a topic which few fully understand.
For example, a "lock free" circular queue one might use to implement a task system to roll your embarassingly parallel programs atop of. Which were indeed lock free - and volatile free - and atomic free - and memory barrier free - and used in parallel from multiple threads...
Even the experts occasionally forget a mutex lock or something. I play catchup with additional static analysis annotations when new compilers expose new options to catch this kind of thing, but...
Static analysis is, of course, very restricted in what it can give you answers to, although I'd still love to have more advanced tools widely available.
What I really consider a game-changer for concurrent programming are the dynamic checkers integrated into recent versions of GCC and Clang (and the Go compilers, too). Those make it super easy to flag data races and don't have any false positives. Caveats apply, naturally, as they can only flag those races that actually happen during a particular execution of your code.
While Valgrind and similar tools could do that for years, the difference with TSan is speed. It's fast enough that you can use it all the time during development. Same for AddressSanitizer and the undefined behaviour checkers.
What I really consider a game-changer for concurrent programming are the dynamic checkers integrated into recent versions of GCC and Clang (and the Go compilers, too). Those make it super easy to flag data races and don't have any false positives. Caveats apply, naturally, as they can only flag those races that actually happen during a particular execution of your code.
While Valgrind and similar tools could do that for years, the difference with TSan is speed. It's fast enough that you can use it all the time during development. Same for AddressSanitizer and the undefined behaviour checkers.
You mistake concurrency with parallelism. If each of your "synchronous" call to a method is an asynchronous message, suddenly, all your program is embarasingly concurrent. Wait... was that what Alan Kay was saying with SmallTalk ? Would we have all lost our mind on RPC and synchronous calls for so long?
Yes. Yes we did.
Yes. Yes we did.
If every call is an async message but the next step in your algorithm always depends upon the result/outcome of that async message you are programming a sequential algorithm wrapped in concurrent clothing.
I would claim that even if you use Erlang and distribute it across multiple processes your program doesn't actually exhibit concurrency. If you want to argue otherwise that's fine, but you just push the problem out: your program is "concurrent" but you gain zero benefits of concurrency, and when you distribute it across multiple cores to try to make it run in parallel it might be concurrent but by definition it will not be parallel.
Either way you want to slice it you have a big problem with this line of thought.
Edit: from my experience it is easy to write Erlang programs that exhibit something not exactly the same as, but similar to, this sort of behavior.
I would claim that even if you use Erlang and distribute it across multiple processes your program doesn't actually exhibit concurrency. If you want to argue otherwise that's fine, but you just push the problem out: your program is "concurrent" but you gain zero benefits of concurrency, and when you distribute it across multiple cores to try to make it run in parallel it might be concurrent but by definition it will not be parallel.
Either way you want to slice it you have a big problem with this line of thought.
Edit: from my experience it is easy to write Erlang programs that exhibit something not exactly the same as, but similar to, this sort of behavior.
"Embarrassingly concurrent" is not embarrassingly parallel. Parallelism is not merely making things concurrent nor making synchronous calls into concurrent async calls.
once you are concurrent, what stop you from being distributed to multiple cores ?
>> Your Erlang program should just run N times faster on an N core processor
> But only if your program is embarrassingly parallel with at least N times available parallelism in the first place! If you have one of those it's already trivial to write a version that runs N times faster on N cores in C, Java, multi-process Python, whatever.
You've made two claims, one irrelevant and one false:
1. only if your program is embarrassingly parallel is irrelevant, because a almost every program is embarrassingly parallel in Erlang. The language is built around concurrency to the point that parts which wouldn't be obviously parallel in another language are in Erlang. Further, slow hashes in crypto have taught us that is actually quite difficult to make something which can't be parallelized.
2. it's trivial to write a program that's faster in X language I'm not sure how toot define trivial, but I've yet to find a language that can communicate between threads as performant-ly. Even languages like Clojure which use similar thread semantics can't do what Erlang can because the underlying threads aren't as lightweight. Spinning up a million threads in Erlang isn't even unusual, whereas in any of the languages you mention it's either crippling-ly slow (Python or Java) or very difficult to synchronize (C most, but Python and Java aren't easy).
> But only if your program is embarrassingly parallel with at least N times available parallelism in the first place! If you have one of those it's already trivial to write a version that runs N times faster on N cores in C, Java, multi-process Python, whatever.
You've made two claims, one irrelevant and one false:
1. only if your program is embarrassingly parallel is irrelevant, because a almost every program is embarrassingly parallel in Erlang. The language is built around concurrency to the point that parts which wouldn't be obviously parallel in another language are in Erlang. Further, slow hashes in crypto have taught us that is actually quite difficult to make something which can't be parallelized.
2. it's trivial to write a program that's faster in X language I'm not sure how toot define trivial, but I've yet to find a language that can communicate between threads as performant-ly. Even languages like Clojure which use similar thread semantics can't do what Erlang can because the underlying threads aren't as lightweight. Spinning up a million threads in Erlang isn't even unusual, whereas in any of the languages you mention it's either crippling-ly slow (Python or Java) or very difficult to synchronize (C most, but Python and Java aren't easy).
I've shipped Erlang code and spoken at Erlang events and even I don't believe that every program in Erlang is embarrassingly parallel. There are very real limits on what you can do.
Erlang has never been a language about amazing performance. It's been about an environment and library that delivers amazing distributed concurrency with shockingly little effort.
This is an amazing accomplishment and should not be undersold. In making unsupported claims about how Erlang has magical parallelism sauce that somehow ignores Ahmdal's law we ignore the real benefits for a fiction.
Erlang has never been a language about amazing performance. It's been about an environment and library that delivers amazing distributed concurrency with shockingly little effort.
This is an amazing accomplishment and should not be undersold. In making unsupported claims about how Erlang has magical parallelism sauce that somehow ignores Ahmdal's law we ignore the real benefits for a fiction.
>only if your program is embarrassingly parallel is irrelevant, because a almost every program is embarrassingly parallel in Erlang. The language is built around concurrency to the point that parts which wouldn't be obviously parallel in another language are in Erlang. Further, slow hashes in crypto have taught us that is actually quite difficult to make something which can't be parallelized.
OK. Not an Erlang user, but I can't let this statement go.
I've studied parallel numerical algorithms. Many/most of them will involve blocking because you're waiting for results from other nodes.
If you're saying Erlang has somehow found a way to do those numerical algorithms without having to wait, then I'd love to see all those textbooks rewritten.
Amdahl's Law reigns supreme.
OK. Not an Erlang user, but I can't let this statement go.
I've studied parallel numerical algorithms. Many/most of them will involve blocking because you're waiting for results from other nodes.
If you're saying Erlang has somehow found a way to do those numerical algorithms without having to wait, then I'd love to see all those textbooks rewritten.
Amdahl's Law reigns supreme.
I'm not devishard, but I parsed his statement slightly differently. He's not saying that Erlang makes things parallel magically. Rather, he's saying that Erlang forces tasks that /could/ be parallel to be parallel by default. Thus, Erlang will tend to maximize the sections of your program that are run in parallel compared to other languages.
Which would make the original commenter's point valid:
>Your Erlang program should just run N times faster on an N core processor
No, it won't. It will only be true for tasks that /could/ be (completely/embarrasingly) parallel (as you say). Which is kind of circular.
>Your Erlang program should just run N times faster on an N core processor
No, it won't. It will only be true for tasks that /could/ be (completely/embarrasingly) parallel (as you say). Which is kind of circular.
I'm interpreting "embarrassingly parallel" to mean that it's obvious the task can be parallelized, and I'm saying that many tasks where this isn't obvious in a more serial language are obvious in Erlang.
No, I'm not claiming Erlang breaks Amdahl's Law. I'm claiming that Amdahl's Law applies less often than people think it does.
No, I'm not claiming Erlang breaks Amdahl's Law. I'm claiming that Amdahl's Law applies less often than people think it does.
Not really. Think about every object you'd have in Java that's being passed around your system. Now imagine each of those objects are their own processes and you're passing around references to them.
Just on that one case, you've taken huge chunks of a linear execution pattern and parallelized it. Now make that your norm and amplify it to everything. Now realize that the message passing allows this mode of operation to spread each part of this workload over not only more cores but more machines across the network.
And then realize that you can deploy updates to this codes individual parts while other parts continue running without taking down the whole system.
Then you get Erlang.
Just on that one case, you've taken huge chunks of a linear execution pattern and parallelized it. Now make that your norm and amplify it to everything. Now realize that the message passing allows this mode of operation to spread each part of this workload over not only more cores but more machines across the network.
And then realize that you can deploy updates to this codes individual parts while other parts continue running without taking down the whole system.
Then you get Erlang.
None of what you said will prevent the need for waiting for the majority of numerical algorithms.
No one's disputing Erlang's prowess at parallelism. What the critic in this thread was saying was that you can only get Nx speedup on an N core processor for a limited set of algorithms. Most parallel algorithms will not fall in this category. Amdahl's law is a general truth - it doesn't matter what your architecture/language is. There is nothing special about Erlang that will make any parallel algorithm scale linearly with nodes.
No one's disputing Erlang's prowess at parallelism. What the critic in this thread was saying was that you can only get Nx speedup on an N core processor for a limited set of algorithms. Most parallel algorithms will not fall in this category. Amdahl's law is a general truth - it doesn't matter what your architecture/language is. There is nothing special about Erlang that will make any parallel algorithm scale linearly with nodes.
On top of that, Erlang has a scheduler which reduces the need to rely on the operating system's concurrency model.
The prescheduler caps the execution window for anything that would block. If one piece of the system would take a long time to finish, it doesn't interfere with the other parts of the system completing their work on schedule. That one blocking piece will finish more slowly, but ever other moving part in the system will keep responding as expected.
>If one piece of the system would take a long time to finish, it doesn't interfere with the other parts of the system completing their work on schedule. That one blocking piece will finish more slowly, but ever other moving part in the system will keep responding as expected.
That's how it works in pretty much any language that supports message passing. I used to do MPI programming in C. Nothing you said is not true for MPI in C. Later I did some MPI in Python. True there as well. If you have MPI in any language, it is true for that language.
That's how it works in pretty much any language that supports message passing. I used to do MPI programming in C. Nothing you said is not true for MPI in C. Later I did some MPI in Python. True there as well. If you have MPI in any language, it is true for that language.
My understanding is that those languages rely on cooperative scheduling within a thread, meaning that the running code has to relinquish control to the scheduler. Threads themselves are prescheduled at the OS layer but OS threads are much heavier and limited in how many can be running. A Java thread is 1024kb for example, compared to an Erlang process that's 0.5kb.
>My understanding is that those languages rely on cooperative scheduling within a thread
MPI doesn't require threads.
I think people here are confusing parallel programming with multithreaded programming. One is a subset of the other.
MPI doesn't require threads.
I think people here are confusing parallel programming with multithreaded programming. One is a subset of the other.
I'm not saying Amdahl's Law is wrong, I'm saying that it doesn't apply to as many problems as people think it does. I said "almost" for a reason.
> almost every program is embarrassingly parallel in Erlang
I can't agree with that, and it's key to my point. There are some problems which we just don't know how to make embarrassingly parallel. Take something classic like mesh triangulation or mesh refinement. Nobody knows how to make those embarrassingly parallel. If you write it in Erlang, it's still not going to be embarrassingly parallel. And it won't scale linearly to N times faster on N cores no matter which language you write it in.
So it's just not true to say that any Erlang program should scale linearly. If nobody on earth knows how to make mesh refinement scale linearly, how will Erlang do it?
Maybe you mean you wouldn't choose to write those programs in Erlang? Well then I think it's a meaningless claim to say Erlang will linearly scale your program, but only if it is a program which is naturally linearly scalable anyway. Erlang hasn't helped you do anything there so why make a claim about it?
I can't agree with that, and it's key to my point. There are some problems which we just don't know how to make embarrassingly parallel. Take something classic like mesh triangulation or mesh refinement. Nobody knows how to make those embarrassingly parallel. If you write it in Erlang, it's still not going to be embarrassingly parallel. And it won't scale linearly to N times faster on N cores no matter which language you write it in.
So it's just not true to say that any Erlang program should scale linearly. If nobody on earth knows how to make mesh refinement scale linearly, how will Erlang do it?
Maybe you mean you wouldn't choose to write those programs in Erlang? Well then I think it's a meaningless claim to say Erlang will linearly scale your program, but only if it is a program which is naturally linearly scalable anyway. Erlang hasn't helped you do anything there so why make a claim about it?
"almost every program" != "any program"
[deleted]
WTF. How do you turn a problem which cannot be reduced into parallel sub-components into an Erlang program then? Are there just a huge class of computations that cannot be done with Erlang? Of course not.
Of course you can implement an inherently serial algorithm in Erlang. It just won't run N times faster on N cores.
> I've yet to find a language that can communicate between threads as performant-ly.
Minor nitpick: Erlang uses processes, not threads. Your comment doesn't explicitly say that Erlang uses threads though, so perhaps I'm being overly pedantic.
Minor nitpick: Erlang uses processes, not threads. Your comment doesn't explicitly say that Erlang uses threads though, so perhaps I'm being overly pedantic.
But an Erlang process is not the same as an OS process, or even an OS thread. It's managed by the Erlang VM and is very lightweight.
I posted this in another thread on what I like about Erlang (I'll shamelessly copy and paste it here):
* A better concurrency model. Lightweight processes vs callbacks and futures. Or even channels. I find processes as concurrency units maps better to problems I had to solve. So there is less impedance mismatch. That makes a huge deal in a larger project. (eg.: a user request as an isolated process vs a chain of callbacks). OS design got this right years ago -- think how most modern popular operating systems represent concurrency - an isolated process.
* Safety & fault tolerance. Processes are isolated. So you can build very robust systems. That simply puts money in your pocket just due to ops costs. A non-critical / experimental part of backend is crashing at 4am and restarting? No, problem, keep sleeping. Rest of the service will stay up and you can fix it in the morning.
* Debuggability and inspection. BEAM VM comes with built in ability to debug, inspect and trace out of the box. That has saved so much time (and money) over the years.
* Hot code reload. This is a first class feature. We don't rely on it to do upgrades. But it proved invaluable to fix an issues or two for a high value customer without taking down the services. Or simply to add extra logging to monitor a pathological edge case.
Some people might prefer Elixir and that's fine, they are great friends. I personally like Erlang, I think it is simpler. Someone new coming from Python or Ruby might like it better and also it has very beginner friendly community. Jose Valim and and other authors really put user-friendliness and approach-ability at the forefront. I really like that, mad props to them.
* A better concurrency model. Lightweight processes vs callbacks and futures. Or even channels. I find processes as concurrency units maps better to problems I had to solve. So there is less impedance mismatch. That makes a huge deal in a larger project. (eg.: a user request as an isolated process vs a chain of callbacks). OS design got this right years ago -- think how most modern popular operating systems represent concurrency - an isolated process.
* Safety & fault tolerance. Processes are isolated. So you can build very robust systems. That simply puts money in your pocket just due to ops costs. A non-critical / experimental part of backend is crashing at 4am and restarting? No, problem, keep sleeping. Rest of the service will stay up and you can fix it in the morning.
* Debuggability and inspection. BEAM VM comes with built in ability to debug, inspect and trace out of the box. That has saved so much time (and money) over the years.
* Hot code reload. This is a first class feature. We don't rely on it to do upgrades. But it proved invaluable to fix an issues or two for a high value customer without taking down the services. Or simply to add extra logging to monitor a pathological edge case.
Some people might prefer Elixir and that's fine, they are great friends. I personally like Erlang, I think it is simpler. Someone new coming from Python or Ruby might like it better and also it has very beginner friendly community. Jose Valim and and other authors really put user-friendliness and approach-ability at the forefront. I really like that, mad props to them.
Dave Thomas has described Elixir as follows: "Elixir took the Erlang virtual machine, BEAM, and put a sensible face on it. It gives you all the power of Erlang plus a powerful macro system."[1]
[1] https://startlearningelixir.com/elixir-for-rubyists
[1] https://startlearningelixir.com/elixir-for-rubyists
Erlang is a great programming language after one takes the time to understand the principles underlying it and its design. Everything fits very nicely into place. There are definitely things that could be done better, but then that’s the case with most programming languages and technologies after they have accumulated some dust. Elixir on the other hand, in my personal opinion, is a false prophet simply because it looks like something which it isn’t. It looks like Ruby and gives the programmer the feeling that he’s right at home, except it is in fact a very different beast with very different semantics, Erlang semantics, as opposed to Ruby. Therefore, if you want to program on the BEAM (the Erlang virtual machine) you better make sure you actually understand the system, its design, principles and also some of the semantics of Erlang itself, point at which you might as well just learn Erlang. I’m not saying that you cannot learn those things coming from Elixir, I’m just saying that if you want to build systems of reasonable complexity (like the ones that Erlang is known for) on top of the Erlang virtual machine, as opposed to CRUD web applications, then you must understand a lot more than just Elixir or Phoenix.
This is a big part of the reason why I appreciate having learned at least the basics of Erlang before moving on to Elixir, having come from Ruby. It helped establish an appreciation in my brain for declarative programming - something which is possible to do in Ruby, but nowhere close to the extent possible in Erlang or Elixir (or any other language with a similar approach to "pattern matching" as a first-class programming paradigm).
Of course, I tend to feel much more productive in Elixir than Erlang, and thus use Elixir way more often than Erlang directly, but I'm able to come at Elixir with a more Erlangy perspective and approach.
Of course, I tend to feel much more productive in Elixir than Erlang, and thus use Elixir way more often than Erlang directly, but I'm able to come at Elixir with a more Erlangy perspective and approach.
Elixir is prettier than Erlang but it really mess up on certain things.
Erlang have pattern matching via function with same name and you can tell if it's a group of pattern matching with semicolon and period. But with Elixir you can't tell it's just def and end.
Erlang's:
-module(recursive).
-export([fac/1]).
fac(N) when N == 0 -> 1;
fac(N) when N > 0 -> N * fac(N-1).
You can tell fac(N) both are in a group of pattern matching cause ';' and '.'. The '.' denote the last pattern matching function.
Elixir:
defmodule Factorial do
You can't tell because there is no ';' and '.'. This is a trivial case but when your Elixir's module have a tons of function in it, this issue become relevant.
Erlang have pattern matching via function with same name and you can tell if it's a group of pattern matching with semicolon and period. But with Elixir you can't tell it's just def and end.
Erlang's:
-module(recursive).
-export([fac/1]).
fac(N) when N == 0 -> 1;
fac(N) when N > 0 -> N * fac(N-1).
You can tell fac(N) both are in a group of pattern matching cause ';' and '.'. The '.' denote the last pattern matching function.
Elixir:
defmodule Factorial do
def of(0), do: 1
def of(n), do: n * of(n-1)
You can't tell because there is no ';' and '.'. This is a trivial case but when your Elixir's module have a tons of function in it, this issue become relevant.
I honestly don't understand how this messes anything up... the elixir syntax produces something semantically equivalent with a lot less garbage. Opinions may differ I guess, but I find your Elixir example much easier to read than your Erlang example (I program in Elixir, so that's not much of a datapoint).
Most of the garbage is because those aren't the closest equivalents.
This is the closest Erlang equivalent:
This is the closest Erlang equivalent:
fac(0) -> 1;
fac(N) -> N * fac(N-1).It's not only that. I think the bigger issue is that because of it's Ruby-like syntax it encourages newcomers to think in Ruby rather than in Erlang. Rather than emphasising what's the core of Erlang it attracts people with it's syntax, which is the least important aspect of a language.
I think you make the Erlang definition look unnecessarily awful.
fac(0) -> 1;
fac(N) -> N * fac(N-1).
Or is there a particular advantage to write it using guards?
fac(0) -> 1;
fac(N) -> N * fac(N-1).
Or is there a particular advantage to write it using guards?
It's an example from http://learnyousomeerlang.com
having a semantically significant distinction between ; and . is unnecessarily awful. That kind of crap is one of the reasons Elixir has so much support.
What you said is equivalent to a Python programmer saying that adding semantical value to the “end” keyword in Ruby is unnecessarily awful because you can achieve the same by giving meaning to the indentation level. It’s an entirely subjective matter.
I would also like to point out that Erlang has also very good support and has seen adoption in some very critical systems as opposed to CRUD web applications which is the main domain of Elixir. Most of Ericsson’s products use Erlang to a certain degree, there are a lot of banking systems and aviation systems which make use of Erlang as well, quite a few Internet companies use it to great success, and many more.
And by the way, the semantic meaning of “;” and “.” is an awful lot similar to their use in the English language, you are blowing it out of proportions. This is a trivial thing which you learn after a 10 minutes introduction to Erlang. For me, personally, if one has a problem understanding the meaning of “;” and “.” or learning a new syntax for that matter, I can easily conclude that I probably should not give that person any decision power in designing systems.
I would also like to point out that Erlang has also very good support and has seen adoption in some very critical systems as opposed to CRUD web applications which is the main domain of Elixir. Most of Ericsson’s products use Erlang to a certain degree, there are a lot of banking systems and aviation systems which make use of Erlang as well, quite a few Internet companies use it to great success, and many more.
And by the way, the semantic meaning of “;” and “.” is an awful lot similar to their use in the English language, you are blowing it out of proportions. This is a trivial thing which you learn after a 10 minutes introduction to Erlang. For me, personally, if one has a problem understanding the meaning of “;” and “.” or learning a new syntax for that matter, I can easily conclude that I probably should not give that person any decision power in designing systems.
How does that matter? Anyone who uses this defines them in groups one after another. You aren't looking at the punctuation at the end of the line, you're looking at the following line to see if it's the same function name.
It matter because you're assuming they will group it. IIRC Erlang forces you to group it.
It's an incredibly safe assumption to make, though. This isn't enough of a problem in practice to amount to a negative for elixir or a pro for erlang.
I strongly agree. I've found it tremendously convenient to have an "alien" syntax (Erlang) to help me think Erlang-y.
(Now, of course, when I see more "traditional" C/Ruby/Perl/Python/Rust syntax, I can't help but roll my eyes, but that's a slightly different subject.)
(Now, of course, when I see more "traditional" C/Ruby/Perl/Python/Rust syntax, I can't help but roll my eyes, but that's a slightly different subject.)
In other words, Elixir is the rocket that will make Erlang go big.
Well, it's a skin on the rocket that will make it big. While they generally say that Elixir runs on top of the Erlang VM there happens to be a big fat Erlang/OTP layer in-between which Elixir and its libraries make full use of. This is just smart of course but not mentioning doesn't paint the whole picture.
I'd say Elixir is the skin, like you say: without it, all your rocket pieces are out in the open and not particularly aerodynamic, and they'll start breaking off and you won't be going to space today. There are other skins, though, like LFE and Erlang itself.
The Erlang VM is the engine. You can make a beautiful, sleek, aerodynamic rocket, but it ain't going anywhere unless you literally light a fire under its butt. It helps that BEAM helps the rocket go fast :)
Of course, you need some sort of structural piece to hold your engine and your aerodynamic skin in place, lest your rocket crumples itself up into a ball at launch. That's where OTP comes in: providing a robust structure for your rocket.
Last but not least, you need to launch it. Rebar and Mix work reasonably well as launch clamps in this really contrived metaphor. You also have exrm (or Distillery nowadays, I guess) that works as the VAB, in which your rocket is put together so it can be launched.
The Erlang VM is the engine. You can make a beautiful, sleek, aerodynamic rocket, but it ain't going anywhere unless you literally light a fire under its butt. It helps that BEAM helps the rocket go fast :)
Of course, you need some sort of structural piece to hold your engine and your aerodynamic skin in place, lest your rocket crumples itself up into a ball at launch. That's where OTP comes in: providing a robust structure for your rocket.
Last but not least, you need to launch it. Rebar and Mix work reasonably well as launch clamps in this really contrived metaphor. You also have exrm (or Distillery nowadays, I guess) that works as the VAB, in which your rocket is put together so it can be launched.
Except that “rockets” built with Erlang have been aerodynamic, production-ready, and deployed some 20 years before Elixir has even existed and they are still in use today. They did not start breaking off and, yes, they actually went to space, as opposed to what you imply in a very sneaky way in your first statement. If you want to be fair, you can say that Elixir is another skin of Erlang, the other way around does not hold.
That's exactly what I said, though: "There are other skins, though, like LFE and Erlang itself".
I personally consider Elixir to have fewer aerodynamic bumps and other sources of drag than Erlang itself, but that's just, like, my opinion, man. Obviously people will disagree with that, and that's fine.
I personally consider Elixir to have fewer aerodynamic bumps and other sources of drag than Erlang itself, but that's just, like, my opinion, man. Obviously people will disagree with that, and that's fine.
Wow, rvirding! Thank you for everything. Distributed software is actually a pleasure to write thanks in large part to your the many years of work.
The rocket analogy is a good one, too. What Erlang accomplishes is massive, and Elixir uses and benefits from all of it.
I don't want to downplay what Elixir accomplishes, either ... to use a SpaceX analogy, Elixir and its ecosystem are like the guidance systems and deployable struts that let you land rockets repeatably on a barge in the middle of the ocean. It uses macros to make solving problems with rockets easier and more widespread, and maybe it also increases our ambition because of it. (I'm thinking about projects like phoenix_pubsub especially Tracker, ecto, the 1.4 registry, etc).
The rocket analogy is a good one, too. What Erlang accomplishes is massive, and Elixir uses and benefits from all of it.
I don't want to downplay what Elixir accomplishes, either ... to use a SpaceX analogy, Elixir and its ecosystem are like the guidance systems and deployable struts that let you land rockets repeatably on a barge in the middle of the ocean. It uses macros to make solving problems with rockets easier and more widespread, and maybe it also increases our ambition because of it. (I'm thinking about projects like phoenix_pubsub especially Tracker, ecto, the 1.4 registry, etc).
Alan Kay once said (at an Erlang conference):
"By the time the software engineering of a language gets in good shape, the language has become obsolete in terms of 'needed expressiveness'. [Therefore] in a 'Real' computer science, the best languages should serve as the 'assembly code' for the next generation of expression!" [1]
He then proposed the hypothetical question: "What should Erlang be an assembly code for?" [2]
I'm not sure if he knew about Elixir..
[1] https://youtu.be/fhOHn9TClXY?t=30m39s [2] https://youtu.be/fhOHn9TClXY?t=36m49s
"By the time the software engineering of a language gets in good shape, the language has become obsolete in terms of 'needed expressiveness'. [Therefore] in a 'Real' computer science, the best languages should serve as the 'assembly code' for the next generation of expression!" [1]
He then proposed the hypothetical question: "What should Erlang be an assembly code for?" [2]
I'm not sure if he knew about Elixir..
[1] https://youtu.be/fhOHn9TClXY?t=30m39s [2] https://youtu.be/fhOHn9TClXY?t=36m49s
There's also a very nice video explaining certain aspects of Elixir by Dave Thomas: https://youtu.be/5hDVftaPQwY
There’s also “Erlang the Movie”; the designers of Erlang demo some of the qualities of the Erlang system in a delightful way: https://youtu.be/xrIjfIjssLE
It says Elixir uses the Actor model, but doesn't Erlang use Communicating Sequential Processes (CSP)?
Not really.
Experience after designing and deploying systems that run 100,000 transactions/sec per box: don't bother with engineers that cannot achieve proficiency in Erlang. Elixir is a crutch, and not something you want to show up with in 400 meters hurdles race. You will look pathetic.
Experience after designing and deploying systems that run 100,000 transactions/sec per box: don't bother with engineers that cannot achieve proficiency in Erlang. Elixir is a crutch, and not something you want to show up with in 400 meters hurdles race. You will look pathetic.
https://blog.discordapp.com/how-discord-handles-push-request...
Erlang has its uses and Elixir has its uses. Dismissing Elixir's userbase as 'pathetic' is pretty rude and a disservice to a pretty incredible language. See the Discord article where they clearly demonstrate Elixir and Genstage are ready for production and can be used to write performant services.
I get that you really like Erlang and its quirks, but maybe there are more mature ways of expressing this love than calling Elixir users 'pathetic'.
Erlang has its uses and Elixir has its uses. Dismissing Elixir's userbase as 'pathetic' is pretty rude and a disservice to a pretty incredible language. See the Discord article where they clearly demonstrate Elixir and Genstage are ready for production and can be used to write performant services.
I get that you really like Erlang and its quirks, but maybe there are more mature ways of expressing this love than calling Elixir users 'pathetic'.
Unrelated to your discussion with the parent, but I just wanted to point out that the fact that a system which was implemented in Elixir and runs on the Erlang virtual machine is able to sustain that traffic (and here I am referring to the link you posted) is and should be regarded as, largely, a quality of the Erlang virtual machine, and only to a very small extent to the programming language that it has been implemented in, if any at all. The fact that a system implemented in Elixir can sustain a large quantity of traffic while running on the Erlang virtual machine does not prove much about Elixir itself, other than the fact that it’s usable; you can implement the exact same system in Erlang also, or any other programming language for that matter. There is no inherent quality of Elixir itself that makes programs handle large amounts of traffic. That’s a quality of the Erlang virtual machine; Elixir is just syntax.
Thanks for your clarification. I think 'just syntax' is a bit dismissive of the importance that well-designed syntax has on being able to reason clearly in a certain way. Elixir's "just syntax" has made it so easy for me to understand FP concepts like pattern matching and higher order functions. I've struggled with these concepts before while trying to learn Haskell, Erlang and many other FP languages. Does that make me a bad programmer? The parent commenter would certainly think so.
I totally take your point that everything written in Elixir could be written in Erlang. That much is a fact. However, my point in saying they serve different uses is saying that better syntax in itself isn't nothing. If a language enables many people to grasp actor-based concurrency in an easy to understand fashion, can we reduce that to "just syntax"? I guess my point is that introducing syntactic clarity is in itself a massive feature that shouldn't be dismissed.
I hope that further explains my intent!
I totally take your point that everything written in Elixir could be written in Erlang. That much is a fact. However, my point in saying they serve different uses is saying that better syntax in itself isn't nothing. If a language enables many people to grasp actor-based concurrency in an easy to understand fashion, can we reduce that to "just syntax"? I guess my point is that introducing syntactic clarity is in itself a massive feature that shouldn't be dismissed.
I hope that further explains my intent!
My point was not that everything which is written in Elixir can be written in Erlang but mostly that using a reference to a system that sustains a large amount of traffic which is implemented in Elixir and runs on top of the Erlang virtual machine as an argument for how incredible Elixir is, is naive at best, as Elixir is just a layer on top of many Erlang libraries and also the Erlang virtual machine. Yes, syntax can matter, but you can discuss that later when you discuss maintainability and other software design and maintenance issues. You cannot discuss syntax and then argue that your program sustains large amounts of traffic; you can, but it’s pointless.
And to answer your first question, although I thought I was very clear that I do not want to take part in your discussion with the parent: no, I do not think that you are a bad programmer for learning Elixir, I think if you were to only learn Elixir, call it a day, and then spread the word of how incredible Elixir is, you would be, however. Programming languages are just that, programming languages. The more you know the better and there is no silver bullet. As you will get back to Haskell you will probably come to appreciate its powers. Also, the fact that you have not grasped Haskell’s syntax when you have first tried does not make it a poor functional programming language and it certainly does not make Elixir a better designed functional programming language. That’s just your perception because Elixir was, for you, easier to learn. The fact that I cannot comprehend particle physics does not necessarily make particle physics a badly designed model.
And to answer your first question, although I thought I was very clear that I do not want to take part in your discussion with the parent: no, I do not think that you are a bad programmer for learning Elixir, I think if you were to only learn Elixir, call it a day, and then spread the word of how incredible Elixir is, you would be, however. Programming languages are just that, programming languages. The more you know the better and there is no silver bullet. As you will get back to Haskell you will probably come to appreciate its powers. Also, the fact that you have not grasped Haskell’s syntax when you have first tried does not make it a poor functional programming language and it certainly does not make Elixir a better designed functional programming language. That’s just your perception because Elixir was, for you, easier to learn. The fact that I cannot comprehend particle physics does not necessarily make particle physics a badly designed model.
Thanks for your reply! All points you make are incredibly valid, but I still have some pushback:
But where would you stop?
If Elixir makes it easier to write fast, concurrent for many people than Erlang, that totally can be discussed. The way you separate developer satisfaction/well-designed syntax from the VM it runs on top of is a separation I dispute.
Obviously these comparisons will never be 100% valid, but I think they make my point: Will you say that Scala is no different (or better) from Java? Will you say that Dart or Elm are no different (or better) from JavaScript? Simply because their underlying architecture is the same?
I think there is some hostility I am sensing from Erlang programmers where there is none given in return. I don't hate Erlang. I'm simply saying a language that can leverage the performance of the Erlang VM shouldn't be dismissed. Again, I never ever said I thought Elixir was better than Erlang. I was just responding to the parent commenter's assertion that all alchemists are pathetic.
Also, I think the implication that may have entered my writing was that because I personally found Elixir easier to learn (and it would be difficult to argue that I am alone in this), it is somehow a better language than Haskell, Erlang, and many other FP languages out there. If that was the implication you drew, I apologise for my a mistake in my writing. Since I learned Elixir, I'm now taking a course in Haskell at my college to better understand these programming languages. Just because I called Elixir incredible, doesn't mean I believe it's the end all be all language!
Again, thanks for your comment. I learned a lot from it.
But where would you stop?
If Elixir makes it easier to write fast, concurrent for many people than Erlang, that totally can be discussed. The way you separate developer satisfaction/well-designed syntax from the VM it runs on top of is a separation I dispute.
Obviously these comparisons will never be 100% valid, but I think they make my point: Will you say that Scala is no different (or better) from Java? Will you say that Dart or Elm are no different (or better) from JavaScript? Simply because their underlying architecture is the same?
I think there is some hostility I am sensing from Erlang programmers where there is none given in return. I don't hate Erlang. I'm simply saying a language that can leverage the performance of the Erlang VM shouldn't be dismissed. Again, I never ever said I thought Elixir was better than Erlang. I was just responding to the parent commenter's assertion that all alchemists are pathetic.
Also, I think the implication that may have entered my writing was that because I personally found Elixir easier to learn (and it would be difficult to argue that I am alone in this), it is somehow a better language than Haskell, Erlang, and many other FP languages out there. If that was the implication you drew, I apologise for my a mistake in my writing. Since I learned Elixir, I'm now taking a course in Haskell at my college to better understand these programming languages. Just because I called Elixir incredible, doesn't mean I believe it's the end all be all language!
Again, thanks for your comment. I learned a lot from it.
Why would you stop learning?
You misunderstand me. I did not say that Elixir is not different from Erlang, what I said already two times and this is the third time is that you cannot send me to some case study about how code deployed on the Erlang virtual machine performs well at high traffic and imply that the performance seen is a quality of Elixir, because it is not. Can we agree on this? That is the one and only point that I was trying to make, and if you read carefully and want to understand I think I have expressed that thought in a quite clear way. It is also a very factual truth, not a subjective matter, when we discuss syntax preference we do not discuss throughput because throughput is not a quality of the syntax. Even less so in the case of Elixir which runs on top of the Erlang virtual machine. Performance could be discussed as a function of syntax if, for instance, Elixir compiled to virtual machine instructions for the Erlang virtual machine and then had its own layer of optimizations etc. and then there would be some inherent quality of the Elixir syntax which makes those optimizations easier. But the fact of the matter is that Elixir compiles to Erlang which is then compiled to instructions for the virtual machine. Yes, Elixir it’s just syntax, I suggest you read its source code and not take my word for it. There is nothing incredible that Elixir has invented, it just looks like Ruby which made it easier to read by ex-Ruby programmers, that’s all.
Yes, of course I am biased to have a problem towards Elixir programmers making extraordinary claims when they lack cultural knowledge and brag about things that have not been invented by Elixir in any way but are qualities of a system which they very much enjoy criticizing in ignorance. I have even heard Elixir programmers talking about a so-called Elixir virtual machine which I find amazing and tells me a lot about their community and culture.
You misunderstand me. I did not say that Elixir is not different from Erlang, what I said already two times and this is the third time is that you cannot send me to some case study about how code deployed on the Erlang virtual machine performs well at high traffic and imply that the performance seen is a quality of Elixir, because it is not. Can we agree on this? That is the one and only point that I was trying to make, and if you read carefully and want to understand I think I have expressed that thought in a quite clear way. It is also a very factual truth, not a subjective matter, when we discuss syntax preference we do not discuss throughput because throughput is not a quality of the syntax. Even less so in the case of Elixir which runs on top of the Erlang virtual machine. Performance could be discussed as a function of syntax if, for instance, Elixir compiled to virtual machine instructions for the Erlang virtual machine and then had its own layer of optimizations etc. and then there would be some inherent quality of the Elixir syntax which makes those optimizations easier. But the fact of the matter is that Elixir compiles to Erlang which is then compiled to instructions for the virtual machine. Yes, Elixir it’s just syntax, I suggest you read its source code and not take my word for it. There is nothing incredible that Elixir has invented, it just looks like Ruby which made it easier to read by ex-Ruby programmers, that’s all.
Yes, of course I am biased to have a problem towards Elixir programmers making extraordinary claims when they lack cultural knowledge and brag about things that have not been invented by Elixir in any way but are qualities of a system which they very much enjoy criticizing in ignorance. I have even heard Elixir programmers talking about a so-called Elixir virtual machine which I find amazing and tells me a lot about their community and culture.
You literally just suggested that someone who writes semantically identical code in a more readable language will look pathetic to you. I think you're confusing nerd-cred with engineering ability. For the record, I will employ the person who prefers to write in Elixir but can work with legacy Erlang before I will employ someone who thinks there's special magic in an obtuse syntax...
I'd say it's extremely subjective which language is more readable, for some that might be Elixir and for others it's Erlang.
A practical language based on well-researched first principles. Details in the armstrong_thesis_2003.pdf
* java is unsuitable
* no pthreads, please (wrong concept)
* immutable data (no locks - no problem)
* isolated lightweight processes (share nothing)
* communication by message-passing (grom Kay's OOP)
* dynamic typing (good-enough, quick prototyping)
* pattern matching (on receive)
* supervision hierarchies (fault tolerance)
* reusable components for server development
I've not found immutable data to be the big win people tout it as. In most of what I do acting on stale data is just as bad, which means I wind up needing a lock anyway.
Doing a < b and taking an action based on that result is as harmful with stale data.
Doing a < b and taking an action based on that result is as harmful with stale data.
I'd like to tell you that stale data will always be a thing in any serious application because light only travels so fast. Therefore, you'll need to have sensible logic in the face of stale data either way, so it's not so bad as you think.
...and while that's true, I'd still like to hear the opinion of someone more knowledgeable because it's an issue that's been bugging me too.
...and while that's true, I'd still like to hear the opinion of someone more knowledgeable because it's an issue that's been bugging me too.
This is a fair point: immutable data is just part of the solution. To get the win, you need to toss out all references to the old data when you update it: then you ensure that you can't access "stale" data. This requires some discipline, either manual (e.g., reusing the same variable name for the old and new bindings to ensure the stale binding is shadowed) or automatic (e.g., linear types).
But the data can't be stale, because `a` and `b` are immutable
In the case of Erlang, how would immutable data help? The code all run in single thread execution path in each process; there's no need for lock anyway. Each process can't access data of another process.
Immutable data (along with per process heaps) makes the garbage collection very simple.
Isn't one of pony's strengths, that it's doing GC better?
As a developer writing code, I don't particularly care whether GC is simple or complicate, as long as it works correctly. That's the compiler writer's job.
A fundamental strength of Erlang is that as many things as possible are simple, which makes it easy to know that it works correctly and under what conditions it doesn't work so well [1]. The compiler writer isn't necessarily the GC writer either; although, it's probably the same couple of people in Erlang.
[1] If you touch a lot of ref counted binaries, without generating enough garbage on the process heap to trigger GC. GC time scales with process heap, if you get a large enough backlog in your mailbox, GC may reduce the throughput to the point where you can't recover, although doing selective receives if you don't trigger the optimization for new references[2] is much more painful than GC scaling.
[2] http://erlang.org/pipermail/erlang-questions/2010-June/05175...
EDIT: from slides about r19 GC [3], it seems that both of the GC issues I mentioned have had targeted changes: for refcounted binary issues, there's now a 'virtual binary heap' to help trigger GC; for GC with a big mailbox, there are different allocation strategies available which may help.
[3] https://www.erlang-solutions.com/blog/erlang-19-0-garbage-co...
[1] If you touch a lot of ref counted binaries, without generating enough garbage on the process heap to trigger GC. GC time scales with process heap, if you get a large enough backlog in your mailbox, GC may reduce the throughput to the point where you can't recover, although doing selective receives if you don't trigger the optimization for new references[2] is much more painful than GC scaling.
[2] http://erlang.org/pipermail/erlang-questions/2010-June/05175...
EDIT: from slides about r19 GC [3], it seems that both of the GC issues I mentioned have had targeted changes: for refcounted binary issues, there's now a 'virtual binary heap' to help trigger GC; for GC with a big mailbox, there are different allocation strategies available which may help.
[3] https://www.erlang-solutions.com/blog/erlang-19-0-garbage-co...
This was back in a brief period when Erlang was being hyped by people as a potential Next Big Thing. Unfortunately it never really got that much momentum though, until Elixir and Phoenix came along.
I've always thought that Erlang would be more successful if it had a better deployment story, along the lines of Go or Rust: e.g. if you had the option of baking an Erlang runtime into a static binary. As it stands, software like ejabberd and rabbitmq are pretty ugly in OS package form (because distro maintainers think such packages need to rely on a common, usually very old, Erlang runtime, instead of allowing them to vendor it in as you're supposed to.)
Now that I think of it, Canonical's push toward "snaps" (sandboxed and vendored—but not fully containerized—software bundles, as a package format) might bring a flourishing of Erlang-based application software, given that it's much more compatible with Erlang's style of release-management.
Now that I think of it, Canonical's push toward "snaps" (sandboxed and vendored—but not fully containerized—software bundles, as a package format) might bring a flourishing of Erlang-based application software, given that it's much more compatible with Erlang's style of release-management.
I think that would help, but in my opinion, Erlang's syntax and overall projected surface has always been its biggest hurdle to wide adoption. You can have the most amazing, powerful language ever, but it still won't receive popular adoption if you're too strange and you can only attract a certain type of highly-skilled senior devs.
I'm not a fan of Elixir's quasi-Ruby syntax — I think something closer to Go or Nim would have served it better — but it's certainly a step in the right direction.
The Erlang toolchain has a ton of ergonomic issues that could have been polished down a long time ago, but haven't been. Erlang simply feels oddly quirky and antiquated for anyone used to modern GNU tools or modern REPLs.
For example: Lack of Readline support (needs rlwrap to be usable, at least on Ubuntu), lack of GNU-style long options, the obnoxious ctrl-C behaviour that doesn't respect Unix conventions and gives you a prompt with confusing options (quick, what's the difference between "abort" and "kill"?) that seem aimed at developers and are completely wrong if you're just a user of an Erlang tool. (Couldn't they have relegated this to USR1 or something instead of INT, like a normal program?) On the server, the epmd process is a thorn in the side of any system administrator. Erlang devs also seem to think that emitting Erlang stack traces is a good replacement for proper English error messages, and if you're not an Erlang dev, you haven't seen stack traces until you've seen the kind of monstrous, obscure, deeply nested contextual dumps that Erlang programs can produce.
And so on. Little things, but important things that can completely kill the joy when you're a potential adopter.
Erlang shouldn't need Elixir to modernize, but there's probably very little incentive within the community to change quirks that they're all used to dealing with by now.
(I encountered similar issues with OCaml, which has many parallels to Erlang: Quirky, odd syntax, antiquated toolchain, etc. Facebook's cleanup effort [1] looks very promising.)
[1] https://facebook.github.io/reason/
I'm not a fan of Elixir's quasi-Ruby syntax — I think something closer to Go or Nim would have served it better — but it's certainly a step in the right direction.
The Erlang toolchain has a ton of ergonomic issues that could have been polished down a long time ago, but haven't been. Erlang simply feels oddly quirky and antiquated for anyone used to modern GNU tools or modern REPLs.
For example: Lack of Readline support (needs rlwrap to be usable, at least on Ubuntu), lack of GNU-style long options, the obnoxious ctrl-C behaviour that doesn't respect Unix conventions and gives you a prompt with confusing options (quick, what's the difference between "abort" and "kill"?) that seem aimed at developers and are completely wrong if you're just a user of an Erlang tool. (Couldn't they have relegated this to USR1 or something instead of INT, like a normal program?) On the server, the epmd process is a thorn in the side of any system administrator. Erlang devs also seem to think that emitting Erlang stack traces is a good replacement for proper English error messages, and if you're not an Erlang dev, you haven't seen stack traces until you've seen the kind of monstrous, obscure, deeply nested contextual dumps that Erlang programs can produce.
And so on. Little things, but important things that can completely kill the joy when you're a potential adopter.
Erlang shouldn't need Elixir to modernize, but there's probably very little incentive within the community to change quirks that they're all used to dealing with by now.
(I encountered similar issues with OCaml, which has many parallels to Erlang: Quirky, odd syntax, antiquated toolchain, etc. Facebook's cleanup effort [1] looks very promising.)
[1] https://facebook.github.io/reason/
Oh, sure, there are plenty of things getting in the way of developers being excited about using Erlang and thus deciding, in a bottom-up sense, to build more stuff in Erlang.
But I think Erlang's use-case isn't really the type of software developers get excited about developing, either way. The type of software Erlang is "best at" (and where it would reduce codebase size the most) is exactly the type of big Enterprise-y bloatware-apps and business-process servers that make Java's market-share so large.
Because Enterprise software gets designed top-down, how excited developers are to be using the (dictated) language aren't very relevant. How easy the language is to learn is relevant; how many engineers you can either hire or train to code in the language is relevant. But whether they enjoy their day-to-day lives writing in the language isn't, really.
Thus, I think, why things like deployability are more important to Erlang: for the type of software projects Erlang would be good for, considerations like deployability (and maintainability, remote debugging, easy hot-patching, etc.) are what guide language choice. Erlang wins on most of those fronts (as it should: its OTP framework is effectively "30 years of ops best-practices constraining development"), but for various reasons, [non-embedded] deployment has always been a pain-point.
But I think Erlang's use-case isn't really the type of software developers get excited about developing, either way. The type of software Erlang is "best at" (and where it would reduce codebase size the most) is exactly the type of big Enterprise-y bloatware-apps and business-process servers that make Java's market-share so large.
Because Enterprise software gets designed top-down, how excited developers are to be using the (dictated) language aren't very relevant. How easy the language is to learn is relevant; how many engineers you can either hire or train to code in the language is relevant. But whether they enjoy their day-to-day lives writing in the language isn't, really.
Thus, I think, why things like deployability are more important to Erlang: for the type of software projects Erlang would be good for, considerations like deployability (and maintainability, remote debugging, easy hot-patching, etc.) are what guide language choice. Erlang wins on most of those fronts (as it should: its OTP framework is effectively "30 years of ops best-practices constraining development"), but for various reasons, [non-embedded] deployment has always been a pain-point.
But did it have to become niche? I think Erlang could very well have branched out to become a more even general-purpose language, rather than doubling down on distributed computing to the exclusion of more quotidian tasks.
Sure, Erlang will never be able to compete with certain languages such as C and C++ for many use cases. But I often grab Go to create small command-line tools, or do some minor parallel data processing where a lighter language like Ruby will not do well. There's nothing in Erlang that conceptually prevents it from being a general-purpose language; it's just that its ergonomics don't really "scale down" to the stupid, simple stuff.
I remember, years ago, trying to write a basic parallel non-OTP log-processing pipeline for some log files, thinking Erlang would be ideal... and being surprised at the number of roadblocks I had to deal with. Around the same time, Tim Bray went through the same process, but spent a lot more effort on it than I [1].
[1] http://www.tbray.org/ongoing/When/200x/2007/09/22/Erlang
Sure, Erlang will never be able to compete with certain languages such as C and C++ for many use cases. But I often grab Go to create small command-line tools, or do some minor parallel data processing where a lighter language like Ruby will not do well. There's nothing in Erlang that conceptually prevents it from being a general-purpose language; it's just that its ergonomics don't really "scale down" to the stupid, simple stuff.
I remember, years ago, trying to write a basic parallel non-OTP log-processing pipeline for some log files, thinking Erlang would be ideal... and being surprised at the number of roadblocks I had to deal with. Around the same time, Tim Bray went through the same process, but spent a lot more effort on it than I [1].
[1] http://www.tbray.org/ongoing/When/200x/2007/09/22/Erlang
I agree; I've gotten so used to thinking "oh, it'd be annoying to code Erlang for that, I'll just use Ruby" that now Elixir exists and "#!/usr/bin/env elixir" works quite well, it never occurs to me to use it.
(Though, even in that setup, there's no way to get an effect equivalent to requiring un-bundled gems from your script in Ruby.)
(Though, even in that setup, there's no way to get an effect equivalent to requiring un-bundled gems from your script in Ruby.)
I don't think it helps that the largest non-Ericsson commercial entity that attempts to curate and bounding-box the community has a kind of identity crisis around whether or not it's a services or products company.
For whatever reason they didn't do much in the way to create and maintain modern tooling to make on-boarding easier, which would have facilitated increased adoption, and lead to more revenue potential. Neither relx, nor rebar, nor erlang.mk, etc. were born out of said commercial entity, despite those kinds of things being perfect candidates for a services company to produce to make their own lives and the lives of potential new users easier.
There have been some huge high-profile Erlang use cases that would normally feed a Silicon Valley/Hacker News style hype-cycle, but somehow those came and went with almost nothing to show for it. It almost feels like if Elixir hadn't shown up on the scene to attract new users and splinter off a chunk of the massive Ruby community that the Erlang side of the world would all but dead already. Sometimes I wonder how it is one goes about blowing a 15+ year technology lead and 10 years of huge and publicly visible wins, but that's sorta what's happened.
In the curation of said community there's also been an overtone of nostalgia for "the good old days" & "yeah, that part is supposed to be painful", and a sort of long-running tone deafness around on-boarding, usability, general-developer-ecosystem-friendliness, etc. complaints that certainly could have had a lot more done to be ameliorated by the commercial entities who stood to gain the most by Erlang having broader adoption.
Frankly, much like Elixir breathing a bit of new life into aura around Erlang, if it weren't for people like Fred, Tristan, Alisdair, Garrett and a few others (who again aren't part of the core supports for Erlang commercially) being really passionate against all odds about making Erlang easier to use, and to explore new use cases, and to meticulously and engagingly document all of it... Erlang would probably be dead already.
All of which might be overly harsh, but I'm having a very acute existential crisis around Erlang at the moment. I love it. I love developing in it. I love introducing it to teams and projects. And as time goes on I feel rapidly and increasingly more guilty about that given the decline it seems to be in and has been in if I'm being honest with myself in retrospect.
Blargh :-(
edit: look at the increasingly dwindling sponsors list of Erlang Factory San Francisco over the last several years as an indicator of the above rant/cry-for-help.
For whatever reason they didn't do much in the way to create and maintain modern tooling to make on-boarding easier, which would have facilitated increased adoption, and lead to more revenue potential. Neither relx, nor rebar, nor erlang.mk, etc. were born out of said commercial entity, despite those kinds of things being perfect candidates for a services company to produce to make their own lives and the lives of potential new users easier.
There have been some huge high-profile Erlang use cases that would normally feed a Silicon Valley/Hacker News style hype-cycle, but somehow those came and went with almost nothing to show for it. It almost feels like if Elixir hadn't shown up on the scene to attract new users and splinter off a chunk of the massive Ruby community that the Erlang side of the world would all but dead already. Sometimes I wonder how it is one goes about blowing a 15+ year technology lead and 10 years of huge and publicly visible wins, but that's sorta what's happened.
In the curation of said community there's also been an overtone of nostalgia for "the good old days" & "yeah, that part is supposed to be painful", and a sort of long-running tone deafness around on-boarding, usability, general-developer-ecosystem-friendliness, etc. complaints that certainly could have had a lot more done to be ameliorated by the commercial entities who stood to gain the most by Erlang having broader adoption.
Frankly, much like Elixir breathing a bit of new life into aura around Erlang, if it weren't for people like Fred, Tristan, Alisdair, Garrett and a few others (who again aren't part of the core supports for Erlang commercially) being really passionate against all odds about making Erlang easier to use, and to explore new use cases, and to meticulously and engagingly document all of it... Erlang would probably be dead already.
All of which might be overly harsh, but I'm having a very acute existential crisis around Erlang at the moment. I love it. I love developing in it. I love introducing it to teams and projects. And as time goes on I feel rapidly and increasingly more guilty about that given the decline it seems to be in and has been in if I'm being honest with myself in retrospect.
Blargh :-(
edit: look at the increasingly dwindling sponsors list of Erlang Factory San Francisco over the last several years as an indicator of the above rant/cry-for-help.
Thanks for the insight. Out of interest, what company are you referring to in your first paragraph?
Erlang Solutions
Once I wrapped my head around it, I found Elixir to be much simpler than other programming languages. I'd rather use Elixir to build things if there aren't any blockers to doing so.
As a long-time Erlang user, I think the above is a lot closer to why people always seemed to have some trouble with it. Lots of little weird things.
Part of this is that Erlang kind of wants to be its own little world - something that it shares to some degree with Lisp and Smalltalk (images!).
Part of this is that Erlang kind of wants to be its own little world - something that it shares to some degree with Lisp and Smalltalk (images!).
I think it's a mind-expanding experience to set up an Erlang release on an "immutable infrastructure" system like CoreOS, with the Erlang "ssh" application enabled (instead of using ERTS's remsh system), and then keep that system up for weeks/months by doing maintenance through said SSH REPL. It becomes very clear, when you do so, that Erlang really is its own little sandboxed OS, and that the "Erlang Runtime System" is really a kind of standalone virtualization system for running that OS on other OSes.
Honestly, I think that if Ericsson was just starting to build Erlang today, it'd look less like a self-contained emulator, and more like a virtual-machine image. (Or more than one: maybe an Erlang unikernel image ala Erlang-on-Xen, and then companion POSIX VMs to spawn ports on, making the Erlang cluster into its own Kubernetes-alike.)
Honestly, I think that if Ericsson was just starting to build Erlang today, it'd look less like a self-contained emulator, and more like a virtual-machine image. (Or more than one: maybe an Erlang unikernel image ala Erlang-on-Xen, and then companion POSIX VMs to spawn ports on, making the Erlang cluster into its own Kubernetes-alike.)
I feel like Robert Virding once said that Erlang-as-OS was more or less one of their original plans (I wish I could remember the source, but it was during a discussion of the Erlang JCL mode in the REPL).
EDIT: found the link http://erlang.org/pipermail/erlang-questions/2011-October/06...
EDIT: found the link http://erlang.org/pipermail/erlang-questions/2011-October/06...
That's a great point — there's an impedance mismatch between Erlang and the rest of the world.
It doesn't just have its own syntax, it has its own idiosynchratic, somewhat alien way of doing almost everything that is already established on Unix. The difference feels almost as stark as Linux vs. Windows.
It doesn't just have its own syntax, it has its own idiosynchratic, somewhat alien way of doing almost everything that is already established on Unix. The difference feels almost as stark as Linux vs. Windows.
I've been thinking about this deployment thing because I'm experimenting with writing an "end user" application in Elixir + C port drivers. I don't see a particular reason one couldn't do something similar to "py2exe" and bundle everything into a single executable, with a custom loader to load modules from embedded resources. It sounds like a fun project, and one I might try out...
edit: There's this (abandoned, for reference): http://www.oocities.org/erlang_journal/sae.html
edit: There's this (abandoned, for reference): http://www.oocities.org/erlang_journal/sae.html
Yes, I blame the distros. Should have always just used release bundled as target systems. Though releases weren't always so easy to build with Erlang before relx and rebar3 (full disclosure: I'm biased on that).
I also played around with an idea of bunding the release into a self extracting bundled executable too https://gist.github.com/tsloughter/d62aad6d67b263e69275376b1...
I also played around with an idea of bunding the release into a self extracting bundled executable too https://gist.github.com/tsloughter/d62aad6d67b263e69275376b1...
It does that. This is an erlang release, and that is something that erlang does really well. The problem lie more on the tooling that made it harder than it should have been. This is what Elixir bring to the table.
An Erlang release is not a static binary. It certainly is (or starts off as) a self-contained directory you can just unzip onto a server—but for the kind of software I'm talking about (RabbitMQ, CouchDB, Riak, etc.) you don't tend to install these things this way. You expect to install them using your distro's package manager. And, until very recently, distros did horrible things to those packages to force them into the FreeDesktop FHS mold. (And it still doesn't entirely work; the Erlang concept of priv dirs, for example, just doesn't map well onto even a reasonably-similar directory like /var/lib.)
That's what I mean by "deployability": the hell that is trying to convert your Erlang release into a distro package so that users can simply "apt-get install" it and have the distro maintain it for you; and so other software packages can declare package-level dependencies on it.
The reason that true static binaries ala Go would be better, is simply that it would stop the distro maintainers from messing with everything to quite the same degree. In such a setup, there wouldn't be a "system Erlang runtime", because Erlang would just be a baked-in part of /usr/bin/riak or /usr/bin/ejabberd. (Which wouldn't stop distros from making Erlang SDK packages for things like an Elixir package to depend on, but they wouldn't be involved at all in the dependency tree of the Erlang-release-derived packages on the system.)
That's what I mean by "deployability": the hell that is trying to convert your Erlang release into a distro package so that users can simply "apt-get install" it and have the distro maintain it for you; and so other software packages can declare package-level dependencies on it.
The reason that true static binaries ala Go would be better, is simply that it would stop the distro maintainers from messing with everything to quite the same degree. In such a setup, there wouldn't be a "system Erlang runtime", because Erlang would just be a baked-in part of /usr/bin/riak or /usr/bin/ejabberd. (Which wouldn't stop distros from making Erlang SDK packages for things like an Elixir package to depend on, but they wouldn't be involved at all in the dependency tree of the Erlang-release-derived packages on the system.)
Well there is something to do that. Exrm has some plugin to output a .deb. And it could be done for other package format. Once again, it is a tooling problem.
The kind of .deb produced by Exrm (or by just packing up the release using FPM or the like) would never be accepted upstream by a distro maintainer. This is a political problem, in that distro vendors have strong philosophical stances on how to do packaging, and those stances make packaging Erlang programs to suit their tastes really hard (and frequently impossible for newer software, since they say things like "you must depend on the system erlang package rather than embedding ERTS; and the erlang package on LTS will be held at R16, so it doesn't support maps.")
Of course, when people install your software "manually", you can distribute it however you like—a zip of an Erlang release, like a Java JAR, is already quite a convenient format for such installations all on its own. But when people want to install something that depends on your software (like, say, OpenStack depends on RabbitMQ), they don't want to spend nearly as much time thinking about your package. They don't know you exist, and they don't want to know you exist—they want the distro to handle that detail for them. So you're stuck fighting the distro. (And things like OpenStack are stuck doing non-distro-packaged releases to avoid the problem.)
Which brings me back to my original point: "snaps" are promising not because they sandbox packages, but because they're a countervailing philosophical movement on how software should be packaged. A distro that supports snaps is a distro that has made room in its heart for packages that don't want to break things down into little FHS-arranged dep-graph-divided bits. It gives them an out, a place to put these packages rather than just denying them outright.
Of course, when people install your software "manually", you can distribute it however you like—a zip of an Erlang release, like a Java JAR, is already quite a convenient format for such installations all on its own. But when people want to install something that depends on your software (like, say, OpenStack depends on RabbitMQ), they don't want to spend nearly as much time thinking about your package. They don't know you exist, and they don't want to know you exist—they want the distro to handle that detail for them. So you're stuck fighting the distro. (And things like OpenStack are stuck doing non-distro-packaged releases to avoid the problem.)
Which brings me back to my original point: "snaps" are promising not because they sandbox packages, but because they're a countervailing philosophical movement on how software should be packaged. A distro that supports snaps is a distro that has made room in its heart for packages that don't want to break things down into little FHS-arranged dep-graph-divided bits. It gives them an out, a place to put these packages rather than just denying them outright.
please, more sources and links? i would love to package up an entire elixir app as an rpm to a rhel7 target that has no internet connection. so the package/binary needs to have everything including erlang. i am not afraid of compiling from source. is the answer distillery or something else entirely?
Does it need to be an RPM or just something easy to install office?
The release process I've used for Erlang before packaged things up into a folder with everything you needed to run.
The release process I've used for Erlang before packaged things up into a folder with everything you needed to run.
rpm is preferred. the unix teams i deal with and the checkboxes they have to check.. err i mean they have to comply with some restrictrictions so from what i understand so far, rpm package would require the fewest amount of hoops to go through
Distillery will give you a tarball. I do not know if there are plugin/backend for .rpm for it, but should be doable.
Similar thing for Ruby: it wasn't popular until Ruby on Rails. It may be just
that most programmers need to write dynamic websites, as opposed to network
services (most likely with custom protocols).
Here's my Stack Overflow theory:
Programmers writing dynamic websites are likely to post questions on Stack Overflow. Questions on Stack Overflow create mindshare, mindshare attracts more serious programmers who can improve the ecosystem by writing generic network services. Language prospers, cycle repeats.
JavaScript is the most popular language on Stack Overflow and React is the fastest growing technology.
Without the dynamic website programmers the cycle short circuits and you end up with a terrible "new user experience" from lack of answered beginner questions and lack of interest building on the non-existent ecosystem.
Programmers writing dynamic websites are likely to post questions on Stack Overflow. Questions on Stack Overflow create mindshare, mindshare attracts more serious programmers who can improve the ecosystem by writing generic network services. Language prospers, cycle repeats.
JavaScript is the most popular language on Stack Overflow and React is the fastest growing technology.
Without the dynamic website programmers the cycle short circuits and you end up with a terrible "new user experience" from lack of answered beginner questions and lack of interest building on the non-existent ecosystem.
I'm not well versed in the entire (phone) network space. Ericson switch uptime (powered by Erlang) is always cited and admittedly initially drew me to the whole ecosystem. Well mostly the "it's used in a network that has basically worked all these years so it must be rather robust one way or another" line of thinking.
However what I'm wondering is...there's other vendors and Ericson isn't powering the entire network. What software stacks are used by other vendors? I'm assuming it's mostly C-ish stuff with some ASM sprinkles like in Cisco IOS?
However what I'm wondering is...there's other vendors and Ericson isn't powering the entire network. What software stacks are used by other vendors? I'm assuming it's mostly C-ish stuff with some ASM sprinkles like in Cisco IOS?
Ericson makes a switch that some other networks use. That switch contains lots of Erlang inside it.
Incidentally, does anyone have a screenshot of the "all Erlang" HN front page of similar vintage (perhaps 2009 or so)?
I think the "more cores == faster" benefit is overstated.
Without jit Erlang is so slow that languages without concurrency support will kill it. It's even more overstated when something like Go can mostly scale as well over multiple cores but be x times faster while doing so. Not to mention it's 10 years later and we haven't seen massively parallel architectures take off.
I suspect other languages are going to fold the good parts of Erlang/BEAM into them and limit its long-term adoption.
Without jit Erlang is so slow that languages without concurrency support will kill it. It's even more overstated when something like Go can mostly scale as well over multiple cores but be x times faster while doing so. Not to mention it's 10 years later and we haven't seen massively parallel architectures take off.
I suspect other languages are going to fold the good parts of Erlang/BEAM into them and limit its long-term adoption.
I think the "more cores == faster" benefit is overstated.
I've tried to create servers for massively multiplayer games in Clojure and Go. What I've found is that it "takes some doing" to be able to efficiently use many processors in parallel. You can't just slap on a pmap call in Clojure or do the equivalent cheesy channel trick in Golang and have good utilization of all your cores in parallel.
It's even more overstated when something like Go can mostly scale as well over multiple cores but be x times faster while doing so.
For what value of "mostly scale as well?" For me, that means doing a little extra design/architecting to break out your Golang server process into multiple worker processes.
it's 10 years later and we haven't seen massively parallel architectures take off.
Because there are still barriers that are a bit too large. You either wind up having to do weird things to do what amounts to efficiently passing data between cores, or you end up using a slower language that's a bit far from the computing mainstream.
I've tried to create servers for massively multiplayer games in Clojure and Go. What I've found is that it "takes some doing" to be able to efficiently use many processors in parallel. You can't just slap on a pmap call in Clojure or do the equivalent cheesy channel trick in Golang and have good utilization of all your cores in parallel.
It's even more overstated when something like Go can mostly scale as well over multiple cores but be x times faster while doing so.
For what value of "mostly scale as well?" For me, that means doing a little extra design/architecting to break out your Golang server process into multiple worker processes.
it's 10 years later and we haven't seen massively parallel architectures take off.
Because there are still barriers that are a bit too large. You either wind up having to do weird things to do what amounts to efficiently passing data between cores, or you end up using a slower language that's a bit far from the computing mainstream.
Can you elaborate on how you implemented those servers? Would you still use Clojure or Go for them now? If not, what alternative do you suggest?
My first attempt was a server for a dungeon crawler that superficially looked like a rogue-like, but was actually 12 frames/sec realtime instead of turn based. It started out on Clojure, but I afterwards ported it to Go. In this server, the world was divided into 80x24 subgrids that did most of their processing in parallel. Processing would happen in two stages: 1) local data processing, where each entity got updated in a loop, sending out updates to clients and 2) collision resolution/movement. If I had to do it over again, I would relax the grid's invariant properties, and make each 80x24 subgrid completely independent, which would eliminate the collision resolution step.
In my current server, I'm basically sharding my space-game into star systems, with free travel between star systems though "Hyperspace." There is a farm of "worker processes" that is coordinated by a master server. Any instance of a star system can be idempotently spawned when a client is attached to it. < https://www.emergencevector.com >
I think Go is a good option for writing a game server. You'd have to be pretty profligate to make the GC pause overrun one 16.66 ms tick, and if you miss a tick here and there, who cares?
In my current server, I'm basically sharding my space-game into star systems, with free travel between star systems though "Hyperspace." There is a farm of "worker processes" that is coordinated by a master server. Any instance of a star system can be idempotently spawned when a client is attached to it. < https://www.emergencevector.com >
I think Go is a good option for writing a game server. You'd have to be pretty profligate to make the GC pause overrun one 16.66 ms tick, and if you miss a tick here and there, who cares?
Why did you leave Clojure?
Clojure weenies on IRC. To be fair, the Go weenies aren't much better. If you like hanging out on IRC so that you can emotionally abuse newbies, ask yourself what you're doing with your life.
Fascinating, so it was purely a communal decision.
Interesting. Thanks for the detailed response!
I've always enjoyed writing MUDs (similar to roguelikes), and I found Clojure's killer feature to be immutability, rather than some of the fancier concurrency features like STM. Operations that consume the most resources in a MUD tend to be very amenable to stale, but consistent snapshots, which immutability makes pretty easy to do. My goal was more about responsiveness than throughput though.
> In this server, the world was divided into 80x24 subgrids that did most of their processing in parallel.
Is there any reason why you do update() in parallel? There is certainly some interesting challenges to solve there but did you have a reason to make it parallel other than intellectual curiosity (e.g. better performance)?
Afaik a game loop's update() is not the intense part in the game loop (maybe that's different for games with lots of world information update, like roguelikes), but rather render() and IO() are so I am wondering what your design thoughts are.
Is there any reason why you do update() in parallel? There is certainly some interesting challenges to solve there but did you have a reason to make it parallel other than intellectual curiosity (e.g. better performance)?
Afaik a game loop's update() is not the intense part in the game loop (maybe that's different for games with lots of world information update, like roguelikes), but rather render() and IO() are so I am wondering what your design thoughts are.
Is there any reason why you do update() in parallel?
I wanted to come up with an architecture that could support an arbitrarily large world given enough resources. Up to a certain density (from 150 to 250 users per "subgrid") I was close to succeeding.
Afaik a game loop's update() is not the intense part in the game loop but rather render() and IO()
"Render" in the context of a server like this is the IO. One thing you should know, is that this game supported Conway's Life cellular automata at 12 frames/second, completely shared and multiplayer.
I wanted to come up with an architecture that could support an arbitrarily large world given enough resources. Up to a certain density (from 150 to 250 users per "subgrid") I was close to succeeding.
Afaik a game loop's update() is not the intense part in the game loop but rather render() and IO()
"Render" in the context of a server like this is the IO. One thing you should know, is that this game supported Conway's Life cellular automata at 12 frames/second, completely shared and multiplayer.
[deleted]
Unfortunately, I think you're correct.
Anecdote: I used Erlang, over the course of several years, for very-high-level bot behavior and multi-player mission control, where performance was much less important than the things Erlang provides, but I still had issues with it. Nonetheless, I felt it had given me great leverage, and I thanked Joe Armstrong profusely, when I met him at a conference. Later at the con, I asked a panel whether there were active efforts at improving performance, perhaps with a JIT. They seemed to take it as an attack, and suggested that if I need performance, I should use another language. I mostly write in Clojure now.
Anecdote: I used Erlang, over the course of several years, for very-high-level bot behavior and multi-player mission control, where performance was much less important than the things Erlang provides, but I still had issues with it. Nonetheless, I felt it had given me great leverage, and I thanked Joe Armstrong profusely, when I met him at a conference. Later at the con, I asked a panel whether there were active efforts at improving performance, perhaps with a JIT. They seemed to take it as an attack, and suggested that if I need performance, I should use another language. I mostly write in Clojure now.
(facepalm) that somewhat closely mirrors my experience. It's a bit baffling.
> I think the "more cores == faster" benefit is overstated.
I find it is not. But of course the right tool for the right job. Multiplying matrices as a single batch process is probably not going to work very well. But I saw it works exceptionally for highly concurrent workload.
This is not just "I ran a quick benchmarks on my laptop and then extrapolated by using an multiplier". Distributed systems work funny and they don't scale linearly. Sometimes a very fast, small sequential program will hit a wall and barf on itself when faced with multiple concurrent requests.
The other secret sauce is not multi-core capability but process isolation. The reason is this: a large distributed system is only as useful / practical as it is fault tolerant. It doesn't matter if you run hand tunes assembly routine and process 1M requests per second, then it crashes every day and you have hours of debugging and downtime. The average throughput or uptime will be pretty bad.
This is one key insight into what makes Erlang (and Elixir) a good tool for distributed systems - fault tolerance is built into the core of the platform.
I find it is not. But of course the right tool for the right job. Multiplying matrices as a single batch process is probably not going to work very well. But I saw it works exceptionally for highly concurrent workload.
This is not just "I ran a quick benchmarks on my laptop and then extrapolated by using an multiplier". Distributed systems work funny and they don't scale linearly. Sometimes a very fast, small sequential program will hit a wall and barf on itself when faced with multiple concurrent requests.
The other secret sauce is not multi-core capability but process isolation. The reason is this: a large distributed system is only as useful / practical as it is fault tolerant. It doesn't matter if you run hand tunes assembly routine and process 1M requests per second, then it crashes every day and you have hours of debugging and downtime. The average throughput or uptime will be pretty bad.
This is one key insight into what makes Erlang (and Elixir) a good tool for distributed systems - fault tolerance is built into the core of the platform.
> ...It's even more overstated when something like Go can mostly scale as well over multiple cores but be x times faster while doing so...
> I suspect other languages are going to fold the good parts of Erlang/BEAM into them and limit its long-term adoption.
Comparing a bare-metal-but-with-GC language like Go to a miniature operating system like Erlang isn't quite fair -- in comparison, Go is missing an actor model, buffered CSP, crash supervision, hot upgrades, remote console/tracing/debugging, and built-in clustering (to name a few).
Once other languages adopt "enough" of the BEAM featureset, they'll have similar performance.
Comparing a bare-metal-but-with-GC language like Go to a miniature operating system like Erlang isn't quite fair -- in comparison, Go is missing an actor model, buffered CSP, crash supervision, hot upgrades, remote console/tracing/debugging, and built-in clustering (to name a few).
Once other languages adopt "enough" of the BEAM featureset, they'll have similar performance.
Hot upgrades and remote debugging are front in center in lisp and it has excellent performance. Even so, there's no general consensus among programmers about whether hot code reloading is preferred to static binaries.
The actor model is a choice of implementation. Crash supervision stands out, but that's not necessary and is part of the Erlang culture - let it crash. It's not as important in other languages with a different underlying belief about errors.
What features mean BEAM must necessarily be slow?
Just to be clear: I really like Erlang. It obviously has different features than a lot of languages and most are aimed towards a certain goal: uptime. But if you can give just a little on that (and you might not have to at all), you can get most of those advantages with a different feature set.
I wonder out of the Erlang experiment which are the objectively better features and which are just really cool?
The actor model is a choice of implementation. Crash supervision stands out, but that's not necessary and is part of the Erlang culture - let it crash. It's not as important in other languages with a different underlying belief about errors.
What features mean BEAM must necessarily be slow?
Just to be clear: I really like Erlang. It obviously has different features than a lot of languages and most are aimed towards a certain goal: uptime. But if you can give just a little on that (and you might not have to at all), you can get most of those advantages with a different feature set.
I wonder out of the Erlang experiment which are the objectively better features and which are just really cool?
> fold the good parts of Erlang/BEAM into them
A lot easier said than done, because there are a lot of really fundamental decisions about Erlang that make it what it is. You can't just slap on a scheduler for instance, or Erlang style processes.
Go is probably the closest thing, and it's still lacking some things like the supervision tree.
Erlang is certainly not 'the answer' in terms of computationally intensive anything, for the foreseeable future.
A lot easier said than done, because there are a lot of really fundamental decisions about Erlang that make it what it is. You can't just slap on a scheduler for instance, or Erlang style processes.
Go is probably the closest thing, and it's still lacking some things like the supervision tree.
Erlang is certainly not 'the answer' in terms of computationally intensive anything, for the foreseeable future.
> Go is probably the closest thing, and it's still lacking some things like the supervision tree.
From what I've read, Go's runtime just isn't compatible with the kind fault-tolerance that Erlang provides (nor was it meant to). It lacks process monitoring, lacks process isolation, has a pause-the-world garbage collector, etc.
Not a knock against Go, just pointing out that the design goals were different.
From what I've read, Go's runtime just isn't compatible with the kind fault-tolerance that Erlang provides (nor was it meant to). It lacks process monitoring, lacks process isolation, has a pause-the-world garbage collector, etc.
Not a knock against Go, just pointing out that the design goals were different.
so many people (including me from time to time) misunderstand "computaionally expensive" that I think Elixir/erlang/beam will serve a lot of systems/apps well for a long time until they really see a need for "intense" computation
Oh, absolutely - you can do tons of things with Erlang, just as people do with Ruby, Python, PHP, Tcl, etc...
But that article is a bit hype-y.
But that article is a bit hype-y.
well, of course it is ;) consider the source.
The ways that massively parallel architectures have taken off are in GPU programming and they interact with them using the languages that they would have already done. C/C++ libraries that either use them directly or provide an interface to Python / Lua / Java etc...
The slowness of Erlang was a conscious decision. You're supposed to offload all the computationally intensive things to native code. That may not be ideal for your use case and a PITA, but it's how the language and runtime was designed. I would guess Erlang would do (if it isn't already doing) well in HPC as the thing that coordinates all the Fortran code.
Agreed. I don't think there is a big fuss about Erlang anymore. Still it's an interesting language from an acidemic perspective if not from a practical one.
Erlang is certainly inaccessible, but I don't think academic is really the right word for it. It was born out of very practical business needs, and the reason it is still around is mainly because it has done an excellent job of serving those use cases.
What they haven't done (or to the extent that other languages have) is make the language and runtime accessible to the broader programming community. The result is that the people who (start to) use Erlang are the kind of people for whom no other tool will help them solve their problems.
Although I have not used Elixir, it seems to me that it was designed as a language that is easy enough to adopt for the wider programming community, while still leveraging the concurrency, fault-tolerance, etc. provided by BEAM and OTP. On top of that, the Elixir community has worked hard to provide the tooling and support developers typically expect these days[0].
This is not to say that Erlang should be radically altered or abandoned. I like the Erlang aesthetic and ethos, and I think the community knows the way it needs to evolve. It is just that I expect more people to come to BEAM and OTP through Elixir than Erlang, and I think it is that platform that should really be the focus.
[0] to their credit, the Erlang community has definitely improved in this area.
What they haven't done (or to the extent that other languages have) is make the language and runtime accessible to the broader programming community. The result is that the people who (start to) use Erlang are the kind of people for whom no other tool will help them solve their problems.
Although I have not used Elixir, it seems to me that it was designed as a language that is easy enough to adopt for the wider programming community, while still leveraging the concurrency, fault-tolerance, etc. provided by BEAM and OTP. On top of that, the Elixir community has worked hard to provide the tooling and support developers typically expect these days[0].
This is not to say that Erlang should be radically altered or abandoned. I like the Erlang aesthetic and ethos, and I think the community knows the way it needs to evolve. It is just that I expect more people to come to BEAM and OTP through Elixir than Erlang, and I think it is that platform that should really be the focus.
[0] to their credit, the Erlang community has definitely improved in this area.
Erlang is made for IoT.
Tens of millions of concurrent unrelated clients, embarrassingly parallelizable, scales linearly with number of cores, as these share nothing.
Tens of millions of concurrent unrelated clients, embarrassingly parallelizable, scales linearly with number of cores, as these share nothing.
Are there online classes (eg MOOCs) teaching Erlang? I looked at Coursera, EdX, and Udacity recently and didn't find anything.
There are some good introductory videos at https://www.cs.kent.ac.uk/ErlangMasterClasses/.
For folks who like Erlang, also check out Pony.
How to make sure I don't read your post:
> Our site has been optimized for use with newer browsers. We also require that your browser has JavaScript enabled.
> It looks as if your browser has JavaScript disabled.
> This site has information about enabling JavaScript, if that's something you want to do.
Really? I am just trying read a few paragraphs of text on a fairly static-looking site.
> Our site has been optimized for use with newer browsers. We also require that your browser has JavaScript enabled.
> It looks as if your browser has JavaScript disabled.
> This site has information about enabling JavaScript, if that's something you want to do.
Really? I am just trying read a few paragraphs of text on a fairly static-looking site.
It's even worse than that. It is static. It would work without javascript if they didn't put
<noscript>
<meta http-equiv="refresh" content="2;url=https://pragprog.com/no_js">
</noscript>As a slight followup: In the Advanced settings panel for NoScript, there's an option to disable <meta> redirects in <noscript> blocks.
I actually saw the text just fine before that stupid error message showed up.
"How do programmers fix these problems? They don’t. They pray." :D LOLed at this!!
> How do programmers fix these problems? They don’t. They pray.
Ah! No, Joe, that's what you do.
Most of developers have figured out how to do concurrency and parallelism with mutable state and locks. It's not as hard as you make it sound and it's supported in a lot of languages which, as opposed to Erlang, are statically typed, hence much more suitable to the kind of large scale software you describe.
And yes, that was already true in 2007.
Ah! No, Joe, that's what you do.
Most of developers have figured out how to do concurrency and parallelism with mutable state and locks. It's not as hard as you make it sound and it's supported in a lot of languages which, as opposed to Erlang, are statically typed, hence much more suitable to the kind of large scale software you describe.
And yes, that was already true in 2007.
>> How do programmers fix these problems? They don’t. They pray.
>Ah! No, Joe, that's what you do.
I don't know if Joe is a religious man, but I don't believe beseeching the Almighty was one of the reasons listed for why an Ericsson switch was able to achieve nine 9's of availability.
> Most of developers have figured out how to do concurrency and parallelism with mutable state and locks.
It appears to me that the industry has moved to a combination of promises, message passing, thread-safe data structures, transactions, etc., instead of code that directly manipulates locks.
>...as opposed to Erlang, are statically typed, hence much more suitable to the kind of large scale software you describe.
Depends on what you mean for scale. If you mean "suitable for gargantuan code bases and development teams" then it might not be the best. If you mean "suitable for processing large volumes of requests/events with low latency" then Erlang should be fine.
Types are nice, and I try to heavily leverage the type analysis tools that are available for Erlang (e.g. dialyzer). Java & C# (the two main static languages) are OK, but presence of null pointer exceptions and absence of sum types, intersection types, tuples, etc. really makes righting terse, type-safe code a chore.
>Ah! No, Joe, that's what you do.
I don't know if Joe is a religious man, but I don't believe beseeching the Almighty was one of the reasons listed for why an Ericsson switch was able to achieve nine 9's of availability.
> Most of developers have figured out how to do concurrency and parallelism with mutable state and locks.
It appears to me that the industry has moved to a combination of promises, message passing, thread-safe data structures, transactions, etc., instead of code that directly manipulates locks.
>...as opposed to Erlang, are statically typed, hence much more suitable to the kind of large scale software you describe.
Depends on what you mean for scale. If you mean "suitable for gargantuan code bases and development teams" then it might not be the best. If you mean "suitable for processing large volumes of requests/events with low latency" then Erlang should be fine.
Types are nice, and I try to heavily leverage the type analysis tools that are available for Erlang (e.g. dialyzer). Java & C# (the two main static languages) are OK, but presence of null pointer exceptions and absence of sum types, intersection types, tuples, etc. really makes righting terse, type-safe code a chore.
I think that's where Scala shines. I don't know if you have tried it, but it solves a bunch of the problem you enumerate with Java & C.
Sure, it can be hard to read because you can write the same thing in tens of different ways and some people abuse it, but if you don't and your team does not, you end up with a beautiful code base.
Sure, it can be hard to read because you can write the same thing in tens of different ways and some people abuse it, but if you don't and your team does not, you end up with a beautiful code base.
I believe it does. The only issue I take is that I believe null objects are still possible. Kotlin and Ceylon both correctly addressed this (but lack some of the sophisticated type features found in Scala).
> Most of developers have figured out how to do concurrency and parallelism with mutable state and locks.
Based on my experience of almost a decade in the embedded/safety space, this isn't true. Based on my experience as an end user of software on computers, phones, and tablets, I'm also not inspired by the crashes and performance snags I see. Even when it's safe, it's often slower than necessary thanks to unneeded locks by being too conservative (or due to refactorings which made the locks unnecessary but still present).
I agree it's not too hard. But it requires more thought and deliberation to make concurrent code in C/C++/Java than in Erlang than most developers seem willing to commit to the task. Paired with the built-in features of OTP to the Erlang platform, it beats the stock distribution of many other languages for reliability/safety of concurrent code. Those depending on external libraries can never seem to agree on just which one to use. It's too faddish to be reliable for maintenance over the next decade or five.
Based on my experience of almost a decade in the embedded/safety space, this isn't true. Based on my experience as an end user of software on computers, phones, and tablets, I'm also not inspired by the crashes and performance snags I see. Even when it's safe, it's often slower than necessary thanks to unneeded locks by being too conservative (or due to refactorings which made the locks unnecessary but still present).
I agree it's not too hard. But it requires more thought and deliberation to make concurrent code in C/C++/Java than in Erlang than most developers seem willing to commit to the task. Paired with the built-in features of OTP to the Erlang platform, it beats the stock distribution of many other languages for reliability/safety of concurrent code. Those depending on external libraries can never seem to agree on just which one to use. It's too faddish to be reliable for maintenance over the next decade or five.
Do you realize how silly this sounds? You are disregarding not only Erlang, but decades of research and development on concurrent programming best practices. Immutability has been promoted for parallel programming since the 1970s. For the past fifteen years, even languages like C++ have moved to promote this style of programming.
(Also, Erlang has a compile-time static type checker, if type safety is something you're after.)
(Also, Erlang has a compile-time static type checker, if type safety is something you're after.)
Immutability is not a silver bullet.
One reason is RAM consumption: while Facebook has bought tremendous amount or RAM for caching content so those extra copies of objects don’t hurt, desktop or embedded apps have different resource constraints.
Another reason is performance. Memory allocation/deallocation is not free. RAM bandwidth for copying those objects isn’t free. Also mutable objects generally more cache-friendly, a modified object stays in the same RAM address i.e. stays cached.
One reason is RAM consumption: while Facebook has bought tremendous amount or RAM for caching content so those extra copies of objects don’t hurt, desktop or embedded apps have different resource constraints.
Another reason is performance. Memory allocation/deallocation is not free. RAM bandwidth for copying those objects isn’t free. Also mutable objects generally more cache-friendly, a modified object stays in the same RAM address i.e. stays cached.
"One reason is RAM consumption"
You can have immutability without data duplication; in an immutable-only environment, every process/thread/whatever that uses a specific object can safely reuse a pointer to that object and always know that the object represented by that pointer cannot change under their noses. It's only when you need to "modify" that object (read: create a new copy) where you start to see RAM consumption issues, and even that can be mitigated.
Linked lists are an example of where this works remarkably well; if you have a list (E→D→C→B→A) and want to append F to that list, you can do so trivially without needing to mutate or copy the original (since all you need for the new list is F and a pointer to E). Same if you wanted to replace E with F; just have F point to D instead.
You can have immutability without data duplication; in an immutable-only environment, every process/thread/whatever that uses a specific object can safely reuse a pointer to that object and always know that the object represented by that pointer cannot change under their noses. It's only when you need to "modify" that object (read: create a new copy) where you start to see RAM consumption issues, and even that can be mitigated.
Linked lists are an example of where this works remarkably well; if you have a list (E→D→C→B→A) and want to append F to that list, you can do so trivially without needing to mutate or copy the original (since all you need for the new list is F and a pointer to E). Same if you wanted to replace E with F; just have F point to D instead.
It’s all good in theory, not in practice.
> only when you need to "modify" that object
Well, unless you’re modifying your data, almost all data structures are fine for multithreaded access.
> Linked lists are an example of where this works remarkably well
Linked lists are an example of what you don’t want to use if you care about performance even a tiny bit.
On modern hardware, random memory access is slow. A ballpark estimate is 100-200 CPU cycles just to read a single value (16 bytes for dual-channel DDR) from RAM. With a linked list you pay that cost with each iteration to the next list element. Very expensive.
> only when you need to "modify" that object
Well, unless you’re modifying your data, almost all data structures are fine for multithreaded access.
> Linked lists are an example of where this works remarkably well
Linked lists are an example of what you don’t want to use if you care about performance even a tiny bit.
On modern hardware, random memory access is slow. A ballpark estimate is 100-200 CPU cycles just to read a single value (16 bytes for dual-channel DDR) from RAM. With a linked list you pay that cost with each iteration to the next list element. Very expensive.
Are you referring to Dialyzer? I've always wanted to learn Erlang but can't give up on types.
Yup, Dialyzer. Works with Elixir too (there's even a Dialyxir mix task) and gives me all the type checking I want -- which admittedly isn't much.
Personally, even when I do want something resembling type checking, I've found pattern matching / destructuring to be more than sufficient. With Elixir's structs, this is trivial:
Between this and protocols, I very rarely feel the need to reach for something like Dialyzer. YMMV, of course.
def some_function(SomeStruct{} = some_struct) do
some_struct.foo + some_struct.bar
end
def some_function(SomeStruct{foo: foo, bar: bar}) do
foo + bar
end
Between this and protocols, I very rarely feel the need to reach for something like Dialyzer. YMMV, of course.
Pattern matching is fantastic. Also, there are guards:
As an aside, comparing:
def handle_info(msg, state) when is_tuple(msg) do
# ...
end
As an aside, comparing:
{:ok, port} = Port.open(...)
port, err := port.open(...)
if err != nil {
panic(err)
}
def handle_info([], state = {ppid, _port}) do
Supervisor.stop(ppid)
{:noreply, state}
end
I'm well aware of the literature.
The fact is that still today, we have millions and millions of lines of code that work in parallel and a crushing majority of it is operating in mutable languages (C++, Java, Javascript, C#, ...).
Pushing immutable languages as the silver bullet to solve concurrency problems is trying to solve a problem that nobody has.
We've moved on to higher abstraction grounds (fork join, work stealing, coroutines, etc...), Erlang is still stuck in the past.
The fact is that still today, we have millions and millions of lines of code that work in parallel and a crushing majority of it is operating in mutable languages (C++, Java, Javascript, C#, ...).
Pushing immutable languages as the silver bullet to solve concurrency problems is trying to solve a problem that nobody has.
We've moved on to higher abstraction grounds (fork join, work stealing, coroutines, etc...), Erlang is still stuck in the past.
I think immutability isn't really that important for concurrency in Erlang, since there's conceptually no shared state between Erlang processes -- if you use the mutable process dictionary, no other process could see it anyway[1]. In reality there is ets which is mutable with locks, and refcounted binaries which become immutable once there's more than one reference.
That Erlang forces you to build a shared nothing system is the real win for concurrency. Of course, you can do that with other languages too.
> We've moved on to higher abstraction grounds (fork join, work stealing, coroutines, etc...), Erlang is still stuck in the past.
Erlang can do fork/join, the schedulers can work steal, and erlang processes are approximately equal to coroutines -- I don't understand what you think is stuck in the past (other than the Prolog like syntax, but you get used to that).
Immutability is a big win for garbage collection and memory allocation however, since there are no circular references, mark and sweep isn't required and a simple (generational) copying collector can be used instead.
[1] actually you can look at other processes' dictionaries if you want, but that requires a lock on the destination process.
That Erlang forces you to build a shared nothing system is the real win for concurrency. Of course, you can do that with other languages too.
> We've moved on to higher abstraction grounds (fork join, work stealing, coroutines, etc...), Erlang is still stuck in the past.
Erlang can do fork/join, the schedulers can work steal, and erlang processes are approximately equal to coroutines -- I don't understand what you think is stuck in the past (other than the Prolog like syntax, but you get used to that).
Immutability is a big win for garbage collection and memory allocation however, since there are no circular references, mark and sweep isn't required and a simple (generational) copying collector can be used instead.
[1] actually you can look at other processes' dictionaries if you want, but that requires a lock on the destination process.
That still does not works. I have to maintain that code everyday. It does not work.
If I am totally honest. Most of the code where I was pretty sure I had figured out the whole mutable states and locks thing has another developer looking at it right now and cursing me under his breath.
I'm investigating an issue with a deployment where a process keeps crashing with the log.fatal("not reachable"). I already fixed 3 potential race conditions and still not sure if and when I'll get my next email basically telling me "haha it's still reachable". Yesterday my neighbor came to my door to ask if everything is okay because I cried "aber warum?" (but why?) out loud.
tl;dr: Don't be that programmer :)
tl;dr: Don't be that programmer :)
You have a more optimistic view of the average level of skill and knowledge of developers than I would. Basic concurrency bugs strike all the time, and a lot of people don't know what they don't know, until they've been burned a few times.
> how to do concurrency and parallelism with mutable state and locks. It's not as hard as you make it sound
This has to be sarcasm, right?
This has to be sarcasm, right?
But only if your program is embarrassingly parallel with at least N times available parallelism in the first place! If you have one of those it's already trivial to write a version that runs N times faster on N cores in C, Java, multi-process Python, whatever.
If your program has sequential or less parallel phases or needs to communicate then you are subject to Amdahl's law like you always were.
Armstrong has that claim in the Erlang book and I was gobsmacked to see it written down with no caveat or mention of the limits from Amdahl's law whatsoever. I was sure it was a joke and it would be followed by 'ha ha ha of course not - nobody knows how to achieve that despite decades of intensive research', but no it's a serious claim made with a straight face. Erlang's solved it!
Erlang helps you write parallel programs... but only if you program is entirely parallel in the first place.