The Erlang Rationale (2008) [pdf](drive.google.com)
drive.google.com
The Erlang Rationale (2008) [pdf]
https://drive.google.com/file/d/1zKsOgwZJ_YZ1bY3b3gNRjAxpn6VneR8b/view
40 comments
When I did Distributed Systems in Uni, it was in C (huuuge mess) and when I visited another Uni, they joked that they taught the course in Java, because "we're not savages".
I keep thinking how much more we could've actually learned about DS if we weren't fighting fires with C.
I keep thinking how much more we could've actually learned about DS if we weren't fighting fires with C.
I regard all distributed systems not written in a BEAM language with suspicion. Just seems a bit weird not to use a tech stack designed specifically for your problem. It's not everything, of course, but it's definitely a strike against a project
As I had to look it up: BEAM is the erlang virtual machine. A number of languages can be happily compiled to BEAM:
https://news.ycombinator.com/item?id=13446521
https://news.ycombinator.com/item?id=13446521
Over 20 languages, in fact.
https://gist.github.com/macintux/6349828#alternative-languag...
https://gist.github.com/macintux/6349828#alternative-languag...
It could be very difficult to implement an industrial-strength
Java on BEAM :-(
Java on BEAM :-(
I'd think so, given the immutability, lightweight processes, and other Erlang-specific features baked into the BEAM.
That's one reason I like Erlang and the BEAM so much: it's an opinionated language running on an opinionated VM that are made for each other, not a general-purpose system that has to be all things to all people.
The JVM is perfectly adequate as a kitchen sink. The BEAM doesn't need to be.
That's one reason I like Erlang and the BEAM so much: it's an opinionated language running on an opinionated VM that are made for each other, not a general-purpose system that has to be all things to all people.
The JVM is perfectly adequate as a kitchen sink. The BEAM doesn't need to be.
Although the JVM has many limitations, it has important capabilities
that are lacking in BEAM. The limitations of BEAM are severe,
especially for implementing Universal Intelligent Systems,
which is the primary challenge of Computer Science for this decade.
See the following:
Project Liftoff: Universal Intelligent Systems (UIS) by 2030
https://papers.ssrn.com/abstract=3428114
that are lacking in BEAM. The limitations of BEAM are severe,
especially for implementing Universal Intelligent Systems,
which is the primary challenge of Computer Science for this decade.
See the following:
Project Liftoff: Universal Intelligent Systems (UIS) by 2030
https://papers.ssrn.com/abstract=3428114
[deleted]
Can you explain a bit more how exactly this string representation works, and how it differs from how strings are represented in Haskell? Also, what are you applying Prolog to these days?
The "naive" and straight-forward way to internally represent a non-empty list is to use a structure of the form (CAR . CDR), where car is the first element of the list, and CDR is the rest of the list. So, you need 3 memory "cells" (i.e., pointers, addresses) for each element in the list: First, the indication that this is a structure of this form (a "cons-cell", in Lisp terminology), then, a pointer to the first element (or that first element itself, if it can be represented in a single tagged cell), and then a pointer to the remainder of the list, which is again of this shape.
On 64-bit systems, a cell takes 8 bytes. Therefore, when representing a list of bytes, this representation incurs a 3×8 = 24-fold overhead (!) over a plain sequence of bytes in memory. For example, to represent a 1GB file (which is not very large by contemporary standards) in memory, this representation takes 24GB, which is unacceptable. There are ways to compress this slightly (by "slightly", I mean a small factor, such as a factor of 2), but the fundamental problem remains: Instead of using bytes directly, this representation uses one cell per byte, which is an 8-fold overhead.
This is an overarching issue in all languages that represent strings as lists (lists of characters, or lists of integers), such as Erlang, Haskell and Prolog. At the same time, lists – especially lists of characters – are the desired representation we want to use in programs in these languages, because it means that the built-in functionality for reasoning about lists becomes directly available for strings too, and it helps to keep the number of language concepts small.
So, how to solve it? Ideally, certainly not by adding another data type to the language ("binary object", "binary string" etc.), because that would forfeit the advantages we get from using lists, and would also make the language harder to teach and learn.
Therefore, we ideally solve this with a better internal representation of strings: We internally (i.e., in the virtual machine), represent strings as sequences of plain bytes. Not cells, but direct bytes, using UTF-8 encoding. And when the virtual machine encounters this type internally, then it acts as if a list were encountered, making this array of bytes appear as a list (of characters, or of integers) to programmers. Internally, the system can "tell" whether a sequence of bytes denotes such a list, for example because they are allocated in a dedicated region of the memory, and suitably tagged pointers are used to refer to such sequences, as is the case for other data types such as large integers.
This much more compact representation was first explained by Ulrich Neumerkel in an issue for Scryer Prolog:
https://github.com/mthom/scryer-prolog/issues/24
The general idea can be applied to Haskell and Erlang too.
In another issue, Ulrich explained in more detail how to extend it to partial lists, i.e., Prolog terms that are not lists, but can still become lists:
https://github.com/mthom/scryer-prolog/issues/95
This latter issue is unique to logic programming languages like Prolog.
Trealla Prolog has already taken the idea to its extreme in that it uses mmap to map an entire file as is into memory when parsing, so that parsing from files becomes extremely efficient: The data from disk appears as a sequence of bytes in memory (the underlying operating system performs the mapping), and the Prolog virtual machine ensures that the byte sequence appears as a list of characters to Prolog programs. So, finally, we can use Prolog for its intended application: efficient, general and convenient parsing of text.
PostScript is another good existing example of this combination: Internally, the PostScript interpreter can represent strings very compactly, while to PostScript programs, strings can be used seamlessly like lists. For instance, we get:
On 64-bit systems, a cell takes 8 bytes. Therefore, when representing a list of bytes, this representation incurs a 3×8 = 24-fold overhead (!) over a plain sequence of bytes in memory. For example, to represent a 1GB file (which is not very large by contemporary standards) in memory, this representation takes 24GB, which is unacceptable. There are ways to compress this slightly (by "slightly", I mean a small factor, such as a factor of 2), but the fundamental problem remains: Instead of using bytes directly, this representation uses one cell per byte, which is an 8-fold overhead.
This is an overarching issue in all languages that represent strings as lists (lists of characters, or lists of integers), such as Erlang, Haskell and Prolog. At the same time, lists – especially lists of characters – are the desired representation we want to use in programs in these languages, because it means that the built-in functionality for reasoning about lists becomes directly available for strings too, and it helps to keep the number of language concepts small.
So, how to solve it? Ideally, certainly not by adding another data type to the language ("binary object", "binary string" etc.), because that would forfeit the advantages we get from using lists, and would also make the language harder to teach and learn.
Therefore, we ideally solve this with a better internal representation of strings: We internally (i.e., in the virtual machine), represent strings as sequences of plain bytes. Not cells, but direct bytes, using UTF-8 encoding. And when the virtual machine encounters this type internally, then it acts as if a list were encountered, making this array of bytes appear as a list (of characters, or of integers) to programmers. Internally, the system can "tell" whether a sequence of bytes denotes such a list, for example because they are allocated in a dedicated region of the memory, and suitably tagged pointers are used to refer to such sequences, as is the case for other data types such as large integers.
This much more compact representation was first explained by Ulrich Neumerkel in an issue for Scryer Prolog:
https://github.com/mthom/scryer-prolog/issues/24
The general idea can be applied to Haskell and Erlang too.
In another issue, Ulrich explained in more detail how to extend it to partial lists, i.e., Prolog terms that are not lists, but can still become lists:
https://github.com/mthom/scryer-prolog/issues/95
This latter issue is unique to logic programming languages like Prolog.
Trealla Prolog has already taken the idea to its extreme in that it uses mmap to map an entire file as is into memory when parsing, so that parsing from files becomes extremely efficient: The data from disk appears as a sequence of bytes in memory (the underlying operating system performs the mapping), and the Prolog virtual machine ensures that the byte sequence appears as a list of characters to Prolog programs. So, finally, we can use Prolog for its intended application: efficient, general and convenient parsing of text.
PostScript is another good existing example of this combination: Internally, the PostScript interpreter can represent strings very compactly, while to PostScript programs, strings can be used seamlessly like lists. For instance, we get:
GS>(hello) { = } forall
104
101
108
108
111
Thanks for the explanation.
This makes sense. In Haskell, strict ByteStrings are a pointer, plus an offset, plus a length. You can call the unpack function on them, which provides a lazy list (it reads from the pointer in blocks) of the characters pointed to. There is also a library that provides lazy ByteStrings from mmapped files.
The Prolog stuff sounds quite interesting. I suppose you haven't published anything about the cross-border stuff? Quite a while ago I was using linear programming to build a home energy optimisation system, modelling the workings of a battery, solar panel, energy usage and energy tariffs. The whole thing was a big min-cost flow network. The output was a series of commands to hopefully optimise energy usage.
I defined the programme with glpk-hs (a Haskell library which can output in the CPLEX format) and then ran the programmes with cbc. It was very complex, and I later realised that what I really needed was a more general system which combined linear and logic.
Someone on HN mentioned ECLiPSe, but I see that you mention SWI quite a bit on your website. What can you recommend for the sort of thing I was doing? I am not an expert in linear programming, and I also found all the options of CBC a bit overwhelming.
This makes sense. In Haskell, strict ByteStrings are a pointer, plus an offset, plus a length. You can call the unpack function on them, which provides a lazy list (it reads from the pointer in blocks) of the characters pointed to. There is also a library that provides lazy ByteStrings from mmapped files.
The Prolog stuff sounds quite interesting. I suppose you haven't published anything about the cross-border stuff? Quite a while ago I was using linear programming to build a home energy optimisation system, modelling the workings of a battery, solar panel, energy usage and energy tariffs. The whole thing was a big min-cost flow network. The output was a series of commands to hopefully optimise energy usage.
I defined the programme with glpk-hs (a Haskell library which can output in the CPLEX format) and then ran the programmes with cbc. It was very complex, and I later realised that what I really needed was a more general system which combined linear and logic.
Someone on HN mentioned ECLiPSe, but I see that you mention SWI quite a bit on your website. What can you recommend for the sort of thing I was doing? I am not an expert in linear programming, and I also found all the options of CBC a bit overwhelming.
The key attraction of the representation I mentioned is that on the level of Prolog programs, the compactly represented lists appear like any other lists, no matter how they are internally represented. So, internally, it is a sequence of raw bytes, and to Prolog programmers, it appears as a Prolog list of characters.
In Haskell, as long as we are forced to use ByteString or similar data types to get this efficient representation, the situation is not as convenient: ByteString is not the same type as String, and we have to convert manually between them, i.e., within Haskell programs. So, this causes overhead for programmers: We have to learn a new type with new interfaces, call an unpack function etc. A conceptually simpler solution would be to reduce the number of different types in Haskell programs, and instead implement String (i.e., [Char]) more efficiently internally so that it can be directly used instead.
For comparison, in Trealla Prolog, when we write the string "hello", then it is automatically internally represented in the efficient way as raw bytes. And on the Prolog level, it appears as a list of characters, corresponding to a String in Haskell:
So, the key attraction is that this efficient representation is transparent to the programmer in the sense that lists that are represented in this way are semantically completely indistinguishable from lists that are represented in the naive way internally. To Prolog programs, they have the exact same type, shape, length etc., they are only represented differently internally in the virtual machine. This idea can be used in Erlang and Haskell too to represent strings.
Currently, SICStus Prolog is the only Prolog system that can be reliably used for linear programming with constraints and is fast enough, see its library(clpq). I hope that a free Prolog system with a similar feature set will eventually become available. Currently, Scryer Prolog is looking very promising, especially due to the mentioned efficient internal representation of characters and its strong commitment to the Prolog ISO standard. There is ongoing work on a simplex library for Scryer Prolog, please see this issue and the linked file:
https://github.com/mthom/scryer-prolog/issues/463
The library contains functionality for solving transportation and assignment problems via network flow algorithms. This sounds similar to your use case, and I hope it works for you.
In Haskell, as long as we are forced to use ByteString or similar data types to get this efficient representation, the situation is not as convenient: ByteString is not the same type as String, and we have to convert manually between them, i.e., within Haskell programs. So, this causes overhead for programmers: We have to learn a new type with new interfaces, call an unpack function etc. A conceptually simpler solution would be to reduce the number of different types in Haskell programs, and instead implement String (i.e., [Char]) more efficiently internally so that it can be directly used instead.
For comparison, in Trealla Prolog, when we write the string "hello", then it is automatically internally represented in the efficient way as raw bytes. And on the Prolog level, it appears as a list of characters, corresponding to a String in Haskell:
?- "hello" = [h,e,l,l,o].
true.
So, the key attraction is that this efficient representation is transparent to the programmer in the sense that lists that are represented in this way are semantically completely indistinguishable from lists that are represented in the naive way internally. To Prolog programs, they have the exact same type, shape, length etc., they are only represented differently internally in the virtual machine. This idea can be used in Erlang and Haskell too to represent strings.
Currently, SICStus Prolog is the only Prolog system that can be reliably used for linear programming with constraints and is fast enough, see its library(clpq). I hope that a free Prolog system with a similar feature set will eventually become available. Currently, Scryer Prolog is looking very promising, especially due to the mentioned efficient internal representation of characters and its strong commitment to the Prolog ISO standard. There is ongoing work on a simplex library for Scryer Prolog, please see this issue and the linked file:
https://github.com/mthom/scryer-prolog/issues/463
The library contains functionality for solving transportation and assignment problems via network flow algorithms. This sounds similar to your use case, and I hope it works for you.
I think it is nice in Haskell to retain the ability to control the underlying representation of a string. The laziness allows the native linked list representation without the user having to know about the underlying representation. Your memory mapping function can simply return a lazy linked list of chars in Haskell as well. If the underlying representation turns out to be inefficient on the current system, then that can always be tuned by the user. Maybe in Prolog it is sufficient to leave all of that to the runtime system.
Thanks for the Prolog tips. I had a look at clpqr, and could not find any reference to boolean variables and operators. Maybe clpqr intersects with another system whose docs do refer to boolean variables. I could not see how to combine clpqr with clpb. Basically what I would like is for certain linear constraints to be active only if other chosen boolean expressions are true.
Example: Choose energy price based on maximum power usage over past month. There are some Japanese electricity tariffs which do this. There would be a partition of maximum power usage ranges, and each range is associated with a price. Associate a binary variable to each range. Ensure exactly one of the variables is true (sum = 1). Create a range constraint over the maximum power usage for each partition, and multiply the bounds of that constraint with the associated binary variable. When a binary variable is set to zero, you have a trivial constraint.
It is indeed possible to represent boolean constraints as linear constraints over binary variables and develop a DSL to translate in this direction. I am wondering whether a system exists which allows easy expression of linear and boolean combined, and which provides tools for solving them efficiently.
Thanks for the Prolog tips. I had a look at clpqr, and could not find any reference to boolean variables and operators. Maybe clpqr intersects with another system whose docs do refer to boolean variables. I could not see how to combine clpqr with clpb. Basically what I would like is for certain linear constraints to be active only if other chosen boolean expressions are true.
Example: Choose energy price based on maximum power usage over past month. There are some Japanese electricity tariffs which do this. There would be a partition of maximum power usage ranges, and each range is associated with a price. Associate a binary variable to each range. Ensure exactly one of the variables is true (sum = 1). Create a range constraint over the maximum power usage for each partition, and multiply the bounds of that constraint with the associated binary variable. When a binary variable is set to zero, you have a trivial constraint.
It is indeed possible to represent boolean constraints as linear constraints over binary variables and develop a DSL to translate in this direction. I am wondering whether a system exists which allows easy expression of linear and boolean combined, and which provides tools for solving them efficiently.
Is this equivalent to a vector backed list?
it's a linked list of utf codepoints. It's actually a terrible representation for most purposes. More modern parts of the BEAM (e.g. tcp/udp) typically give the option to emit a "binary", which is a immutable contiguous memory region of bytes, and Elixir, which also runs on the BEAM defaults to using "Strings" (cap S) which are "binary"s, but semantically interpreted to contain UTF-8 encoded utf codepoints.
For some things, though lists are REALLY good. BEAM languages constructs are generally immutable, so if you pass your string (or String), to a function that then changes the contents, it doesn't affect the value seen by the original function after it returns. This is a GOOD thing as it reduces the cognitive burden.
Conversely, performance of binaries in this immutability regime can get hairy because as you start concatenating them or editing them you have to make tons of immutable copies of stuff, which is no fun for anyone. (Literally last month we had a problem in prod because megabytes-sized JSON strings were being copied around after minor appends).
The middleground is something that erlang calls an "iolist" which is a (list of (iolist or binaries or utf codepoints)). Elixir has "iodata" which is (iolist or binary) This is nice, because you have o(1) append, o(1) prepend, and for long stretches that aren't going to change, you can use binaries instead of awkward utf codepoint list structures.
For some things, though lists are REALLY good. BEAM languages constructs are generally immutable, so if you pass your string (or String), to a function that then changes the contents, it doesn't affect the value seen by the original function after it returns. This is a GOOD thing as it reduces the cognitive burden.
Conversely, performance of binaries in this immutability regime can get hairy because as you start concatenating them or editing them you have to make tons of immutable copies of stuff, which is no fun for anyone. (Literally last month we had a problem in prod because megabytes-sized JSON strings were being copied around after minor appends).
The middleground is something that erlang calls an "iolist" which is a (list of (iolist or binaries or utf codepoints)). Elixir has "iodata" which is (iolist or binary) This is nice, because you have o(1) append, o(1) prepend, and for long stretches that aren't going to change, you can use binaries instead of awkward utf codepoint list structures.
Haskell allows you to build a (lazy) linked list of characters sourced in blocks from strict ByteStrings, which are pointers to bytes with an offset and length. Leaving out the garbage collection of the cons cells (I would hope that the cons cells are reused in some situations, but I don't know the specifics), I think this is quite a nice interface.
iolist sounds like lazy ByteStrings in Haskell. They are lists of strict ByteStrings.
iolist sounds like lazy ByteStrings in Haskell. They are lists of strict ByteStrings.
I had the pleasure of using Erlang in Data Structures, Graph Theory and Distributed Systems classes in university, had been lucky with the professor.
Erlang made a fundamentally good design decision in reinventing
the Actor paradigm [Hewitt, Bishop, and Steiger 1973].
However, many of the language design decisions were compromised.
It's amazing that Armstrong, et. al. got as far as they did
without formal foundational principles. Unfortunately, Erlang
still lacks a rigorous formal foundation :-(
the Actor paradigm [Hewitt, Bishop, and Steiger 1973].
However, many of the language design decisions were compromised.
It's amazing that Armstrong, et. al. got as far as they did
without formal foundational principles. Unfortunately, Erlang
still lacks a rigorous formal foundation :-(
The author says that he sees no need to include types in the language. There have been efforts to create typed languages which run on the Erlang BEAM virtual machine. I would be interested to know more about why the author thought that types were not useful.
From one perspective types (and I mean proofs encoded in types) is a compile time measure against bugs, while supervisors is a runtime measure. It's much easier and more productive to not do proofs and rely on runtime architecture for reliability, it adds relatively little complexity and asks very little from developers, it stays almost the same as regular software development. And it's still necessary to do regardless whether you have proofs or not.
And types like machine types are of course irrelevant for BEAM, well, maybe they'll start to matter a bit with JIT, but probably still not, BEAM with JIT is still not going to be a performance miracle.
And types like machine types are of course irrelevant for BEAM, well, maybe they'll start to matter a bit with JIT, but probably still not, BEAM with JIT is still not going to be a performance miracle.
Strong types are very important for concurrency.
However, by themselves, strong types do not provide proofs.
See the following for proofs of strongly-typed concurrent systems:
https://papers.ssrn.com/abstract=3418003
Strongly-typed programming languages need to provide
additional capabilities for error processing and recovery.
However, by themselves, strong types do not provide proofs.
See the following for proofs of strongly-typed concurrent systems:
https://papers.ssrn.com/abstract=3418003
Strongly-typed programming languages need to provide
additional capabilities for error processing and recovery.
Thanks for posting. I can't find it now, but we mention somewhere in the paper that typing makes it more difficult to fake messages. Typing allows you to make stronger proofs over the properties and behaviour of actor systems. Sorry if I've missed the point: What are the other direct implications to security which typing brings?
Excellent question! Types can provide very strong security
for messages by having the type system do crypto so that
applications can benefit without requiring application
programmers to manage keys and crypto code.
for messages by having the type system do crypto so that
applications can benefit without requiring application
programmers to manage keys and crypto code.
Thanks. That makes sense.
You are very welcome.
There is additional information available here:
https://professorhewitt.blogspot.com/
There is additional information available here:
https://professorhewitt.blogspot.com/
Without trying to sound too flamebaity, as typed versus untyped has been discussed endlessly, I would like to see a study which compares development time and reliability between a Erlang and another typed language running on BEAM. That is, typed versus untyped for the sort of systems which are built with Erlang.
So I can say from experience, none of the production issues we ran into in two and a half years of Erlang would have been solved by better typing.
That said, our own practices meant we included a lot of unit and API level tests, and some of the issues I personally caught and fixed when testing might have been avoided by typing, and were, eventually, when we added Dialyzer about a year and a half into the project.
Dialyzer itself was worth it to me only insofar as type specs gave me a language to express the typing I was going for, rather than leaving it implicit. It also helped me understand what my colleagues were trying to do in some cases. The explicitness helped me feel better about what I was doing as well; in hindsight I don't think it sped up development, nor did it reduce bugs (though it likely reduced testing time...but that's probably a wash given the time to get the spec right, and update it as data structures evolved).
Can't speak to using another language on the BEAM, whose typing isn't optimistic like Dialyzer, but my experience with Dialyzer, at least, left me feeling subjectively better about the code, but objectively and in hindsight I don't feel it gave us measurable improvement. Just more confidence. Which may be enough of a gain; confidence in the system is what you want when you're optimizing for reliability, even if it's not really a gain on that front.
That said, our own practices meant we included a lot of unit and API level tests, and some of the issues I personally caught and fixed when testing might have been avoided by typing, and were, eventually, when we added Dialyzer about a year and a half into the project.
Dialyzer itself was worth it to me only insofar as type specs gave me a language to express the typing I was going for, rather than leaving it implicit. It also helped me understand what my colleagues were trying to do in some cases. The explicitness helped me feel better about what I was doing as well; in hindsight I don't think it sped up development, nor did it reduce bugs (though it likely reduced testing time...but that's probably a wash given the time to get the spec right, and update it as data structures evolved).
Can't speak to using another language on the BEAM, whose typing isn't optimistic like Dialyzer, but my experience with Dialyzer, at least, left me feeling subjectively better about the code, but objectively and in hindsight I don't feel it gave us measurable improvement. Just more confidence. Which may be enough of a gain; confidence in the system is what you want when you're optimizing for reliability, even if it's not really a gain on that front.
After years of using statically type checked languages like Haskell, I’ve observed that I am quite lazy. If I went to a language without static type checking today, I wouldn’t trust myself to provide the right function arguments in the right order all the time, or to send the right message types, especially after refactoring. It would be a nightmare for me to have to write unit tests for this. Maybe that would improve over time, or maybe I’m just a crap programmer.
Well, as strong a type system as Haskell likely prevents enough bugs in the code to let you feel safe without unit tests. That's what I'm talking about; that -feeling- of safety.
Do any of these languages help prevent you from writing code that simply does the wrong thing? No. You multiplied instead of adding; no static type checker is going to catch that, because you semantically told the computer to do the wrong thing.
That's why I wrote extensive tests; I needed to make sure the code did the right thing, reliably. It's also why we chose Erlang; supervisor trees and immutability meant that as long as we tested and felt good about the happy path, the unhappy paths would generally take care of themselves.
As an aside, in Haskell, you actively have to make tradeoffs around laziness, too. Both these data types are Integer -> I either have to create a custom data type to distinguish them (increasing level of effort as a dev), or I risk getting them in the wrong order if they're both passed into the same function. Named parameters might be a better solution here, and that can be done regardless of typing.
Do any of these languages help prevent you from writing code that simply does the wrong thing? No. You multiplied instead of adding; no static type checker is going to catch that, because you semantically told the computer to do the wrong thing.
That's why I wrote extensive tests; I needed to make sure the code did the right thing, reliably. It's also why we chose Erlang; supervisor trees and immutability meant that as long as we tested and felt good about the happy path, the unhappy paths would generally take care of themselves.
As an aside, in Haskell, you actively have to make tradeoffs around laziness, too. Both these data types are Integer -> I either have to create a custom data type to distinguish them (increasing level of effort as a dev), or I risk getting them in the wrong order if they're both passed into the same function. Named parameters might be a better solution here, and that can be done regardless of typing.
Yes. I agree that you certainly need to write tests for algorithmic code unless you are formally proving it. I definitely would not feel safe about a data structure I wrote in Haskell without writing a large suite of tests. If your tests cover a good portion of the code, then errors arising from incorrect arguments will be quickly found. It's nice to have that eliminated by the compiler when possible though.
Excellent point!
Attempts are being made to graft types onto Erlang. However,
they are limited by compatibility requirements with existing
large existing programs and libraries :-(
Attempts are being made to graft types onto Erlang. However,
they are limited by compatibility requirements with existing
large existing programs and libraries :-(
This is an excellent summary of rationale for
Erlang with the goals stated at the beginning of the article!
However, the Erlang language itself is now rather dated :-(
In order to implement next generation Universal
Intelligent Systems [Hewitt 2019], the following extensions
to Erlang are needed:
• Automatic reclamation of processes. For example,
Erlang uses Process Identifiers (PIDs) to communicate between
processes. However, a process that has a process identifier
(say ProcessIdentifier) of a process with which it
communicates can launch a denial of service attack to kill
that process using exit(ProcessIdentifier, kill). An Erlang
process can be orphaned if its Process Identifier (PID)
becomes inaccessible. By default, the send operation in
Erlang always succeeds (even if the target is a non-existing process).
• Strong parameterized types with no type Any.
• Region of mutual exclusion for an Erlang process.
An Erlang process does not have a region of mutual exclusion,
which can lead to internal conflicts within processes.
• Behavior change using variables in a region of mutual
exclusion instead of having to create auxiliary processes to
hold state with callbacks. [Swain 2014] In particular,
o Better support for holes in the region of mutual exclusion
of an Actor implementation. For example, in Erlang it is
necessary for application programmers to explicitly code a
request handler for each re-entry into the region of mutual
exclusion potentially enabling cyberattacks from outside the
process.
o Better ways to upgrade Actors in place without losing work in progress.
Erlang with the goals stated at the beginning of the article!
However, the Erlang language itself is now rather dated :-(
In order to implement next generation Universal
Intelligent Systems [Hewitt 2019], the following extensions
to Erlang are needed:
• Automatic reclamation of processes. For example,
Erlang uses Process Identifiers (PIDs) to communicate between
processes. However, a process that has a process identifier
(say ProcessIdentifier) of a process with which it
communicates can launch a denial of service attack to kill
that process using exit(ProcessIdentifier, kill). An Erlang
process can be orphaned if its Process Identifier (PID)
becomes inaccessible. By default, the send operation in
Erlang always succeeds (even if the target is a non-existing process).
• Strong parameterized types with no type Any.
• Region of mutual exclusion for an Erlang process.
An Erlang process does not have a region of mutual exclusion,
which can lead to internal conflicts within processes.
• Behavior change using variables in a region of mutual
exclusion instead of having to create auxiliary processes to
hold state with callbacks. [Swain 2014] In particular,
o Better support for holes in the region of mutual exclusion
of an Actor implementation. For example, in Erlang it is
necessary for application programmers to explicitly code a
request handler for each re-entry into the region of mutual
exclusion potentially enabling cyberattacks from outside the
process.
o Better ways to upgrade Actors in place without losing work in progress.
"However, a process that has a process identifier
(say ProcessIdentifier) of a process with which it
communicates can launch a denial of service attack to kill
that process"
The BEAM does not promise (or provide) secure isolation between processes within a single VM. It's even called out in their bug report policy:
https://github.com/erlang/otp/wiki/FAQ:-What-kind-of-patches...
"An Erlang process does not have a region of mutual exclusion, which can lead to internal conflicts within processes."
I don't follow the use of "mutual exclusion" in this statement; the only thing that can write to a process's stack or heap is the process - what kind of "conflicts" do you mean?
"For example, in Erlang it is necessary for application programmers to explicitly code a request handler for each re-entry into the region of mutual exclusion"
This sentence doesn't help either, because the previous statement was that the processes don't _have_ the region this objects to writing handlers for entering...
The BEAM does not promise (or provide) secure isolation between processes within a single VM. It's even called out in their bug report policy:
https://github.com/erlang/otp/wiki/FAQ:-What-kind-of-patches...
"An Erlang process does not have a region of mutual exclusion, which can lead to internal conflicts within processes."
I don't follow the use of "mutual exclusion" in this statement; the only thing that can write to a process's stack or heap is the process - what kind of "conflicts" do you mean?
"For example, in Erlang it is necessary for application programmers to explicitly code a request handler for each re-entry into the region of mutual exclusion"
This sentence doesn't help either, because the previous statement was that the processes don't _have_ the region this objects to writing handlers for entering...
In general, an Actor receiving a message enters its region of
mutual exclusion (which can have holes), which is explained
here:
https://papers.ssrn.com/abstract=3459566
Is there a rigorous formalization of Erlang message passing?
mutual exclusion (which can have holes), which is explained
here:
https://papers.ssrn.com/abstract=3459566
Is there a rigorous formalization of Erlang message passing?
I think Prof. Hewitt should one of these days actually write a working system in Erlang or Elixir. I suggest to Carl that he write something easy (and potentially useful), like a DNS server, as a starting point. Raft might also be fun, that's about 350-500 lines to do as it is in the paper, and maybe about 600 with the upgrades necessary to get a correct consensus protocol.
There are a number of very useful systems written for BEAM!
Consensus protocols are too slow for Universal Intelligent Systems (UIS) :-(
See the following for UIS:
https://papers.ssrn.com/abstract=3428114
Consensus protocols are too slow for Universal Intelligent Systems (UIS) :-(
See the following for UIS:
https://papers.ssrn.com/abstract=3428114
Yeah but have you written one? Give it a shot sometime. It's fun, I promise. You'd be surprised at how far an imperfect actor system can go and how easy actors (or in this case accidental-almost-actors) makes things!
The Actor paradigm is indeed very powerful. It has been
reinvented many times since it was first published in 1973.
For example, Erlang was a reinvention that put the paradigm
into many practical applications :-)
However, only more recently has it been characterized up to a
unique isomorphism. See the following article:
https://papers.ssrn.com/abstract=3459566
The most important code to be written now is for the
foundations of Universal Intelligent Systems :-)
reinvented many times since it was first published in 1973.
For example, Erlang was a reinvention that put the paradigm
into many practical applications :-)
However, only more recently has it been characterized up to a
unique isomorphism. See the following article:
https://papers.ssrn.com/abstract=3459566
The most important code to be written now is for the
foundations of Universal Intelligent Systems :-)
[deleted]
“Very early on we decided that strings were lists. This was partly environment as many of the languages we were using, such as Prolog, use lists and partly practical as lists are a very powerful data type.”
Lists are extremely convenient to handle in Erlang, and they are an ideal representation of text for many purposes. This convenient representation can also be implemented efficiently, in such a way that compact UTF-8 based encoding is used internally, while the string appears as a list to programmers. Two recent Prolog systems, Scryer Prolog and Trealla Prolog, do this already. It is to be hoped that future Erlang and also Haskell implementations will adopt this idea to combine convenience with efficiency, eliminating the need for additional types to compactly represent binary objects, and allowing fast reading from files via mmap.
In university, we had a distributed systems class with many very involved exercises. They were involved mostly because we had to use Java: Out of interest, I also solved the exercises in Erlang, and that usually allowed much shorter and more convenient solutions.