Advanced Programming Languages (2009)(matt.might.net)
matt.might.net
Advanced Programming Languages (2009)
http://matt.might.net/articles/best-programming-languages/
138 comments
I've seen a definite shift in what "artificial intelligence" means.
Norvig's PAIP is all about manipulating symbols. It's so fundamental that in any other language you'd have to Greenspun an interpreter first, just to solve the same problems in the same way.
Today, AI seems to mostly mean matrices. If that's all you need, then you can do it in any language, especially if you're just wrapping old C or Fortran libraries.
I haven't done any new-style AI but I get the feeling that most "research" is in finding new nails to pound with the same hammer. That's a legitimate area of research, but it also should be no surprise that the concept of "AI research" expanded so much in recent years because it changed from "learn to build your own lightsaber" to "pick up a hammer and hit something with it".
Norvig's PAIP is all about manipulating symbols. It's so fundamental that in any other language you'd have to Greenspun an interpreter first, just to solve the same problems in the same way.
Today, AI seems to mostly mean matrices. If that's all you need, then you can do it in any language, especially if you're just wrapping old C or Fortran libraries.
I haven't done any new-style AI but I get the feeling that most "research" is in finding new nails to pound with the same hammer. That's a legitimate area of research, but it also should be no surprise that the concept of "AI research" expanded so much in recent years because it changed from "learn to build your own lightsaber" to "pick up a hammer and hit something with it".
Behind matrices, there is a lot of symbolic processing in the data cleanup part. Most of the time, Python suffices.
Most of the time, people does not leave Python when it is not satisfactory. They stick with it because they have no clue, most of the time.
For example, it is not quite natural for Python folks to even think about creating rewrite rules (as in Haskell) for programs to stay at higher level of expression. E.g., if you use dictionary with default value, why not lift all operations up (inline, so to say) and avoid calling these that are overloaded? It bit me many times, yet other users of Python just gave me blank stare when I tried to explain these things.
Most of the time, people does not leave Python when it is not satisfactory. They stick with it because they have no clue, most of the time.
For example, it is not quite natural for Python folks to even think about creating rewrite rules (as in Haskell) for programs to stay at higher level of expression. E.g., if you use dictionary with default value, why not lift all operations up (inline, so to say) and avoid calling these that are overloaded? It bit me many times, yet other users of Python just gave me blank stare when I tried to explain these things.
Sorry, can you explain what you mean here? I am always trying to get better at both python and functional programming, but I don't know what you mean about lifting operations up/inlining them in this case with defaultdicts.
Take a look here to get a feel of what rewrite rules are: https://downloads.haskell.org/~ghc/7.0.1/docs/html/users_gui...
Haskell has two very nice libraries, bytestring and vector which use rewrite rules to have performance comparable to C code.
Operations on dictionaries with default values implemented using inheritance from dictionary. This means that instead of just having conditional expression on the right side I have a call to virtual function and there is an expression on the right side of assignment or return statement.
Most of OO virtual machines can JIT things like these into efficient machine code, by specializing. The same is also quite possible with bytecode like in CPython - by optimization in the compiler or by introduction of rewrite rules like above for library implementors to specify rules that apply to any (transformed) user code.
Haskell has two very nice libraries, bytestring and vector which use rewrite rules to have performance comparable to C code.
Operations on dictionaries with default values implemented using inheritance from dictionary. This means that instead of just having conditional expression on the right side I have a call to virtual function and there is an expression on the right side of assignment or return statement.
Most of OO virtual machines can JIT things like these into efficient machine code, by specializing. The same is also quite possible with bytecode like in CPython - by optimization in the compiler or by introduction of rewrite rules like above for library implementors to specify rules that apply to any (transformed) user code.
So something like Cython or numba?
What are these? Do they support rewrite rules on programs?
I meant them in the context of your last paragraph; they are tools that optimize Python for performance. Cython is a compiled superset of Python, while numba is a JIT compiler that works on a subset of Python.
I must admit I don't quite understand what rewrite rules actually are; are they akin to macros?
I must admit I don't quite understand what rewrite rules actually are; are they akin to macros?
Macros have local scope. They are evaluated to program text only once, probably, lazily.
Rewrite rules fire when compiler encounters a chance for them to fire, at any point of (optimizing) program transformation.
Due to this fact, rewrite rules are not local - compiler can transform the inner structure of a program changing the locality of program statements.
Rewrite rules fire when compiler encounters a chance for them to fire, at any point of (optimizing) program transformation.
Due to this fact, rewrite rules are not local - compiler can transform the inner structure of a program changing the locality of program statements.
bs = map g xs
... many other lines of program text not reassigning bs
as = map f bs
So they -- like macros -- rewrite the source code into something different before the compilation, but -- unlike macros -- only do that under certain circumstances. Would this be a sensible approximation? I'm just trying to understand what they're doing, not why or how. Thank you!
Ooh, good observation. I never really thought about how many problems I've had explaining why I'm lifting something or how it makes things better.
There seems to be a misconception among non machine learning programmers that ML is all about multiplying matrices. In reality, there is a lot of architecture involved and having a programming languages that facilitates that would be enormously helpful.
Yes, there is a lot of multiplying matrices. But you need to think of a matrix as like a <div> or something and multiplying matrices and placing two <div>s next to each other. Then, think of how to create a dynamic website from that. There is a lot of architecturing involved other than multiplying matrices.
Yes, there is a lot of multiplying matrices. But you need to think of a matrix as like a <div> or something and multiplying matrices and placing two <div>s next to each other. Then, think of how to create a dynamic website from that. There is a lot of architecturing involved other than multiplying matrices.
> why aren't the "more expressive" (which in this post seems to be a synonym for "functional") languages used in these fields?
90% of all programmers are cargo-culters whose knowledge of programming is gained from the answers they cut-and-paste from Stack Exchange or Quora. This is as true of "data scientisis" as it is of "web developers." The result is that there is a network effect making certain answers determinant of the language or monthly flavour of xxxScript that gets pasted most often, and reduces the domain in which further answers are examined.
If you want it in simpler terms: the new dudes looked around and saw all the cool kids were using Python and only a few boffins (UK) or nerds (USA) were using anything else, so they knew who to crib the answers from if they also wanted to look valley cool.
90% of all programmers are cargo-culters whose knowledge of programming is gained from the answers they cut-and-paste from Stack Exchange or Quora. This is as true of "data scientisis" as it is of "web developers." The result is that there is a network effect making certain answers determinant of the language or monthly flavour of xxxScript that gets pasted most often, and reduces the domain in which further answers are examined.
If you want it in simpler terms: the new dudes looked around and saw all the cool kids were using Python and only a few boffins (UK) or nerds (USA) were using anything else, so they knew who to crib the answers from if they also wanted to look valley cool.
If anything, Python in science is the opposite of cargo cult - it's more like pragmatic to a fault.
Its ubiquituous in academia/science, has a low barrier to entry, and a large amount and arguably often unmatched quality of specialized libraries/frameworks/tools for all kinds of scientific fields. So the first go to language/ecosystem will of course almost always be python.
I'm definitely not the biggest fan of Python (the language). And there certainly are more elegant/fitting languages for many problem classes. But the thing is, from my experience many people in the field are quite aware of this. But it simply doesn't matter if it's the best language from a developer POV. It just has to be good enough to get the job done, apart from this the points above are simply more imporant.
And it's hard to understate the importance of ubiquity in this case:
Python is the code equivalent of a lingua franca in science.
Its ubiquituous in academia/science, has a low barrier to entry, and a large amount and arguably often unmatched quality of specialized libraries/frameworks/tools for all kinds of scientific fields. So the first go to language/ecosystem will of course almost always be python.
I'm definitely not the biggest fan of Python (the language). And there certainly are more elegant/fitting languages for many problem classes. But the thing is, from my experience many people in the field are quite aware of this. But it simply doesn't matter if it's the best language from a developer POV. It just has to be good enough to get the job done, apart from this the points above are simply more imporant.
And it's hard to understate the importance of ubiquity in this case:
Python is the code equivalent of a lingua franca in science.
I think you're leaving out the most salient fact.
Scientists are smart within their fields, but they're usually terrible at programming. I once saw a hundred-page source file (one file for everything) that defined no functions. And of course it was in python.
So if you're arguing that python is good if you want to exchange source code with people... I guess you might do that? Personally, nobody has asked me for my code in quite a while, so I do all my work in Calysto scheme (which is written in python and can accesss the whole library ecosystem).
If, on the other hand, you're saying scientists' opinions on what constitutes a good programming language ought to be taken seriously, then you're wrong. Scientists are crappy programmers for the same reason they're crappy writers. Programming and writing are tools they use to get their real work done. Most of them master the minimum they need and spend most of their time in the lab
Scientists are smart within their fields, but they're usually terrible at programming. I once saw a hundred-page source file (one file for everything) that defined no functions. And of course it was in python.
So if you're arguing that python is good if you want to exchange source code with people... I guess you might do that? Personally, nobody has asked me for my code in quite a while, so I do all my work in Calysto scheme (which is written in python and can accesss the whole library ecosystem).
If, on the other hand, you're saying scientists' opinions on what constitutes a good programming language ought to be taken seriously, then you're wrong. Scientists are crappy programmers for the same reason they're crappy writers. Programming and writing are tools they use to get their real work done. Most of them master the minimum they need and spend most of their time in the lab
"large numbers of libraries" is almost a subgenre of cargo cult. You're not programming, you're using code you assume does a thing because either it says so or you saw someone else do; but not doing it yourself. Python is paying my mortgage right now and I enjoy using it, but the community around it is no more enlightened with respect to cargo culting than any other I've seen.
Yes. So many developers think, I string together some APIs into something that looks like an app, I've done something. Forgetting those libraries are designed/tested to work only on whatever subset of the problem space they were used for previously. They probably don't handle your case; they don't have explicit capacity promises; they don't work together (don't respect each others backpressure signals, don't understand each others error spaces etc).
I read open source, then extract the bits of IP I think was useful. Often rewriting it to fit my case. Its useful source, but as an example to help me write my app.
{ Please don't use Linux as a strawman (You rewrote Linux?! I don't believe it!) That's the single largest, most successful (along with maybe 5 others) open source project. But absolutely not representative of the 1M other projects out there. }
I read open source, then extract the bits of IP I think was useful. Often rewriting it to fit my case. Its useful source, but as an example to help me write my app.
{ Please don't use Linux as a strawman (You rewrote Linux?! I don't believe it!) That's the single largest, most successful (along with maybe 5 others) open source project. But absolutely not representative of the 1M other projects out there. }
Seems to me like if you want to look cool nowadays you would use Rust not Python.
Different communities have different needs. Data scientists, or physicists, or biologists don't care about memory safety, the programming language is a tool to run a computation for a limited time (not a server). The tool should not get in the way of fast iteration and rapid prototyping which many languages do in varying amount: C with memory management, Haskell with category theory, Rust with borrow checking, C++ with being C++, ...
Basically you are criticizing data scientists for choosing to not invent a new thing every time and reusing an established ecosystem.
Different communities have different needs. Data scientists, or physicists, or biologists don't care about memory safety, the programming language is a tool to run a computation for a limited time (not a server). The tool should not get in the way of fast iteration and rapid prototyping which many languages do in varying amount: C with memory management, Haskell with category theory, Rust with borrow checking, C++ with being C++, ...
Basically you are criticizing data scientists for choosing to not invent a new thing every time and reusing an established ecosystem.
So why aren't they using Clojure?
I agree that people typically want to get from start to solving their problem as quickly as possible, and don't care about things that don't obviously contribute to that. But that doesn't mean they are perfect rational agents, or immune to cargo-culting, or that a little time invested in learning something else wouldn't pay off.
I agree that people typically want to get from start to solving their problem as quickly as possible, and don't care about things that don't obviously contribute to that. But that doesn't mean they are perfect rational agents, or immune to cargo-culting, or that a little time invested in learning something else wouldn't pay off.
What does Clojure bring?
Here is my data science stack https://github.com/mratsim/arch-Data-Science
Let's go over it:
- Is there a mature tensor/multidimensional array library in Clojure? Is there a Dataframe library?
- Are they compatible with common formats: .npy, .csv, .hdf5 and sqlite databases
- Is there a fast PCA and DBSCAN algorithm?
- Can you pass those arrays to XGBoost or LightGBM for Gradient Boosting based classification or regression.
- Do they support out-of-core processing (i.e. memory-mapped arrays) for processing that takes more than your RAM (happens often with geolocalisation)
- Can you use PyTorch through Clojure? Can you use Tensorflow? What's the GPU computing story
- Does your ML library support things like One-Hot-Encoding, cross-validation? Or more advanced like Target Encoding?
- NLP: stanfordNLP wrapper? spacy? Huggingfaces? Rasa-nlu? sentencepieces?
- Deep learning: Do you support image augmentation like Nvidia DALi or Albumentations? Mixed precision training like Nvidia APEx
- Vision: How is the OpenCV support? Does it interface with Clojure multidimensional arrays? Do you have morphological operations like erosion or dilation?
- Visualization: is there a dataviz library like plotly, ggplot2 or Altair/Vega?
And more importantly
- Are there tutorials and courses in Clojure from beginner to advanced concept? Even Julia is struggling with that.
Here is my data science stack https://github.com/mratsim/arch-Data-Science
Let's go over it:
- Is there a mature tensor/multidimensional array library in Clojure? Is there a Dataframe library?
- Are they compatible with common formats: .npy, .csv, .hdf5 and sqlite databases
- Is there a fast PCA and DBSCAN algorithm?
- Can you pass those arrays to XGBoost or LightGBM for Gradient Boosting based classification or regression.
- Do they support out-of-core processing (i.e. memory-mapped arrays) for processing that takes more than your RAM (happens often with geolocalisation)
- Can you use PyTorch through Clojure? Can you use Tensorflow? What's the GPU computing story
- Does your ML library support things like One-Hot-Encoding, cross-validation? Or more advanced like Target Encoding?
- NLP: stanfordNLP wrapper? spacy? Huggingfaces? Rasa-nlu? sentencepieces?
- Deep learning: Do you support image augmentation like Nvidia DALi or Albumentations? Mixed precision training like Nvidia APEx
- Vision: How is the OpenCV support? Does it interface with Clojure multidimensional arrays? Do you have morphological operations like erosion or dilation?
- Visualization: is there a dataviz library like plotly, ggplot2 or Altair/Vega?
And more importantly
- Are there tutorials and courses in Clojure from beginner to advanced concept? Even Julia is struggling with that.
In other words, you'd consider switching to Java if it was exactly like the system you already use.
And note the question you didn't ask: does Java do anything better than python? Didn't even cross your mind.
And note the question you didn't ask: does Java do anything better than python? Didn't even cross your mind.
The ceremony to start using Java is significantly worse than Python or Clojure.
Science is about functions, not objects, a language that requires all functions to be associated with an object is a non-starter.
Science is about functions, not objects, a language that requires all functions to be associated with an object is a non-starter.
He wouldn't because Java gets in your way much more than Python does. A better language needs to offer these things AND be a better language.
Agreed that Java is an obnoxious language. I think originally the grandparent and I were talking about Clojure, which uses the Java library ecosystem, and I started talking about Java libs.
Clojure is certainly the context I'd use Java's machine learning libraries in.
Clojure is certainly the context I'd use Java's machine learning libraries in.
Well then, instead of putting words in his mouth like "exactly the same", go down his list.
> - Is there a mature tensor/multidimensional array library in Clojure? Is there a Dataframe library?
Is there one in Java? If so, what?
And so on with his other items. It doesn't have to be exactly the same (offering him the benefit of the doubt), but it has to offer substantially the same functionality with substantially the same performance. So, you want to use Java libraries from Clojure? Go for it. Where are the things he needs?
> - Is there a mature tensor/multidimensional array library in Clojure? Is there a Dataframe library?
Is there one in Java? If so, what?
And so on with his other items. It doesn't have to be exactly the same (offering him the benefit of the doubt), but it has to offer substantially the same functionality with substantially the same performance. So, you want to use Java libraries from Clojure? Go for it. Where are the things he needs?
Not sure what they cover, but there are surely libraries available,
https://deeplearning4j.org/
https://github.com/originrose/cortex
http://www.onyxplatform.org/
https://neanderthal.uncomplicate.org/
https://clojurecuda.uncomplicate.org/
https://clojurecl.uncomplicate.org/
https://github.com/uncomplicate/bayadera
https://devblogs.nvidia.com/grcuda-a-polyglot-language-bindi...
Also since Pytorch 1.4 there is initial support for Java, built on top of the new pure C++ API.
https://deeplearning4j.org/
https://github.com/originrose/cortex
http://www.onyxplatform.org/
https://neanderthal.uncomplicate.org/
https://clojurecuda.uncomplicate.org/
https://clojurecl.uncomplicate.org/
https://github.com/uncomplicate/bayadera
https://devblogs.nvidia.com/grcuda-a-polyglot-language-bindi...
Also since Pytorch 1.4 there is initial support for Java, built on top of the new pure C++ API.
Given the choice between python with an almost free ffi to c and Fortran libraries for numeric processing with a syntax that's so easy we teach it to children under the age of ten and clojure which is sits in the centre of the universe feeding on all memory it comes across, you wonder why people don't choose clojure?
To backproject the causation of popularity of a given ecosystem and language towards cultural relativism and sociological in-groups rather than pragmatics just seems like a post-hoc rationalization. But, to take your comment in good faith (I don't want to dismiss it) I wonder if there really are generational dynamics at play here.
Earlier on, software engineering wasn't as glamorous as it is today, from either a material or cultural standpoint. I even recall a staggering difference now versus twenty years ago, when I first got into it, which even then was right before the bust of the first dot com bubble. While ostensibly by that point, the field of software engineering as a creative pursuit and an intellectual space had already thoroughly lost its pioneer spirit to the metaphorical urban and suburban "gentrifiers", there's no denying that the field itself continued to grow, and even at that point is a small fraction of what it is today. If software has truly eaten the world, isn't it possible that the languages and ecosystem that survived and reached dominance outcompeted the other ones?
That is to say, what makes you believe that another language is more fit to be in the position where Python is? While I will use whatever language is necessitated by the job (and have never had issues doing that) and even prefer the aesthetics and "handling" of Ruby and modern JS, Python is still the tool I reach for most often when I need to get some general purpose scripting or systems design task done. The ease with which I can go from zero to prototype and prototype to production and production to pain-free maintenance is unparalleled. The only two ecosystems that are remotely close are JavaScript and Java, and ostensibly Rails for web dev. Nothing else is competitive.
What would make you dispute this?
Earlier on, software engineering wasn't as glamorous as it is today, from either a material or cultural standpoint. I even recall a staggering difference now versus twenty years ago, when I first got into it, which even then was right before the bust of the first dot com bubble. While ostensibly by that point, the field of software engineering as a creative pursuit and an intellectual space had already thoroughly lost its pioneer spirit to the metaphorical urban and suburban "gentrifiers", there's no denying that the field itself continued to grow, and even at that point is a small fraction of what it is today. If software has truly eaten the world, isn't it possible that the languages and ecosystem that survived and reached dominance outcompeted the other ones?
That is to say, what makes you believe that another language is more fit to be in the position where Python is? While I will use whatever language is necessitated by the job (and have never had issues doing that) and even prefer the aesthetics and "handling" of Ruby and modern JS, Python is still the tool I reach for most often when I need to get some general purpose scripting or systems design task done. The ease with which I can go from zero to prototype and prototype to production and production to pain-free maintenance is unparalleled. The only two ecosystems that are remotely close are JavaScript and Java, and ostensibly Rails for web dev. Nothing else is competitive.
What would make you dispute this?
PrincipalSkinner.png:
Is Haskell a bad language?
No, it's the people getting actual work done that are wrong.
Is Haskell a bad language?
No, it's the people getting actual work done that are wrong.
[deleted]
I think it needs to be said that Haskell is just not actually all that ergonomic for mutably banging on buffers to solve linear algebra problems. I am a partial differential equations guy, so I have tried.
In ML as in PDE, correctness ultimately springs from performance, be it processing more data or using a finer mesh. So at the end of the day impedance matching with imperative linear algebra is pretty much a must.
In ML as in PDE, correctness ultimately springs from performance, be it processing more data or using a finer mesh. So at the end of the day impedance matching with imperative linear algebra is pretty much a must.
> why aren't the "more expressive" (which in this post seems to be a synonym for "functional") languages used in these fields
The real answer is probably inertia, but ...
I think ultimately there isn't a ton of actual "programming" happening in ML, more implementation. People like Python. It's simple, it's clean, it's elegant and it's performant when it needs to be. I'm sure a lot of people who have thought "man why don't I just re-write this in [language]?" stopped short because they soon asked themselves "wait, why?" Most of your work is on either side of the algorithm (data - [ml] - analysis).
The real answer is probably inertia, but ...
I think ultimately there isn't a ton of actual "programming" happening in ML, more implementation. People like Python. It's simple, it's clean, it's elegant and it's performant when it needs to be. I'm sure a lot of people who have thought "man why don't I just re-write this in [language]?" stopped short because they soon asked themselves "wait, why?" Most of your work is on either side of the algorithm (data - [ml] - analysis).
Functional languages are pretty good for the cleaning and analysis process as well. This blogpost in particular blew my mind:
http://evelinag.com/blog/2015/11-18-f-tackles-james-bond/
http://evelinag.com/blog/2015/11-18-f-tackles-james-bond/
Oh I think functional languages would be great for a lot of this, my point is the middle bit ends up being such an inconsequential part of the process that people end up not bothering and just using whatever's out there.
The current way to get a lot of efficient computation on a modern computer is to engage the GPU of said machine. This means that any language hosted on the CPU is going to be at a disadvantage.
In short: you are better off writing a compiler toward the GPU and use it as a compilation target. And a lot of things argues that language benefits from being a functional one, given how GPUs tend to work on data.
If there were no prior work, and if we have the full GPU specs, I'd reckon you want languages which are strong in program translation: Haskell or OCaml comes to mind.
Elixir---as well as its Erlang sibling---are really good at handling problems where there is concurrency involved. It is subtly different from parallel computation in that you often have many threads of execution all wanting to do different things at the same time. It is the same space e.g., Go occupies really well.
In short: you are better off writing a compiler toward the GPU and use it as a compilation target. And a lot of things argues that language benefits from being a functional one, given how GPUs tend to work on data.
If there were no prior work, and if we have the full GPU specs, I'd reckon you want languages which are strong in program translation: Haskell or OCaml comes to mind.
Elixir---as well as its Erlang sibling---are really good at handling problems where there is concurrency involved. It is subtly different from parallel computation in that you often have many threads of execution all wanting to do different things at the same time. It is the same space e.g., Go occupies really well.
As someone who has been writing hand-optimised GPU code for a while now, I'd argue that this high level view (and thus an appropriate functional high-level language) can get you reasonably decent performance on a GPU for little effort for many highly parallel problems.
And there are still plenty of cases where this will leave performance improvements of an order of magnitude on the table, because the compiler isn't smart enough to work out the optimal sequencing of operations. Often the computational core which benefits from running on a GPU is small and its complexity low enough that, if performance is a priority, you are better off writing it in an imperative low-level language as a library which can be accessed from higher level code.
And there are still plenty of cases where this will leave performance improvements of an order of magnitude on the table, because the compiler isn't smart enough to work out the optimal sequencing of operations. Often the computational core which benefits from running on a GPU is small and its complexity low enough that, if performance is a priority, you are better off writing it in an imperative low-level language as a library which can be accessed from higher level code.
Do you have any examples of such cases?
I'm asking because I'm part of the team that's working on the Futhark language[0], which is exactly what you're describing: a high-level array language in a functional style, targeting the GPU. We're always on the lookout for interesting applications or benchmarks. In particular, we're interested in knowing exactly where our shortcomings are, compared to hand-written GPU code, so that we may work on improving them. For the curious, we think we're doing reasonably well compared to benchmark suites such as Rodinia[1], though there are still cases where we cannot keep up.
[0]: https://futhark-lang.org/ [1]: https://futhark-lang.org/performance.html
I'm asking because I'm part of the team that's working on the Futhark language[0], which is exactly what you're describing: a high-level array language in a functional style, targeting the GPU. We're always on the lookout for interesting applications or benchmarks. In particular, we're interested in knowing exactly where our shortcomings are, compared to hand-written GPU code, so that we may work on improving them. For the curious, we think we're doing reasonably well compared to benchmark suites such as Rodinia[1], though there are still cases where we cannot keep up.
[0]: https://futhark-lang.org/ [1]: https://futhark-lang.org/performance.html
> The current way to get a lot of efficient computation on a modern computer is to engage the GPU of said machine. This means that any language hosted on the CPU is going to be at a disadvantage.
That's only true if your calculations are mostly the kind of things that take advantage of GPUs, like working with deep neural nets or graphics. For most other things using multiple CPU cores will be better.
That's only true if your calculations are mostly the kind of things that take advantage of GPUs, like working with deep neural nets or graphics. For most other things using multiple CPU cores will be better.
why aren't the "more expressive" (which in this post seems to be a synonym for "functional") languages used in these fields
Because very often the best solution does not win, which is why we are still using x86(_64), Unix, etc.
Python is a good example of worse is better. A functional language could provide more elegant abstractions. But for most people the ML APIs that Python provides are good enough. By taking a pragmatic approach, Python was faster to capture some niches, including machine learning and data science. Now we are at a point that even if another language had better abstractions and had similar library support, it would face an uphill battle because of inertia. Why would someone invest in learning another language for machine learning when Python is good enough?
To me it seems that only a language that has both a good model development story and a good deployment story stands a chance to compete with Python, since deployment (especially on mobile/embedded devices) is a weak point of the Python ML ecosystem.
Because very often the best solution does not win, which is why we are still using x86(_64), Unix, etc.
Python is a good example of worse is better. A functional language could provide more elegant abstractions. But for most people the ML APIs that Python provides are good enough. By taking a pragmatic approach, Python was faster to capture some niches, including machine learning and data science. Now we are at a point that even if another language had better abstractions and had similar library support, it would face an uphill battle because of inertia. Why would someone invest in learning another language for machine learning when Python is good enough?
To me it seems that only a language that has both a good model development story and a good deployment story stands a chance to compete with Python, since deployment (especially on mobile/embedded devices) is a weak point of the Python ML ecosystem.
Python prioritized pragmatism over expressive purity. That alone put it way ahead of many other languages.
Attention to language interop (especially with C) and a decision not to try and be all things to all programmers was another reason for its success - particularly in data science which relies upon heavily optimized numerical processing libraries that are mostly best written in C, C++ or Fortran.
Attention to language interop (especially with C) and a decision not to try and be all things to all programmers was another reason for its success - particularly in data science which relies upon heavily optimized numerical processing libraries that are mostly best written in C, C++ or Fortran.
"Worse is better" is a myth - it all depends of your definition of better.
The reason why Unix and Python win is simplicity. For many people simple == better.
Software is so easy to create and mold that many get trapped in excess complexity. Not being able to write complex software on a platform such as Unix or Python is therefore a feature, not a bug. It prevents you from ever getting trapped in the wet sands of complexity.
If there is a silver bullet in software engineering it is simplicity - and aggressive pursuit thereof.
The reason why Unix and Python win is simplicity. For many people simple == better.
Software is so easy to create and mold that many get trapped in excess complexity. Not being able to write complex software on a platform such as Unix or Python is therefore a feature, not a bug. It prevents you from ever getting trapped in the wet sands of complexity.
If there is a silver bullet in software engineering it is simplicity - and aggressive pursuit thereof.
Simple is better. People however tend to not agree on what's simple.
I think that's a bit too strong of a statement. In my opinion, it's not going to be possible to represent every solution to a problem in a simple manner when reality is so mind-bogglingly complex. Good tools should allow you to face natural complexity head-on, instead of trying to fit the outside world into a "simple" Bed of Procrustes.
It is actually the opposite: if you let complexity creep in, you have lost.
To model complex systems you have to keep them as simple as possible.
To model complex systems you have to keep them as simple as possible.
As simple as possible, but no simpler. As the previous comment was getting at, there is often a point of irreducible complexity. Modelling that efficiently is a mark of a good language.
> "Worse is better" is a myth - it all depends of your definition of better.
Isn't that the whole point of worse is better?
Isn't that the whole point of worse is better?
[deleted]
Not being able to write complex software on a platform such as Unix or Python is therefore a feature, not a bug.
Except that Python is not that. Multiple inheritance: check. Operator overloading: check. Functional programming: at least some of that (for-comprehensions, map, filter, reduce provided by the language). Exceptions: you got it. Coroutines: sure.
Python provides all the rope to 'get trapped in the wet sands of complexity'. Add the lack of static typing and you can write impermeable systems. There are a lot of Python code bases out which are exactly that. Go is a simple language, Python is not.
Worse is better" is a myth
Richard P. Gabriel:
The lesson to be learned from this is that it is often undesirable to go for the right thing first. It is better to get half of the right thing available so that it spreads like a virus. Once people are hooked on it, take the time to improve it to 90% of the right thing.
Or as aptly summarized by Wikipedia:
Therefore, the worse-is-better software first will gain acceptance, second will condition its users to expect less, and third will be improved to a point that is almost the right thing.
Which is exactly the history of Python (and Unix), compared to 'better' languages of the same era. Note that 'worse' is not necessarily used in a derogatory way. It just means that it is often better to get a good-enough, working, simple implementation out there that quickly adapts and spreads, than making that unicorn perfect system. If you look at the early Python versions and how Python spread over the years, it's a prototypical 'worse is better' example.
Except that Python is not that. Multiple inheritance: check. Operator overloading: check. Functional programming: at least some of that (for-comprehensions, map, filter, reduce provided by the language). Exceptions: you got it. Coroutines: sure.
Python provides all the rope to 'get trapped in the wet sands of complexity'. Add the lack of static typing and you can write impermeable systems. There are a lot of Python code bases out which are exactly that. Go is a simple language, Python is not.
Worse is better" is a myth
Richard P. Gabriel:
The lesson to be learned from this is that it is often undesirable to go for the right thing first. It is better to get half of the right thing available so that it spreads like a virus. Once people are hooked on it, take the time to improve it to 90% of the right thing.
Or as aptly summarized by Wikipedia:
Therefore, the worse-is-better software first will gain acceptance, second will condition its users to expect less, and third will be improved to a point that is almost the right thing.
Which is exactly the history of Python (and Unix), compared to 'better' languages of the same era. Note that 'worse' is not necessarily used in a derogatory way. It just means that it is often better to get a good-enough, working, simple implementation out there that quickly adapts and spreads, than making that unicorn perfect system. If you look at the early Python versions and how Python spread over the years, it's a prototypical 'worse is better' example.
You don't need to use multiple inheritance, coroutines or anything advanced in Python and still be productive. Many non computer science data analysts do exactly that and are very successful with it.
Go and other languages listed in the original articles on the other hand require you to understand the full complexity before you even begin or spend most of your time deep in documentation if you decide to learn as you go. Static type language simply doesn't allow you to not understand the type system.
This effect is especially enhanced in Scala, Haskell and similar ones where you will not finish a thing until you get a degree in category theory or Odersky theory.
'Worse' might not be intended to sound bad, but it surely does. It seems it is used to describe Unix and Python by that cohort of programmers whose intention is not to get shit done but to develop some mathematical theory of everything.
People who care about delivering results would not call them 'worse' compared to current alternatives in the first place.
There definitely are better alternatives on the horizon and they are better because they bring more simplicity to the table but not at the cost of giving up intuitiveness.
Go and other languages listed in the original articles on the other hand require you to understand the full complexity before you even begin or spend most of your time deep in documentation if you decide to learn as you go. Static type language simply doesn't allow you to not understand the type system.
This effect is especially enhanced in Scala, Haskell and similar ones where you will not finish a thing until you get a degree in category theory or Odersky theory.
'Worse' might not be intended to sound bad, but it surely does. It seems it is used to describe Unix and Python by that cohort of programmers whose intention is not to get shit done but to develop some mathematical theory of everything.
People who care about delivering results would not call them 'worse' compared to current alternatives in the first place.
There definitely are better alternatives on the horizon and they are better because they bring more simplicity to the table but not at the cost of giving up intuitiveness.
> This effect is especially enhanced in Scala, Haskell and similar ones where you will not finish a thing until you get a degree in category theory or Odersky theory.
This is simply not true.
This is simply not true.
>>Therefore, the worse-is-better software first will gain acceptance, second will condition its users to expect less, and third will be improved to a point that is almost the right thing.
>Which is exactly the history of Python (and Unix)
I was thinking Python (and Windows) :v
>Which is exactly the history of Python (and Unix)
I was thinking Python (and Windows) :v
Honestly I think this is a little bit too abstract.
Imo the answer is that only one mainstream language bothered to emulate MATLAB's plotting API to the best of its ability.
Imo the answer is that only one mainstream language bothered to emulate MATLAB's plotting API to the best of its ability.
Little tangent, but what do you consider better than Unix? I don’t have much experience with anything else so I’m curious.
Anything based on the workstation concepts explored at Xerox PARC, and further pursued at ETHZ, TI and Genera.
Bitsavers has all the necessary papers and manuals, and computer museum as well.
Here are some video sessions.
"Yesterday's Computer of Tomorrow: The Xerox Alto │Smalltalk-76 Demo"
https://www.youtube.com/watch?v=NqKyHEJe9_w
"Alto System Project: Dan Ingalls demonstrates Smalltalk"
https://www.youtube.com/watch?v=uknEhXyZgsg
"Eric Bier Demonstrates Cedar"
https://www.youtube.com/watch?v=z_dt7NG38V4
"The Final Demonstration of the Xerox 'Star' computer"
https://www.youtube.com/watch?v=_OwG_rQ_Hqw
The end of the road from Plan 9, Inferno.
http://www.vitanuova.com/inferno/index.html
http://doc.cat-v.org/inferno/4th_edition/
Bitsavers has all the necessary papers and manuals, and computer museum as well.
Here are some video sessions.
"Yesterday's Computer of Tomorrow: The Xerox Alto │Smalltalk-76 Demo"
https://www.youtube.com/watch?v=NqKyHEJe9_w
"Alto System Project: Dan Ingalls demonstrates Smalltalk"
https://www.youtube.com/watch?v=uknEhXyZgsg
"Eric Bier Demonstrates Cedar"
https://www.youtube.com/watch?v=z_dt7NG38V4
"The Final Demonstration of the Xerox 'Star' computer"
https://www.youtube.com/watch?v=_OwG_rQ_Hqw
The end of the road from Plan 9, Inferno.
http://www.vitanuova.com/inferno/index.html
http://doc.cat-v.org/inferno/4th_edition/
Small tangent: It's sad that the Mother of All Demos still looks like the future of computing. Google Docs and its siblings come close, but they still don't match it in scope.
Besides what pjmlp said. I was thinking of Plan 9, VMS, NT (the kernel, which takes a lot if ideas from VMS), L4.
Of course, out of these, NT did succeed wildly as well did L4 (e.g. iOS secure enclave).
Some improve in the architecture, some in user land, some in both :).
Of course, out of these, NT did succeed wildly as well did L4 (e.g. iOS secure enclave).
Some improve in the architecture, some in user land, some in both :).
Two things. Effort and support. You can probably wrap BLAS in any language and make a framework out of that. With some more effort you can add support for GPUs and make your framework much faster. But why would you when most of that is already available? Some might really hate Python but learning Python just for the framework would probably be easier then making a ML framework for the language you like. And Python wasn't even dominant until Tensorflow. Torch used to be a good competetor.
Many people make the mistake of promoting the language instead of the framework.
And the next thing is support. When some big company releases a ML framework and starts using it for themselves people start using it for their own projects. It's mostly because people think that it will last longer then a month. There were lots of dead python ML projects before Tensorflow.
So if you can get a big company to release a ML framework for your favourite language it will surely gain at least some market share.
The language almost doesn't matter. I say almost because big companies are more likely to choose popular languages.
Many people make the mistake of promoting the language instead of the framework.
And the next thing is support. When some big company releases a ML framework and starts using it for themselves people start using it for their own projects. It's mostly because people think that it will last longer then a month. There were lots of dead python ML projects before Tensorflow.
So if you can get a big company to release a ML framework for your favourite language it will surely gain at least some market share.
The language almost doesn't matter. I say almost because big companies are more likely to choose popular languages.
> why aren't the "more expressive" (which in this post seems to be a synonym for "functional")
Nobody else seems to have tackled this specific issue so I'd like to.
Matt's use of "expressive" is not whimsical or full of personal ideology. He uses Felleisen's definition of "expressiveness" when comparing programming languages [0][1]. The work is a continuation of Landin's earlier efforts to formalize language comparison. From the text (where Felleisen is comparing two languages of "different size", which means that one provides more first-class "constructs" than the other):
> Here "more expressive" means that the translation of a program with occurrences of one of the constructs c_i to the smaller language requires a global reorganization of the entire program.
The paper goes on to provide a mathematical framework by which to make these comparisons.
The languages Matt lists are more expressive than many non-functional languages for various reasons. Each of them has some functionality that, if you were to remove it, would require a "global reorganization of the entire program" to achieve the same goal. For example: if you have a Haskell program and remove automatic currying, you may now have to rewrite almost all of your functions and call sites to accommodate (perhaps by explicitly encoding delays, or by using continuations, etc.). Python does not feature automatic currying, so in this respect Haskell is more expressive than Python — there is a method in Haskell in which you can express your code which you cannot use in Python.
Note that the relationship is not predicated entirely upon individual constructs: it is possible for two languages to be compared and to find that each has expressive capability that the other lacks. However, by and large the functional languages Matt lists have focused specifically on increases in expressive power compared to the more "traditional" languages that most programmers use daily. (Haskell did this explicitly by design, but the others Matt lists came to it by way of exploring alternate approaches to problems and arrived at increased expressiveness incidentally.)
Later in the paper, Felleisen touches on Turing completeness (a common point brought up in these discussions):
> If a programming language can represent all computable functions (on the integers), it contains a functionally equivalent counterpart to each program in a more expressive language. This raises the question as to what advantages there are to programming in the more expressive language when equivalent programs in the simpler language already exist. [...] > > [...W]e have come to the conclusion that programs in less expressive languages exhibit repeated occurrences of programming patterns, and that this pattern-oriented style is detrimental to the programming process.
So this provides motivation for why one might want to investigate these "more expressive" languages. Being able to reduce code duplication is (almost) always a good thing!
---
[0] https://doi.org/10.1016/0167-6423(91)90036-W
[1] I know this to be the case because I've talked to Matt about this very article. :) The Felleisen paper is pretty popular in the PL communities Matt used to hang out in.
Nobody else seems to have tackled this specific issue so I'd like to.
Matt's use of "expressive" is not whimsical or full of personal ideology. He uses Felleisen's definition of "expressiveness" when comparing programming languages [0][1]. The work is a continuation of Landin's earlier efforts to formalize language comparison. From the text (where Felleisen is comparing two languages of "different size", which means that one provides more first-class "constructs" than the other):
> Here "more expressive" means that the translation of a program with occurrences of one of the constructs c_i to the smaller language requires a global reorganization of the entire program.
The paper goes on to provide a mathematical framework by which to make these comparisons.
The languages Matt lists are more expressive than many non-functional languages for various reasons. Each of them has some functionality that, if you were to remove it, would require a "global reorganization of the entire program" to achieve the same goal. For example: if you have a Haskell program and remove automatic currying, you may now have to rewrite almost all of your functions and call sites to accommodate (perhaps by explicitly encoding delays, or by using continuations, etc.). Python does not feature automatic currying, so in this respect Haskell is more expressive than Python — there is a method in Haskell in which you can express your code which you cannot use in Python.
Note that the relationship is not predicated entirely upon individual constructs: it is possible for two languages to be compared and to find that each has expressive capability that the other lacks. However, by and large the functional languages Matt lists have focused specifically on increases in expressive power compared to the more "traditional" languages that most programmers use daily. (Haskell did this explicitly by design, but the others Matt lists came to it by way of exploring alternate approaches to problems and arrived at increased expressiveness incidentally.)
Later in the paper, Felleisen touches on Turing completeness (a common point brought up in these discussions):
> If a programming language can represent all computable functions (on the integers), it contains a functionally equivalent counterpart to each program in a more expressive language. This raises the question as to what advantages there are to programming in the more expressive language when equivalent programs in the simpler language already exist. [...] > > [...W]e have come to the conclusion that programs in less expressive languages exhibit repeated occurrences of programming patterns, and that this pattern-oriented style is detrimental to the programming process.
So this provides motivation for why one might want to investigate these "more expressive" languages. Being able to reduce code duplication is (almost) always a good thing!
---
[0] https://doi.org/10.1016/0167-6423(91)90036-W
[1] I know this to be the case because I've talked to Matt about this very article. :) The Felleisen paper is pretty popular in the PL communities Matt used to hang out in.
AI, NLP and ML are bottleneck first by memory accesses and second by CPU.
Whatever language you end up using, the low-level implementation must allow you to precisely control memory layout.
This is what is being done with C++ + Python but pure Elixir doesn't allow you to control memory layout and will kill your performance.
Whatever language you end up using, the low-level implementation must allow you to precisely control memory layout.
This is what is being done with C++ + Python but pure Elixir doesn't allow you to control memory layout and will kill your performance.
Elixir is a BEAM language and therefore not designed for computational throughput.
That's true - it's why I said you need a native library (something like numpy, but as a NIF) to back it up.
There's Matrex, which is based on CBLAS and has some integration with IElixir, Jupyter's kernel for Elixir.
Also someone seems to be doing interesting work to integrate with GPU at https://github.com/sasagawa888/deeppipe2
Of course that's not to say that Elixir is ready to replace Python tomorrow. I just wanted to mention interesting projects in this area.
Also someone seems to be doing interesting work to integrate with GPU at https://github.com/sasagawa888/deeppipe2
Of course that's not to say that Elixir is ready to replace Python tomorrow. I just wanted to mention interesting projects in this area.
I think the main issue you face here is that there is already a big C/C++/Fortran ecosystem around distributed computation, one that happens to be very well impedance-matched to numerical linear algebra, and one with which experts in numerical linear algebra are intimately familiar (MPI).
Why bother with Erlang as a thin wrapper over C++ when C++ has the necessary facilities already?
Why bother with Erlang as a thin wrapper over C++ when C++ has the necessary facilities already?
What is it specifically about BEAM languages that make them not good for computational throughput?
Immutability and thread-safety - if you want to get a value from one concurrent process to another, you have to copy it and send it to the other process. You can circumvent this by using a smart-pointer / reference to mutable data, which is manipulated by native code. One example would be a ResourceArc[0] in Rustler.
[0] https://docs.rs/rustler/0.22.0-rc.0/rustler/resource/struct....
[0] https://docs.rs/rustler/0.22.0-rc.0/rustler/resource/struct....
libraries, examples, classes, scaling architecture, gpu integration, ease of syntax.
mostly the same reason new developers dont use functional languages for anything else either.
mostly the same reason new developers dont use functional languages for anything else either.
"If, on the other hand, you're interested in enlightenment, academic research or a start-up, the criterion by which you should choose your next language is not employability, but expressiveness."
There is a big difference between enlightenment, academic research and a start-up.
For both academic research and start-ups, employability is a very important component:
- Doing academic research on "non-employable" languages decreases the impact of your research and your own employability;
- In a start-up, you want to pick a language where you can be productive with your team members, with an healthy community, and where you won't have hiring problems because you're using an "advanced" programming language.
There is a big difference between enlightenment, academic research and a start-up.
For both academic research and start-ups, employability is a very important component:
- Doing academic research on "non-employable" languages decreases the impact of your research and your own employability;
- In a start-up, you want to pick a language where you can be productive with your team members, with an healthy community, and where you won't have hiring problems because you're using an "advanced" programming language.
> - Doing academic research on "non-employable" languages decreases the impact of your research and your own employability
That is not a metric for choice of language for academic research at all. The academic research itself is a job, paid with public money, and you better not hinder it with your career plans. Most computer science research is not on languages itself, so you need a language expressive enough to allow you study whatever computer science concept you need to study; it's mostly C and Haskell around me, but I can't imagine my friends going into embedded C development - the C they write is very different, as are the practices.
That is not a metric for choice of language for academic research at all. The academic research itself is a job, paid with public money, and you better not hinder it with your career plans. Most computer science research is not on languages itself, so you need a language expressive enough to allow you study whatever computer science concept you need to study; it's mostly C and Haskell around me, but I can't imagine my friends going into embedded C development - the C they write is very different, as are the practices.
I was referring to academic research in software engineering or PL related areas.
I think your view of academic research is nice but perhaps a bit naive. Academic research in most countries is filled with pressure from private money to achieve valuable outputs to feed the private sector.
I think your view of academic research is nice but perhaps a bit naive. Academic research in most countries is filled with pressure from private money to achieve valuable outputs to feed the private sector.
> Academic research in most countries is filled with pressure from private money to achieve valuable outputs to feed the private sector.
This is not my experience working in PL research, personally. Mostly government grants here (US) that are either wholly research-oriented ("Build a cool thing because it improves the research community") or government-oriented ("Develop this technique because it will provide us with better (cybernetic) defenses). Admittedly I'm pretty new, but the funding does not seem generally to come from or be focused on developments for the private sector.
I imagine it would be different in compilers, though, for example. That's certainly an area that the private sector has a more vested interest in, due to direct applicability.
This is not my experience working in PL research, personally. Mostly government grants here (US) that are either wholly research-oriented ("Build a cool thing because it improves the research community") or government-oriented ("Develop this technique because it will provide us with better (cybernetic) defenses). Admittedly I'm pretty new, but the funding does not seem generally to come from or be focused on developments for the private sector.
I imagine it would be different in compilers, though, for example. That's certainly an area that the private sector has a more vested interest in, due to direct applicability.
Huh, that's not the reality around me, in CZ/SK/DE/AT/PL in EU.
In US universities in machine learning fields, a lot of labs work with private companies or do interships during summer at private companies and the grad students eventually graduate and get a job at the private company. Software engineering research might be similar but not sure.
All the economists working in Stata would like to talk to you about their employability
I am not sure what economists at a commercial firm have to do with computer scientists doing academical research.
There are some groups with large code-bases that will want "employable language" experience (mine was one), but, at least in my field, outside of the groups with big simulation packages used by other groups, the job description is mostly bodging out a nice-looking .eps file and maybe some favorable-looking benchmark numbers any way you feel like.
Every developers should know at least two languages (more is of course better, but two is the minimum for a professional). The optimal way to select:
(1) Learn the language which is most prominent or employable in your preferred industry (e.g. Java for enterprises, Python for science, JavaScript for frontend).
(2) Learn the language most different to the first, regardless of its popularity and employability. Eg. Scheme if you know Java, Haskell if you know Python, Assembler if you know JavaScript. (There will probably be some disagreement about which languages are to most different to each other, but you get my point.)
(1) Learn the language which is most prominent or employable in your preferred industry (e.g. Java for enterprises, Python for science, JavaScript for frontend).
(2) Learn the language most different to the first, regardless of its popularity and employability. Eg. Scheme if you know Java, Haskell if you know Python, Assembler if you know JavaScript. (There will probably be some disagreement about which languages are to most different to each other, but you get my point.)
I broadly agree with you here. I believe what you're saying is that once you're economically productive in your first language, you get the most bang out of your second language by picking a second language that encourages concepts and idioms not encouraged by your first language. In this way, your second language is really there to improve the way you think about your first language.
However, I would word (2) a bit differently, otherwise tons of people will be learning INTERCAL and BrainFuck. For a second language, consider only languages that are at least popular enough to have good learning resources. (This is a pretty low bar... there are probably hundreds of such languages.) Look at which concepts and idioms are encouraged by each language, ignore those concepts and idioms encouraged by your first language, and try and score the usefulness of the remaining concepts and idioms. Do a bit of research reading up on the pros/cons of the idioms and concepts to help you score them.
The goal here is to pick up concepts and idioms by learning your second language, which will then diversify the way you think about your first language.
I would absolutely look at the machine code generated by one of your languages (or bytecode, if none of your languages compile to native code or have a JIT that can dump out assembly) to understand what's going on, but I wouldn't suggest learning to write assembler as one of your two first languages. Most of the usefulness of assembler can be gained from learning to read it.
However, I would word (2) a bit differently, otherwise tons of people will be learning INTERCAL and BrainFuck. For a second language, consider only languages that are at least popular enough to have good learning resources. (This is a pretty low bar... there are probably hundreds of such languages.) Look at which concepts and idioms are encouraged by each language, ignore those concepts and idioms encouraged by your first language, and try and score the usefulness of the remaining concepts and idioms. Do a bit of research reading up on the pros/cons of the idioms and concepts to help you score them.
The goal here is to pick up concepts and idioms by learning your second language, which will then diversify the way you think about your first language.
I would absolutely look at the machine code generated by one of your languages (or bytecode, if none of your languages compile to native code or have a JIT that can dump out assembly) to understand what's going on, but I wouldn't suggest learning to write assembler as one of your two first languages. Most of the usefulness of assembler can be gained from learning to read it.
I think Python and C are simply a must, whatever the language you use in your daily job and whatever else you might want to learn. You cannot seriously call yourself a programmer if you are afraid of C, and Python is too useful as an ad-hoc tool to be ignored.
I totally disagree with gate-keeping like "you are not a serious programmer if you don't know C". You are a serious programmer if you program stuff that works and achieves it purpose.
If you don't understand the concepts of C - things like pointers - then you don't actually know how computers work and how programming languages are implemented (i.e. how the JavaScript you write actually happens) so, yeah, you're probably not a serious programmer. Copy-paste code from Stack Overflow? Sure. Debug a complex problem or optimize a piece of code? Unlikely.
> Copy-paste code from Stack Overflow? Sure. Debug a complex problem or optimize a piece of code? Unlikely.
What a ridiculous thing to say. One doesn't need to know how programming languages are implemented or 'how pointers work' or even know C in order to be a proficient programmer. This is just nonsensical gatekeeping.
What a ridiculous thing to say. One doesn't need to know how programming languages are implemented or 'how pointers work' or even know C in order to be a proficient programmer. This is just nonsensical gatekeeping.
> If you don't understand the concepts of C - things like pointers - then [...] you're probably not a serious programmer.
This is true, but doesn't actually require knowing (or even having heard of) C specifically.
This is true, but doesn't actually require knowing (or even having heard of) C specifically.
Unsafe Rust is at least as close to the hardware as C is, if not somewhat closer (C doesn't have 8- or 16-bit arithmetic operations, and I believe unsafe Rust doesn't have the strict aliasing rules of C).
Arguing that you specifically have to learn C to understand hardware is the gatekeeping I dislike the most, especially because it turns out that C itself can have a pretty bad view of hardware.
Arguing that you specifically have to learn C to understand hardware is the gatekeeping I dislike the most, especially because it turns out that C itself can have a pretty bad view of hardware.
Sure, which is why I wrote understand not know. I doubt many Rust programmers would have issues writing C (basically just forget about ownership and write plain functions) unlike say Python or JavaScript (or even Java) programmers.
I started my career as C dev, working on driver level software. These days I want to stay as far away from hardware as possible. Hardware is horrible, bugs, bugs, bugs, and more bugs.
If you don't understand assembler then you don't understand how a computer works. C is a higher level language which just happen to be more primitive than most.
Knowing C is fine, but thinking C is somehow "the real world" is dangerous, since then you don't understand that the abstractions in C (like pointers or the stack) are just design choices with certain trade-off and that alternatives are possible.
Are you suggesting someone who knows say Haskell but not C is not able to debug their own programs?
Knowing C is fine, but thinking C is somehow "the real world" is dangerous, since then you don't understand that the abstractions in C (like pointers or the stack) are just design choices with certain trade-off and that alternatives are possible.
Are you suggesting someone who knows say Haskell but not C is not able to debug their own programs?
Sure, I agree. I just think the impact is less, since e.g. computers are optimized for C (there's special instructions for manipulating C-like stack). I think of more importance is the concept of registers, which C mostly abstracts away - definitely useful to know, but probably less performance impact than pointers vs. objects.
Nice to know that those early years doing Z80, 8086, 68000 and Turbo Pascal TSRs and demos wasn't programming.
Koshkin wrote "if you are afraid of C" and IMO if you are writing assembly or Turbo Pascal you wont (or at least, shouldn't) be afraid of C. Turbo Pascal is pretty much at the same "level" as C with slightly more stronger typing and a few extra bells and whistles, but things that cause people to be afraid of C, like pointers and manual memory management, are still there.
Yeah, I commented on purpose, because that is what people should learn, pointers and manual memory management concepts, the language to learn them should be secondary.
Those that learn to abstract concepts from how they get expressed on the language of the jour, are better prepared for change in this fashion industry.
I also never understood what it means being afraid of programming languages.
Those that learn to abstract concepts from how they get expressed on the language of the jour, are better prepared for change in this fashion industry.
I also never understood what it means being afraid of programming languages.
[deleted]
> I think Python and C are simply a must, whatever the language you use in your daily job and whatever else you might want to learn. You cannot seriously call yourself a programmer if you are afraid of C, and Python is too useful as an ad-hoc tool to be ignored.
I haven't ever found Python to be useful since I started program. Perl, Bash, Ruby, JavaScript, etc. work equally well for one off scripts and Haskell works much better for mathematical calculations. For large programs I would never consider using Python. Maybe if I did machine learning my opinion would be different.
I haven't ever found Python to be useful since I started program. Perl, Bash, Ruby, JavaScript, etc. work equally well for one off scripts and Haskell works much better for mathematical calculations. For large programs I would never consider using Python. Maybe if I did machine learning my opinion would be different.
A good dogma I think fits roughly in the same category as your opinion is a quote from Alan Perlis:
“A language that doesn't affect the way you think about programming is not worth knowing.”
“A language that doesn't affect the way you think about programming is not worth knowing.”
JS changed almost nothing about how I view languages, nor did it teach me much, outside of what several libraries have. That does not mean it was not incredibly useful to know, both monetarily, and for what it enables me to do, when necessary.
Any programing language that does not increase what you are able to do is not worth knowing. That increase may come from greater knowledge, different ways of looking at things, or just from different and/or better libraries.
> Assembler if you know JavaScript
Maybe go with Rust instead.
Maybe go with Rust instead.
This article might need a date added. It mentions Ruby and Perl as "hot right now"...
> the criterion by which you should choose your next language is not employability, but expressiveness.
What a weird choice.
If you are looking for an out-of-the-mainstream language, you can't really tell right away if a language is expressive or not before you actually dive in.
Myself, I go for "exotic looking". I read the brief description of the language, look at a few snippets, and the weirder it looks, or the most foreign the main concepts are, the more I'll be interested in studying it for a few hours.
The point is not to be expressive, it's to stretch my brain.
What a weird choice.
If you are looking for an out-of-the-mainstream language, you can't really tell right away if a language is expressive or not before you actually dive in.
Myself, I go for "exotic looking". I read the brief description of the language, look at a few snippets, and the weirder it looks, or the most foreign the main concepts are, the more I'll be interested in studying it for a few hours.
The point is not to be expressive, it's to stretch my brain.
I’m making an effort to learn Scala. It’s enough like Java that I can write as I would traditionally so I quickly wrote a Word Search puzzle generator.
Next I’m going to try and write a second version in a more functional way. Then a third...
First version: https://github.com/melling/WordSearchScala/blob/master/WordS...
Next I’m going to try and write a second version in a more functional way. Then a third...
First version: https://github.com/melling/WordSearchScala/blob/master/WordS...
It would be interesting to see how the Scala section would change after the release of Java 8 and especially the release of Kotlin. Java 8 introduced the bare minimum functional programming support in order for the language to be used functionally. Not to mention libraries came out like reactive streams which essentially lets you write your entire code as chained function calls inside of an error monad.
Kotlin then made functional programming a first class citizen. Java is clearly recognizable inside of Kotlin under a bunch of layers of abstraction and syntactic sugar. Once you learn Java you can learn Kotlin with minimal effort and add more language features to your toolbelt as you get more Kotlin experience -- similar to what the author says about Scala.
Kotlin is probably not as advanced as Scala, and admittedly I've never used Scala, but it does seem to be taking some of Scala's market share.
Kotlin then made functional programming a first class citizen. Java is clearly recognizable inside of Kotlin under a bunch of layers of abstraction and syntactic sugar. Once you learn Java you can learn Kotlin with minimal effort and add more language features to your toolbelt as you get more Kotlin experience -- similar to what the author says about Scala.
Kotlin is probably not as advanced as Scala, and admittedly I've never used Scala, but it does seem to be taking some of Scala's market share.
"Kotlin then made functional programming a first class citizen" - what do you mean by this?
In Java if you want to map a function over a list you write `list.stream().map().collect()` and in Kotlin you write `list.map()`. The functional programming APIs in Java are atrocious. Lambdas are much easier to write thanks to anaphoric macros and other syntactic sugar. Data structures are immutable by default. A big one that isn't mentioned often is that functions can exist outside of classes so your code is not as tied to Java's style of OOP.
Not OP, but I assume they meant: Kotlin encourages you to solve problems functionally, whereas Java merely allows you to do so. Kotlin provides features like top-level function declarations (don't need to be declared as methods of a class) and functions-as-arguments, both of which make it easier to think functions first.
> it is possible to write an amb macro that "chooses" the right argument to make a subsequent assert statement be true.
I'm not too familiar with Scheme's macro feature but how would one go about that? I was under the impression with macros you get to define what ever syntax is part if the call, but here in order to achieve what the author mentions the macro would need to be able to look at surrounding code, to look ahead for the assert. How is this possible? Is he maybe talking of a compiler plugin or a macro (not shown) that encloses the entire expression, e.g. `(make-assert-true-macro (let ...{the code snippet}... ))`?
I'm not too familiar with Scheme's macro feature but how would one go about that? I was under the impression with macros you get to define what ever syntax is part if the call, but here in order to achieve what the author mentions the macro would need to be able to look at surrounding code, to look ahead for the assert. How is this possible? Is he maybe talking of a compiler plugin or a macro (not shown) that encloses the entire expression, e.g. `(make-assert-true-macro (let ...{the code snippet}... ))`?
I'm just thinking out aloud, so I'm probably -- if not completely wrong -- at best sketchy on details! That being said, `amb` could be a function/macro that uses a continuation in such a way that, when a failure is encountered (however that may be defined), it can backtrace and try with the next value. By using continuations with some encapsulation of the call stack, it doesn't need to envelope the surrounding code and can just "rewind".
Sicp, the ultimate.
https://mitpress.mit.edu/sites/default/files/sicp/full-text/...
https://mitpress.mit.edu/sites/default/files/sicp/full-text/...
You can implement 'amb' using either continuations or monads.
Back in 2005, I wrote a very short Ruby version of 'amb' using continuations: http://www.randomhacks.net/2005/10/11/amb-operator/ This approach works if your code contains no side effects, because it involves backtracking and restarting the computation as needed. But continuations are a heavyweight feature that place significant constraints on language implementation, so few languages support them.
You can also implement 'amb' using monads: http://www.randomhacks.net/2007/03/12/monads-in-15-minutes/ (Yes, this is yet another monad tutorial. Sorry.) This benefits from a "do notation" syntax, which could be implemented using macros.
Back in 2005, I wrote a very short Ruby version of 'amb' using continuations: http://www.randomhacks.net/2005/10/11/amb-operator/ This approach works if your code contains no side effects, because it involves backtracking and restarting the computation as needed. But continuations are a heavyweight feature that place significant constraints on language implementation, so few languages support them.
You can also implement 'amb' using monads: http://www.randomhacks.net/2007/03/12/monads-in-15-minutes/ (Yes, this is yet another monad tutorial. Sorry.) This benefits from a "do notation" syntax, which could be implemented using macros.
It is possible to implement it in a lot of languages.
https://www.rosettacode.org/wiki/Amb
https://www.rosettacode.org/wiki/Amb
Finally, an article about PL's that doesn't mention Go or Rust!
(Oh, wait... the article is from 2009? Never mind then. Carry on.)
(Oh, wait... the article is from 2009? Never mind then. Carry on.)
As many have indicated, this article is pretty dated now.
In some ways it argues for a thorium nuclear power plant when subsidized wind is available. Yeah thorium is great, but we know how to do wind and the buy-in is cheap right now. You'd have to invest a lot in thorium to make it worth it.
Forgive the crude analogy.
Also, the people building ML models and doing data science aren't always ubber functional programmers, so Python is easier for them to get going with. Just like the people writing Enterprise software find Java to be fine.
In some ways it argues for a thorium nuclear power plant when subsidized wind is available. Yeah thorium is great, but we know how to do wind and the buy-in is cheap right now. You'd have to invest a lot in thorium to make it worth it.
Forgive the crude analogy.
Also, the people building ML models and doing data science aren't always ubber functional programmers, so Python is easier for them to get going with. Just like the people writing Enterprise software find Java to be fine.
> once the program compiles, it's almost certainly correct.
I'm pretty sure I've heard/read that remark about more than one language. It's interesting how that "almost certainly" often means "around one percent of the time" or the definition of "correct" is moved to a different goalpost.
Meaning, of course, that ultimately it is no different from any other programming language: great coders will write great code and bad coders will write bad code.
I'm pretty sure I've heard/read that remark about more than one language. It's interesting how that "almost certainly" often means "around one percent of the time" or the definition of "correct" is moved to a different goalpost.
Meaning, of course, that ultimately it is no different from any other programming language: great coders will write great code and bad coders will write bad code.
I'm really curious what 'advanced' will mean in 2030 ?
procedural programming fully gone and most programs being about metalevel self-adapting (ml or else) parallel agents ?
procedural programming fully gone and most programs being about metalevel self-adapting (ml or else) parallel agents ?
I'm only being half tongue in cheek here, but if you follow recent patterns with the growth of functional languages, perhaps find something being mentioned in the computer science journals in the 80s or 90s and imagine it at a bigger scale and with a few twists ;-)
A reinvented version of APL? Some twist on visual programming? Prolog-style logic programming with a built-in graph database? Or just stick 'quantum' on the front of anything?
If you care to do such spelunking and guessing yourself, the ACM's currently open library is a big help! For example: https://dl.acm.org/action/doSearch?fillQuickSearch=false&exp...
A reinvented version of APL? Some twist on visual programming? Prolog-style logic programming with a built-in graph database? Or just stick 'quantum' on the front of anything?
If you care to do such spelunking and guessing yourself, the ACM's currently open library is a big help! For example: https://dl.acm.org/action/doSearch?fillQuickSearch=false&exp...
Yeah good point, although I think the FP revival is reaching top of the current wave. And you know, time and new things have a tendancy to be out of the ordinary.
One thing that seems very near reality, is logical auto completion or partial proover edition. Right now and IDE/typechecker will verify there's a logical path between A and B, but maybe they'll try some capped graph search to find possible paths when you need to compose some functions. I know I've seen similar things at very small scale in high end cgi software. Their node editors will introduce conversion nodes if you try to add scalar to a vector.
One thing that seems very near reality, is logical auto completion or partial proover edition. Right now and IDE/typechecker will verify there's a logical path between A and B, but maybe they'll try some capped graph search to find possible paths when you need to compose some functions. I know I've seen similar things at very small scale in high end cgi software. Their node editors will introduce conversion nodes if you try to add scalar to a vector.
I don't have the oomph to write a whole big thing this morning but uh, here are some links that are relevant IMO:
http://conal.net/papers/compiling-to-categories/
https://www.categoricaldata.net/
https://terminusdb.com/blog/ (It's a little market-ey, dig around for the meat. Graph DB in Prolog & Rust, "git for data", etc.)
https://github.com/Co-dfns/Co-dfns
edit: Ah it's you agumonkey (I don't check usernames before replying.) Apologies if I sound like a broken record in re: the above topics.
http://conal.net/papers/compiling-to-categories/
https://www.categoricaldata.net/
https://terminusdb.com/blog/ (It's a little market-ey, dig around for the meat. Graph DB in Prolog & Rust, "git for data", etc.)
https://github.com/Co-dfns/Co-dfns
edit: Ah it's you agumonkey (I don't check usernames before replying.) Apologies if I sound like a broken record in re: the above topics.
> procedural programming fully gone
No chance. Not by 2030, anyway. (If you just want to define "advanced" that way, you can define it that way today...)
> most programs being about metalevel self-adapting (ml or else) parallel agents
Also no - not by 2030, anyway.
No chance. Not by 2030, anyway. (If you just want to define "advanced" that way, you can define it that way today...)
> most programs being about metalevel self-adapting (ml or else) parallel agents
Also no - not by 2030, anyway.
> I'm really curious what 'advanced' will mean in 2030 ?
Javascript on the GPU
Javascript on the GPU
That's mainstream advanced, not advanced^2
Well, you only gave ten years for advancement.
I love learning new languages. But I have yet to see anything that looks like the enlightenment in the article. Most languages gives you different ways to accomplish the same tasks. Some languages make some tasks easier and others hard. It doesn't really matter weather the language is hot, popular or obscure.
Rust is an advanced language that might go mainstream.
I wouldn't call Rust an advanced language. It has some functional-inspired features (everything is an expression, ADTs, composition/pipes with iterators), but the language's raison d'etre is zero-cost abstractions, and many of the features of Haskell and the like require a GC and costly runtime checks. See this post on why Rust doesn't have functors, monads, and the like:
https://varkor.github.io/blog/2018/08/28/feasible-functors-i...
https://varkor.github.io/blog/2018/08/28/feasible-functors-i...
Affine types are definitely an advanced PL feature, and the compile time checking in Rust is largely based on them. ("Affine" meaning that values of such a type cannot be implicitly copied, but can be implicitly dropped. One reason why Rust doesn't yet have functors or monads is that it's not very clear to what extent these would usefully generalize to such a construct.)
Those zero-cost abstractions on Rust are pretty advanced, and superior to anything comparable in any other widely used language.
Your complaint stands basically to calling Rust a low-level language... what it is. It has advanced low level features, that bring productivity for some code that is completely different form what those high-level ones you point are concerned with.
Your complaint stands basically to calling Rust a low-level language... what it is. It has advanced low level features, that bring productivity for some code that is completely different form what those high-level ones you point are concerned with.
Especially in fast changing domains like tech, essays and blog posts must state publish dates.
[deleted]
I'm sick of the functional programming people telling us that they are better than we are.
I have written lines of code in Scheme, Haskell, and Common Lisp. I regularly code in Clojure. I like writing in functional languages too.
But every language makes trade-offs. The imperative languages generally try to optimize for readability and speed. The functional languages are often optimized for expression, the ability to easily express yourself to the interpreter or compiler. There is a trade-off though. These languages all trade high expressibility for slow execution times and steep learning curves. This trade-off might well be worth the effort especially if the team is small but it is a trade-off. It's not like it's objectively better to always program in the functional languages.
I have written lines of code in Scheme, Haskell, and Common Lisp. I regularly code in Clojure. I like writing in functional languages too.
But every language makes trade-offs. The imperative languages generally try to optimize for readability and speed. The functional languages are often optimized for expression, the ability to easily express yourself to the interpreter or compiler. There is a trade-off though. These languages all trade high expressibility for slow execution times and steep learning curves. This trade-off might well be worth the effort especially if the team is small but it is a trade-off. It's not like it's objectively better to always program in the functional languages.
Most people are already familiar with procedural programming. If you want to grow as a developer, don't just learn more of the same, learns something that challenges you to work in a different way. If I only knew functional programming, I don't think I'd understand half of what I understand now. It's the fact that they accomplish the same things in very different ways that allows me to see what it is I'm really trying to achieve in programming and see different ways of doing that. The author even goes so far as to call these niche. That's what they are: weird little languages that do things very differently from what you're used to. There is no air of superiority in the linked article.
Common Lisp doesn't belong on that list as it isn't particularly functional in general and is rather fast.
You can have expressive, easy to read, and performant, but most people have never seen StandardML before.
You can have expressive, easy to read, and performant, but most people have never seen StandardML before.
2009
(The article is undated but was present on the Internet Archive in 2009, https://web.archive.org/web/20090106175608/http://matt.might...)
(The article is undated but was present on the Internet Archive in 2009, https://web.archive.org/web/20090106175608/http://matt.might...)
Many previous threads as well: https://hn.algolia.com/?dateRange=all&page=0&prefix=true&que...
All the posts on Matt's blog are pretty old. I'd asked him to please add dates to the articles to make this more clear but he was (and always is) very busy so I guess it didn't happen haha. I think the most recent post is at least a couple years old, though he does update the rest of the site (publications, current work, etc.).
Was going to say, the recommendations seem rather dated.
What would you recommend currently?
So he recommends: "C++, Java and C#--and probably Python, Ruby, PHP and Perl too.... .. Haskell, Scala, ML and Scheme"
If you're going to learn that many languages, then for the current market, I would learn (in this order):
Python, JavaScript, Java or C#, C, Clojure / F# / Haskell, Rust.
If you're going to learn that many languages, then for the current market, I would learn (in this order):
Python, JavaScript, Java or C#, C, Clojure / F# / Haskell, Rust.
In the spirit as written (i.e., not necessarily my personal from-scratch list), drop Perl and Ruby [1], add Javascript (the language itself; not necessarily any specific framework). Possibly add Rust to the list of currently-somewhat-exotic languages... depends on how the author feels, it doesn't 100% fit but it's not completely foreign either. (If you squint hard enough, it as arguably the closest current entrant in the "Haskell, but practical to actually use" contest... note I'm not saying it is the Platonic ideal of such things, but the closest currently existing thing.) Go is debatably getting into the C++/Java/C# set. I'd also add a note that you don't need to deliberately learn all of them; one from a given set is enough, pick up any specific one you may need later on demand. Possible exception for C++ which is so funky that I'm not sure you can just "pick it up", but it's been completely possibly to build careers in the industry without ever using it. I'm 23 years in and still yet to touch it professionally.
[1] Neither may be "dead" but at this point I'd have a hard time recommending either to a young person looking to get started. Python and Javascript are both clearly better choices for that case in 2020; if you do encounter a Perl or Ruby codebase you'll be able to pivot just fine from a strong grounding in Python or JS.
[1] Neither may be "dead" but at this point I'd have a hard time recommending either to a young person looking to get started. Python and Javascript are both clearly better choices for that case in 2020; if you do encounter a Perl or Ruby codebase you'll be able to pivot just fine from a strong grounding in Python or JS.
If you've internalized many of the interesting ideas of Haskell, check out Idris: https://www.idris-lang.org/
It builds on the ideas in Haskell (and similar languages) and adds proofs and first class types to the language. With it you can use the type checker to prove almost any invariant about your code that you care about. So the obvious stuff is there, prove that I never access an array out of bounds, sure. But also: prove that this code obeys this network protocol (e.g. never sends header before a successful handshake, never sends body before header has completed, etc).
Ordinarily the best that you can do is expose an API that enforces these constraints, but nothing is checking that the internals of the implementation of that API uphold the constraints. With Idris you can move that into the type system, so that changes that would introduce a protocol violation fail at compile time.
It's wild! And that's really only scratching the surface. It's still firmly in academia land, but the ideas present are fascinating and definitely worth puzzling through.
The book is super good. It's well written, and introduces the ideas gradually but in a satisfying way: https://www.manning.com/books/type-driven-development-with-i...
It builds on the ideas in Haskell (and similar languages) and adds proofs and first class types to the language. With it you can use the type checker to prove almost any invariant about your code that you care about. So the obvious stuff is there, prove that I never access an array out of bounds, sure. But also: prove that this code obeys this network protocol (e.g. never sends header before a successful handshake, never sends body before header has completed, etc).
Ordinarily the best that you can do is expose an API that enforces these constraints, but nothing is checking that the internals of the implementation of that API uphold the constraints. With Idris you can move that into the type system, so that changes that would introduce a protocol violation fail at compile time.
It's wild! And that's really only scratching the surface. It's still firmly in academia land, but the ideas present are fascinating and definitely worth puzzling through.
The book is super good. It's well written, and introduces the ideas gradually but in a satisfying way: https://www.manning.com/books/type-driven-development-with-i...
This is definitely a dated article, since the obvious choice today would be Python, but I have to wonder: why aren't the "more expressive" (which in this post seems to be a synonym for "functional") languages used in these fields? I don't have much experience with ML languages, but it seems to me that something like Elixir (where the whole language is centered around pipes, parallelization, and operating on data) would be ideal for data science with a performant native library to back it up.