Pattern matching accepted for Python(lwn.net)
lwn.net
Pattern matching accepted for Python
https://lwn.net/Articles/845480/
538 comments
At first, I overlooked the excellent LWN writeup on the feature linked in TFA, so I'm linking it here:
https://lwn.net/Articles/838600/
https://lwn.net/Articles/838600/
Core dev Larry Hastings [0] puts it well. The cost-benefit case for this complicated language feature is limited.
"I dislike the syntax and semantics expressed in PEP 634. I see the match statement as a DSL contrived to look like Python, and to be used inside of Python, but with very different semantics. When you enter a PEP 634 match statement, the rules of the language change completely, and code that looks like existing Python code does something surprisingly very different. It also adds unprecedented new rules to Python, e.g. you can replace one expression with another in the exact same spot in your code, and if one has dots and the other doesn’t, the semantics of what the statement expresses changes completely. And it changes to yet a third set of semantics if you replace the expression with a single _.
I think the bar for adding new syntax to Python at this point in its life should be set very high. The language is already conceptually pretty large, and every new feature means new concepts one must learn if one is to read an arbitrary blob of someone else’s Python code. The bigger the new syntax, the higher the bar should become, and so the bigger payoff the new syntax has to provide. To me, pattern matching doesn’t seem like it’s anywhere near big enough a win to be worth its enormous new conceptual load."
This while we have elephants in the room such as packaging. Researching best practices to move away from setup.py right now takes you down a rabbit hole of (excellent) blog posts, and yet you still need a setup.py shim to use editable installs, because the new model simply doesn't yet support this fundamental feature.
I can't afford to spend days immersing myself in packaging to the point of writing a PEP, but would help pay someone to do it well. I can see no way to fund packaging efforts directly on the PSF donations page (edit: see comment). It's great to see Pip improving but there is still not even a coherent guide that I can find for packaging using up-to-date best practices. This appears to be because the best practices are currently slightly broken.
[0] https://discuss.python.org/t/gauging-sentiment-on-pattern-ma...
"I dislike the syntax and semantics expressed in PEP 634. I see the match statement as a DSL contrived to look like Python, and to be used inside of Python, but with very different semantics. When you enter a PEP 634 match statement, the rules of the language change completely, and code that looks like existing Python code does something surprisingly very different. It also adds unprecedented new rules to Python, e.g. you can replace one expression with another in the exact same spot in your code, and if one has dots and the other doesn’t, the semantics of what the statement expresses changes completely. And it changes to yet a third set of semantics if you replace the expression with a single _.
I think the bar for adding new syntax to Python at this point in its life should be set very high. The language is already conceptually pretty large, and every new feature means new concepts one must learn if one is to read an arbitrary blob of someone else’s Python code. The bigger the new syntax, the higher the bar should become, and so the bigger payoff the new syntax has to provide. To me, pattern matching doesn’t seem like it’s anywhere near big enough a win to be worth its enormous new conceptual load."
This while we have elephants in the room such as packaging. Researching best practices to move away from setup.py right now takes you down a rabbit hole of (excellent) blog posts, and yet you still need a setup.py shim to use editable installs, because the new model simply doesn't yet support this fundamental feature.
I can't afford to spend days immersing myself in packaging to the point of writing a PEP, but would help pay someone to do it well. I can see no way to fund packaging efforts directly on the PSF donations page (edit: see comment). It's great to see Pip improving but there is still not even a coherent guide that I can find for packaging using up-to-date best practices. This appears to be because the best practices are currently slightly broken.
[0] https://discuss.python.org/t/gauging-sentiment-on-pattern-ma...
> This while we have elephants in the room such as packaging.
There is a part of me that wonders, at this point, if basically every new addition to the language itself is secretly just a medium for procrastinating on figuring out what to do about setup.py.
I'm not as down on this PEP as some other seem to be, mind. It's just that, when I think about my actual pain points with python, this language addition starts to look like a very, very fancy bike shed.
There is a part of me that wonders, at this point, if basically every new addition to the language itself is secretly just a medium for procrastinating on figuring out what to do about setup.py.
I'm not as down on this PEP as some other seem to be, mind. It's just that, when I think about my actual pain points with python, this language addition starts to look like a very, very fancy bike shed.
The people that work on packaging don't overlap much with those that work on the core language. The latter don't seem to care about it much, as far as I can tell.
Have you taken a look at [PEP 517](https://www.python.org/dev/peps/pep-0517/) ? It enables other tools to replace setup.py (e.g., poetry is pretty nice for making easy-to-package-and-publish pure python libraries).
I sort of wonder why I see so much users complaining about package management in Python, meanwhile I'm having a fantastic journey since 2008 with it, with over 50 published open source packages. "Sort of" because I'm suspecting that the users in question just do not want to integrate upstream changes continuously, so, they expect the package manager to help them procrastinating on dependency updates, which has proven to lead to disasters such as npm install, and I'm kind of worried that this is the direction they have been taking.
But I admit I use setupmeta for auto definition of the version, and it just makes setup.py even better, but that's basically the only thing I like to add to setup.py, because it simplifies package publishing scripts. I haven't found any feature in pip to verify the gpg signatures that it allows us to upload packages with (python setup.py sdist upload --sign).
As for pattern matching is not specific to Python, it's available in many other languages and is a joy to use in OCaml, I see no reason why Python would not have pattern matching.
But I admit I use setupmeta for auto definition of the version, and it just makes setup.py even better, but that's basically the only thing I like to add to setup.py, because it simplifies package publishing scripts. I haven't found any feature in pip to verify the gpg signatures that it allows us to upload packages with (python setup.py sdist upload --sign).
As for pattern matching is not specific to Python, it's available in many other languages and is a joy to use in OCaml, I see no reason why Python would not have pattern matching.
Anything that breaks and changes semantics should not be allowed into the language. Let Python be Python, not a Frankenstein's monster of ideas like C++. If it were an idea that were Pythonic, you would not see the confusing examples I've seen in the comments. C++ is the poster child of trying to do too much with the language and it losing itself due to death-by-committee. It's very sad that Python has started down this road.
We seem to be seeing a paradigm clash. On one side, there are people who are concerned with whether or not a feature is desirable. On the other side, there are people who are concerned with whether or not a feature is desirable and also Pythonic.
> On one side, there are people who are concerned with whether or not a feature is desirable.
The thing is this: You can add something which, in isolation, seems desirable and positive -- but in the greater picture, is a net negative due to the complexity it adds.
People might say that those who do not like the pattern matching syntax are not obliged to use it. But when developing code in longer-running projects, far more code is read than written. Adding syntax, especially with complex edge cases, especially from languages which use concepts that are at the core quite alien to Pythons main concepts, adds a burden which is difficult to justify.
The thing is this: You can add something which, in isolation, seems desirable and positive -- but in the greater picture, is a net negative due to the complexity it adds.
People might say that those who do not like the pattern matching syntax are not obliged to use it. But when developing code in longer-running projects, far more code is read than written. Adding syntax, especially with complex edge cases, especially from languages which use concepts that are at the core quite alien to Pythons main concepts, adds a burden which is difficult to justify.
Very much so. I run into this with Clojure. It has so many different ways to skin every cat, each with its own unique blend of quirks, that it can be quite difficult to understand other people's code, and, by extension, use the language in a team setting.
That sort of experience leaves me thinking that this is a very dangerous turn to take for a language whose core ecological niche is, "Easy for professionals who don't have a BS in CS to understand and use productively." Lines 2 and 13 of PEP 20 are core to why Python, of all languages, came to dominate that niche. I am beginning to fear that the Python core team, being composed primarily of software engineers, is ill-equipped to properly understand that.
That sort of experience leaves me thinking that this is a very dangerous turn to take for a language whose core ecological niche is, "Easy for professionals who don't have a BS in CS to understand and use productively." Lines 2 and 13 of PEP 20 are core to why Python, of all languages, came to dominate that niche. I am beginning to fear that the Python core team, being composed primarily of software engineers, is ill-equipped to properly understand that.
Yap. It doesn't make sense to destroy the language just to get in a particular feature. You don't need a language to do everything. It needs to be good at everything it's meant to be good at.
I very much want to agree with you. Only that I do not know any more what "Pythonic" is supposed to mean.
One thing that Larry Hastings refers to seems often to be underestimated - readability.
It seems nice to be able to do code golfing and use pattern matching to reduce an expression from maybe 50 lines to 10.
But what matters far more is that one can read code easily, without guessing, and without consulting definitions of edge cases. Even in smaller projects, one will read 10 times more code than one writes. In larger projects and as a senior programmer, that could be a factor 100 or 1000. Not that unusual to work one week through a bug in someone else's code, and fix it by changing a single line. As code becomes more complex, it becomes really important to understand exactly what it means, without guessing. This is key for writing robust, reliable and correct code (and this is perhaps why the ML and functional languages, which stress correctness, tend to be quite compact).
And while it might be satisfying puzzle-solving for smart and easily bored people, like you and me, to write that pattern matching code and reduce its length, it is just not feasible to read through all the PEPs describing surprising syntactic edge cases in a larger code base.
One thing that Larry Hastings refers to seems often to be underestimated - readability.
It seems nice to be able to do code golfing and use pattern matching to reduce an expression from maybe 50 lines to 10.
But what matters far more is that one can read code easily, without guessing, and without consulting definitions of edge cases. Even in smaller projects, one will read 10 times more code than one writes. In larger projects and as a senior programmer, that could be a factor 100 or 1000. Not that unusual to work one week through a bug in someone else's code, and fix it by changing a single line. As code becomes more complex, it becomes really important to understand exactly what it means, without guessing. This is key for writing robust, reliable and correct code (and this is perhaps why the ML and functional languages, which stress correctness, tend to be quite compact).
And while it might be satisfying puzzle-solving for smart and easily bored people, like you and me, to write that pattern matching code and reduce its length, it is just not feasible to read through all the PEPs describing surprising syntactic edge cases in a larger code base.
I can only agree. Compared to other languages I find Python increasingly difficult to reason about, mainly due to its dynamicity. If the language complexity increases as well from now on I don't think I will use Python unless absolutely necessary.
Meanwhile, Julia supports pattern matching due to macros: https://github.com/kmsquire/Match.jl
Meanwhile, Julia supports pattern matching due to macros: https://github.com/kmsquire/Match.jl
Unfortunately Match.jl has gone unmaintained for a while, but we do have MLStyle.jl
https://github.com/thautwarm/MLStyle.jl
https://thautwarm.github.io/MLStyle.jl/latest/syntax/pattern...
https://github.com/thautwarm/MLStyle.jl
https://thautwarm.github.io/MLStyle.jl/latest/syntax/pattern...
That looks great, thanks.
Some of the english documentation can be quite hard to read, but the code is very useful. Submitting PRs to improve the documentation is welcomed by maintainers.
Pattern matching is the brainchild of ML. Python, being a multi-paradigm language with the strong functional side, missed this simple in concept and powerful in practice language concept.
> Python, being a multi-paradigm language with the strong functional side
Coming back to that, just a reminder that lambdas in Python are still gimped, closures do not work as expected because of -- scoping, and core developers in Python 3 tried to remove with "map" and "filter" tools that are considered quite essential for functional programming.
Coming back to that, just a reminder that lambdas in Python are still gimped, closures do not work as expected because of -- scoping, and core developers in Python 3 tried to remove with "map" and "filter" tools that are considered quite essential for functional programming.
I actually wish they had done so.
As someone who switches between Python and functional languages, I find Python's "map" and "filter" to be a trap, and have taken to scrupulously avoiding them. The problem is that I expect those functions to be pure, and, in Python, they aren't. They actually can't be, not even in principle, because their domain and range include a core datatype that cannot be interacted with in a pure manner: generators. A generator will change its own state every time you touch it. For example:
(Which isn't to say that you can't successfully use some functional techniques in Python. But it's best done in a measured, pragmatic way.)
As someone who switches between Python and functional languages, I find Python's "map" and "filter" to be a trap, and have taken to scrupulously avoiding them. The problem is that I expect those functions to be pure, and, in Python, they aren't. They actually can't be, not even in principle, because their domain and range include a core datatype that cannot be interacted with in a pure manner: generators. A generator will change its own state every time you touch it. For example:
>>> seq = (x for x in range(1, 11))
>>> list(filter(lambda x: x % 2 == 0, seq))
[2, 4, 6, 8, 10]
>>> list(filter(lambda x: x % 2 == 1, seq))
[]
(Which isn't to say that you can't successfully use some functional techniques in Python. But it's best done in a measured, pragmatic way.)
To be fair, that's a foot gun in haskell as well. Using lists non-linearly like this in haskell gives you the correct results but at a 10x performance tax or worse because it can't optimize into a loop anymore.
At least to me, a footgun goes beyond a mere performance gotcha. There's a whole category difference between, "If you do this, the compiler may not be able to optimize your code as well," and, "If you do this, your code may produce incorrect results."
A good example why immutability by default seems to be the right thing - in Clojure, "seq" would not have been modified by the first filter expression:
> (Which isn't to say that you can't successfully use some functional techniques in Python. But it's best done in a measured, pragmatic way.)
A great example how it is done right is Python's numpy package. The people who created that knew about functional languages and APL (which fits nicely in since Python's predecessor ABC had some APL smell). The obviously knew what they were doing, and created a highly usable combination of a general data type and powerful operations on it.
user=> (def seq_ (range 1 11))
user=> (filter (fn [x] (== (mod x 2) 0)) seq_)
(2 4 6 8 10)
user=> (filter (fn [x] (== (mod x 1) 0)) seq_)
(1 2 3 4 5 6 7 8 9 10)
user=> (filter even? seq_)
(2 4 6 8 10)
user=> (filter odd? seq_)
(1 3 5 7 9)
> (Which isn't to say that you can't successfully use some functional techniques in Python. But it's best done in a measured, pragmatic way.)
A great example how it is done right is Python's numpy package. The people who created that knew about functional languages and APL (which fits nicely in since Python's predecessor ABC had some APL smell). The obviously knew what they were doing, and created a highly usable combination of a general data type and powerful operations on it.
I found this very surprising so asked a Python-knowledgeable acquaintance who mentioned that this works as expected
>>> seq = [x for x in range(1,11)]
>>> list(filter(lambda x: x % 2 == 0, seq))
[2, 4, 6, 8, 10]
>>> list(filter(lambda x: x % 2 == 1, seq))
[1, 3, 5, 7, 9]Right. Because it's doing two different things. One is working with lists, the other is working with lazy calculations.
A common functional pattern is to do lazy calculations, so that you don't have to store every result in memory all at once. The subtext I'm getting at is that a language that has footguns that make it dangerous (for reasons of correctness, if not performance) to reuse and compose lazy calculations is a language that is fundamentally ill-suited to actual functional programming.
Which is fine! Python's already a great procedural-OO language. It's arguably the best procedural-OO language. Which is a big part of why it's taken over data science, business intelligence, and operations. Those are, incidentally, the domains where I feel just about the least need to program in a functional paradigm. And, in the domains where I do have a strong preference for FP, Python is already a poor choice for plenty of other reasons. (For example, the global interpreter lock eliminates one of the key practical benefits.) No amount of risky, scarring cosmetic surgery is going to change that.
A common functional pattern is to do lazy calculations, so that you don't have to store every result in memory all at once. The subtext I'm getting at is that a language that has footguns that make it dangerous (for reasons of correctness, if not performance) to reuse and compose lazy calculations is a language that is fundamentally ill-suited to actual functional programming.
Which is fine! Python's already a great procedural-OO language. It's arguably the best procedural-OO language. Which is a big part of why it's taken over data science, business intelligence, and operations. Those are, incidentally, the domains where I feel just about the least need to program in a functional paradigm. And, in the domains where I do have a strong preference for FP, Python is already a poor choice for plenty of other reasons. (For example, the global interpreter lock eliminates one of the key practical benefits.) No amount of risky, scarring cosmetic surgery is going to change that.
OK I see, the very short sequence in your example was a stand-in for a potentially infinite one. I got nerd-sniped into understanding what was happening as I found the behavior surprising.
It is very surprising.
And that is sort of another layer to why I think that people trying to turn Python into a functional language should just simmer down. A core principle in functional programming is that you should be able to swap out different implementations of an abstraction without changing the correctness of the program. That's not really something that can be enforced by most functional languages, but it's at least a precedent they set in their standard libraries. But Python's standard library has a different way of doing things, and favors different approaches to abstraction. And those conventions make the Python ecosystem a hostile environment for functional programming.
Hylang is another interesting example. It's a cool language. But watching how it evolved is slightly disheartening. It sought to be a lisp for Python, but the compromises they needed to make to get the language to interact well with Python have hurt a lot of its lisp feel, and make it a kind of peculiar language to try to learn as someone with a lisp background. IIRC, Python's lack of block scoping was an elephant in the room for that project, too.
And that is sort of another layer to why I think that people trying to turn Python into a functional language should just simmer down. A core principle in functional programming is that you should be able to swap out different implementations of an abstraction without changing the correctness of the program. That's not really something that can be enforced by most functional languages, but it's at least a precedent they set in their standard libraries. But Python's standard library has a different way of doing things, and favors different approaches to abstraction. And those conventions make the Python ecosystem a hostile environment for functional programming.
Hylang is another interesting example. It's a cool language. But watching how it evolved is slightly disheartening. It sought to be a lisp for Python, but the compromises they needed to make to get the language to interact well with Python have hurt a lot of its lisp feel, and make it a kind of peculiar language to try to learn as someone with a lisp background. IIRC, Python's lack of block scoping was an elephant in the room for that project, too.
> Python, being a multi-paradigm language with the strong functional side
I would doubt that. Surely, things like Numpy are written in a functional fashion, but Python relies very much on statements, iteration, things not being an expression, almost every named symbol except string and number literals being mutable, and there are no block-scoped name bindings which are essential to functional languages.
And the attempt to add the latter to Python might end in a far bigger train wreck than C++ is already.
Mixing OOP and functional style works, more or less, for Scala, but everyone agrees that Scala is a hugely complex language. And in difference to Python, it has immutable values.
What could turn out better would be to create a new language which runs interoperable in the same VM (much like Clojure runs alongside Java and can call into it). And that new language, perhaps with the file extension ".pfpy", would be almost purely functional, perhaps like a Scheme or Hy, without these dreaded parentheses. That would at least leverage Python's existing libraries.
I would doubt that. Surely, things like Numpy are written in a functional fashion, but Python relies very much on statements, iteration, things not being an expression, almost every named symbol except string and number literals being mutable, and there are no block-scoped name bindings which are essential to functional languages.
And the attempt to add the latter to Python might end in a far bigger train wreck than C++ is already.
Mixing OOP and functional style works, more or less, for Scala, but everyone agrees that Scala is a hugely complex language. And in difference to Python, it has immutable values.
What could turn out better would be to create a new language which runs interoperable in the same VM (much like Clojure runs alongside Java and can call into it). And that new language, perhaps with the file extension ".pfpy", would be almost purely functional, perhaps like a Scheme or Hy, without these dreaded parentheses. That would at least leverage Python's existing libraries.
> This while we have elephants in the room such as packaging. Researching best practices to move away from setup.py right now takes you down a rabbit hole of (excellent) blog posts, and yet you still need a setup.py shim to use editable installs, because the new model simply doesn't yet support this fundamental feature.
You can do editable installs with poetry, I do it every day.
Just run this: \rm -rv dist/; poetry build --format sdist && tar --wildcards -xvf dist/.tar.gz -O '/setup.py' > setup.py && pip3 install --prefix="${HOME}/.local/" --editable .
More details here: https://github.com/python-poetry/poetry/issues/34#issuecomme...
You can do editable installs with poetry, I do it every day.
Just run this: \rm -rv dist/; poetry build --format sdist && tar --wildcards -xvf dist/.tar.gz -O '/setup.py' > setup.py && pip3 install --prefix="${HOME}/.local/" --editable .
More details here: https://github.com/python-poetry/poetry/issues/34#issuecomme...
I do editable installs every day with setup.py – what is your point? Poetry is a very interesting third party solution to these issues, but not a best practice. Best practices matter in packaging.
Struggling a bit with Python and JS-related packaging stuff recently, I'm also wondering: in a clean-room environment, what does good Python packaging look like? Is it "get a single binary" stuff?
PyOxidizer feels like the big win, but a part of me wonders if the original sin of allowing aribtrary code execution during installs really stops a "wrapper" tool from getting this right. https://pyoxidizer.readthedocs.io/en/v0.9.0/index.html
PyOxidizer feels like the big win, but a part of me wonders if the original sin of allowing aribtrary code execution during installs really stops a "wrapper" tool from getting this right. https://pyoxidizer.readthedocs.io/en/v0.9.0/index.html
The gold standard for dynamic interpreted language package management is Yarn, and to a lesser extent npm. They are both cross platform, allows for easy publishing and building of native code dependencies, and support each package having its own conflicting dependencies, which means you don't have to do SAT solving. Furthermore, the metadata is held outside of the package, so package selection is much quicker than Python which requires downloading the entire library first and parsing the requirements.
Nope, cpan still is the gold standard.
Packages are tested before installation, and there is no need for hacks like pyenv or broken ruby envs. It also plays nicely with vendor packaging. No sat solving needed, deps are resolved dynamically, powered by simple Makefiles, not special baroque and limited declarations syntax.
Packages are tested before installation, and there is no need for hacks like pyenv or broken ruby envs. It also plays nicely with vendor packaging. No sat solving needed, deps are resolved dynamically, powered by simple Makefiles, not special baroque and limited declarations syntax.
> support each package having its own conflicting dependencies
So that you don't have to integrate upstream changes continuously... Not a way to build a sane ecosystem if you ask me.
> package selection is much quicker than Python
So is it because it downloads multiple versions of the same dependencies that it's so much slower than pip? Or is there more?
So that you don't have to integrate upstream changes continuously... Not a way to build a sane ecosystem if you ask me.
> package selection is much quicker than Python
So is it because it downloads multiple versions of the same dependencies that it's so much slower than pip? Or is there more?
Is there a difference between package management for dynamic/static languages?
Because without this arbitrary distinction npm is pretty much around last place on a ranking between package managers.
Did you try the "Sponsor" link in pypi.org ? It will take you there: https://pypi.org/sponsor/
> The Python Software Foundation is receiving $407,000 USD to support work on pip in 2020. Thank you to Mozilla (through its Mozilla Open Source Support Awards) and to the Chan Zuckerberg Initiative for this funding!
> The Python Software Foundation is receiving $407,000 USD to support work on pip in 2020. Thank you to Mozilla (through its Mozilla Open Source Support Awards) and to the Chan Zuckerberg Initiative for this funding!
They should give that to the meson author instead. He knows what he's doing and has a clue about C/C++.
Really, the whole distutils is a fancy way to produce a simple zip archive and put it in a well know location in site-packages.
meson has the build figured out, it's just a way of installing that archive with a bit of metadata.
Really, the whole distutils is a fancy way to produce a simple zip archive and put it in a well know location in site-packages.
meson has the build figured out, it's just a way of installing that archive with a bit of metadata.
Thanks, donated.
This gives the impression that Python, similar to C++. might have entered a competition between popular languages which one can accumulate the most popular features.
Obviously, pattern matching comes from the ML family and functional Lisps like Clojure. What makes it a difficult integration into Python is that in languages such as Rust, OCaml, Haskell, but also Racket and Clojure, almost everything is an expression, and name bindings are, apart from a few exceptions, always scoped. Consequently, pattern matching is an expression, not a statement.
A similar issue is that Python 3 tried to become more "lazy", similar to how Haskell and Clojure are - this is the reason for replacing some list results with generators, which is a subtle breaking change. Lazy evaluation of sequences is nice-to-have on servers but its importance in Haskell and Clojure comes from them being functional languages which are geared towards a "pure" (side-effect-free) style, in which they differ a lot from Python.
My impression is also that over time, Python has really absorbed a huge number of features from functional Lisp dialects. This might seem surprising. Well, here are some things that modern Lisps like SBCL, Schemes and functional languages on the one hand side, and Python 3 do have in common:
* a Read-Eval-Print Loop (REPL)
* strong dynamic typing
* automatic memory management
* memory safety
* exceptions
* a wide choice of built-in data types: lists, strings, vectors, arrays, dictionaries / hash maps, tuples, sets
* keyword arguments and optional arguments in functions
* handling of names in scopes and names spaces
* closures and lambda functions
* list comprehensions
* pattern matching (limited support for tuples and lists in Python)
* Unicode strings
* arbitrarily long integers
* complex numbers
* rational numbers
* number type are part of a hierarchical type hierarchy (numeric tower)
* empty sequences, containers and strings are logically false
* support for threads (but no real parallelism in Python)
* low-level bit operations (a bit limited in Python)
* easy way to call into C code
* type annotations
* if ... else can be used as an expression
* string formatting is a mini language
* hexadecimal, octal and binary literals
* standard functions for functional programming like map and filter
* support for OOP (e.g. by the Common Lisp Object System)
* support for asynchronous execution
What is notably missing from this list is parallelism (as opposed to concurrency); Python simply does not support it in a practical way, while some functional languages do support it extremely well.
The net result of this creeping featuritis appears to be that Python, which is still classified as "beginner friendly", is now actually significantly more complex than a language like Racket or Clojure, perhaps because these features are often not integrated that well. I even think that Rust, while clearly being targeted at a very different domain, is more streamlined and well-composed than Python.
What is also worth mentioning is that these functional languages have seen steady improvements in compilers and performance of generated code, with the result that Rust code is now frequently at least as fast as C code, SBCL is within arms reach of C, and Clojure and Racket are about in the same league as Java - while Python code in comparison continues to be extremely slow:
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
Obviously, pattern matching comes from the ML family and functional Lisps like Clojure. What makes it a difficult integration into Python is that in languages such as Rust, OCaml, Haskell, but also Racket and Clojure, almost everything is an expression, and name bindings are, apart from a few exceptions, always scoped. Consequently, pattern matching is an expression, not a statement.
A similar issue is that Python 3 tried to become more "lazy", similar to how Haskell and Clojure are - this is the reason for replacing some list results with generators, which is a subtle breaking change. Lazy evaluation of sequences is nice-to-have on servers but its importance in Haskell and Clojure comes from them being functional languages which are geared towards a "pure" (side-effect-free) style, in which they differ a lot from Python.
My impression is also that over time, Python has really absorbed a huge number of features from functional Lisp dialects. This might seem surprising. Well, here are some things that modern Lisps like SBCL, Schemes and functional languages on the one hand side, and Python 3 do have in common:
* a Read-Eval-Print Loop (REPL)
* strong dynamic typing
* automatic memory management
* memory safety
* exceptions
* a wide choice of built-in data types: lists, strings, vectors, arrays, dictionaries / hash maps, tuples, sets
* keyword arguments and optional arguments in functions
* handling of names in scopes and names spaces
* closures and lambda functions
* list comprehensions
* pattern matching (limited support for tuples and lists in Python)
* Unicode strings
* arbitrarily long integers
* complex numbers
* rational numbers
* number type are part of a hierarchical type hierarchy (numeric tower)
* empty sequences, containers and strings are logically false
* support for threads (but no real parallelism in Python)
* low-level bit operations (a bit limited in Python)
* easy way to call into C code
* type annotations
* if ... else can be used as an expression
* string formatting is a mini language
* hexadecimal, octal and binary literals
* standard functions for functional programming like map and filter
* support for OOP (e.g. by the Common Lisp Object System)
* support for asynchronous execution
What is notably missing from this list is parallelism (as opposed to concurrency); Python simply does not support it in a practical way, while some functional languages do support it extremely well.
The net result of this creeping featuritis appears to be that Python, which is still classified as "beginner friendly", is now actually significantly more complex than a language like Racket or Clojure, perhaps because these features are often not integrated that well. I even think that Rust, while clearly being targeted at a very different domain, is more streamlined and well-composed than Python.
What is also worth mentioning is that these functional languages have seen steady improvements in compilers and performance of generated code, with the result that Rust code is now frequently at least as fast as C code, SBCL is within arms reach of C, and Clojure and Racket are about in the same league as Java - while Python code in comparison continues to be extremely slow:
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
https://benchmarksgame-team.pages.debian.net/benchmarksgame/...
Has it absorbed features from functional Lisp dialects, or does it just have features in common?
Early Python was inspired by ABC and C. ABC has a Python-like REPL, strong dynamic typing, automatic memory management, memory safety, a nice collection of built-in types, arbitrarily long integers, and rational numbers. C has low-level bit operations, (an easy way to call into C code,) a ternary expression-level if-else operator (and make no mistake, Python's expression-if is a ternary operator that's distinct from the statement-if-else), and hexadecimal and octal number literals.
There is at least a little influence from functional Lisps (Mypy lists Typed Racket as one of its influences), but a lot of what you list was taken from different languages, or is distinctly un-Lisp-like in Python, or was in Python from the start rather than absorbed over time, or is just obvious enough to put in a very high-level language to be reinvented.
It's also important to distinguish between internal complexity and user complexity. Arbitrarily long integers are complex to implement, but easier to use than fixed-length integers. Even features that do have a lot of user-facing complexity can be very easy to use in the common case. Python is hideously complex if you explore all the tiny details, but I'm not sure that it's all that complex to use. But I haven't used Clojure and Racket so I can't really comment on them.
> I even think that Rust, while clearly being targeted at a very different domain, is more streamlined and well-composed than Python.
I think I agree. But Rust has the benefit of only dealing with a measly five years of backward compatibility. Python has accumulated complexity, but the alternative would have been stagnation. If Python hadn't significantly changed since 1996 it would be more streamlined but also dead.
> What is also worth mentioning is that these functional languages have seen steady improvements in compilers and performance of generated code, with the result that Rust code is now frequently at least as fast as C code
I don't think Rust suffers from the issues that make functional languages hard to compile, so that might be a bad example. In Rust code it's unambiguous where memory lives. It has functional features augmenting a procedural model, rather than a functional model that has to be brought down to the level of procedural execution. So it might be "merely" as hard to optimize as C++.
Early Python was inspired by ABC and C. ABC has a Python-like REPL, strong dynamic typing, automatic memory management, memory safety, a nice collection of built-in types, arbitrarily long integers, and rational numbers. C has low-level bit operations, (an easy way to call into C code,) a ternary expression-level if-else operator (and make no mistake, Python's expression-if is a ternary operator that's distinct from the statement-if-else), and hexadecimal and octal number literals.
There is at least a little influence from functional Lisps (Mypy lists Typed Racket as one of its influences), but a lot of what you list was taken from different languages, or is distinctly un-Lisp-like in Python, or was in Python from the start rather than absorbed over time, or is just obvious enough to put in a very high-level language to be reinvented.
It's also important to distinguish between internal complexity and user complexity. Arbitrarily long integers are complex to implement, but easier to use than fixed-length integers. Even features that do have a lot of user-facing complexity can be very easy to use in the common case. Python is hideously complex if you explore all the tiny details, but I'm not sure that it's all that complex to use. But I haven't used Clojure and Racket so I can't really comment on them.
> I even think that Rust, while clearly being targeted at a very different domain, is more streamlined and well-composed than Python.
I think I agree. But Rust has the benefit of only dealing with a measly five years of backward compatibility. Python has accumulated complexity, but the alternative would have been stagnation. If Python hadn't significantly changed since 1996 it would be more streamlined but also dead.
> What is also worth mentioning is that these functional languages have seen steady improvements in compilers and performance of generated code, with the result that Rust code is now frequently at least as fast as C code
I don't think Rust suffers from the issues that make functional languages hard to compile, so that might be a bad example. In Rust code it's unambiguous where memory lives. It has functional features augmenting a procedural model, rather than a functional model that has to be brought down to the level of procedural execution. So it might be "merely" as hard to optimize as C++.
> C has low-level bit operations,
As an unrelated fun fact, Common Lisp has more low-level bit operations than C, such as "and complement of integer a with integer b" or "exclusive nor":
http://www.lispworks.com/documentation/HyperSpec/Body/f_loga...
It also has logcount, which is the counterpart to the implementation-specific popcount() in C.
As an unrelated fun fact, Common Lisp has more low-level bit operations than C, such as "and complement of integer a with integer b" or "exclusive nor":
http://www.lispworks.com/documentation/HyperSpec/Body/f_loga...
It also has logcount, which is the counterpart to the implementation-specific popcount() in C.
> but a lot of what you list was taken from different languages, or is distinctly un-Lisp-like in Python, or was in Python from the start rather than absorbed over time, or is just obvious enough to put in a very high-level language to be reinvented.
Here, Python clearly borrows from functional languages. And there are basically two families of functional languages: Strongly and statically typed languages like ML, OCaml, Haskell, Scala, F#, and on the other hand, dynamically typed Lisps and Schemes.
My point is that all these adopted features are present in the latter category.
Here, Python clearly borrows from functional languages. And there are basically two families of functional languages: Strongly and statically typed languages like ML, OCaml, Haskell, Scala, F#, and on the other hand, dynamically typed Lisps and Schemes.
My point is that all these adopted features are present in the latter category.
How many of these features are present in neither statically typed functional languages nor dynamically typed procedural languages?
My impression is that Python has a) a lot of bog-standard dynamic features, and b) a few functional features (like most languages nowadays).
Group a) overlaps with functional Lisps, but no more than with ABC and Perl and Lua, so functional Lisps are not a great reference point.
Group b) overlaps with functional Lisps, but no more than with ML and Haskell, or even modern fundamentally-procedural languages like Kotlin(?) and Rust, so functional Lisps still aren't a great reference point.
It's mostly parallel evolution. It can be interesting to compare Python to functional Lisps because similarities are similarities no matter where they come from.
But I don't think that functional Lisps neatly slot into an explanation as to why Python looks the way it does. In a world where functional Lisps didn't exist Python might not have looked all that different. In a world where ABC and Modula didn't exist Python would have looked very different, if it existed at all.
My impression is that Python has a) a lot of bog-standard dynamic features, and b) a few functional features (like most languages nowadays).
Group a) overlaps with functional Lisps, but no more than with ABC and Perl and Lua, so functional Lisps are not a great reference point.
Group b) overlaps with functional Lisps, but no more than with ML and Haskell, or even modern fundamentally-procedural languages like Kotlin(?) and Rust, so functional Lisps still aren't a great reference point.
It's mostly parallel evolution. It can be interesting to compare Python to functional Lisps because similarities are similarities no matter where they come from.
But I don't think that functional Lisps neatly slot into an explanation as to why Python looks the way it does. In a world where functional Lisps didn't exist Python might not have looked all that different. In a world where ABC and Modula didn't exist Python would have looked very different, if it existed at all.
> Group b) overlaps with functional Lisps, but no more than with ML and Haskell, or even modern fundamentally-procedural languages like Kotlin(?) and Rust, so functional Lisps still aren't a great reference point.
Both of them stem from Lambda calculus. The difference between ML languages and Lisps is the type system. To do functional operations like map, foldl, filter, reduce in compiled ML-style languages with strong static typing, one needs a rather strong and somewhat complex type system. When you try that at home with a weaker type system, like C++ has for example, the result is messy and not at all pleasant to write.
Lisps/Schemes do the equivalent thing with strong dynamic typing, and good Lisp compilers doing a lot of type inference for speed.
> It's mostly parallel evolution. It can be interesting to compare Python to functional Lisps because similarities are similarities no matter where they come from.
Lisps (and, for the field of numerical computation, also APL and its successors) had and continue to have a lot of influence. They are basically at the origin of the language tree of functional programming. The MLs are a notable fork and apart from that there are basically no new original developments. I would not count that other languages like Java or C++ pick up some FP features like lambdas, too.
What's however interesting is the amount of features that Python 3 has now in common with Lisps. Lisps are minimalist languages - the have only a limited number of features which fit together extremely well.
And if all these features adopted were not basically arbitrary, unconnected, and easy to bolt-on, why has Python such a notably bad performance and -- similar as C++ -- such an explosion in complexity?
Both of them stem from Lambda calculus. The difference between ML languages and Lisps is the type system. To do functional operations like map, foldl, filter, reduce in compiled ML-style languages with strong static typing, one needs a rather strong and somewhat complex type system. When you try that at home with a weaker type system, like C++ has for example, the result is messy and not at all pleasant to write.
Lisps/Schemes do the equivalent thing with strong dynamic typing, and good Lisp compilers doing a lot of type inference for speed.
> It's mostly parallel evolution. It can be interesting to compare Python to functional Lisps because similarities are similarities no matter where they come from.
Lisps (and, for the field of numerical computation, also APL and its successors) had and continue to have a lot of influence. They are basically at the origin of the language tree of functional programming. The MLs are a notable fork and apart from that there are basically no new original developments. I would not count that other languages like Java or C++ pick up some FP features like lambdas, too.
What's however interesting is the amount of features that Python 3 has now in common with Lisps. Lisps are minimalist languages - the have only a limited number of features which fit together extremely well.
And if all these features adopted were not basically arbitrary, unconnected, and easy to bolt-on, why has Python such a notably bad performance and -- similar as C++ -- such an explosion in complexity?
Map, filter and lambda were originally suggested by a Lisp programmer, so those do show functional Lisp heritage. (I don't know of similar cases.) But they're a small part of the language. Comprehensions are now emphasized more, and they come from Haskell, and probably SETL originally—no Lisp there.
> They are basically at the origin of the language tree of functional programming.
That's fair. But that only covers Python's functional features, which aren't that numerous.
> if all these features adopted were not basically arbitrary, unconnected, and easy to bolt-on
I never said they weren't! I just don't think they're sourced from functional Lisps.
>why has Python such a notably bad performance
Because it's very dynamic, not afraid to expose deep implementation details, and deliberately kept simple and unoptimized. In the words of Guido van Rossum: "Python is about having the simplest, dumbest compiler imaginable."
Even if you wanted to, it's hard to optimize when somewhere down the stack someone might call sys._getframe() and start poking at the variables twenty frames up. That's not quite a language design problem.
PyPy is faster than CPython but it goes to great lengths to stay compatible with CPython's implementation details. A while ago I ran a toy program that generated its own bytecode on PyPy, to see what would happen, and to my surprise it just worked. I imagine that constrains them. V8 isn't bytecode-compatible with JavaScriptCore, at least to my knowledge.
The most pressing problems with Python's performance have more to do with implementation than with high-level language design.
PHP is the king of arbitrary, unconnected, bolted-on features, and it's pretty fast nowadays. Not much worse than Racket, eyeballing benchmarksgame, and sometimes better.
> and -- similar as C++ -- such an explosion in complexity?
I'm not so sure that it does. I'm given to understand the problem with C++ is that its features compose badly and interact in nasty ways. Do Python's? Looking at previously new features, I mainly see people complaining about f-strings and the walrus operator, but those are simple syntactic sugar that doesn't do anything crazy.
Instead of an explosion in complexity, I think there's merely a steady growth in size. People complain that it's becoming harder to keep the whole language in your head, and that outdated language features pile up. I think those are fair concerns. But these new features don't make the language slower (it was already slow), and they don't complicate other features with their mere existence.
The growth isn't even that fast. Take a look at https://docs.python.org/3.9/whatsnew/3.9.html . I wouldn't call it explosive.
I don't know enough C++ to program in it, but the existence of operator= seems like a special kind of hell that nothing in Python compares to.
> They are basically at the origin of the language tree of functional programming.
That's fair. But that only covers Python's functional features, which aren't that numerous.
> if all these features adopted were not basically arbitrary, unconnected, and easy to bolt-on
I never said they weren't! I just don't think they're sourced from functional Lisps.
>why has Python such a notably bad performance
Because it's very dynamic, not afraid to expose deep implementation details, and deliberately kept simple and unoptimized. In the words of Guido van Rossum: "Python is about having the simplest, dumbest compiler imaginable."
Even if you wanted to, it's hard to optimize when somewhere down the stack someone might call sys._getframe() and start poking at the variables twenty frames up. That's not quite a language design problem.
PyPy is faster than CPython but it goes to great lengths to stay compatible with CPython's implementation details. A while ago I ran a toy program that generated its own bytecode on PyPy, to see what would happen, and to my surprise it just worked. I imagine that constrains them. V8 isn't bytecode-compatible with JavaScriptCore, at least to my knowledge.
The most pressing problems with Python's performance have more to do with implementation than with high-level language design.
PHP is the king of arbitrary, unconnected, bolted-on features, and it's pretty fast nowadays. Not much worse than Racket, eyeballing benchmarksgame, and sometimes better.
> and -- similar as C++ -- such an explosion in complexity?
I'm not so sure that it does. I'm given to understand the problem with C++ is that its features compose badly and interact in nasty ways. Do Python's? Looking at previously new features, I mainly see people complaining about f-strings and the walrus operator, but those are simple syntactic sugar that doesn't do anything crazy.
Instead of an explosion in complexity, I think there's merely a steady growth in size. People complain that it's becoming harder to keep the whole language in your head, and that outdated language features pile up. I think those are fair concerns. But these new features don't make the language slower (it was already slow), and they don't complicate other features with their mere existence.
The growth isn't even that fast. Take a look at https://docs.python.org/3.9/whatsnew/3.9.html . I wouldn't call it explosive.
I don't know enough C++ to program in it, but the existence of operator= seems like a special kind of hell that nothing in Python compares to.
> Even if you wanted to, it's hard to optimize when somewhere down the stack someone might call sys._getframe() and start poking at the variables twenty frames up. That's not quite a language design problem.
It's hard to optimize only if you accept the tenet that it sys._getframe(), and all its uses, must continue to work exactly the same in optimized code.
Instead, you can just declare that that it (and any related sort of anti-pattern of the same ilk) won't work in optimized code. If you want the speed from optimized compiling of some code, then do not do to those things in that particular code.
The programmer can also be given fine-grained tools over optimization, so as to be able to choose how much is done where, on at least a function-by-function basis, if not statement or expression.
It's not written in stone that compiled code must behave exactly as interpreted code in every last regard, or that optimized code must behave as unoptimized code, in every regard. They behave the same in those ways which are selected as requirements and documented, and that's it.
In C in a GNU environment, I suspect your Glibc backtrace() function won't work very well if the code is compiled with -fomit-frame-pointer.
In the abstract semantics of C++, there are situations where the existence of temporary objects is implied. These objects are of a programmer-defined class type and can have constructors and destructors with side-effects. Yet, C++ allows compete freedom in optimizing away temporary objects.
The compiler could help by diagnosing situations, as much as possible, when it's not able to preserve this kind of semantics. Like if a sys._getframe call is being compiled with optimizations that rule it out, a warning could be issued that it won't work, and the generated code for it could blow up at run-time, if stepped on.
One way in which compiled code in a dynamic language could differ from interpreted code (or less "vigorously" compiled code) is safety. For that, you want some fine-grained, explicit switch, which expresses "in this block of code it's okay to make certain unsafe assumptions about values and types". Then, he optimizer removes checks from the generated code, or chooses unsafe primitives from the VM instruction set.
The code will then behave differently under conditions where the assumptions are violated. The unoptimized code will gracefully detect the problems, whereas the vigorously compiled code will behave erratically.
This entire view can nicely take into account programmer skill levels. Advanced optimization simply isn't foisted onto programmers of all skill levels, so then they have to grapple with issues they don't understand, with impaired ability to debug. You make it opt-in. People debug their programs to maturity without it and then gradually introduce it in places that are identified as bottlenecks.
It's hard to optimize only if you accept the tenet that it sys._getframe(), and all its uses, must continue to work exactly the same in optimized code.
Instead, you can just declare that that it (and any related sort of anti-pattern of the same ilk) won't work in optimized code. If you want the speed from optimized compiling of some code, then do not do to those things in that particular code.
The programmer can also be given fine-grained tools over optimization, so as to be able to choose how much is done where, on at least a function-by-function basis, if not statement or expression.
It's not written in stone that compiled code must behave exactly as interpreted code in every last regard, or that optimized code must behave as unoptimized code, in every regard. They behave the same in those ways which are selected as requirements and documented, and that's it.
In C in a GNU environment, I suspect your Glibc backtrace() function won't work very well if the code is compiled with -fomit-frame-pointer.
In the abstract semantics of C++, there are situations where the existence of temporary objects is implied. These objects are of a programmer-defined class type and can have constructors and destructors with side-effects. Yet, C++ allows compete freedom in optimizing away temporary objects.
The compiler could help by diagnosing situations, as much as possible, when it's not able to preserve this kind of semantics. Like if a sys._getframe call is being compiled with optimizations that rule it out, a warning could be issued that it won't work, and the generated code for it could blow up at run-time, if stepped on.
One way in which compiled code in a dynamic language could differ from interpreted code (or less "vigorously" compiled code) is safety. For that, you want some fine-grained, explicit switch, which expresses "in this block of code it's okay to make certain unsafe assumptions about values and types". Then, he optimizer removes checks from the generated code, or chooses unsafe primitives from the VM instruction set.
The code will then behave differently under conditions where the assumptions are violated. The unoptimized code will gracefully detect the problems, whereas the vigorously compiled code will behave erratically.
This entire view can nicely take into account programmer skill levels. Advanced optimization simply isn't foisted onto programmers of all skill levels, so then they have to grapple with issues they don't understand, with impaired ability to debug. You make it opt-in. People debug their programs to maturity without it and then gradually introduce it in places that are identified as bottlenecks.
> In C in a GNU environment, I suspect your Glibc backtrace() function won't work very well if the code is compiled with -fomit-frame-pointer.
Actually, backtraces work correctly without explicit framepointers (in a typical GNU environment using ELF+DWARF).
The general concept has existed in DWARF since version 2 in 1992. The mechanism used for this is known as Call Frame Information (CFI)[0][1] — not to be confused with Control Flow Integrity, which is unrelated.
Here's some example libgcc code that evaluates CFI metadata[2]; there's similar logic in the libunwind component of llvm[3].
Burning a register on a frame pointer is a big deal on i386 and somewhat less so on amd64; there are other platforms where the impact is even lower. So, just know that you don't have to include FPs to be able to get stack traces.
If you're interested in how to apply these directives to hand-written assembler routines, there are some nice examples in [0].
[0]: https://www.imperialviolet.org/2017/01/18/cfi.html
[1]: https://sourceware.org/binutils/docs/as/CFI-directives.html
[2]: https://github.com/gcc-mirror/gcc/blob/master/libgcc/unwind-...
[3]: https://github.com/llvm/llvm-project/blob/main/libunwind/src...
Actually, backtraces work correctly without explicit framepointers (in a typical GNU environment using ELF+DWARF).
The general concept has existed in DWARF since version 2 in 1992. The mechanism used for this is known as Call Frame Information (CFI)[0][1] — not to be confused with Control Flow Integrity, which is unrelated.
Here's some example libgcc code that evaluates CFI metadata[2]; there's similar logic in the libunwind component of llvm[3].
Burning a register on a frame pointer is a big deal on i386 and somewhat less so on amd64; there are other platforms where the impact is even lower. So, just know that you don't have to include FPs to be able to get stack traces.
If you're interested in how to apply these directives to hand-written assembler routines, there are some nice examples in [0].
[0]: https://www.imperialviolet.org/2017/01/18/cfi.html
[1]: https://sourceware.org/binutils/docs/as/CFI-directives.html
[2]: https://github.com/gcc-mirror/gcc/blob/master/libgcc/unwind-...
[3]: https://github.com/llvm/llvm-project/blob/main/libunwind/src...
> I don't think Rust suffers from the issues that make functional languages hard to compile, so that might be a bad example.
The issue is that in functional languages, the compiler has more information and can reliably rely on more assumptions, thus it can make more optimized code. This is why also Common Lisp can compile to very fast code (in a few of the cited micro-benchmarks, faster than Java).
The issue is that in functional languages, the compiler has more information and can reliably rely on more assumptions, thus it can make more optimized code. This is why also Common Lisp can compile to very fast code (in a few of the cited micro-benchmarks, faster than Java).
Yes, a lot of these features are hard to integrate into Python because it has peculiar (to say the least) namespaces.
You just need to follow the mailing lists [1] and see that many core developers themselves are caught by some namespace abnormality.
It is a pity that Greenspun's 10th rule is in full effect again and we're stuck with an inferior Lisp/ML, especially in the scientific sector.
[1] Or rather the archives, since they are now de facto corporate owned. There is no free discussion any more and people are afraid to post.
You just need to follow the mailing lists [1] and see that many core developers themselves are caught by some namespace abnormality.
It is a pity that Greenspun's 10th rule is in full effect again and we're stuck with an inferior Lisp/ML, especially in the scientific sector.
[1] Or rather the archives, since they are now de facto corporate owned. There is no free discussion any more and people are afraid to post.
> and we're stuck with an inferior Lisp/ML, especially in the scientific sector.
You will love Julia.
Here are some links:
https://julialang.org/blog/2012/02/why-we-created-julia/
Julia: Dynamism and Performance Reconciled by Design (https://dl.acm.org/doi/pdf/10.1145/3276490) <= This is great and surprisingly approachable.
https://opensourc.es/blog/basics-multiple-dispatch/
And when you start finding things that you miss, Julia and the community got you with excellent Metaprogramming support.
https://github.com/thautwarm/MLStyle.jl
https://github.com/MikeInnes/Lazy.jl
https://github.com/jkrumbiegel/Chain.jl
You will love Julia.
Here are some links:
https://julialang.org/blog/2012/02/why-we-created-julia/
Julia: Dynamism and Performance Reconciled by Design (https://dl.acm.org/doi/pdf/10.1145/3276490) <= This is great and surprisingly approachable.
https://opensourc.es/blog/basics-multiple-dispatch/
And when you start finding things that you miss, Julia and the community got you with excellent Metaprogramming support.
https://github.com/thautwarm/MLStyle.jl
https://github.com/MikeInnes/Lazy.jl
https://github.com/jkrumbiegel/Chain.jl
I know really little about Python except that it was heavily adopted by Google.
However for C++, I think that corporatization is having a strong negative influence on the language, which leads to it being stuffed with poorly integrated features which nobody really overlooks any more.
However for C++, I think that corporatization is having a strong negative influence on the language, which leads to it being stuffed with poorly integrated features which nobody really overlooks any more.
I would not call this “improving”:
https://twitter.com/mildmojo/status/1354514025414066178?s=20
https://twitter.com/szferi/status/1352208874657472512?s=20
https://www.reddit.com/r/Python/comments/kfxibk/pypi_xmlrpc_...
And relatedly they have a massive spam package problem:
https://www.zdnet.com/article/pypi-gitlab-dealing-with-spam-...
https://twitter.com/mildmojo/status/1354514025414066178?s=20
https://twitter.com/szferi/status/1352208874657472512?s=20
https://www.reddit.com/r/Python/comments/kfxibk/pypi_xmlrpc_...
And relatedly they have a massive spam package problem:
https://www.zdnet.com/article/pypi-gitlab-dealing-with-spam-...
Are you on the wrong article discussion? Or is this just a personal axe to grind?
Honestly I'm not sure what all the fuss is about. I just skimmed over PEP 634 and PEP 636 (the accompanying tutorial) and I'm actually kind of excited to use this. I've missed this feature after using langs like Rust, Haskell, and Scala. People have pointed out some surprising edge cases such as this:
I, for one, am excited to start using this syntax.
NOT_FOUND = 404
match get_status():
case NOT_FOUND:
# this actually assigns to the global NOT_FOUND
# and is effectively the default case
case _:
# this never gets triggered
class Status(Enum):
NOT_FOUND = 404
match get_status():
case Status.NOT_FOUND:
# works as expected
...
def func_with_default(foo={}):
if "bar" not in foo:
# "bar" is set in the module-level dict that
# was initialized when the func def was parsed,
# not in a dict that is created per call
foo["bar"] = ...
...
I, for one, am excited to start using this syntax.
A previous mistake does not make a convincing argument for making a new mistake. Also, the mutable default arguments problem is quite difficult to solve in any other way. Python never copies things for you.
Pattern matching is a desired feature, but the shortcomings are too costly for what it provides. You just outlined yourself the quickest foot-gun I've ever seen.
My second problem is that this violates many of the invariants you rely on while programming Python. f(x) always gives you the value of applying f to x. Nope, not in case. | means bitwise or. Nope, not in case. a means the value of a. NOPE. Not in case.
Pattern matching is a desired feature, but the shortcomings are too costly for what it provides. You just outlined yourself the quickest foot-gun I've ever seen.
My second problem is that this violates many of the invariants you rely on while programming Python. f(x) always gives you the value of applying f to x. Nope, not in case. | means bitwise or. Nope, not in case. a means the value of a. NOPE. Not in case.
> Also, the mutable default arguments problem is quite difficult to solve in any other way. Python never copies things for you.
Why would it need to copy anything? What's wrong with just evaluating the default argument expression every time the function is called with a default argument rather than once at definition-time? That's how it works in other languages.
Why would it need to copy anything? What's wrong with just evaluating the default argument expression every time the function is called with a default argument rather than once at definition-time? That's how it works in other languages.
The variables used in the expressions might not be in scope (in fact they usually aren't). Also, I'm rather sure that's how it works in C++ (which by accident copies arguments instead of leaving a reference, but the single time default argument evaluation holds).
> The variables used in the expressions might not be in scope (in fact they usually aren't).
That's solved easily enough by evaluating the expression in the scope where it was defined (again that's what other languages do).
> Also, I'm rather sure that's how it works in C++
Default arguments in C++ work as I described: If you define `void f(int arg = g(x)) {}`, then calling `f()` n times will call `g` n times as well (using the `x` that was in scope when `f` was defined) - and if you never call `f`, `g` isn't called either.
An example demonstrating this: https://ideone.com/vs26Oq
That's solved easily enough by evaluating the expression in the scope where it was defined (again that's what other languages do).
> Also, I'm rather sure that's how it works in C++
Default arguments in C++ work as I described: If you define `void f(int arg = g(x)) {}`, then calling `f()` n times will call `g` n times as well (using the `x` that was in scope when `f` was defined) - and if you never call `f`, `g` isn't called either.
An example demonstrating this: https://ideone.com/vs26Oq
That’s a good idea, I like that!
A variable name has never referred to the value of it in expressions where it is being assigned. In 'def f(a,b,c)' or '(a,*_) = x' the term 'a' doesn't refer to the value of a either. A match statement in this sense is just an assignment that has a graceful failing mechanism (rather than e.g. '(a,*_) = x' which throws for empty lists).
And I think the ship on '|' has sailed with the dictionaries.
And I think the ship on '|' has sailed with the dictionaries.
Isn't it the same in ML though (using the same syntax in different contexts) ? Compare `| [a, b] => a + b` and `let my_list = [a, b]`. Same in JS, `let {a, b} = {a: 1, b: 2}` and `let d = {a, b}`. It's weird at first but then you just get used to it.
edit: updated list name since pipe ("|") and small L ("l") look kind similar in the code snippets.
edit: updated list name since pipe ("|") and small L ("l") look kind similar in the code snippets.
> the mutable default arguments problem is quite difficult to solve in any other way. Python never copies things for you.
Why would it be a problem to do a copy in this case?
> f(x) always gives you the value of applying f to x
Not if it is preceded by "def"
> | means bitwise or.
Not if it had been overloaded, like many popular libraries do.
> a means the value of a
Not if it is followed by "="
How is that different from the new pattern matching syntax ?
Why would it be a problem to do a copy in this case?
> f(x) always gives you the value of applying f to x
Not if it is preceded by "def"
> | means bitwise or.
Not if it had been overloaded, like many popular libraries do.
> a means the value of a
Not if it is followed by "="
How is that different from the new pattern matching syntax ?
> Why would it be a problem to do a copy in this case?
How are copy semantics even defined unambiguously?
> Not if it is preceded by "def"
In that case it is analogous, def f(x) DEFINES what f(x) means. For class definitions it's different, and it is indeed debatable if `class A(SuperClass)` is good syntax. I would have preferred `class A from SuperClass`, but that ship has sailed.
> | means bitwise or. Not if it had been overloaded, like many popular libraries do.
You missed the point willingly I think. See sibling comment. You cannot overload its meaning in a match, and the meaning is not related to what it otherwise means (bitwise or, set union, etc.).
> a means the value of a Not if it is followed by "="
And again, this is by analogy. a = 1 means that you substitute a with 1 wherever a occurs. Not so with the pattern matching syntax. If we had a = 1 then a == 1. Where is this analogy for pattern matching? How do you even make the comparison? You do not.
All of your counterpoints have reasonable analogies within the syntax itself (eg: the inverse of a function call is a function definition).
How are copy semantics even defined unambiguously?
> Not if it is preceded by "def"
In that case it is analogous, def f(x) DEFINES what f(x) means. For class definitions it's different, and it is indeed debatable if `class A(SuperClass)` is good syntax. I would have preferred `class A from SuperClass`, but that ship has sailed.
> | means bitwise or. Not if it had been overloaded, like many popular libraries do.
You missed the point willingly I think. See sibling comment. You cannot overload its meaning in a match, and the meaning is not related to what it otherwise means (bitwise or, set union, etc.).
> a means the value of a Not if it is followed by "="
And again, this is by analogy. a = 1 means that you substitute a with 1 wherever a occurs. Not so with the pattern matching syntax. If we had a = 1 then a == 1. Where is this analogy for pattern matching? How do you even make the comparison? You do not.
All of your counterpoints have reasonable analogies within the syntax itself (eg: the inverse of a function call is a function definition).
"f(x)" and "|" inside a regular expression mean something different. Similarly here, pattern matching has its own sub-language (a DSL).
Coincidentally, regular expressions are not part of the Python syntax. They're just string literals.
Are you being intentionally ironic here, or do you not know that regex DSL is written only inside of strings in python, not within the Python's syntax?
For what it's worth, I tried a modified version of the first example you've used above in the sandbox:
NOT_FOUND = 404
match get_status():
case NOT_FOUND:
print("Here")
# this actually assigns to the global NOT_FOUND
# and is effectively the default case
case _:
# this never gets triggered
print("There")
File "<ipython-input-13-5264d8327114>", line 3
case NOT_FOUND:
^
SyntaxError: name capture 'NOT_FOUND' makes remaining patterns unreachableThat in itself is weird though. Why throw a syntax error there but not here?:
def f(x):
return 1
y = x + 1
return yYou’ve braver than me, I had the same thoughts but didn’t dare post in here.
Not entirely surprised by the negative responses and I can see there are gotchas here, which is always thorny. But everyone seems to be picking simple uninteresting use cases so they can knock then down.
This feature would seem to be for cases where you need to do more complex matching.
In terms of the variable assignment, surely the whole point of this feature is to give you nice assignment based on matching, rather than using it as an if or switch statements?
Not entirely surprised by the negative responses and I can see there are gotchas here, which is always thorny. But everyone seems to be picking simple uninteresting use cases so they can knock then down.
This feature would seem to be for cases where you need to do more complex matching.
In terms of the variable assignment, surely the whole point of this feature is to give you nice assignment based on matching, rather than using it as an if or switch statements?
> Yeah, I agree that's a bit ugly.
It's not about ugliness, it's about clarity and robustness. It's about not ingraining landmines into the language, whether beautiful or ugly. This is just asking to be blown up in people's faces over and over again.
It's not about ugliness, it's about clarity and robustness. It's about not ingraining landmines into the language, whether beautiful or ugly. This is just asking to be blown up in people's faces over and over again.
The behaviour my intuition would have expected is that if a variable is currently in scope, then it is considered as a literal match, otherwise it's a capture match.
I suppose there's an obvious flaw to that behaviour, though.
I suppose there's an obvious flaw to that behaviour, though.
No, that's not robust. The idea was simple - to use explicit marker like:
case == FOO: case is True:
That was part of PEP642, but it ended up containing more other questionable things. I hope that particular point will be separated and implemented.
case == FOO: case is True:
That was part of PEP642, but it ended up containing more other questionable things. I hope that particular point will be separated and implemented.
[deleted]
module-level dict?... why...
Uh...that's my point. Everyone new to Python has to eventually learn not to do that.
It's a simple rule: Default arguments are evaluated at function definition time (when the interpreter reaches the 'def' statement) not at the time the function is called.
It's not a "module-level dict", you can create functions inside other functions:
It's not a "module-level dict", you can create functions inside other functions:
def foo():
def bar(a={}):
...
return bar
I understand this simple rule, and I understand closures, of course, it's just using closure semantics for default arguments doesn't make any sense, aside from some optimization maybe, and pretty dangerous. Other approaches are possible, see ruby, for example, default arguments are evaluated basically as if they're already in the body of running function, every time, only when they're not passed, and with the same lexical scope as the function/method.
And what, python retained this behavior even in 3.x? omg.
Ah, sorry for over-explaining.
The folk over at openAI do something similar in order to create a registry, so that you can initialize objects using a name.
IMO the match statement has some very unintuitive behaviour:
match status:
case 404:
return "Not found"
not_found = 404
match status:
case not_found:
return "Not found"
Yes, and this is incredibly disappointing.
Couldn't we achieve the same functionality with a little less ambiguity using the following syntax?:
Couldn't we achieve the same functionality with a little less ambiguity using the following syntax?:
not_found = 404
match status:
case not_found:
return "Not found"
case _ as foo:
do_something(foo)
return "Match anything"
match json_pet:
case {"type": "cat", "name": _ as name, "pattern": _ as pattern}:
return Cat(name, pattern)
case {"type": "dog", "name": _ as name, "breed": _ as breed}:
return Dog(name, breed)
case _:
raise ValueError("Not a suitable pet")I dig it. 'as' primes my brain to something going into a scoped context.
It’s fine. Python just introduced expression assignment. Why not use that?
match subject
case [first := _, rest := _]
return [rest, first]That works too but I guess I am still getting used to the walrus operator. What really bugs me about this PEP is they are going through all this effort to introduce match but they don't have match assignments, which is personally one of my favorite things about match.
Well get to Perl eventually
Note that the original PEP622 relied on ":=" initially, but they found some corner case with it (apparently, operator precedence related) and switched to "as" in PEP634.
I'm still wary about this change making it in to Python, but I like this suggestion. It makes the assignment clear. The way it's currently specified would definitely trip me at some point.
I like your proposal. The `as` keyword is very readable to me.
I would have preferred `as` instead of the walrus operator too.
I would have preferred `as` instead of the walrus operator too.
This would be really confusing to anyone who is used with match statements in other languages
For 2 seconds until they learn what it means. If anything, it's more explicit what exactly it does.
There are 100s of things in Python who would be confusing to people used to other languages, this is not one of them.
There are 100s of things in Python who would be confusing to people used to other languages, this is not one of them.
The general tripping up of binding vs mutation vs assignment vs initialization is a pervasive Python issue. This just continues to double down on exacerbating the problem.
Comparing garbage syntax to worse syntax and saying “it’s not as bad as that” isn’t the way one improves upon a language.
Which is neither here, nor there.
This syntax is clearly an improvemen to the PEP syntax.
And parent said it's "confusing". Well, it's less confusing than the PEP syntax.
Nobody compared it to not adding it at all, or was concerned with improving the language aside from the pattern matching case here.
This syntax is clearly an improvemen to the PEP syntax.
And parent said it's "confusing". Well, it's less confusing than the PEP syntax.
Nobody compared it to not adding it at all, or was concerned with improving the language aside from the pattern matching case here.
Almost everyone in this discussion is making exactly those comparisons. Including yourself. When you're discussing usability issues due to changes to the syntax, the perspective of non-exclusive developers vs full time Python devs doesn't change the underlying context of the discussion regarding the change in usability.
And I stand by my position that defending a bad decision because of the existence of worse decisions is a really piss poor counterargument.
Disclaimer: I'm a language designer myself so I know first hand how hard it is to get these things right.
And I stand by my position that defending a bad decision because of the existence of worse decisions is a really piss poor counterargument.
Disclaimer: I'm a language designer myself so I know first hand how hard it is to get these things right.
>And I stand by my position that defending a bad decision because of the existence of worse decisions is a really piss poor counterargument.
This thread was just about the two alternatives (the PEP and explicit capture), not about the PEP in general, or about defending the PEP or even saying that the better alternative is "good". We just say it's better than the PEP. Not sure how you read that into what we wrote.
>Disclaimer: I'm a language designer myself so I know first hand how hard it is to get these things right.
Then go make that argument in some thread in this post discussing the PEP proposal in general?
This thread was just about the two alternatives (the PEP and explicit capture), not about the PEP in general, or about defending the PEP or even saying that the better alternative is "good". We just say it's better than the PEP. Not sure how you read that into what we wrote.
>Disclaimer: I'm a language designer myself so I know first hand how hard it is to get these things right.
Then go make that argument in some thread in this post discussing the PEP proposal in general?
> This thread was just about the two alternatives (the PEP and explicit capture), not about the PEP in general, or about defending the PEP or even saying that the better alternative is "good". We just say it's better than the PEP. Not sure how you read that into what we wrote.
Bullshit. You said, and I quote, "There are 100s of things in Python who would be confusing". That's what I was responding to. And my point stands: just because there are other pain points in Python doesn't mean we should accept more pain points into it.
> Then go make that argument in some thread in this post discussing the PEP proposal in general?
I’d prefer to make that argument in the thread where I was already talking you about the PEP proposal in general. ;)
Bullshit. You said, and I quote, "There are 100s of things in Python who would be confusing". That's what I was responding to. And my point stands: just because there are other pain points in Python doesn't mean we should accept more pain points into it.
> Then go make that argument in some thread in this post discussing the PEP proposal in general?
I’d prefer to make that argument in the thread where I was already talking you about the PEP proposal in general. ;)
I think the general criticism of the match statement is just baggage from C overexposure. See the keyword "case" and the brain immediately snaps to thinking we're in a bog-standard C-style switch statement.
It's not a switch! Nowhere does it say switch. It's structural pattern matching!
EDIT: The lack of block scoped variable thing does seem like a wart right enough.
It's not a switch! Nowhere does it say switch. It's structural pattern matching!
EDIT: The lack of block scoped variable thing does seem like a wart right enough.
OK, but from a functional programming point of view (where structural pattern matching comes from), "case" should bind a value to a name, not mutate the value of an existing variable. That seems nuts to me.
Does it shadow the outside variable or mutate it?
Mutates it. Python `match` statements do not introduce a new scope.
Ouch, that's indeed pretty bad. I do expect `not_found = status` (that's how pattern matching works in several other languages), but it should be in its own scope, so that `not_found` is still `404` outside of the `match` block!
Python doesn't have block scope. This can lead to some surprising behaviour:
x = -1
for x in range(7):
if x == 6:
print(x, ': for x inside loop')
print(x, ': x in outer')
6 : for x inside loop
6 : x in outer
It would make even more sense for Python not to adopt language features that can't be made to behave consistently with the language's existing syntax and semantics without introducing horrendous footguns.
I've been programming in dialects of ML for much longer than Python, so I absolutely appreciate the attraction of this feature. But, the opinion I'm coming to as I think more about this is:
I've been programming in dialects of ML for much longer than Python, so I absolutely appreciate the attraction of this feature. But, the opinion I'm coming to as I think more about this is:
1. I *love* pattern matching.
2. But not in Python.Exactly. At this point it feels like Python is LARPing at being an ML. Without even block scoping, this is just asking for trouble.
To be fair, lots of languages have been moving closer to ML recently (and new languages tend to look more ML-like than in the past). That includes the adoption of pattern matching, but also things like value objects, first-class functions, sum types, named tuples, algebraic datatypes, type inference, etc.
I don't think that's a bad thing. I do think care should be taken when incorporating features from other languages, to see how it interacts with existing features, what the best form of that feature would be, and perhaps whether some different feature could achieve a similar goal.
(For the latter point, I find it unfortunate that languages which already contain 'try/catch' have been introducing 'yield' and 'async/await' as separate features; rather than generalising to 'shift/reset' and implementing all three as libraries)
I don't think that's a bad thing. I do think care should be taken when incorporating features from other languages, to see how it interacts with existing features, what the best form of that feature would be, and perhaps whether some different feature could achieve a similar goal.
(For the latter point, I find it unfortunate that languages which already contain 'try/catch' have been introducing 'yield' and 'async/await' as separate features; rather than generalising to 'shift/reset' and implementing all three as libraries)
Interested in block-level scoping in Python? Please post on the python-ideas mailing list. Thanks.
No, not interested in block-level scoping in Python. Why on earth would I ask a programming language I rely on for getting important work done to introduce so massive a breaking change as changing its scoping style?
"I don't think feature X is a good fit for Python because it interacts poorly with existing Python features Y and Z" is not a tacit statement of support for changing features Y and Z. It's a statement that means exactly what it says.
Like I alluded to in my grandparent post, I use other languages that are not Python, and like their features, too. That does not necessarily mean I want all my favorite features from those pulled into Python. Nor do I want my favorite features from Python pulled into other languages.
The best analogy I can think of is a college friend once developed this sort of transitive theory of food that went, "If A tastes good with B, and B tastes good with C, then A must taste good with C." This resulted in a number of questionable concoctions, like serving cottage cheese on top of chocolate cake because they both taste good with strawberries.
"I don't think feature X is a good fit for Python because it interacts poorly with existing Python features Y and Z" is not a tacit statement of support for changing features Y and Z. It's a statement that means exactly what it says.
Like I alluded to in my grandparent post, I use other languages that are not Python, and like their features, too. That does not necessarily mean I want all my favorite features from those pulled into Python. Nor do I want my favorite features from Python pulled into other languages.
The best analogy I can think of is a college friend once developed this sort of transitive theory of food that went, "If A tastes good with B, and B tastes good with C, then A must taste good with C." This resulted in a number of questionable concoctions, like serving cottage cheese on top of chocolate cake because they both taste good with strawberries.
Actually, the opposite. Call me an old crank but I feel like Python should stay Python and not try to (badly) tack on ideas from ML.
For my part, as I maintain some complex academic research software, I would be much more interested in continuing
support for Python 2, and more comprehensive numerical, math and vector libraries for Racket.
I haven't been programming in languages with pattern matching longer than Python and I still agree with you. Pattern matching is awesome, but it doesn't suit Python at all. I hope they reconsider adding it.
No, we'll rather fix issues people see with it.
Interested in block-level scoping in Python? Please post on the python-ideas mailing list. Thanks.
Interested in block-level scoping in Python? Please post on the python-ideas mailing list. Thanks.
> It would make even more sense for Python not to adopt language features that can't be made to behave consistently with the language's existing syntax and semantics
Current match behavior is consistent with existing syntax and semantics.
And no, not adopting new features known to be generally useful across programming languages is not "better". Instead, better to continue elaborating the language even after the initial pattern matching support is added.
Interested in block-level scoping in Python? Please post on the python-ideas mailing list.
Current match behavior is consistent with existing syntax and semantics.
And no, not adopting new features known to be generally useful across programming languages is not "better". Instead, better to continue elaborating the language even after the initial pattern matching support is added.
Interested in block-level scoping in Python? Please post on the python-ideas mailing list.
Why did you chop the last four words off of that sentence you quoted? All it accomplishes is making it so that the response rebuts a statement that wasn't actually made.
I don't know if I even know what "being consistent" would mean given the example here:
https://news.ycombinator.com/item?id=26082225
We don't shadow variables when they are re-used in comprehensions for example:
https://news.ycombinator.com/item?id=26082225
We don't shadow variables when they are re-used in comprehensions for example:
>>> i = 4
>>> [i for i in range(10)]
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
>>> i
4
> We don't shadow variables when they are re-used in comprehensions for example:
I think "for example" makes this sound more general than it is. List comprehensions are one of the rare few constructs that introduce a new scope. The match block is more syntactically similar to if/while/etc. blocks, and so it makes sense that it would follow similar scoping rules.
This is not to say that I agree with the design as a whole. I think they should have went with:
I think "for example" makes this sound more general than it is. List comprehensions are one of the rare few constructs that introduce a new scope. The match block is more syntactically similar to if/while/etc. blocks, and so it makes sense that it would follow similar scoping rules.
This is not to say that I agree with the design as a whole. I think they should have went with:
case obj.some_value:
...
case some_value: # behaves the same way as the previous
...
case _ as variable:
...
case (variable := _): # an alternative to the previous syntax
...
> The match block is more syntactically similar to if/while/etc. blocks, and so it makes sense that it would follow similar scoping rules.
I would argue that a stack of 'case's are a 'lambda' (and in particular a single 'case x: ...' is equivalent), and hence they should have function scoping like 'lambda' does.
'match' is a distraction; it's just a function call written backwards, i.e. `match x: f` is equivalent to `f(x)` (in this case Python happens to require `f` to be implemented using the stack-of-'case's form, but that seems incidental; similar to how 'raise' and 'except' are a general form of control flow, despite Python requiring values to be wrapped in something descended from 'Exception')
I would argue that a stack of 'case's are a 'lambda' (and in particular a single 'case x: ...' is equivalent), and hence they should have function scoping like 'lambda' does.
'match' is a distraction; it's just a function call written backwards, i.e. `match x: f` is equivalent to `f(x)` (in this case Python happens to require `f` to be implemented using the stack-of-'case's form, but that seems incidental; similar to how 'raise' and 'except' are a general form of control flow, despite Python requiring values to be wrapped in something descended from 'Exception')
> I think "for example" makes this sound more general than it is. List comprehensions are one of the rare few constructs that introduce a new scope.
The rarity of the construct doesn't matter honestly. Are you saying that list comprehensions should _not_ have introduced a new scope because it was unusual?
> The match block is more syntactically similar to if/while/etc. blocks, and so it makes sense that it would follow similar scoping rules.
Syntactically yes, but semantically it is very different and the semantics must be taken into account as well. The example given at the top of this thread (case 404: and case variable:) is enough to convince me that having variable scoping is a brain-dead obvious requirement.
> I think the complete situation is messy, but it's not the variable scoping rules' fault.
I agree with that statement. I think that improving the design/semantics would be more effective than just adding some more scoping rules in. In fact, I don't think this PEP should have been accepted in this current form at all. But given the current design, block-level scoping is appropriate. Given another design like maybe those you mention might not require that, but I think focusing on the fact that python doesn't have block-level scoping makes no sense. The python language before this PEP is not the same as the one after this PEP. The new one should have block-level scoping in this instance.
The rarity of the construct doesn't matter honestly. Are you saying that list comprehensions should _not_ have introduced a new scope because it was unusual?
> The match block is more syntactically similar to if/while/etc. blocks, and so it makes sense that it would follow similar scoping rules.
Syntactically yes, but semantically it is very different and the semantics must be taken into account as well. The example given at the top of this thread (case 404: and case variable:) is enough to convince me that having variable scoping is a brain-dead obvious requirement.
> I think the complete situation is messy, but it's not the variable scoping rules' fault.
I agree with that statement. I think that improving the design/semantics would be more effective than just adding some more scoping rules in. In fact, I don't think this PEP should have been accepted in this current form at all. But given the current design, block-level scoping is appropriate. Given another design like maybe those you mention might not require that, but I think focusing on the fact that python doesn't have block-level scoping makes no sense. The python language before this PEP is not the same as the one after this PEP. The new one should have block-level scoping in this instance.
> I think they should have just gone the whole way and _not_ mutated outside variables. There seems to be little consistent with this addition to the language.
Interested in block-level scoping in Python? Please post on the python-ideas mailing list. Thanks.
Interested in block-level scoping in Python? Please post on the python-ideas mailing list. Thanks.
[deleted]
In those other languages, if `not_found` is equal to 404 before the match block, how do you write a clause that only matches if `status` is 404?
Do you have to use the integer literal 404 instead of the named integer `not_found`?
Do you have to use the integer literal 404 instead of the named integer `not_found`?
Elixir uses the “pin operator”
“Use the pin operator ^ when you want to pattern match against a variable’s existing value rather than rebinding the variable.”
“Use the pin operator ^ when you want to pattern match against a variable’s existing value rather than rebinding the variable.”
iex> not_found = 404
404
iex> ^not_found = 200
** (MatchError) no match of right hand side value: 200
Exactly, Elixir is usually the only example given. So, how it makes sense to blame Python here is unclear. There's a proposal to add similar "sigils" to Python pattern matching, it just didn't make it yet.
Depends on the language, but a lot of them use guard clauses (so case ... if cond =>).
[deleted]
Pretty much the same as in the PEP: https://www.python.org/dev/peps/pep-0636/#adding-conditions-...
In OCaml and F# you can use `when`:
match status with
| n when n = not_found -> ...I don't get it, isn't that exactly what it's doing here?
It's mutating (not shadowing) `not_found` with the value of `status`. That can cause trouble if you rely on `not_found` keeping the initial value later somewhere. Which you would, e.g. with the global constant of `NOT_FOUND`.
Honestly I think the issue is so troublesome only if there's a single case to match, though. With more expected cases it should cause pretty obvious bugs (easy to catch).
Honestly I think the issue is so troublesome only if there's a single case to match, though. With more expected cases it should cause pretty obvious bugs (easy to catch).
[deleted]
Reassigning not mutating
Reassigning to a variable is how you mutate a variable, isn't it?
The value being [im]mutable is a different topic entirely.
The value being [im]mutable is a different topic entirely.
There aren't actually "variables" in Python in the sense of named values, instead there are namespaces where values are stored and looked up under string identifier keys. (Just to add spice, some values, namely function and class objects, do get their names embedded in them, but this is only used for debugging. They are stored in enclosing namespace dicts like other values.):
>>> def f(): pass
>>> g = f
>>> g
<function f at 0x000001B4A686E0D0>
>>> locals()
{'__annotations__': {},
'__builtins__': <module 'builtins' (built-in)>,
'__doc__': None,
'__loader__': <class '_frozen_importlib.BuiltinImporter'>,
'__name__': '__main__',
'__package__': None,
'__spec__': None,
'f': <function f at 0x000001B4A686E0D0>,
'g': <function f at 0x000001B4A686E0D0>}Wait, what's your definition of variable?
Because what you're describing fits perfectly into what I would call a variable. There is a mapping from an identifier to a slot that can store a value (or reference), and that mapping is stored in a specific scope. I would call that mapping a variable.
I'm not sure exactly why you mentioned objects that have names embedded in them. Is that relevant to the definition you're using?
Because what you're describing fits perfectly into what I would call a variable. There is a mapping from an identifier to a slot that can store a value (or reference), and that mapping is stored in a specific scope. I would call that mapping a variable.
I'm not sure exactly why you mentioned objects that have names embedded in them. Is that relevant to the definition you're using?
> Wait, what's your definition of variable?
With full formality, the definition of "variable" depends on context. In assembly language and C variables are names for storage locations in RAM, in Python they are name bindings in a dict data structure.
One distinction we could make is whether the names are compiled away or kept and used at runtime.
In any event, the important thing is to keep clear in one's mind the semantics of the language one is using. In Python you have values (objects of various types: ints, strings, tuples, lists, functions, classes, instances of classes, types, etc.) some of which are mutable and others are immutable, and you have namespaces: dicts that map strings to values. These namespaces have nothing to do with the location of the values in RAM.
So it doesn't really make sense in Python to speak of "mutate a variable", you can mutate (some) values, and rebind values to names (new or old).
> I'm not sure exactly why you mentioned objects that have names embedded in them. Is that relevant to the definition you're using?
Not really, it's just another little point that sometimes confuses some people when they are coming to grips with the idea that in Python values are not associated with names except by namespace bindings. There are some values (function and class objects) that do get associated with names.
With full formality, the definition of "variable" depends on context. In assembly language and C variables are names for storage locations in RAM, in Python they are name bindings in a dict data structure.
One distinction we could make is whether the names are compiled away or kept and used at runtime.
In any event, the important thing is to keep clear in one's mind the semantics of the language one is using. In Python you have values (objects of various types: ints, strings, tuples, lists, functions, classes, instances of classes, types, etc.) some of which are mutable and others are immutable, and you have namespaces: dicts that map strings to values. These namespaces have nothing to do with the location of the values in RAM.
So it doesn't really make sense in Python to speak of "mutate a variable", you can mutate (some) values, and rebind values to names (new or old).
> I'm not sure exactly why you mentioned objects that have names embedded in them. Is that relevant to the definition you're using?
Not really, it's just another little point that sometimes confuses some people when they are coming to grips with the idea that in Python values are not associated with names except by namespace bindings. There are some values (function and class objects) that do get associated with names.
I suppose, but it's very strange to say "Assigning to a variable is classified as mutating the variable in some languages but not others, even though the underlying mechanism works exactly the same way."
Shouldn't terms like this be cross-language?
Shouldn't terms like this be cross-language?
The underlying mechanism doesn't work the same though, e.g. in C assigning to a variable "mutates" the contents of RAM, in Python it mutates the namespace dictionary (which of course also results in some RAM contents changing too, but through a relatively elaborate abstraction.)
> Shouldn't terms like this be cross-language?
Variables in C are different beasties than variables in Python, and variables in Prolog are different from both, and none of those are like variables in math. It's a source of confusion that we use the word "variable" for a host of similar concepts. The word "type" is similarly overloaded in CS. (See https://www.cs.kent.ac.uk/people/staff/srk21/research/papers... )
FWIW, I'm just explaining what wendyshu was on about above. :)
> Shouldn't terms like this be cross-language?
Variables in C are different beasties than variables in Python, and variables in Prolog are different from both, and none of those are like variables in math. It's a source of confusion that we use the word "variable" for a host of similar concepts. The word "type" is similarly overloaded in CS. (See https://www.cs.kent.ac.uk/people/staff/srk21/research/papers... )
FWIW, I'm just explaining what wendyshu was on about above. :)
[deleted]
[deleted]
Reading the discussion, I think the criticism is more motivated by the subtle but crucial differences to most functional programming languages.
I was thinking more about the common criticism that something like `case Point2d(x, y):` "looks like an instantiation" and hence an equality check.
I actually replied to the wrong comment after reading several that visually looked similar at the time, so apologise for causing confusion in this subthread.
I actually replied to the wrong comment after reading several that visually looked similar at the time, so apologise for causing confusion in this subthread.
[deleted]
> EDIT: The lack of block scoped variable thing does seem like a wart right enough.
This is not specific to "match" statement, but is a general issue in Python. And thus, needs to have a general solution, orthogonal to pattern matching. Are you interested? Please post to the python-ideas mailing list.
This is not specific to "match" statement, but is a general issue in Python. And thus, needs to have a general solution, orthogonal to pattern matching. Are you interested? Please post to the python-ideas mailing list.
[deleted]
Oh, yikes. I understand what's happening here but this is going to bite a lot of people.
I might be misreading the grammar but it looks like you can produce the desired effect by using an attribute, e.g. the following would perform an equality check, not an assignment:
I might be misreading the grammar but it looks like you can produce the desired effect by using an attribute, e.g. the following would perform an equality check, not an assignment:
match status:
case requests.codes.not_found:
return "Not found"
The inconsistency just makes it worse and will lead to even more bugs. I foresee lint rules prohibiting match statements in the near future.
Couldn't they have used "case" and "capture" clauses instead?
match status:
case not_found:
return "Not found"
capture success_code:
return "Returned success code: %s" % success_code
A separate statement doesn't compose with complex patterns. I'd prefer always requiring the walrus operator to capture:
match response:
case (not_found, msg := _):
return f"Not found: {msg}"
case (error, "No swizzle available"):
return "We lack swizzle. :-("
case pair := (_, "Swizzle"):
log(f"Unexpected swizzle: {pair}")
return f"Swizzle?"
case ((code := _), _):
return f"Success: {code}"Yes, apparently your example will work correctly.
Now think what will happen if you need to move that not_found variable to the same file as that code (so it will no longer be a dotted name). If you do it manually you need to be extra careful, if you use an automatic tool either it will reject the change or will need to create a dummy class or something in the process.
Now think what will happen if you need to move that not_found variable to the same file as that code (so it will no longer be a dotted name). If you do it manually you need to be extra careful, if you use an automatic tool either it will reject the change or will need to create a dummy class or something in the process.
[deleted]
That's not too bad if it would be a syntax error to either set or shadow an existing variable with the match statement. Apparently it isn't, which is concerning. Personally I think I may have preferred something like:
You know, a lot of potentially confusing behavior would be avoided if programming languages had the sense to make variables read-only by default and disallow variable shadowing altogether.
match status:
case == not_found: # Check for equality
...
match status:
case as not_found: # bind status to variable not_found
...
You know, a lot of potentially confusing behavior would be avoided if programming languages had the sense to make variables read-only by default and disallow variable shadowing altogether.
I really like shadowing, since it prevents me making mistakes all over the place by referring to the wrong thing. If I introduce a new name, I have two names cluttering up my namespace, and might pick the wrong one by mistake; for example if I validate 'myInput' to get 'myValidatedInput', later on I can still refer to 'myInput', which would be a mistake, and may end up bypassing the validation. On the other hand, I can shadow 'myInput' with the validated result, meaning that (a) I can no longer refer to the value I no longer want, (b) there's only one suitable "input" in scope, so it's easier to do things correctly, (c) I don't have to juggle multiple names and (d) it's pure and immutable, and hence easier to reason about than statements (like 'del(myInput)' or 'myInput = validate(myInput)'.
>I really like shadowing, since it prevents me making mistakes all over the place by referring to the wrong thing. If I introduce a new name, I have two names cluttering up my namespace, and might pick the wrong one by mistake;
Compared to having two versions of the same name, one shadowing another?
Compared to having two versions of the same name, one shadowing another?
Yes. For example:
1) It declares that the lambda doesn't depend on the argument of 'neighbourhood'
2) It prevents us referring to that argument by mistake
Compare it to a non-shadowing version:
This version doesn't make any declaration like (1), so the computer can't help me find or fix the problem; it's a perfectly valid program. A static type checker like mypy wouldn't help me either, since 'position' and 'neighbour' are presumably both the same type.
It's not even clear to a human that there's anything wrong with this code. The problem would only arise during testing (we hope!), and the logic error would have to be manually narrowed-down to this function. Even if we correctly diagnose that the 'if' is returning a different variable than it was checking, the fix is still ambiguous. We could do this:
def neighbourhood(position):
return map(
lambda position: EDGE if position is None else position.absolute,
position.neighbours
)
1) It declares that the lambda doesn't depend on the argument of 'neighbourhood'
2) It prevents us referring to that argument by mistake
Compare it to a non-shadowing version:
def neighbourhood(position):
return map(
lambda neighbour: EDGE if neighbour is None else position.absolute,
position.neighbours
)
This version doesn't make any declaration like (1), so the computer can't help me find or fix the problem; it's a perfectly valid program. A static type checker like mypy wouldn't help me either, since 'position' and 'neighbour' are presumably both the same type.
It's not even clear to a human that there's anything wrong with this code. The problem would only arise during testing (we hope!), and the logic error would have to be manually narrowed-down to this function. Even if we correctly diagnose that the 'if' is returning a different variable than it was checking, the fix is still ambiguous. We could do this:
EDGE if position is None else position.absolute
EDGE if neighbour is None else neighbour.absolute
> Oops, I've accidentally written 'position.absolute' instead of 'neighbour.absolute'!
I'm going to be honest here, the number of times I've made that kind of mistake is absolutely dwarfed by the number of times I have used the wrong variable because I had accidentally shadowed it.
Neither mistake is super common, but I can't recall ever writing 'position.absolute' instead of 'neighbour.absolute' unless I legitimately needed both position and neighbour in scope and the problem was hard to reason about. I can recall accidentally reusing a variable like 'x' as an iteration variable and then using the wrong 'x' because I forgot, and I can also recall misunderstanding what some piece of code did because I thought 'x' was referring to the outer scope but I had missed that it was shadowed by another declaration. Shadowing has caused me many more problems than it solved, at least in my own experience.
I'm going to be honest here, the number of times I've made that kind of mistake is absolutely dwarfed by the number of times I have used the wrong variable because I had accidentally shadowed it.
Neither mistake is super common, but I can't recall ever writing 'position.absolute' instead of 'neighbour.absolute' unless I legitimately needed both position and neighbour in scope and the problem was hard to reason about. I can recall accidentally reusing a variable like 'x' as an iteration variable and then using the wrong 'x' because I forgot, and I can also recall misunderstanding what some piece of code did because I thought 'x' was referring to the outer scope but I had missed that it was shadowed by another declaration. Shadowing has caused me many more problems than it solved, at least in my own experience.
>Oops, I've accidentally written 'position.absolute' instead of 'neighbour.absolute'!
That's a contrived example though, if I ever saw one.
I don't think that's the kind of issue people commonly have, compared to misuse of shadowed variable further down the scope.
And for your example, a better solution would be for the close to declare what it wants to use from its environment. Python doesn't allow this syntax, but some languages do:
def neighbourhood(position): return map( lambda neighbour use (): EDGE if neighbour is None else position.absolute, position.neighbours )
Now the compiler can again warn you, since you're only allowed to use neighbour in the lambda.
That's a contrived example though, if I ever saw one.
I don't think that's the kind of issue people commonly have, compared to misuse of shadowed variable further down the scope.
And for your example, a better solution would be for the close to declare what it wants to use from its environment. Python doesn't allow this syntax, but some languages do:
def neighbourhood(position): return map( lambda neighbour use (): EDGE if neighbour is None else position.absolute, position.neighbours )
Now the compiler can again warn you, since you're only allowed to use neighbour in the lambda.
I've been scanning the docs and this syntax is hard for me to understand, however your version makes sense right on first glance. I'd go with that.
[deleted]
That's how structural pattern matching works in ML-like languages, where it's been in use for 40+ years.
Don't those languages typically have immutable variables so you would not be able to rebind a constant by accident?
This is usually orthogonal to mutability.
In other languages that support match, whether functional or not, you are not changing the value of the variable, but you are shadowing/rebinding the identifier inside the scope of the match clause.
In other languages that support match, whether functional or not, you are not changing the value of the variable, but you are shadowing/rebinding the identifier inside the scope of the match clause.
Python `match` statements do not introduce a new scope.
The second example does indeed change the value of `not_found`.
The second example does indeed change the value of `not_found`.
What's the rationale for not introducing a new scope?
The only clause I can find in the PEP is
> A capture pattern always succeeds. It binds the subject value to the name using the scoping rules for name binding established for named expressions in PEP 572. (Summary: the name becomes a local variable in the closest containing function scope unless there's an applicable nonlocal or global statement.)
This seems... incredibly bad.
The only clause I can find in the PEP is
> A capture pattern always succeeds. It binds the subject value to the name using the scoping rules for name binding established for named expressions in PEP 572. (Summary: the name becomes a local variable in the closest containing function scope unless there's an applicable nonlocal or global statement.)
This seems... incredibly bad.
It's consistent with how scoping works in loops and `with` clauses. Agreed that it's more problematic here, though.
That works the same as the rest of the language, doesn't it?
(That is, for, if, while, etc. also don't introduce new scopes.)
(That is, for, if, while, etc. also don't introduce new scopes.)
"Introducing a new scope" is not a concept that exists in Python, you only have function scope and module scope.
Having only function, module, class, generator, etc. scope in Python before this PEP might have made sense, but they really should have added pattern matching scope to keep things sane here.
There are some exceptions to this - for example, the iteration variable(s) in a list comprehension are only in scope within the comprehension.
Worth noting that this was _fixed_ in Python 3.
In Python 2.7 you get this:
In Python 2.7 you get this:
>>> my_list = [x for x in 'hello']
>>> x
'o'Thanks, I didn't realize that. That makes their choice really baffling.
The explanation is that (in Python 3) list comprehensions are really just syntactic sugar for generator expressions. In Python, "scope" is really a synonym for "dictionary attached to some object", so generators can have local variables (since they have their own internal scope), but purely syntactic constructs cannot.
and class scope
So you can easily create a variable in a match case
Oh dear, that is indeed very bad.
That's a very problematic design decision, imo...
[deleted]
Doesn’t other languages have block scope and shadowing is not an issue, but in Python it will reassign instead of shadowing?
In my mind that is not well thought out.
In my mind that is not well thought out.
Interested in block-level scoping in Python? Please post on the python-ideas mailing list. Thanks.
[deleted]
In Rust no
Apparently it's reassigning the existing not_found variable from the outer scope, not binding a new not_found variable in an inner scope.
[deleted]
This is how pattern matching works in any language (except for some niche languages that allow nonlinear patterns, like Curry).
Many of the comments here reveal a bizarre parochialism.
Many of the comments here reveal a bizarre parochialism.
Not the scoping rule. For example, in Haskell:
According to the Python semantics we would get `(bar, bar)`, since we only have one `x` variable. When the case pattern succeeds, the existing `x` is updated to refer to `bar`. Hence we get the `bar` we expect in the first element, but we also get `bar` as the second element, since that `x` variable was updated. (Note that Python guarantees that tuple elements are evaluated from left to right).
let x = foo
in (
case bar of
x -> x,
x
)
According to the Python semantics we would get `(bar, bar)`, since we only have one `x` variable. When the case pattern succeeds, the existing `x` is updated to refer to `bar`. Hence we get the `bar` we expect in the first element, but we also get `bar` as the second element, since that `x` variable was updated. (Note that Python guarantees that tuple elements are evaluated from left to right).
That's just an inevitable consequence of the fact that, in Python, "scope" is synonymous with "dictionary attached to some object." This is already how for-loops work.
It's not "inevitable". They could have required local variables in case statements to be local to that case statement. It would have required changes to the "scope is synonymous with dictionary attached to some object" idea or maybe it would have required a dictionary to be attached to a case statement. I personally think local scope should have been viewed as a hard requirement if they were to introduce this to the language.
Interested in block-level scoping in Python? Please post on the python-ideas mailing list. Thanks.
I'm interested in sane semantics. In this case, that calls for block-level scoping. Those who introduced pattern matching should have understood that the lack of block-level scoping _before_ this PEP does in no way support the continuing of the status quo. The language after this PEP has changed and has turned into one where block-level scoping is appropriate in this case.
I'm honestly _not_ interested in block-level scoping in this case because I would _never_ have wanted this PEP to be accepted. This feature was quite controversial on the various python mailing lists, and yet the steering committee accepted it anyway. The steering committee might consider leading with a bit more humility and _not_ accepting such controversial PEPs. This is an example of language devolution and not evolution.
I'm honestly _not_ interested in block-level scoping in this case because I would _never_ have wanted this PEP to be accepted. This feature was quite controversial on the various python mailing lists, and yet the steering committee accepted it anyway. The steering committee might consider leading with a bit more humility and _not_ accepting such controversial PEPs. This is an example of language devolution and not evolution.
It occurs to me that there's a nice way to understand this from what's happened in Scala.
Scala has always had built-in syntax for pattern-matching, like:
For example we can write:
When it comes to Python, it doesn't even need to be a discussion about block scope; it's equally valid to think of this as function scope (like Python already supports), where `case` acts like `lambda`, except we can define a single function as a combination of multiple `case`s (like in the Scala above).
Scala has always had built-in syntax for pattern-matching, like:
foo match {
case bar => ...
case baz => ...
}
For example we can write:
val f: PartialFunction[A, B] = {
case bar => ...
case baz => ...
}
f(foo)
val f: PartialFunction[A, B] = {
case bar => ...
case baz => ...
}
foo.pipe(f)
foo match {
case bar => ...
case baz => ...
}
foo.pipe({
case bar => ...
case baz => ...
})
When it comes to Python, it doesn't even need to be a discussion about block scope; it's equally valid to think of this as function scope (like Python already supports), where `case` acts like `lambda`, except we can define a single function as a combination of multiple `case`s (like in the Scala above).
As said many times already, then you have the opposite problem - how to get value from "inner" to "outer" scope. If we talk about function scope, then it requires "nonlocal" declaration in the inner scope. From Python, too many declaration like that are syntactic litter. It has a scoping discipline which allows to avoid them in most cases, and that works great in 90% of cases (popularity of Python and amount of code written in it is there proof).
Yes, there're still remaining 10%, and pattern matching kinda drew attention to those 10%. I'm interested to address those, and invite other interested parties to discuss/work together on that. The meeting place is python-ideas mailing list.
Yes, there're still remaining 10%, and pattern matching kinda drew attention to those 10%. I'm interested to address those, and invite other interested parties to discuss/work together on that. The meeting place is python-ideas mailing list.
> If we talk about function scope
Note that I'm not simply saying 'match should have function scope', I'm saying that 'case' is literally a function definition. Hence functions defined using the 'case' keywork should work the same as functions defined using other keywords ('def', 'lambda' or 'class').
> you have the opposite problem - how to get value from "inner" to "outer" scope
The same way as if we defined the function using 'lambda' or 'def' or 'class'
> it requires "nonlocal" declaration in the inner scope
That's not a general solution, since it doesn't work in 'lambda'; although this exposes the existing problem that there is already a difference between functions defined using 'def'/'class' and functions defined using 'lambda'. Adding yet another way to define functions ('case') which defines functions that act in yet another different way just makes that worse.
Note that I'm not simply saying 'match should have function scope', I'm saying that 'case' is literally a function definition. Hence functions defined using the 'case' keywork should work the same as functions defined using other keywords ('def', 'lambda' or 'class').
> you have the opposite problem - how to get value from "inner" to "outer" scope
The same way as if we defined the function using 'lambda' or 'def' or 'class'
> it requires "nonlocal" declaration in the inner scope
That's not a general solution, since it doesn't work in 'lambda'; although this exposes the existing problem that there is already a difference between functions defined using 'def'/'class' and functions defined using 'lambda'. Adding yet another way to define functions ('case') which defines functions that act in yet another different way just makes that worse.
> I'm saying that 'case' is literally a function definition
And I don't agree with saying it like that. I would agree with "a 'case' could be seen as a function definition". In other words, that's just one possible way to look at it, among others.
Note that from PoV of the functional programming, everything is a function. And block scope is actually recursively lexical lambda.
And OTOH function inlining is a baseline program transformation. Currently in Python, whether a syntactic element (not explicitly a function) gets implemented as a function is an implementation detail. For example, comprehension happen to be implemented as functions. But just as well they could be inlined.
Note that function calls are generally expensive, and even more so in Python. Thus, any optimizing Python implementation would inline whenever it makes sense (called once is obviously such a case). (CPython hardly can be called an optimizing impl, though since 3.8, there's noticeable work on that).
And I don't agree with saying it like that. I would agree with "a 'case' could be seen as a function definition". In other words, that's just one possible way to look at it, among others.
Note that from PoV of the functional programming, everything is a function. And block scope is actually recursively lexical lambda.
And OTOH function inlining is a baseline program transformation. Currently in Python, whether a syntactic element (not explicitly a function) gets implemented as a function is an implementation detail. For example, comprehension happen to be implemented as functions. But just as well they could be inlined.
Note that function calls are generally expensive, and even more so in Python. Thus, any optimizing Python implementation would inline whenever it makes sense (called once is obviously such a case). (CPython hardly can be called an optimizing impl, though since 3.8, there's noticeable work on that).
I mentioned that in other comments, and can repeat again, there were 2 choices: a) add initial pattern matching to reference Python implementation; b) throw all the work into /dev/null and get back to dark ages where pattern matching is implemented in hacky ways by disparate libs and macros. Common sense won, and a) was chosen. Pattern matching will be definitely elaborated further.
> This feature was quite controversial on the various python mailing lists
I'm also on various Python lists, and what I saw that various details were controversial, not pattern matching itself. Mostly, people wanted pattern matching to be better right from the start, just like many people here. Well, I also want Linux version 234536464576.3.1-final-forever, but instead run 5.4.0 currently, and install new versions from time to time. The same is essentially with Python too.
> This feature was quite controversial on the various python mailing lists
I'm also on various Python lists, and what I saw that various details were controversial, not pattern matching itself. Mostly, people wanted pattern matching to be better right from the start, just like many people here. Well, I also want Linux version 234536464576.3.1-final-forever, but instead run 5.4.0 currently, and install new versions from time to time. The same is essentially with Python too.
> throw all the work into /dev/null and get back to dark ages where pattern matching is implemented in hacky ways by disparate libs and macros.
How does not accepting this PEP throw anything away? It's literally right there. It's still hosted there on the PEP site. Those who want pattern matching can continue to refine the work. "Common sense" requires understanding the current work is a sunk cost and in no way supports its introduction into the language.
> I'm also on various Python lists, and what I saw that various details were controversial, not pattern matching itself.
The details of the PEP are the problem, not the idea. Not accepting this PEP is not the same as rejecting pattern matching. This is only one possible implementation of pattern matching. It's also a bad one and one that makes the language worse. Rejecting this PEP allows a better implementation in the future.
How does not accepting this PEP throw anything away? It's literally right there. It's still hosted there on the PEP site. Those who want pattern matching can continue to refine the work. "Common sense" requires understanding the current work is a sunk cost and in no way supports its introduction into the language.
> I'm also on various Python lists, and what I saw that various details were controversial, not pattern matching itself.
The details of the PEP are the problem, not the idea. Not accepting this PEP is not the same as rejecting pattern matching. This is only one possible implementation of pattern matching. It's also a bad one and one that makes the language worse. Rejecting this PEP allows a better implementation in the future.
> It's still hosted there on the PEP site.
On the PEP site, https://www.python.org/dev/peps/ , there're a lot of deadlocked PEPs, some of them a good and better would have been within, than without.
> Rejecting this PEP allows a better implementation in the future.
Let's count - 3rd-party patmatching libs for Python exists for 10-15 years. And only now some of those people who did their work as "third parties" came to do it inside mainstream Python.
The "future" you talk about is on the order of a decade. (Decade(s) is for example a timespan between 1st attempts to add string interpolation and f-strings landing).
I myself was ardent critic of PEP622/PEP634. I find situation with requiring "case Cls.CONST:" to match against constants to be unacceptable. But I'm pragmatic guy, and had to agree that it can be resolved later. The core pattern matching support added isn't bad at all. Could have been better. Best is the enemy of good.
On the PEP site, https://www.python.org/dev/peps/ , there're a lot of deadlocked PEPs, some of them a good and better would have been within, than without.
> Rejecting this PEP allows a better implementation in the future.
Let's count - 3rd-party patmatching libs for Python exists for 10-15 years. And only now some of those people who did their work as "third parties" came to do it inside mainstream Python.
The "future" you talk about is on the order of a decade. (Decade(s) is for example a timespan between 1st attempts to add string interpolation and f-strings landing).
I myself was ardent critic of PEP622/PEP634. I find situation with requiring "case Cls.CONST:" to match against constants to be unacceptable. But I'm pragmatic guy, and had to agree that it can be resolved later. The core pattern matching support added isn't bad at all. Could have been better. Best is the enemy of good.
> On the PEP site, https://www.python.org/dev/peps/ , there're a lot of deadlocked PEPs, some of them a good and better would have been within, than without.
If it's deadlocked, it really _shouldn't_ be added.
> Let's count - 3rd-party patmatching libs for Python exists for 10-15 years. And only now some of those people who did their work as "third parties" came to do it inside mainstream Python.
What's wrong with multiple implementations? Maybe people want different things? Besides the implementations' existence shows that lack of language support isn't something that blocks the use of pattern matching. Also moving it into the language doesn't mean people will work on that one implementation. Haven't you heard that packages go to the standard library to die? Why would it be any different in the python language. Besides I'm sure that the 3rd party libs will continue to be used anyway.
> But I'm pragmatic guy, and had to agree that it can be resolved later. The core pattern matching support added isn't bad at all. Could have been better. Best is the enemy of good.
I'm pragmatic too. I understand that I can do everything that this PEP introduces without the change to the language. I also understand that this PEP could continue to be worked on and improved. It's true that best is the enemy of good. I (and obviously many others here) believe that this is _bad_.
If it's deadlocked, it really _shouldn't_ be added.
> Let's count - 3rd-party patmatching libs for Python exists for 10-15 years. And only now some of those people who did their work as "third parties" came to do it inside mainstream Python.
What's wrong with multiple implementations? Maybe people want different things? Besides the implementations' existence shows that lack of language support isn't something that blocks the use of pattern matching. Also moving it into the language doesn't mean people will work on that one implementation. Haven't you heard that packages go to the standard library to die? Why would it be any different in the python language. Besides I'm sure that the 3rd party libs will continue to be used anyway.
> But I'm pragmatic guy, and had to agree that it can be resolved later. The core pattern matching support added isn't bad at all. Could have been better. Best is the enemy of good.
I'm pragmatic too. I understand that I can do everything that this PEP introduces without the change to the language. I also understand that this PEP could continue to be worked on and improved. It's true that best is the enemy of good. I (and obviously many others here) believe that this is _bad_.
> What's wrong with multiple implementations?
It's absolutely great, and I'm saying that as someone working 5+ years on an alternative Python dialect (exactly with a motto of "down with toxic lead-acid batteries").
> Also moving it into the language doesn't mean people will work on that one implementation.
Only on that one - god forbid. But gather around that particular implementation to make it better and polish rough edges - for sure. (While the rest of impls will remain niche projects unfortunately.)
> I (and obviously many others here) believe that this is _bad_.
And me, and many others, believe it's good ;).
It's absolutely great, and I'm saying that as someone working 5+ years on an alternative Python dialect (exactly with a motto of "down with toxic lead-acid batteries").
> Also moving it into the language doesn't mean people will work on that one implementation.
Only on that one - god forbid. But gather around that particular implementation to make it better and polish rough edges - for sure. (While the rest of impls will remain niche projects unfortunately.)
> I (and obviously many others here) believe that this is _bad_.
And me, and many others, believe it's good ;).
> Let's count - 3rd-party patmatching libs for Python exists for 10-15 years. And only now some of those people who did their work as "third parties" came to do it inside mainstream Python.
Well, somewhat tongue-in-cheek, why not introduce a macro system into Python which allows to experimentally implement such syntactic changes as a library?
Well, somewhat tongue-in-cheek, why not introduce a macro system into Python which allows to experimentally implement such syntactic changes as a library?
First of all, macro systems for Python exist for decades (just as long as pattern matching, and indeed, many patmatching implementations are done as macros). One well-know example of both is https://macropy3.readthedocs.io/en/latest/pattern.html
Secondly, there's a PEP to embrace macros in CPython (instead of pretending they don't exist, and leaving that to external libraries): https://www.python.org/dev/peps/pep-0638/
But the point, you don't need to wait for official PEP to use macros in Python. If you wanted, you could do that yesterday (== decades ago). And I guess in absolute numbers, the same amount of people use macros in Python as in Scheme. It's just in relative figures, it's quite different, given that there're millions of Python users.
Secondly, there's a PEP to embrace macros in CPython (instead of pretending they don't exist, and leaving that to external libraries): https://www.python.org/dev/peps/pep-0638/
But the point, you don't need to wait for official PEP to use macros in Python. If you wanted, you could do that yesterday (== decades ago). And I guess in absolute numbers, the same amount of people use macros in Python as in Scheme. It's just in relative figures, it's quite different, given that there're millions of Python users.
> Well, I also want Linux version 234536464576.3.1-final-forever, but instead run 5.4.0 currently, and install new versions from time to time. The same is essentially with Python too.
Just one thing, if mainline Linux would work like Python in respect to stability of APIs and features, you could start to debug and re-write your system after minor kernel upgrades. Linux does not break APIs, and this is possible because people are very careful what they implement - they will need to support it for an indefinite future.
Of course you can make patched branches and experimental releases of the kernel, these exist, but few people will use them, for good reasons.
Just one thing, if mainline Linux would work like Python in respect to stability of APIs and features, you could start to debug and re-write your system after minor kernel upgrades. Linux does not break APIs, and this is possible because people are very careful what they implement - they will need to support it for an indefinite future.
Of course you can make patched branches and experimental releases of the kernel, these exist, but few people will use them, for good reasons.
The Linux kernel has a document imaginatively called "stable API nonsense": https://www.kernel.org/doc/Documentation/process/stable-api-...
But the talk was not about that, it was about the fact that we want to get "finished software", but as soon as we ourselves deliver software, we vice-versa want to do it step by step, over long period of time. One day, we should get some reflection and self-awareness and understand that other programmers are exactly like ourselves - can't deliver everything at once.
But the talk was not about that, it was about the fact that we want to get "finished software", but as soon as we ourselves deliver software, we vice-versa want to do it step by step, over long period of time. One day, we should get some reflection and self-awareness and understand that other programmers are exactly like ourselves - can't deliver everything at once.
What you cite appears misleading to me - the text by Greg Kroah-Hartman talks very clearly about interfaces within the Linux kernel, not interfaces between kernel and user space, such as the syscall interface, which are stable. If you want to read the position of the lead Linux kernel developer on breaking user space APIs, here it is, in all caps:
https://linuxreviews.org/WE_DO_NOT_BREAK_USERSPACE
And here is the rationale why:
https://unix.stackexchange.com/questions/235335/why-is-there...
- you see that it makes perfect sense for a programming language like Python, too, to make only backwards-compatible changes (except perhaps if there are severe problems with a release).
In the same way, it does not matter how things are implemented within Python, but it matters a lot that the user interfaces, which includes in this case the syntax of the language, are stable.
And the fact that Python contrary to that does break backward compatibility - sometimes even in minor releases -, and continues to do so, is a reason that for my own projects I have come to the point at avoiding python for new stuff. There are other languages which are more stable and give the same flexibility, even at better runtime performance.
https://linuxreviews.org/WE_DO_NOT_BREAK_USERSPACE
And here is the rationale why:
https://unix.stackexchange.com/questions/235335/why-is-there...
- you see that it makes perfect sense for a programming language like Python, too, to make only backwards-compatible changes (except perhaps if there are severe problems with a release).
In the same way, it does not matter how things are implemented within Python, but it matters a lot that the user interfaces, which includes in this case the syntax of the language, are stable.
And the fact that Python contrary to that does break backward compatibility - sometimes even in minor releases -, and continues to do so, is a reason that for my own projects I have come to the point at avoiding python for new stuff. There are other languages which are more stable and give the same flexibility, even at better runtime performance.
> for my own projects I have come to the point at avoiding python for new stuff.
But who are you, do I know you? I know some guy who said that about Python and now develops his own Python-like language. Is that you? Because if you just consumer of existing languages, it's something different, there always will be a new shiny thingy around the corner to lure you.
> There are other languages which are more stable and give the same flexibility, even at better runtime performance.
Yes, but from bird's eye view, all languages are the same, and differences only emphasize similarities. So, in a contrarian move, I decided to stay with Python, and work on adding missing things to it. Because any language has missing things, and Python isn't bad base to start from at all.
But who are you, do I know you? I know some guy who said that about Python and now develops his own Python-like language. Is that you? Because if you just consumer of existing languages, it's something different, there always will be a new shiny thingy around the corner to lure you.
> There are other languages which are more stable and give the same flexibility, even at better runtime performance.
Yes, but from bird's eye view, all languages are the same, and differences only emphasize similarities. So, in a contrarian move, I decided to stay with Python, and work on adding missing things to it. Because any language has missing things, and Python isn't bad base to start from at all.
> you see that it makes perfect sense for a programming language like Python, too, to make only backwards-compatible changes
That's exactly what Python does of course (except when purposely otherwise, like 2->3 transition). And of course, that policy is implemented by humans, which are known to err.
That's exactly what Python does of course (except when purposely otherwise, like 2->3 transition). And of course, that policy is implemented by humans, which are known to err.
> That's just an inevitable consequence of the fact that, in Python, "scope" is synonymous with "dictionary attached to some object."
What object is the scope of a comprehension (in Py 3; in py 2 they don't have their own scope) a dict attached to? And, if you can answer that why could there not be such an object for a pattern match expression?
What object is the scope of a comprehension (in Py 3; in py 2 they don't have their own scope) a dict attached to? And, if you can answer that why could there not be such an object for a pattern match expression?
> What object is the scope of a comprehension (in Py 3; in py 2 they don't have their own scope) a dict attached to?
The generator object that gets created behind the scenes.
> And, if you can answer that why could there not be such an object for a pattern match expression?
There could be, I suppose, just as there could be for "if" or "for". If Python decided to have lexical scoping everywhere, I would be in favor of that (but then people would complain about breaking changes). In lieu of that, I like the consistency.
The generator object that gets created behind the scenes.
> And, if you can answer that why could there not be such an object for a pattern match expression?
There could be, I suppose, just as there could be for "if" or "for". If Python decided to have lexical scoping everywhere, I would be in favor of that (but then people would complain about breaking changes). In lieu of that, I like the consistency.
>The generator object that gets created behind the scenes.
So like a match object that could get crreated behind the scenes?
So like a match object that could get crreated behind the scenes?
If you have automatic block-level scoping, then you have the opposite problem - you need to do extra leg-work to communicate a value to the surrounding scope.
Anyway, anyone agrees that block-level scoping is useful. Interested in block-level scoping in Python? Please post on the python-ideas mailing list. Thanks.
Anyway, anyone agrees that block-level scoping is useful. Interested in block-level scoping in Python? Please post on the python-ideas mailing list. Thanks.
>If you have automatic block-level scoping, then you have the opposite problem - you need to do extra leg-work to communicate a value to the surrounding scope.
In the general case, you just declare the variable in the surrounding scope and then affect it in the lower one, no?
In the general case, you just declare the variable in the surrounding scope and then affect it in the lower one, no?
Right. But the whole idea of Python scoping rules was to not burden people with the need to declare variables (instead, if you define one, it's accessible everywhere in the function).
But yes, block-level scoping (as in C for example) would be useful too in addition to existing whole-function scoping discipline.
Again, I'm looking for similarly-minded people to move this idea forward. If interested, please find me on the python-ideas mailing list for discussing details.
But yes, block-level scoping (as in C for example) would be useful too in addition to existing whole-function scoping discipline.
Again, I'm looking for similarly-minded people to move this idea forward. If interested, please find me on the python-ideas mailing list for discussing details.
> then you have the opposite problem - you need to do extra leg-work to communicate a value to the surrounding scope.
that would be far less likely to break things in an unexpected way, as in "explicit is better than implicit".
I am also wondering whether what is really missing here is perhaps a kind of imperative switch/case statement which has the explicit purpose of changing function variables.
that would be far less likely to break things in an unexpected way, as in "explicit is better than implicit".
I am also wondering whether what is really missing here is perhaps a kind of imperative switch/case statement which has the explicit purpose of changing function variables.
List comprehensions were changed in Python 3 to avoid this, so I don't think it's quite that simple.
In Python 3, list comprehensions use generators behind the scenes.
Well, this "inevitable" consequence also infects pattern matching.
> This is how pattern matching works in any language
No, its not, but no language (at least that I am aware of) except python does pattern matching + local variables + not introducing a new scope with the pattern match.
Ruby is the closest, but it does introduce a new scope while providing a mechanism for binding variables in the containing local scope. (As well as a method to “pin” variables from the containing scope to use them in matches.)
No, its not, but no language (at least that I am aware of) except python does pattern matching + local variables + not introducing a new scope with the pattern match.
Ruby is the closest, but it does introduce a new scope while providing a mechanism for binding variables in the containing local scope. (As well as a method to “pin” variables from the containing scope to use them in matches.)
Not introducing a new scope with a match is unfortunate, but it's also consistent with how every other language feature interacts with scoping.
> (As well as a method to “pin” variables from the containing scope to use them in matches.)
This is a good idea, I agree -- at least for Python, where you would obviously just call __eq__.
EDIT: It looks like you actually can match against constants with this PEP, as long as you access your constant with a dot (e.g., HttpError.NotFound). This seems like a perfectly reasonable solution to me.
> (As well as a method to “pin” variables from the containing scope to use them in matches.)
This is a good idea, I agree -- at least for Python, where you would obviously just call __eq__.
EDIT: It looks like you actually can match against constants with this PEP, as long as you access your constant with a dot (e.g., HttpError.NotFound). This seems like a perfectly reasonable solution to me.
> Not introducing a new scope with a match is unfortunate, but it's also consistent with how every other language feature interacts with scoping.
Except comprehensions, which changed in Py 3 to have their own scope, rather than binding control variables in the surrounding (function or module) scope as in Py 2.
> It looks like you actually can match against constants with this PEP, as long as you access your constant with a dot (e.g., HttpError.NotFound). This seems like a perfectly reasonable solution to me.
It would be except:
* You can't access function-scoped identifiers that way.
* You can't access module-scoped identifiers in the main module that way.
* You can't conveniently reference identifiers in the current module that way. (I think you can use the full module path to qualify the name in the local module, but that's both awkward and brittle to refactoring, and there's pretty much never a reason to do it for any other purpose.)
Except comprehensions, which changed in Py 3 to have their own scope, rather than binding control variables in the surrounding (function or module) scope as in Py 2.
> It looks like you actually can match against constants with this PEP, as long as you access your constant with a dot (e.g., HttpError.NotFound). This seems like a perfectly reasonable solution to me.
It would be except:
* You can't access function-scoped identifiers that way.
* You can't access module-scoped identifiers in the main module that way.
* You can't conveniently reference identifiers in the current module that way. (I think you can use the full module path to qualify the name in the local module, but that's both awkward and brittle to refactoring, and there's pretty much never a reason to do it for any other purpose.)
Interested in block-level scoping in Python? Please post on the python-ideas mailing list. Thanks.
>Many of the comments here reveal a bizarre parochialism.
This seems like a bizarre misunderstanding of Python scoping rules, and how this can be a problem here.
This seems like a bizarre misunderstanding of Python scoping rules, and how this can be a problem here.
In that case, what’s the correct way to write a clause that only matches if `status` is equal to 404? Do we have to use the integer literal 404 instead of a named integer?
It will be to stick your constants in a module and use module.constant to match them. Or use an enum. Or hang them off a class.
Yes, of course. Again, this is how it works in basically every mainstream language with this feature.
Really? It’s best practice to use a magic number rather than a name?
That breaks one of the most fundamental rules for writing good code.
That breaks one of the most fundamental rules for writing good code.
If you really need to compare against a variable, use an "if". The primary benefit of pattern-matching is destructuring.
EDIT: It looks like you actually can match against constants with this PEP, as long as you access your constant with a dot (e.g., HttpError.NotFound). This seems like a perfectly reasonable solution to me.
EDIT: It looks like you actually can match against constants with this PEP, as long as you access your constant with a dot (e.g., HttpError.NotFound). This seems like a perfectly reasonable solution to me.
No it’s not because it’s none obvious and requires a fair amount of boilerplate code. Both of which are usually idioms Python normally tries to avoid.
I guarantee you this will trip up a lot of developers who are either learning the language for the first time or who Python isn’t their primary language.
Worse still, the kind of bugs this will lead to is valid code with unexpected pattern matching, which is a lot harder to debug than invalid code which gets kicked out with a compiler error.
I guarantee you this will trip up a lot of developers who are either learning the language for the first time or who Python isn’t their primary language.
Worse still, the kind of bugs this will lead to is valid code with unexpected pattern matching, which is a lot harder to debug than invalid code which gets kicked out with a compiler error.
There are PatternSynonyms in Haskell.
Are you sure the second one isn’t just declaring not_found as a stand-in for 404 so the case statement two lines below can refer to business logic rather than a “magic” constant?
I would NOT expect for the line “case not_found:” to reassign the status variable to 404 regardless of what it was before.
I can’t see how or why that would be intended behavior.
I would NOT expect for the line “case not_found:” to reassign the status variable to 404 regardless of what it was before.
I can’t see how or why that would be intended behavior.
Why is it performing an implicit assignment instead of an equality check?
A bit part of the appeal of pattern matching in other languages is support for destructuring, where you implicitly assign variables to members of a data structure you're performing a match on. For example, in ML-ish pseudocode:
len([]) = 0 len([first|rest]) = 1 + len(rest)
That's a trivial example. The second PEP (pattern matching tutorial) has several other examples:
https://www.python.org/dev/peps/pep-0636/#matching-multiple-...
So, if you use a variable in a pattern, it's an implicit assignment. If you use a literal, it's a value to be matched against.
I agree that the trivial case (a single variable with no conditions) may be confusing before you know what's going on, but I think the alternative, where a single variable is a comparison but multiple variables (or a structure) is an assignment isn't necessarily better.
len([]) = 0 len([first|rest]) = 1 + len(rest)
That's a trivial example. The second PEP (pattern matching tutorial) has several other examples:
https://www.python.org/dev/peps/pep-0636/#matching-multiple-...
So, if you use a variable in a pattern, it's an implicit assignment. If you use a literal, it's a value to be matched against.
I agree that the trivial case (a single variable with no conditions) may be confusing before you know what's going on, but I think the alternative, where a single variable is a comparison but multiple variables (or a structure) is an assignment isn't necessarily better.
> A bit part of the appeal of pattern matching in other languages is support for destructuring, where you implicitly assign variables to members of a data structure you're performing a match on
I would clarify that to implicitly introduce variables.
Consider the following:
This match syntax seems to be assigning, rather than introducing, which is a shame.
I would clarify that to implicitly introduce variables.
Consider the following:
$ python3
Python 3.8.6 (default, Dec 28 2020, 20:00:05)
[Clang 7.1.0 (tags/RELEASE_710/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> def foo(x):
... return ([x for x in [1, 2, 3]], x, (lambda x: x+1)(x), x)
...
>>> foo(42)
([1, 2, 3], 42, 43, 42)
This match syntax seems to be assigning, rather than introducing, which is a shame.
That looks pretty counter intuitive even after this explanation
[deleted]
[deleted]
I just wrote a comment with the same complain and similar example!
I also think the same as you wrote, the second check will be a hard to spot mistake
I also think the same as you wrote, the second check will be a hard to spot mistake
I think your example makes it clearer that it won't be a very subtle a bug to find, but rather completely broken behaviour where only the first case is ever triggered. That should be much simpler to test against. Granted, this breaks down if you're matching by the number of values, possibly other cases.
To be honest, I feel I like the (sort-of-) "forced namespacing" of matching constants this brings. It should be an easy habit to discipline too, the rule being very simple:
"if you don't want to use literals in your matching, you must plug them in an enumeration"
Too bad that enumeration can't be a PEP 435 enum without adding an ugly `.value` accessor to each case, though.
To be honest, I feel I like the (sort-of-) "forced namespacing" of matching constants this brings. It should be an easy habit to discipline too, the rule being very simple:
"if you don't want to use literals in your matching, you must plug them in an enumeration"
Too bad that enumeration can't be a PEP 435 enum without adding an ugly `.value` accessor to each case, though.
According to the specification pointed to by choeger in this comment https://news.ycombinator.com/item?id=26086589 , failed matches like that can also assign a value to existing variables, which seems even more problematic to me.
Thus, will be fixed sooner or later. (Sad it's not fixed right away, but neither me nor you proposed patches.)
Assignments going wrong is par for the course in python! I still am surprised it is taught to beginners...
[deleted]
[deleted]
That's nuts. Sorry, but ... that's nuts.
I will be waiting until pylint detects these sorts of footguns...
I just read the part about the name binding and its ... totally bonkers. Sorry, but I have no other words for this:
> The implementation may choose to either make persistent bindings for those partial matches or not. User code including a match statement should not rely on the bindings being made for a failed match, but also shouldn't assume that variables are unchanged by a failed match. This part of the behavior is left intentionally unspecified so different implementations can add optimizations, and to prevent introducing semantic restrictions that could limit the extensibility of this feature.
That's basically dynamic scoping instead of lexical scoping only inside the match blocks and only for pattern variables. Who in their right mind comes up with a design like that? That's the javascript route! I get it that proper binding would have been difficult but if language design is too difficult, you should stop and not go one like you don't care about the consequences! This decision will waste thousands of developer hours when searching bugs!
> The implementation may choose to either make persistent bindings for those partial matches or not. User code including a match statement should not rely on the bindings being made for a failed match, but also shouldn't assume that variables are unchanged by a failed match. This part of the behavior is left intentionally unspecified so different implementations can add optimizations, and to prevent introducing semantic restrictions that could limit the extensibility of this feature.
That's basically dynamic scoping instead of lexical scoping only inside the match blocks and only for pattern variables. Who in their right mind comes up with a design like that? That's the javascript route! I get it that proper binding would have been difficult but if language design is too difficult, you should stop and not go one like you don't care about the consequences! This decision will waste thousands of developer hours when searching bugs!
> This part of the behavior is left intentionally unspecified so different implementations can add optimizations, and to prevent introducing semantic restrictions that could limit the extensibility of this feature.
Isn't this more or less what was the argument to allow more undefined behaviour in ANSI C ?
(Edit: I think this also comes from the tension caused by integrating features from functional languages into an imperative language, like explained in more detail here: https://news.ycombinator.com/item?id=26086863 )
Isn't this more or less what was the argument to allow more undefined behaviour in ANSI C ?
(Edit: I think this also comes from the tension caused by integrating features from functional languages into an imperative language, like explained in more detail here: https://news.ycombinator.com/item?id=26086863 )
Unlike Python, C has the excuse of runtime efficiency as a plausible rationale.
Python is big time SLOW, there's a ton of untapped optimisation potential before wandering into UB for sake of speed territory.
Python is big time SLOW, there's a ton of untapped optimisation potential before wandering into UB for sake of speed territory.
They went for "UB" exactly because they understand the current behavior is not ideal, and to allow for better implementation and to change it later.
No worries, this was debated fiercely. In the end, we had to choose between "Python has a reference syntax for pattern matching" vs "Python continues to have disparate 3rd-party extensions for pattern matching, without clear leader".
The essence of the issue you quote is that Python needs block-level scope for variables (in addition to the standard function-level scope). Similar to what JavaScript acquired with "let" and "const" keywords. That's more pervasive issue than just "match" statement of pattern matching, and thus better handled orthogonally to it.
The essence of the issue you quote is that Python needs block-level scope for variables (in addition to the standard function-level scope). Similar to what JavaScript acquired with "let" and "const" keywords. That's more pervasive issue than just "match" statement of pattern matching, and thus better handled orthogonally to it.
> The essence of the issue you quote is that Python needs block-level scope for variables (in addition to the standard function-level scope).
No, it's not. I mean, sure, either general or special-case (like comprehensions already provide, and there is no good reason for matches not to) isolated scope would solve this, but so would bind-on-success-only with existing scoping.
No, it's not. I mean, sure, either general or special-case (like comprehensions already provide, and there is no good reason for matches not to) isolated scope would solve this, but so would bind-on-success-only with existing scoping.
> The essence of the issue you quote is that Python needs block-level scope for variables
In this case, would it not better to fix block scoping first, and introduce the pattern matching feature later?
Also, this is by far not a easily agreeable issue because changing this will break a lot of ugly but working imperative code. This has easily more impact than changing the default string type to Unicode.
For me, adding pattern matching before defining scoping feels like a technical debt issue in a large code base - if you make a change that depends on an improvement, it is much cleaner to make that improvement first.
Also, Python has only one widely-used implementation, and arguably it will be difficult to change behavior of that implementation if people did came to rely on a certain scoping behavior.
In this case, would it not better to fix block scoping first, and introduce the pattern matching feature later?
Also, this is by far not a easily agreeable issue because changing this will break a lot of ugly but working imperative code. This has easily more impact than changing the default string type to Unicode.
For me, adding pattern matching before defining scoping feels like a technical debt issue in a large code base - if you make a change that depends on an improvement, it is much cleaner to make that improvement first.
Also, Python has only one widely-used implementation, and arguably it will be difficult to change behavior of that implementation if people did came to rely on a certain scoping behavior.
> That's basically dynamic scoping instead of lexical scoping
No, it's not a scoping change, the variable in the pattern is definitively in the scope (function or module) containing the pattern match.
It's just non-determinism (or, no specification) as to what, if anything, gets assigned to it as a result of the failed pattern match.
No, it's not a scoping change, the variable in the pattern is definitively in the scope (function or module) containing the pattern match.
It's just non-determinism (or, no specification) as to what, if anything, gets assigned to it as a result of the failed pattern match.
It is. Normally, a match expression can bind variables to values. In lexical scoping, these variables exist inside the match expression. In dynamic scoping they exist outside. The design mistake seems to be to assume that the patterns inside a match expression should be regular expressions, whereas pattern expressions should be a dedicated syntactic category.
Python has lexical scoping in which modules, functions, and comprehensions are the only lexical units that introduce a new scope. The addition of pattern matching, and the UB around whether binding happens for failed matches, do not change that to dynamic scope, in which the course of execution, not the lexical structure, would determine visibility (e.g., variables would be visible in a function called from, even if not nested within, the one in which they were defined.)
guh.
another pylint rule to forbid using same names outside match clauses and inside.
another pylint rule to forbid using same names outside match clauses and inside.
You probably should do that anyway even if the scoping was local.
Funny how they accept this craziness but for the walrus operator there was endless debate
Bike shedding at its worse
Bike shedding at its worse
> The Python steering council has, after some discussion, accepted the controversial proposal to add a pattern-matching primitive to the language.
Controversial is correct. A poll showed a majority (65%) of core developers opposed to this particular proposal, and a plurality (44%) opposed to pattern matching altogether.
https://discuss.python.org/t/gauging-sentiment-on-pattern-ma...
Controversial is correct. A poll showed a majority (65%) of core developers opposed to this particular proposal, and a plurality (44%) opposed to pattern matching altogether.
https://discuss.python.org/t/gauging-sentiment-on-pattern-ma...
It's controversial, granted, but I don't think that's a fair reading of the poll. Here's another:
* A majority (56%) of responders want some form of pattern matching.
* Exactly half of all responders, forming nearly 90% of the above majority, are fine with that form being PEP 634.
* There are differing opinions about supporting PEPs, but a supermajority (70%) of those who agree with PEP 634 are fine with it alone.
The fact that it was possible to express more nuance when agreeing with PEP 634 shouldn't diminish their voice against those who reject the idea altogether.
* A majority (56%) of responders want some form of pattern matching.
* Exactly half of all responders, forming nearly 90% of the above majority, are fine with that form being PEP 634.
* There are differing opinions about supporting PEPs, but a supermajority (70%) of those who agree with PEP 634 are fine with it alone.
The fact that it was possible to express more nuance when agreeing with PEP 634 shouldn't diminish their voice against those who reject the idea altogether.
Going with the decision approved by 34% is a bad move. There's nothing wrong with going with it eventually, but this is a sign that as of the poll, there is no consensus and more discussion / more options should happen.
If you decide to act on this 34% anyway, you are only just telling your community that their opinion doesn't really matter. Which might explain why 60% of the eligible voters did not bother.
If you decide to act on this 34% anyway, you are only just telling your community that their opinion doesn't really matter. Which might explain why 60% of the eligible voters did not bother.
> but this is a sign that as of the poll, there is no consensus
agreed. I'm being pedantic on more specific conclusions, but it wasn't the best move. Even if I'm somewhere between "fine" and "happy" about this, personally.
agreed. I'm being pedantic on more specific conclusions, but it wasn't the best move. Even if I'm somewhere between "fine" and "happy" about this, personally.
> Exactly half of all responders, forming nearly 90% of the above majority, are fine with that form being PEP 634.
That half includes those who voted for "accept 634 + 640" or "accept 634 + 642", whom I doubt are entirely happy with the decision to accept PEP 634 and reject 640 & 642.
That half includes those who voted for "accept 634 + 640" or "accept 634 + 642", whom I doubt are entirely happy with the decision to accept PEP 634 and reject 640 & 642.
My point was that you cannot claim "65% of responders opposed this particular proposal". I understand, they're possibly not entirely happy. But the poll is flawed that way: "accept 634" should not mean "reject 640 & 642". I presented my reading as an alternative, but the poll isn't substantive.
That said, I interpret this part:
> PEP 642’s proposed syntax does not seem like the right way to solve the jagged edges in PEP 634’s syntax
to mean that this is merely an initial spec. Those unhappy with the syntax can avoid using it for now, there are no breaking changes. It took a couple iterations to refine the async syntax too, remember, and (IMHO) we arrived at a clean-enough version of it. I have hope!
That said, I interpret this part:
> PEP 642’s proposed syntax does not seem like the right way to solve the jagged edges in PEP 634’s syntax
to mean that this is merely an initial spec. Those unhappy with the syntax can avoid using it for now, there are no breaking changes. It took a couple iterations to refine the async syntax too, remember, and (IMHO) we arrived at a clean-enough version of it. I have hope!
Maybe adding new syntax should require a 2/3 majority or more.
just wait for a filibuster =)
> 34 voters
Is that not a reasonable sample of the 87 core developers eligible to vote?
https://discuss.python.org/groups/committers
https://discuss.python.org/groups/committers
It is a rather poor sample, and a sign that either more people than those 87 should be allowed to vote or there is something wrong with how the vote was conducted.
An explicit "do not care" or "none of the above" option in that poll would have shed more light into this.
An explicit "do not care" or "none of the above" option in that poll would have shed more light into this.
If they randomly chose 34 people to poll it would be decent sample. But now it's not a sample at all. The responders are not representative of the non-responders.
The poll is from end of November, there was a lot of discussions since then.
People complain on external corporate capture e.g. Microsoft’s embrace, extend, extinguish but I see a more common pattern of introducing internal corporate complexity driving socially powered projects to oblivion. Drupal’s extension system a few years back, Firefox addon ecosystem destruction, Python features after 3.8 and many others.
People that contribute are not infinite and those initial more active contributors get demoralized supporting work over a moving foundation.
Why not put up an effort on speeding things up, removing the GIL. Important and stale issues of the language that users really care about. F strings where good tho. That’s it. Async syntax? Less good. This pattern matching thing? Have fun debugging this.
People that contribute are not infinite and those initial more active contributors get demoralized supporting work over a moving foundation.
Why not put up an effort on speeding things up, removing the GIL. Important and stale issues of the language that users really care about. F strings where good tho. That’s it. Async syntax? Less good. This pattern matching thing? Have fun debugging this.
>Firefox addon ecosystem destruction
Firefox suffered greatly in marketshare because they didn't pull the plug soon enough but rather gained a reputation for slowness and bugginess. Doing nothing isn't a viable strategy when you have competitors (ie: Chrome) who are doing something.
Firefox suffered greatly in marketshare because they didn't pull the plug soon enough but rather gained a reputation for slowness and bugginess. Doing nothing isn't a viable strategy when you have competitors (ie: Chrome) who are doing something.
> Firefox addon ecosystem destruction
I _love_ that I can install an extension without restarting, and as much as I sympathize with the pet projects that got dumped, I think it would have been a huge mistake to hold that up just to keep dead extensions in zombie mode rather than grave mode.
I _love_ that I can install an extension without restarting, and as much as I sympathize with the pet projects that got dumped, I think it would have been a huge mistake to hold that up just to keep dead extensions in zombie mode rather than grave mode.
>I _love_ that I can install an extension without restarting
Why? How many times do you install extensions that restarting the browser to see them would be a concern?
Why? How many times do you install extensions that restarting the browser to see them would be a concern?
A lot of us use url/cookie/etc cleaners and restarting the browser forces us to log back in to sites that we'd rather not want to during the current browser session. OR have the browser set to just delete everything at close.
1) if you want to keep the session, why do you use a session cleaner?
2) how hoften do you install new extensions?
Is this really a problem?
I leave browsers open for days/weeks. I save anything I think I'll need to bookmarks or pocket. I like having it wiped when I close it. Which does force me to log back in to email/reddit/HN/etc. I have encrypted applescripts for that though so it's not too annoying.
So basically: https://xkcd.com/1172/
> Why not put up an effort on speeding things up, removing the GIL.
You are making it look like there is nobody working on those things but there is actually multiple core devs trying to making Python faster.
You are making it look like there is nobody working on those things but there is actually multiple core devs trying to making Python faster.
I always found it funny that there active efforts to speed up the language by adding more of the dreaded GILs to the runtime!
https://bugs.python.org/issue40512
https://bugs.python.org/issue40512
> Python features after 3.8 and many others.
Really, after 3.8? I thought 3.5 was a golden era?
> Why not put up an effort on speeding things up, removing the GIL.
That's exactly what requires corporate backing (removing is easy, fixing all the breakage it causes takes a lot of time and effort).
Really, after 3.8? I thought 3.5 was a golden era?
> Why not put up an effort on speeding things up, removing the GIL.
That's exactly what requires corporate backing (removing is easy, fixing all the breakage it causes takes a lot of time and effort).
Can't wait to see all the errors and questions when users start using this for the missing switch statement, and not realizing it doesn't work for non-dotted constants.
Because if I didn't misunderstood, this is not correct, is it?
Because if I didn't misunderstood, this is not correct, is it?
NOT_FOUND = 404
OK = 200
match getCode():
case NOT_FOUND:
return default()
case OK:
return response()Yup, that's what the PEP seems to suggest.
This along with mutable default values is going to be one of the nastier Python warts.
This along with mutable default values is going to be one of the nastier Python warts.
Not to mention that expressions after the case keyword have completely different semantics from expressions ANYWHERE else. a | b is not the bitwise or of a with b.
This part is truly strange. This example from the tutorial drove it home for me:
What I still don't understand is: does the new syntax magically do this to a user-defined class? i.e. if I say
edit: typo fix
match event.get():
case Click(position=(x, y)):
handle_click_at(x, y)
case KeyPress(key_name="Q") | Quit():
game.quit()
What I still don't understand is: does the new syntax magically do this to a user-defined class? i.e. if I say
foo = Click(position=(x, y))
...
case Click(position=(x, y))
edit: typo fix
You are (sadly) correct. The latter example checks that the type of `event.get()` is a Click, then that it has a `.position` attribute that be restructured into a two-element tuple `(x,y)`. It's equivalent to `ev = event.get(); if isinstance(ev, Click): (x,y) = ev.position`
Correct.
> Patterns may use named constants. These must be dotted names to prevent them from being interpreted as capture variable:
NOT_FOUND would always match as a capture variable, so you'd always return default().
I fully expect this to be covered by linters, but not with further language support.
> Patterns may use named constants. These must be dotted names to prevent them from being interpreted as capture variable:
NOT_FOUND would always match as a capture variable, so you'd always return default().
I fully expect this to be covered by linters, but not with further language support.
yeah, I would also expect the linters to catch this. has some way to allow use of non-dotted constants been considered? something like window.global_variable in JavaScript? Or would I need to do something like this:?
NOT_FOUND = 404
m = {"NOT_FOUND": NOT_FOUND}
match value:
case m["NOT_FOUND"]:
passYou can do
Alternatively, we can avoid using a separate `v` variable and just use `value` directly:
NOT_FOUND = 404
match value:
case v if v == NOT_FOUND:
pass
Alternatively, we can avoid using a separate `v` variable and just use `value` directly:
NOT_FOUND = 404
match value:
case _ if value == NOT_FOUND:
pass
NOT_FOUND = 404
match None:
case _ if value == NOT_FOUND:
pass
() match {
case _ if foo > bar => ...
case _ if foo < bar => ...
case _ if foo == bar => ...
}But all of these alternatives are strictly less intuitive than the original expression. A big reason why I got into Python was because it’s the language that can be “read by anyone”, even those who do not know Python. This latest PEP, coming on the heels of several others like it, shows how alarmingly far from the ideal we’ve come since 2.7.
> But all of these alternatives are strictly less intuitive than the original expression.
I agree; I was just trying to show how pattern guards work. I would only resort to pattern guards if I need a `cond` (like my last example), or on a few cases of a more 'natural' pattern match, e.g.
I agree; I was just trying to show how pattern guards work. I would only resort to pattern guards if I need a `cond` (like my last example), or on a few cases of a more 'natural' pattern match, e.g.
match foo():
case OK(msg):
return success(msg)
case NotFound(_) if isQuery:
return success("No results matched query")
case NotFound(msg):
return failure(msg)
...
return {
NOT_FOUND: default,
OK: response,
}[getCode()]()> A big reason why I got into Python was because it’s the language that can be “read by anyone”, even those who do not know Python.
Yes! For the first 2 decades of its life, people loved Python because it was like “executable pseudocode”. Unfortunately over the last 5-10 years it seems to have given up on that idea, instead becoming more and more like “whitespace-sensitive C++”.
Yes! For the first 2 decades of its life, people loved Python because it was like “executable pseudocode”. Unfortunately over the last 5-10 years it seems to have given up on that idea, instead becoming more and more like “whitespace-sensitive C++”.
I may be wrong, but after reading the grammar I think your example is a syntax error. You can't access an element! A pattern syntax is completely different to a normal expression, and there is apparently no array access.
hmm, so I would need to use some namespace class https://docs.python.org/3.8/library/types.html#types.SimpleN...
Well, of course. It is a pattern matcher. case NOT_FOUND: is a pattern that matches anything and binds it to NOT_FOUND. If you want to match the content of a variable you need to signal that in some way. In scheme that usually means unquote:
(define not-found 404)
(match status
(,not-found (do-stuff))
(else (display "you might believe this is an else clause, but it actually just matches everything and binds it to else")))[deleted]
plus the distinction of using dotted constants makes no sense, that can just mean a variable in another module
I'm mixed on this one. While I love pattern matching in SML and Rust, I don't know if it's right for Python.
Like other comments have said, I loved and learned a lot of Python from looking at a code base and understanding everything immediately because there wasn't much to learn on the language end of things. It was nice and small. Not that the addition of pattern matching is a death sentence for simplicity, but I fear that it may lead to some other extraneous features in the future.
I like powerful, featureful languages, but I personally like Python as a simple language that anyone can learn 80% of in a busy weekend.
Will I use pattern matching? Maybe. I'll definitely play around with it. Now if we could get some algebraic datatypes then I think I'd start contradicting myself...
Like other comments have said, I loved and learned a lot of Python from looking at a code base and understanding everything immediately because there wasn't much to learn on the language end of things. It was nice and small. Not that the addition of pattern matching is a death sentence for simplicity, but I fear that it may lead to some other extraneous features in the future.
I like powerful, featureful languages, but I personally like Python as a simple language that anyone can learn 80% of in a busy weekend.
Will I use pattern matching? Maybe. I'll definitely play around with it. Now if we could get some algebraic datatypes then I think I'd start contradicting myself...
True, but honestly pattern matching is so cool that all languages should have it and ditch switch statements.
Thankfully Python has avoided switch statements. Hopefully pattern-matching will take the place of if/elif/elif/else chains, or dereferencing dictionary literals (e.g. `{case1:foo, case2:bar, ...}[myVal]`)
Interesting discussions over on Twitter about the new parser might be letting out the complexity genie and how some think that might not be the greatest. https://twitter.com/ramalhoorg/status/1358168753100496907
Not really. Its a string of preferences for Python's development. I hope the author pursues those endeavors and encourages others to join him. But they're not an argument against pattern matching.
Talking about preferences still qualifies as interesting discussion.
As someone who hasn’t been following closely, was the PEG parser necessary for pattern matching or is that an unrelated change?
IIRC one of Guido’s reasons for switching to a PEG parser was to enable the addition of pattern matching.
And it allowed some non-intuitive parser limitations to be lifted, like full expressions in decorators (@buttons[0].click) and parentheses in context expressions (https://bugs.python.org/issue12782).
The accepted PEP is 634: https://www.python.org/dev/peps/pep-0634/ and the tutorial has some examples: https://www.python.org/dev/peps/pep-0636/ .
The comments on LWN are just hilarious.
> I'm going to submit a PEP to add a kitchen sink to Python.
I personally would opt for the sink with the snake, as it is more Pythonic, despite not being the nicest and cleanest solution (is that a Pythonic property as well?).
> I'm going to submit a PEP to add a kitchen sink to Python.
I personally would opt for the sink with the snake, as it is more Pythonic, despite not being the nicest and cleanest solution (is that a Pythonic property as well?).
Even better is the reply: "Python already has async"
Mozilla for a time had a kitchen sink (https://bugzilla.mozilla.org/show_bug.cgi?id=122411) at about:kitchensink.
It looked like this: https://www-archive.mozilla.org/docs/web-developer/samples/k...
It looked like this: https://www-archive.mozilla.org/docs/web-developer/samples/k...
> After much deliberation, the Python Steering Council is happy to announce that we have chosen to accept PEP 634, and its companion PEPs 635 and 636, collectively known as the Pattern Matching PEPs
This is why I'm still enamored with Lisp. One doesn't wait around for the high priests to descent from their lofty towers of much deep pontification and debate with shiny, gold tablets inscribed with how the PEPs may be, on behalf of the plebes. One just adds new language feature themselves, eg. pattern matching[1] and software transactional memory[2].
1. https://github.com/guicho271828/trivia
2. https://github.com/cosmos72/stmx
This is why I'm still enamored with Lisp. One doesn't wait around for the high priests to descent from their lofty towers of much deep pontification and debate with shiny, gold tablets inscribed with how the PEPs may be, on behalf of the plebes. One just adds new language feature themselves, eg. pattern matching[1] and software transactional memory[2].
1. https://github.com/guicho271828/trivia
2. https://github.com/cosmos72/stmx
Not only that, but we also know that no one pattern matcher fits all: with or without segment patterns? how to designate pattern variables? should it be extendable? what would we actually be matching? should we actually use a unifier? etc. Pattern matchers are easy to write, anyway.
We also know that they don't always make things clear, so we don't use them everywhere. Programmers using languages that have a pattern matcher "built in" tend to use it all the time, and it's not always a pretty sight. I conjecture that with this addition, Python codebases will become dichotomized: either it will be used everywhere, or it will not be used at all. The latter obviously has a head start :)
We also know that they don't always make things clear, so we don't use them everywhere. Programmers using languages that have a pattern matcher "built in" tend to use it all the time, and it's not always a pretty sight. I conjecture that with this addition, Python codebases will become dichotomized: either it will be used everywhere, or it will not be used at all. The latter obviously has a head start :)
Fine for solo work but what happens if your team all come to work in the morning ecstatic about what they just added to the language? Now you have n problems where n is the number of members in your team.
This is what code review is for though, right? I think any reviewer worth their salt is going to be pretty skeptical about a new language feature being the right way to solve a problem.
New syntax is often a bad idea, but sometimes it's a really good idea. It'd be good to allow experimentation in the wild, and potentially upstream these features later once they've been proved out.
This is why I suggested macros as the feature I'd like to see most in the last python developer survey. Most of the time, defining a macro is bad idea and is a code smell, but sometimes they add very useful language features.
New syntax is often a bad idea, but sometimes it's a really good idea. It'd be good to allow experimentation in the wild, and potentially upstream these features later once they've been proved out.
This is why I suggested macros as the feature I'd like to see most in the last python developer survey. Most of the time, defining a macro is bad idea and is a code smell, but sometimes they add very useful language features.
Pattern matching is one of my favorite things about Haskell. But seeing it done here in a dynamically typed language is a more than little clunky (three indents needed to get to an expression). It would be better to be matched on the parameter list, but again Python’s typing would make that too difficult.
I think it works very well in Elixir which is also dynamically typed. It was there since the beginning though and it's more than just a "match/case" construct, it's built deep into the core of the language, so maybe that makes a difference
The two-space compromise might work:
def handle(command):
match command.split():
case ["quit"]:
print("Goodbye!")
quit_game()
case ["look"]:
current_room.describe()
I have never really pushed hard on style in my software journey. But this is different. That code sample is horrifying.
Pretty cute syntax, but not what I'd wanted. It takes something that is already trivial to write and makes it simpler. Not a lot of "bang for your syntax bucks" so to speak. E.g the interpreter example from the PEP could have been written:
parts = command.split()
if not (0 < len(parts) < 3):
raise TypeError(...)
cmd, arg = parts
if cmd == 'quit':
...
elif cmd == 'get':
character.get(arg, current_room)
elif ...:
...
Your code is buggy, though. It will only ever accept two parts because of `cmd, arg = parts`, so `["quit"]` will fail, and each command might accept a different number of parts, so you can't just test `0 < len(parts) < 3` for all of them. The pattern matching version is harder to get wrong.
Your snippet would be condensed into 1 indentation block with pattern matching. In scheme it'd be even more explicit that we're deconstructing command, therefore much much easier to follow logic. No need to worry about cmd, args, etc.
(match command
(("quit") (display "Goodbye!"))
(("look") (describe current-room))
(("get" obj) (character-get obj current-room))
(("go" dir) (set! current-room (room-neighbour dir)))
(_ (display "Invalid command")))what happens if len(parts)==1 ?
in this case pattern matching would have helped
in this case pattern matching would have helped
Wow, this is perhaps the worst idea introduced to Python in a long time. So many ambiguities, so much inscrutable syntax. What a shame.
Can anyone confirm this is actual pattern matching and not the frankly ridiculous procedural version that was originally proposed?
That proposal, to deliberately hobble pattern matching by making it a statement, was an egregious ideological campaign with genuinely absurd consequences.
That proposal, to deliberately hobble pattern matching by making it a statement, was an egregious ideological campaign with genuinely absurd consequences.
Not sure what you mean, but the accepted PEP does appear to be a statement. What was the alternative?
In many languages match is an expression, so you can do things like `x = match y { ... }`, which is a very common use case.
I don't write much Python these days, but it being a statement rather than an expression is disappointing
It forces you to use a separate "output" variable or `return`.
I don't write much Python these days, but it being a statement rather than an expression is disappointing
It forces you to use a separate "output" variable or `return`.
I presume that the alternative to a statement is an expression. For example, the ternary operator
x = (a if c else b)
if c:
x = a
else:
x = b
match is a statement, but the syntax is very readable.
https://www.python.org/dev/peps/pep-0636/#adding-conditions-...
https://www.python.org/dev/peps/pep-0636/#adding-conditions-...
> the syntax is very readable
Yet it can't be used in a lambda, it can't be assigned to a variable, it can't be used as an argument, it can't be called as a function, it can't be put in a datastructure, it can't be raised, it can't be returned, it can't be yielded, and so on for all the things that can't be done to statements.
Heck, we can't even match on a match!
Yet it can't be used in a lambda, it can't be assigned to a variable, it can't be used as an argument, it can't be called as a function, it can't be put in a datastructure, it can't be raised, it can't be returned, it can't be yielded, and so on for all the things that can't be done to statements.
Heck, we can't even match on a match!
I've been trying to find the motivation for making it a statement. Do you have links to any prior discussion?
I maintain a transpiler that converts python to 6 type safe languages. Keeping match semantics similar makes it easier to transpile.
I maintain a transpiler that converts python to 6 type safe languages. Keeping match semantics similar makes it easier to transpile.
Strong agree that this is hobbled by being a statement.
Pattern matching is great for dispatch and overloading, but because it's a statement there's no way for users to add overloads.
There will have to be some `matches(pattern, object)` function in CPython. Why not make this available to users? Why make patterns syntax, not objects?
Pattern matching is great for dispatch and overloading, but because it's a statement there's no way for users to add overloads.
There will have to be some `matches(pattern, object)` function in CPython. Why not make this available to users? Why make patterns syntax, not objects?
It's this: https://www.python.org/dev/peps/pep-0634/ - not sure which version that is?
Java is also in proces of adding Pattern Matching over the next couple of releases [1].
[1] https://www.infoq.com/articles/java-pattern-matching/
[1] https://www.infoq.com/articles/java-pattern-matching/
I just took a look at this and I am confused as to the actual practical utility of pattern matching in a language that is not strongly-typed to begin with.
My initial take upon stumbling upon the controversy was "why would pattern matching be bad?" because I have only experienced it through the lens of C# 9.0. Upon reviewing PEP 622 and looking at some examples of usage... I am in agreement with the detractors. This just doesn't seem to make sense for python, but I would certainly hate to get in the way of someone's particular use case as long as the new feature doesn't fundamentally cripple the language for others.
IMO, Microsoft nailed it with their pattern matching implementation:
https://docs.microsoft.com/en-us/dotnet/csharp/pattern-match...
My initial take upon stumbling upon the controversy was "why would pattern matching be bad?" because I have only experienced it through the lens of C# 9.0. Upon reviewing PEP 622 and looking at some examples of usage... I am in agreement with the detractors. This just doesn't seem to make sense for python, but I would certainly hate to get in the way of someone's particular use case as long as the new feature doesn't fundamentally cripple the language for others.
IMO, Microsoft nailed it with their pattern matching implementation:
https://docs.microsoft.com/en-us/dotnet/csharp/pattern-match...
Python is strongly-typed. With its built-in type annotation syntax and a checker like mypy or pyright, it is statically-typed, too.
My experience with mypy has been pretty negative.
Not because mypy is bad per se, but it just doesn't flow into a Python workflow naturally. Every time you try to interact with an external module without these annotations you immediately lose all the advantages of statically typed programs.
For me, Python with annotations just doesn't make sense, at that point I would prefer to program in Rust or Java.
Not because mypy is bad per se, but it just doesn't flow into a Python workflow naturally. Every time you try to interact with an external module without these annotations you immediately lose all the advantages of statically typed programs.
For me, Python with annotations just doesn't make sense, at that point I would prefer to program in Rust or Java.
For external modules, you can generate stub files that contain inferred and/or manually supplied annotations[0].
[0] https://mypy.readthedocs.io/en/stable/stubgen.html
[0] https://mypy.readthedocs.io/en/stable/stubgen.html
Type annotation and mypy, while better than nothing, is a hack and doesn't always properly work so in a real codebase you'd be using `# type: ignore` quite often even if all of your downstream code is properly type-annotated and mypy-typechecked.
Most importantly though, it does not make Python statically typed, by no means. It just lets you lint stuff in a semantic fashion (which is helpful, no doubt), improves auto-completion and removes the need of listing type requirements in docstrings.
Most importantly though, it does not make Python statically typed, by no means. It just lets you lint stuff in a semantic fashion (which is helpful, no doubt), improves auto-completion and removes the need of listing type requirements in docstrings.
A bonus for writing strongly typed code and using a small/safe subset of python is that you can transpile it to languages like rust or kotlin
Maybe this will grow on me, but at first glance this looks needlessly complex. I don't think I've even fully comprehended the syntax after staring at for a few minutes: https://www.python.org/dev/peps/pep-0634/#appendix-a-full-gr...
Another step away from the beautiful clarity that Python embodies. Yes, I get what it does, but mostly I see the consequences. The past few years of Python language 'development' have been somewhat alarming to me.
Super mixed feelings about this. I feel like the binding stuff in particular could have been solved with some sort of prefix symbol (like "this is referring to a binding location") without introducing a bunch of semantic weirdness.
But like... hey, finally get a case statement! I imagine the community will coalesce a bit around best practices. And stuff like pylint will catch name shadowing (at least in theory). Almost all pattern matching mechanism have name shadowing issues and it's definitely annoying, but they are able to figure it out.
But like... hey, finally get a case statement! I imagine the community will coalesce a bit around best practices. And stuff like pylint will catch name shadowing (at least in theory). Almost all pattern matching mechanism have name shadowing issues and it's definitely annoying, but they are able to figure it out.
But what if you want to match to value of some variable in the scope? I see no "^pinning" of variables or stuff like that. There's "dotted" syntax, but it works only for constants and fields, not local variables, right? Am I missing something?
IMO the obvious solution would be to allow a bare dot, like .local_variable
But I don't see that, or any other solution for local variables, in the PEP
But I don't see that, or any other solution for local variables, in the PEP
I like it. I was just trying to do some simple "parsing" of a pyproject.toml file. So I wrote the standard crud of, "is it a dictionary? If so, does it have a specific key? What if it's just a string? If so, ..."
Having a dedicated syntax would mean someone could look at the patterns I'm matching and see clearly what the format of the input should be. This seems ideal for handling various user-friendly toml/yaml/json configuration formats and other semi(poorly)-structured data.
I think it also addresses a real weakness in OO: to add behavior per class you either lard up classes with not-really-relevant methods, or create a proxy class, or do the inevitable if/elif isinstance chain.
Having a dedicated syntax would mean someone could look at the patterns I'm matching and see clearly what the format of the input should be. This seems ideal for handling various user-friendly toml/yaml/json configuration formats and other semi(poorly)-structured data.
I think it also addresses a real weakness in OO: to add behavior per class you either lard up classes with not-really-relevant methods, or create a proxy class, or do the inevitable if/elif isinstance chain.
A lot of early Python's appeal was in its simplicity. I miss that era. Soon Python is going to resemble C++'s bloated feature set where you're not supposed to use parts of the language because "that's the old way of doing things".
Edit: Actually that's already true with string formatting.
Edit: Actually that's already true with string formatting.
I understand your feeling and I would tend to agree for things in other domains.. but for a programming language, I find that if:
(1) It doesn't make the language less efficient
(2) The new feature is coherent with the rest of the language
(3) It doesn't make learning the language harder
(4) It brings joy in my life (as f-strings do)
Then.. it's ok :-)
(1) It doesn't make the language less efficient
(2) The new feature is coherent with the rest of the language
(3) It doesn't make learning the language harder
(4) It brings joy in my life (as f-strings do)
Then.. it's ok :-)
There are some 'invisible' downsides to adding new features, which often get overlooked:
- Tooling, code analysers, etc. have to be updated to understand the new features. Tools which are dead or "finished" may stop working.
- The possibility that someone might use a new feature can break guarantees that are relied upon by existing code. An obvious example would be adding try/catch to a language: in languages without this, a function call is guaranteed to either return control to us, or to run indefinitely. Libraries might rely on this to clean up file descriptors, etc. afterwards. Such libraries would be broken by the mere existence of try/catch; even if we never actually use it.
That said, I'm a big fan of pattern-matching, so it would be nice to have in Python, rather than crude approximations like '{pattern1: A, pattern2: B, pattern3: C, ...}[myVal]'
- Tooling, code analysers, etc. have to be updated to understand the new features. Tools which are dead or "finished" may stop working.
- The possibility that someone might use a new feature can break guarantees that are relied upon by existing code. An obvious example would be adding try/catch to a language: in languages without this, a function call is guaranteed to either return control to us, or to run indefinitely. Libraries might rely on this to clean up file descriptors, etc. afterwards. Such libraries would be broken by the mere existence of try/catch; even if we never actually use it.
That said, I'm a big fan of pattern-matching, so it would be nice to have in Python, rather than crude approximations like '{pattern1: A, pattern2: B, pattern3: C, ...}[myVal]'
Pattern matching will definitely make Python harder to learn, though. It’s not particularly coherent with the rest of the language either - the match statement has been described as “a DSL contrived to look like Python, and to be used inside of Python, but with very different semantics” [0].
[0] https://discuss.python.org/t/gauging-sentiment-on-pattern-ma...
[0] https://discuss.python.org/t/gauging-sentiment-on-pattern-ma...
I have never seen f-strings! It's my lucky day! https://xkcd.com/1053/
Time to refactor the project I am currently working on that is loaded with `str.format()`
Time to refactor the project I am currently working on that is loaded with `str.format()`
Take a look at flynt, a tool that can convert older string formatting code to the newer fstring style. I've found it very helpful!
https://github.com/ikamensh/flynt
https://github.com/ikamensh/flynt
Can confirm. I replaced 1.5k instances in our code base in seconds using flynt and it knew the ones that couldn't be swapped out directly. Really great tool. You only ever need it once in your life, but that one time it's amazing.
Having not written Python in a long time until recently, I discovered fstrings. Are they the best choice for (nearly all) string formatting at least in new code? They seem to work pretty similarly to eg Ruby’s string interpolation.
Be aware that the only time this won't work is if you need to re-render the f-string with different context. For example you can do this:
EDIT: Fix syntax in examples.
[0] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
s = "this is {}"
print(s.format("possible"))
for message in messages:
print(f"The message is {message}")
s = f"The message is {message}"
for message in messages:
print(s)
EDIT: Fix syntax in examples.
[0] https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
f-strings are truly amazing. They're very painless. The only gotcha I've run into is accessing a dict from inside one. You need to watch out for your quotations :).
My most recent project makes liberal use of `f"""..."""`
I find it both incredibly humorous and saddening to see Python devs excited about finally getting string interpolation.
My concern with Python’s feature bloat is the opposite. That there aren’t parts you’re not supposed to use - developers are expected to know all of it.
Three string formatting approaches are all in wide use. There are now two different ways to do something as fundamental as assignment. Some classes use explicit methods like __eq__, others hide them by using @dataclass. Async-await adds coloured functions [0], along with “red” versions of numerous language constructs.
That’s not to mention type annotations, which I won’t deny are popular but are incredibly complex.
[0] http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y...
Three string formatting approaches are all in wide use. There are now two different ways to do something as fundamental as assignment. Some classes use explicit methods like __eq__, others hide them by using @dataclass. Async-await adds coloured functions [0], along with “red” versions of numerous language constructs.
That’s not to mention type annotations, which I won’t deny are popular but are incredibly complex.
[0] http://journal.stuffwithstuff.com/2015/02/01/what-color-is-y...
And probably other situations, like perhaps corporate environments, where you're not supposed to use the new more advanced parts because not everyone on the team knows them. Python is evolving into several different languages. "There should be one-- and preferably only one --obvious way to do it." Yeah right. I'm waiting for a PEP that deprecates PEP 20, "The Zen of Python". It's become more and more clear that the core developers don't follow it all anymore.
First off I'll say that I'm very sympathetic to your point of view, and mostly agree with it.
But... after I learned Scala a bunch of years ago, working in any language without pattern matching has felt like significantly more of a chore than it used to be. It's like you've been doing something the hard way for years, someone teaches you an easier, better, more readable way, and just when you get used to it and love it, then tells you that you can't use it anymore.
Certainly that's not true of all possible high-mental-load new features that could be added to Python. But I don't want to believe that features that drastically increase ergonomics shouldn't be added to a language because they make the language less simple.
But... after I learned Scala a bunch of years ago, working in any language without pattern matching has felt like significantly more of a chore than it used to be. It's like you've been doing something the hard way for years, someone teaches you an easier, better, more readable way, and just when you get used to it and love it, then tells you that you can't use it anymore.
Certainly that's not true of all possible high-mental-load new features that could be added to Python. But I don't want to believe that features that drastically increase ergonomics shouldn't be added to a language because they make the language less simple.
I thought I’d learn Perl once so I started reading a book on it. I was shocked that literally the bottom half of every page was footnotes. I gave up after a few pages.
Too much mental overhead. Other people love that kind of stuff though.
Too much mental overhead. Other people love that kind of stuff though.
I was a Perl developer/scripter for the better part of 3 years - wrote a 2-way LDAP<-> HRIS synchronization system in it, complete with customizeable Schemas. Then, 3 or so years into my Perl experience, in which I still needed to look at template code every time I did a HashOfArray/ArrayOfHash, I tripped across some python code explaining how everything was objects.
Within a few seconds I tested that theory, by populating a list with dicts and voila - just worked. Close to the last day I ever touched Perl.
The python language expansions that have come recently, f-strings, walrus operator, and now matching - are great in that they don't make the language more complex - all three of these are easily explainable in a few minutes to the novice, and once they understand it, they can quickly (and profitably) incorporate it into their code.
I wan't there to be a steady drumbeat of these improvements that let me write more elegant code, more concisely.
Try and find a single python developer who would give up their f-strings now.
Within a few seconds I tested that theory, by populating a list with dicts and voila - just worked. Close to the last day I ever touched Perl.
The python language expansions that have come recently, f-strings, walrus operator, and now matching - are great in that they don't make the language more complex - all three of these are easily explainable in a few minutes to the novice, and once they understand it, they can quickly (and profitably) incorporate it into their code.
I wan't there to be a steady drumbeat of these improvements that let me write more elegant code, more concisely.
Try and find a single python developer who would give up their f-strings now.
> in which I still needed to look at template code every time I did a HashOfArray/ArrayOfHash
Honestly, that sounds like someone that didn't bother to learn what's actually going on and understand what the language was doing. I mean, that's fine, we all do that... but I wouldn't blame the language for that. Perl makes nested data structures very easy and obvious once you learn what's going on. You just need to actually learn it and not rely on the little shortcuts Perl allows you to use to never make that next step to learning exactly why.
Perl will "just do what you mean" so much that it papers over some of the early learning friction points enough that they build up to a later point. At that point, people either throw up their hands and move on because it's annoying when some shortcut they've relied on isn't working in this one weird instance, or they learn what's really going on (Hint: it's almost always either about understanding references or context at this stage) and it's no longer a problem.
> Try and find a single python developer who would give up their f-strings now.
Things like f-strings and list comprehensions in Python are the exact thing that Python devs used to point out as too complex in other languages (I mean, f-strings look to be pretty much just string interpolation). I think that's evidence that perhaps a language isn't always best suited for all audiences. Python used to be aimed at learning and easy of understanding and use. Now that it's often more targeted for complex engineering projects, you get stuff sneaking in that was specifically avoided by design initially.
Honestly, that sounds like someone that didn't bother to learn what's actually going on and understand what the language was doing. I mean, that's fine, we all do that... but I wouldn't blame the language for that. Perl makes nested data structures very easy and obvious once you learn what's going on. You just need to actually learn it and not rely on the little shortcuts Perl allows you to use to never make that next step to learning exactly why.
Perl will "just do what you mean" so much that it papers over some of the early learning friction points enough that they build up to a later point. At that point, people either throw up their hands and move on because it's annoying when some shortcut they've relied on isn't working in this one weird instance, or they learn what's really going on (Hint: it's almost always either about understanding references or context at this stage) and it's no longer a problem.
> Try and find a single python developer who would give up their f-strings now.
Things like f-strings and list comprehensions in Python are the exact thing that Python devs used to point out as too complex in other languages (I mean, f-strings look to be pretty much just string interpolation). I think that's evidence that perhaps a language isn't always best suited for all audiences. Python used to be aimed at learning and easy of understanding and use. Now that it's often more targeted for complex engineering projects, you get stuff sneaking in that was specifically avoided by design initially.
"Honestly, that sounds like someone that didn't bother to learn what's actually going" - Yes, exactly! I never learned what was going on. I pinged our JAPH that was kind of my mentor, and asked how he had ever grokked it - and his answer was, he hadn't - he just yanked his template code each time as well.
The point I'm trying to make is not that I'm a good or knowledgeable developer (neither of which I am, despite having written tens of thousands of lines of perl) - but that the core essence of Python, is that people can use it quickly and profitably without being one. The cognitive hurdle to start using objects in python is tiny - and, once you get that - a lot of the stuff that you would hope works, just does. The language is very friendly to novice coders, and it's lack of implicit (for the most part) actions avoids a lot of unclear side effect.
For more complex projects, things like decorators and generators and type-hints, which are advanced, are available if you need them - but you can go a long way (sometimes forever) without ever touching them - that's not the case with simple data structures - you pretty much need to start working with HoA in perl if you want to do complex things, and I know people who have used perl for the better part of a decade, who have never done so - and were blocked from doing more interesting things.
The point I'm trying to make is not that I'm a good or knowledgeable developer (neither of which I am, despite having written tens of thousands of lines of perl) - but that the core essence of Python, is that people can use it quickly and profitably without being one. The cognitive hurdle to start using objects in python is tiny - and, once you get that - a lot of the stuff that you would hope works, just does. The language is very friendly to novice coders, and it's lack of implicit (for the most part) actions avoids a lot of unclear side effect.
For more complex projects, things like decorators and generators and type-hints, which are advanced, are available if you need them - but you can go a long way (sometimes forever) without ever touching them - that's not the case with simple data structures - you pretty much need to start working with HoA in perl if you want to do complex things, and I know people who have used perl for the better part of a decade, who have never done so - and were blocked from doing more interesting things.
That saddens me. It's not actually that complicated, but I think Perl's usage numbers have degraded enough that unless you're frequenting one of the online spots where those knowledgeable about the language gather, you're unlikely to encounter them in the wild anymore, and thus aren't getting a useful explanation.
The simple answer that probably would have solved almost all your problems is that you can use the reference syntax for defining things and it will look like and function the same as JavaScript 95% or more of the time, and the only time you'll need to do anything is when passing it to something that expects an actual array or hash. I mean, you can get away with taking JSON and changing colons into fat arrows and booleans into 0 and 1 and it will just work as a Perl data structure as is like 99.99% of the time. Data structure manipulation works similarly as well for the most part.
The simple answer that probably would have solved almost all your problems is that you can use the reference syntax for defining things and it will look like and function the same as JavaScript 95% or more of the time, and the only time you'll need to do anything is when passing it to something that expects an actual array or hash. I mean, you can get away with taking JSON and changing colons into fat arrows and booleans into 0 and 1 and it will just work as a Perl data structure as is like 99.99% of the time. Data structure manipulation works similarly as well for the most part.
I've always viewed javascript as verbose crippled perl with good vendor support. I mean, `"use strict";` ...
Oh my goodness kbenson - we had this identical conversation 7 years ago on HN. https://news.ycombinator.com/item?id=7869603
Ha, I always wondered how long it would take for me to repeat a conversation here with someone! I figured it was only a matter of time. Now I know how much. :)
Yeah, this. I love Python, and I hated Perl. Even Java was a pleasure compared to Perl.
At one time, a trainer was trying to tell us Ruby was the new hotness (2009 or 2010, I think). I liked Ruby OK, but Python seems cleaner and we stuck with it. I have not felt the slightest regret.
At one time, a trainer was trying to tell us Ruby was the new hotness (2009 or 2010, I think). I liked Ruby OK, but Python seems cleaner and we stuck with it. I have not felt the slightest regret.
sounds like you shoulda been using objects to bind data. Moo is one of the best OO toolkits out there. Can whip up really good long lived, testable, easy to understand stuff fast with extensive use of `lazy => 1`
Thought I'd try to balance out all the Perl hate in the sibling comments.
I still use, and love Perl. If you stick close to the original inspiration of using it to process text, it flows very naturally and isn't hard to read or understand.
I do get that the proliferation of sigils ($,@,%), breadth of built-ins, things like $_, complex dereferencing, and overall terseness are a visual putoff.
It may be that I still like it because it uses mostly the same functions as 'C'...things like stat(), getpwent(), ioctl(), and so forth that were already in my head. And it was way less tedious for strings and text, and no malloc/free to keep track of.
I still use, and love Perl. If you stick close to the original inspiration of using it to process text, it flows very naturally and isn't hard to read or understand.
I do get that the proliferation of sigils ($,@,%), breadth of built-ins, things like $_, complex dereferencing, and overall terseness are a visual putoff.
It may be that I still like it because it uses mostly the same functions as 'C'...things like stat(), getpwent(), ioctl(), and so forth that were already in my head. And it was way less tedious for strings and text, and no malloc/free to keep track of.
I'm glad I'm not the only one around still using and enjoying Perl. I'm just starting to play around in Python now with the plan to rewrite/replace a few older Perl scripts so while I may not get much use out of Perl going forward, it's been great for tasks like log file parsing and reporting.
This is the first time in the history of the internet that anyone has complained about too much documentation.
I programmed in Perl 5 for several years. Ugh. That language was hard to read, and some of the features (list vs. scalar context, for example) were awful. I have never wanted to write another Perl program. Capable, but painful to maintain.
I think this comment is super exaggerated. In my experience pretty much all features in Python 3 by far are really welcome. The only features you're usually not supposed to use are very obvious ones, inspect, metaclasses and nested comprehensions, which are super uncommon and feel like dark magic that you would obviously not want to use, or just shit ugly code in the case of nested comprehensions.
I agree with this up through Python 3.7. The recent updates though seem to be unnecessary or have obtuse syntax.
do you have specific examples? honestly I was against the walrus operator but having played with it for just two seconds I think it's really great and that people are making a bigger deal than needed about this
What's the problem with nested comprehensions? How else do you create bi-dimensional stuff?
Not the same kind of nesting, is it?
Not to mention, you can even reuse the name for extra evil:
[[None for _y in range(y)] for _x in range(x)]
[None for _x in range(x) for _y in range(y)]
Not to mention, you can even reuse the name for extra evil:
[i for i in range(2) for i in range(3)]
[0, 1, 2, 0, 1, 2]Fwiw, I find creating a 2d array and then flattening (`itertools.chain.from_iterable([[None for _y in range(y)] for _x in range(x)])` to be both readable and reasonably concise, though yes explicit loops are also fine.
nice trick.. though if you are relying on the 'generator' aspect of comprehensions this may be problematic (will create in-memory copies)
Yes, I think the general consensus on metaclasses is that if you don't already know what they are, then you probably don't need to use them. There's some metaclasses in Django, but it's pretty easy to see how you're expected to use them from examples, and you don't need to really understand what's going on.
String formatting with f"" has been the new sheriff in town for a while and it is really killer.
Yes, and it also takes approx 11 seconds to learn, since str.format already had exactly the same syntax, just slightly more verbose.
I never understood why they didn't remove the previous ones.
Isn't there supposed to be only one obvious way of doing these things?
Isn't there supposed to be only one obvious way of doing these things?
I would support removing the weird '%' operator, but str.format definitely still has uses. For one you can't make templates using f-strings since they are immediately evaluated.
Python already broke backwards compatibility once. It didn't got over well
You can't just remove things, that'd break stuff!
That is why scala has scalafix[1] where library maintainers can provide rules along with newer versions of libraries that can be used to automatically, either at build time or out of band on the source, translate old constructs/patterns in the new.
See it as deferred refactoring for library maintainers. Library maintainers that work in a monorepo with their users get this refactoring feature for free.
[1]: https://scalacenter.github.io/scalafix
See it as deferred refactoring for library maintainers. Library maintainers that work in a monorepo with their users get this refactoring feature for free.
[1]: https://scalacenter.github.io/scalafix
I have two thoughts on this: One is that tools like scalafix no-doubt rely heavily on static analysis to ensure correctness of their program transformations. I'm not sure a hypothetical "pythonfix" tool will be quite as feasible for untyped code bases.
And also, the issue isn't really with library maintainers when it comes to Python syntax changes. It's with all the user code written in Python. I tried converting my programs from Scala 2.12 from 2.13, and I failed so hard. I couldn't justify the time commitment, even with tools like scalafix to automate stuff. It's just not that smart.
All this is to say, let's not pretend that breaking changes to a programming language are ok. If you have to do break stuff once or twice in the lifetime of your language, fine. But accepting breaking changes as the status-quo is a failure of a language, by definition, IMO.
And also, the issue isn't really with library maintainers when it comes to Python syntax changes. It's with all the user code written in Python. I tried converting my programs from Scala 2.12 from 2.13, and I failed so hard. I couldn't justify the time commitment, even with tools like scalafix to automate stuff. It's just not that smart.
All this is to say, let's not pretend that breaking changes to a programming language are ok. If you have to do break stuff once or twice in the lifetime of your language, fine. But accepting breaking changes as the status-quo is a failure of a language, by definition, IMO.
lib2to3 exists and is used by tools to do this kind of automatic refactoring for python. The idea of having a migration path to the "blessed" way to do things is pretty neat!
As an occasional python developer - I remember you could use `from future import <function>` to use python3 features in python 2.x. perhaps it would be possible to introduce a `past` package to include deprecated language features? of course this would still break compatibility in some cases, but would make it much easier to update them
> perhaps it would be possible to introduce a `past` package to include deprecated language features?
Deprecated language features are, by definition, still present and need no special incantation to miss, though they may produce Deprecation warnings to encourage you to update your code. Removed features would, I guess, be something you could do this for, except that generally the whole benefit of removing features is getting rid of the supporting code and the effort of maintaining it and it's interactions with new code.
Deprecated language features are, by definition, still present and need no special incantation to miss, though they may produce Deprecation warnings to encourage you to update your code. Removed features would, I guess, be something you could do this for, except that generally the whole benefit of removing features is getting rid of the supporting code and the effort of maintaining it and it's interactions with new code.
This is why the language is getting bloated. People shouldn't be so obstinately against breaking changes.
This is quite an alarming attitude that seems to be most common among the Python community. It seems like the 2->3 transition filtered out devs who felt strongly about backwards compatibility and the remaining devs have relatively little patience for objections against breaking changes.
There is a huge benefit to keeping existing code working as it is. Both for businesses and the ecosystem. It benefits everyone if the language does everything it can to keep that intact.
There is a huge benefit to keeping existing code working as it is. Both for businesses and the ecosystem. It benefits everyone if the language does everything it can to keep that intact.
This is the same orange site that thinks the Python 2 to Python 3 breaking changes are still relevant and worth arguing about in 2021.
Yes, f-strings are now the obvious way to format strings _if_ the template is fixed (not variable at runtime) _and_ variables are defined in the current scope. There are many many cases where this does not apply: hence str.format is still required.
greeting = "hello {}"
greet = partial(str.format, greeting)
greet("world")
You can use variables in the formats.
>>> flt = 1.2345
>>> two = 2
>>> f"{flt:.{two}f}"
'1.23'That is the opposite of the problem being posed. The question is how you define `greet` using f-strings.
greet = lambda x: f"hello {x}" greet = lambda x: f"hello {x}"What if "hello {x}" is not a static string but was read from somewhere else?
Surely this would not work:
Surely this would not work:
greeting = "hello {x}"
greet = lambda x: f greetingSomething along the lines of...
code = 'f"{input()} {x}"'
lambda x: eval(code)
code = 'f"here it is {eval(input())}"'
(lambda: eval(code))()
Yes, I'll stick to using str.format :)
That isn't the purpose for f-strings. You still have % and .format
I mean, if you're comfortable with `functools.partial`, lambdas are no great stretch:
greet_fstring = partial(lambda _,s: f"hello {s}", greeting)Why is the example
greet = partial(str.format, greeting)
greet("world")
greeting.format("world")
?The real question is why they didn't just write:
greet = greeting.formatIt was implying you could use the str.format function as argument to other computations, something you cannot do with f-strings.
Or read the string to be formatted from a file, to name another use case.
I believe it's been a cause of care for very easy and expressive string formatting syntax in other languages
I think str.format() was an improvement on from % formatting, and that the f strings are an improvement on str.format(). It's readable, concise, and it's familiar from other scripting languages.
I think the advantage of switch statments and pattern matching is that you know the scope of the thing you're switching or matching on, but you don't have that restriction in if/elif/else blocks. There's nothing keeping me from adding a condition like, `elif day == "Tuesday":`, that has nothing to do with the task at hand. When I see switch or match, I feel like I know what I'm getting into.
I think the advantage of switch statments and pattern matching is that you know the scope of the thing you're switching or matching on, but you don't have that restriction in if/elif/else blocks. There's nothing keeping me from adding a condition like, `elif day == "Tuesday":`, that has nothing to do with the task at hand. When I see switch or match, I feel like I know what I'm getting into.
This argument is totally valid, but I think the problem is that it's universal. It seems like it's equally valid as an argument against any feature. So to really apply it in practice, we need to go further, and come up with some way to quantify the value a new feature brings, so that we can compare that value to the downside of more bloat.
If developers actually knew what simplicity was they'd stop worshipping it.
Brainfuck is an extremely simple language. I mean, it's perfect. Only 8 instructions! Only 1 pointer! Turing complete! Why isn't everyone writing their code in this language?
A bicycle attached to a wheelbarrow is extremely simple transportation. You can get anywhere you need to go, you can carry heavy loads, it's easy to use, easy to fix, cheap. Why would you use anything else?
Wood is a great building material. You can build a building several stories tall, it's cheap, plentiful, easy to work with. We should just standardize all construction on wood.
Water is the simplest beverage. It's clean, healthy, easy to ship, easy to store. We don't need to drink anything else.
Brainfuck is an extremely simple language. I mean, it's perfect. Only 8 instructions! Only 1 pointer! Turing complete! Why isn't everyone writing their code in this language?
A bicycle attached to a wheelbarrow is extremely simple transportation. You can get anywhere you need to go, you can carry heavy loads, it's easy to use, easy to fix, cheap. Why would you use anything else?
Wood is a great building material. You can build a building several stories tall, it's cheap, plentiful, easy to work with. We should just standardize all construction on wood.
Water is the simplest beverage. It's clean, healthy, easy to ship, easy to store. We don't need to drink anything else.
I regularly read cutting edge research papers whose code examples are expressed in lambda calculus, a language from the 1930s with 3 syntax rules and 1 reduction rule.
Not a powerful argument because in 90 years it's still not mainstream and only exists in papers and blog posts. On the other hand pattern matching is widely used. 'Simple' can be 'complicated' in another way, otherwise we'd prefer to write machine code.
> Not a powerful argument because in 90 years it's still not mainstream and only exists in papers and blog posts
https://docs.python.org/3/reference/expressions.html#lambda
https://262.ecma-international.org/11.0/#sec-arrow-function-...
https://docs.oracle.com/javase/specs/jls/se15/html/jls-15.ht...
https://timsong-cpp.github.io/cppwp/n3337/expr.prim.lambda
https://rosettacode.org/wiki/First-class_functions
https://docs.python.org/3/reference/expressions.html#lambda
https://262.ecma-international.org/11.0/#sec-arrow-function-...
https://docs.oracle.com/javase/specs/jls/se15/html/jls-15.ht...
https://timsong-cpp.github.io/cppwp/n3337/expr.prim.lambda
https://rosettacode.org/wiki/First-class_functions
This feature could make a lot of code that people write quite often a fair amount simpler, though. I'm looking forward to this feature quite a bit.
Maybe the problem with C++ is that it tries too hard to preserve backwards compatibility? I think it makes some of the newer features clunkier. If the C++ standards folks did something like Google's Abseil and provided an upgrade tool while discontinuing old features they might have ended up with a better language. I understand something like that might not work for everyone, but from what I've gathered in talking to other devs, there is a strong distaste for the current state of C++.
> Maybe the problem with C++ is that it tries too hard to preserve backwards compatibility?
That's a funny one. The whole history of C++ is being backward compatible, originally with C.
That's a funny one. The whole history of C++ is being backward compatible, originally with C.
It's difficult to balance it out correctly. F-strings were a significant improvement and I still think it was a great idea to include them and trade off some of the language simplicty for the added convenience. This new pattern matching... I'm not convinced yet.
Personally I don't feel f-strings are an improvement at all, but I think I can see why people prefer them. But a significant improvement, that I don't see at all. Can someone explain why e.g.
f'gcd({a}, {b}) = {math.gcd(a, b)}'
'gcd({}, {}) = {}'.format(a, b, math.gcd(a, b))
I find the f-string version better because I read from left to right and find it unnatural when my eyes need to jump around several times over the same expression in order to parse it. In your example that effect isn't too strong because there isn't a lot of additional text in the string and it's very clear what's going on either way.
Sometimes you just want to quote something in a string real quick. In some cases you want to do something slightly more complicated.
And I really dislike format where the arguments are given by position. For short strings it's not going to matter but for long strings you're bound to mess something up sometime. Just imagine writing some kind of long email template and ending up with something like:
And once you insist things are named then f-strings look a lot neater since you get:
And I really dislike format where the arguments are given by position. For short strings it's not going to matter but for long strings you're bound to mess something up sometime. Just imagine writing some kind of long email template and ending up with something like:
'''Dear {}{}
I write you on the {} to inform you...
Kind Regards,
{}'''.format('Mr. ', 'President', '1st of April', 'Ohio', '12', ..., 'Automated mail bot')
And once you insist things are named then f-strings look a lot neater since you get:
f'gcd({a}, {b}) = {math.gcd(a, b)}'
f'gcd({a}, {b}) = {gcd_a_b}'.format(a=a,b=b,gcd_a_b=math.gcd(a, b))
Once you get over five arguments in str.format(), it gets weird. I reverse engineered a product that required a lot of string formatting, and f strings made things more readable and debuggable.
As I see it the less cognitive overhead I have when reading code the better. The more I have to think about the code I am reading the more likely I am to make a mistake because my brain is not a machine. With f" I simply read the code left and right and understand the output as I go. When I see the output I'm interested in I stop and see the context instantly. With .format() I read left and right, stop once I get to what I'm interested in, go back to count the {}s, remember the count, and finally count the arguments. Multiple places to make a mistake. And doing this when refactoring code is how you get bugs.
> In the .format() version, the formatting and the values are separate and that makes it more clear to see both the formatting and the values.
It makes it harder to see which values actually go where though. I really dislike the separation.
It makes it harder to see which values actually go where though. I really dislike the separation.
> I really dislike the separation.
That's obviously a pretty fundamental difference in point-of-view. And it looks like your point-of-view is shared by a large majority.
"The reasonable man adapts himself to the world. The unreasonable man persists in trying to adapt the world to himself. Therefore, all progress depends on the unreasonable man." -- George Bernard Shaw
I try to be reasonable, so I guess I should try to come to terms with f-strings. In any case please don't depend on more progress.
That's obviously a pretty fundamental difference in point-of-view. And it looks like your point-of-view is shared by a large majority.
"The reasonable man adapts himself to the world. The unreasonable man persists in trying to adapt the world to himself. Therefore, all progress depends on the unreasonable man." -- George Bernard Shaw
I try to be reasonable, so I guess I should try to come to terms with f-strings. In any case please don't depend on more progress.
> It makes it harder to see which values actually go where though.
You can name the variables in the format string and then pass the expressions as keyword parameters.
You can name the variables in the format string and then pass the expressions as keyword parameters.
You can do that with f-strings too... but you don't need to because they're generally already clearer.
Thank goodness we still have both ;)
Thank you!
Like you - and I just learned about the existence of f-strings tonight - I find .format() to be much more readable and logical than f-strings.
My first impression reading about f-strings was "ew."
It's not a case of "I don't like it because it's new". To my jaundiced and admittedly subjective eyes, I'm still finding .format() better, precisely because it doesn't lose any clarity in exactly the way you stated.
In other words, I just don't get the point.
Furthermore! f? Really? What the f does f mean? The str() function is the str() function. As well as print(). As well as .format().
f? please
Like you - and I just learned about the existence of f-strings tonight - I find .format() to be much more readable and logical than f-strings.
My first impression reading about f-strings was "ew."
It's not a case of "I don't like it because it's new". To my jaundiced and admittedly subjective eyes, I'm still finding .format() better, precisely because it doesn't lose any clarity in exactly the way you stated.
In other words, I just don't get the point.
Furthermore! f? Really? What the f does f mean? The str() function is the str() function. As well as print(). As well as .format().
f? please
Just wait until you get Python 3.9 and can do:
>>> a, b = 2, 4
>>> f"{math.gcd(a, b) = }"
'math.gcd(a, b) = 2'The first is more readable and performant, the second is much more error prone. One of the things folks complain about in Python is runtime errors, this is one part of the solution.
Is that enough?
Is that enough?
I don't know about performant. If the difference is significant, that can be something to take into account, depending on the situation. Otherwise readability is more important.
So we come to readability, and I have to say that to me .format() is more readable. Honestly.
Regarding runtime errors, I presume you mean .format() doesn't prevent mismatches between the number of placeholders and the number of arguments, as in '{} {}'.format(42)? That's indeed a point in favor of f'', that I hadn't previously considered.
So we come to readability, and I have to say that to me .format() is more readable. Honestly.
Regarding runtime errors, I presume you mean .format() doesn't prevent mismatches between the number of placeholders and the number of arguments, as in '{} {}'.format(42)? That's indeed a point in favor of f'', that I hadn't previously considered.
The eff string uses a special instruction that makes it not need to do expensive function lookups at runtime. These are costly in Python. Better in loops.
Readability is a win in my book. You mention too much is happening in your example string. Ok, perhaps it is. But remember, just because the interpolation supports expressions, doesn't mean you have to put them in there. I'd argue your function call would be better done on the line above, with only the result put in the string. This is the most readable version so far I think:
Yes about placehoders. Also that in some cases of format() each variable is repeated a number of times:
Readability is a win in my book. You mention too much is happening in your example string. Ok, perhaps it is. But remember, just because the interpolation supports expressions, doesn't mean you have to put them in there. I'd argue your function call would be better done on the line above, with only the result put in the string. This is the most readable version so far I think:
c = math.gcd(a, b)
f'gcd({a}, {b}) = {c}'
Yes about placehoders. Also that in some cases of format() each variable is repeated a number of times:
'hello {name}'.format(name=name)
I wouldn't say it is true with string formatting. I would tell a new dev to use the cool new syntax if I saw them using old syntax, but I don't think using the old syntax is painful at all to read.
Already true with string formatting, map/filter versus comprehensions, plenty of other things.
Find me a modern language where that isn't true, though.
Find me a modern language where that isn't true, though.
Obligatory Bjarne Stroustrup quote:
"There are only two kinds of languages: the ones people complain about and the ones nobody uses."
I think it's an interesting point you make, because Python is actually one of the few languages that tried to break away from its past (the whole 2.7 to 3.7 fiasco), but it seems the need for a language to be stable is just much, much greater – so features are just piled onto the existing standard instead.
If COBOL taught me one thing, it's that I'm pretty sure in 60 years time there will still be critical infrastructure running on Python 2.7.
"There are only two kinds of languages: the ones people complain about and the ones nobody uses."
I think it's an interesting point you make, because Python is actually one of the few languages that tried to break away from its past (the whole 2.7 to 3.7 fiasco), but it seems the need for a language to be stable is just much, much greater – so features are just piled onto the existing standard instead.
If COBOL taught me one thing, it's that I'm pretty sure in 60 years time there will still be critical infrastructure running on Python 2.7.
> There are only two kinds of languages: the ones people complain about and the ones nobody uses.
My old boss used to say this thought terminating cliche whenever I encountered a problem with PHP. It's a rather useless form of binary thinking, since it completely ignores scale and severity.
It's similar to statements like "all politicians are bad", which put the likes of Nick Clegg in the same bag as, say, Pol Pot. It's counter-productive to any meaningful discussion.
My old boss used to say this thought terminating cliche whenever I encountered a problem with PHP. It's a rather useless form of binary thinking, since it completely ignores scale and severity.
It's similar to statements like "all politicians are bad", which put the likes of Nick Clegg in the same bag as, say, Pol Pot. It's counter-productive to any meaningful discussion.
I also think using any quote as a thought terminating cliché is flawed. In that case I wouldn't blame the quote though, but the person using it. (I assume your boss probably had an interest in terminating thought. Perhaps the project was invested in the language and it was just not feasible to change? Perhaps he didn't have an answer to your valid criticism? Just speculation...)
In most cases I have encountered people uttering this quote they were using it as a mere observation of the statistical fact that a large group of users will inevitably lead to a larger amount of complaints. And the conclusion was usually something like: "Don't blindly disregard a language, because many people hate it. Always take a closer look if the tool is up to the job."
In most cases I have encountered people uttering this quote they were using it as a mere observation of the statistical fact that a large group of users will inevitably lead to a larger amount of complaints. And the conclusion was usually something like: "Don't blindly disregard a language, because many people hate it. Always take a closer look if the tool is up to the job."
> It's a rather useless form of binary thinking, since it completely ignores scale and severity.
You may also be interested in: "there are 10 kinds of people - those who understand binary, and those who do not"
You may also be interested in: "there are 10 kinds of people - those who understand binary, and those who do not"
[deleted]
The issue was not the breaking changes, the issue was that it was done poorly. In particular that v2 and v3 code could not coexist in the same project.
I think that they did it about as well as they could given the driving motivation behind the change. The semantics of existing syntax changed with Python 3: 'foo' is a string of ascii bytes in Python2, but a string of unicode characters in Python3. Treating a string as both ASCII and Unicode within the same project, depending on which codebase it originated in, seems like a recipe for disaster.
Java => Kotlin is an example of a migration that's relatively painless, by design, and lets you incrementally convert projects over, source file by source file. But the Kotlin designers intentionally left alone certain misfeatures of Java to make that happen. It's still UCS-2 rather than UTF-8, for example, and it syntactically papered over nulls rather than actually fixing them with a true Optional monad like more recent languages, and its coroutine/async/await semantics aren't as elegant as Go or ES6 or Python 3.6. Changing runtime semantics of an existing language ecosystem is hard.
Java => Kotlin is an example of a migration that's relatively painless, by design, and lets you incrementally convert projects over, source file by source file. But the Kotlin designers intentionally left alone certain misfeatures of Java to make that happen. It's still UCS-2 rather than UTF-8, for example, and it syntactically papered over nulls rather than actually fixing them with a true Optional monad like more recent languages, and its coroutine/async/await semantics aren't as elegant as Go or ES6 or Python 3.6. Changing runtime semantics of an existing language ecosystem is hard.
Come on down to golang town. Though I worry when the introduce generics things will spiral out of control.
Go has a different kind of complexity from Python, though. It makes you deal with low-level details like pointers and casting between different-sized integers, none of which are necessary in Python.
I don't really agree. I helped steer my organization to Python more than a decade ago -- its mix of capability and readability made it a great fit, and it remains so. Nothing I saw in this made me worried that people would struggle to read it. F-strings are great, a huge improvement on needing to say .format() all the time, and format was an improvement on the original use of % (more descriptive).
C++ is a wholly different beast. Template metaprogramming achieves great things, but at significant cost to readability.
C++ is a wholly different beast. Template metaprogramming achieves great things, but at significant cost to readability.
That's been true for a very long while - in Python 2.1 (released just shy of 20 years ago), they added "new-style classes", class Foo(object), and for the entire life of Python 2 you weren't supposed to use "old-style classes", class Foo, because they behaved in confusing ways.
(Python 3 got rid of old-style classes and made the two syntaxes equivalent.)
(Python 3 got rid of old-style classes and made the two syntaxes equivalent.)
I need to justify six digits in Germany - we're not the Bay Area.
sorry for the downvote: please propose solutions that allow the language to evolve and not stagnate.
example: pylint-type tools that flag old ways of doing things, and suggest replacement syntax (e.g. https://pycodequ.al/docs/pylint-messages/r1717-consider-usin... )
example: pylint-type tools that flag old ways of doing things, and suggest replacement syntax (e.g. https://pycodequ.al/docs/pylint-messages/r1717-consider-usin... )
I wouldn't treat 'evolution' as an inherently good thing; i.e. change for its own sake.
My first thought whenever someone proposes changing a language is "could that be done with a library instead?". If it's possible then I think libraries are preferable to language changes; if it's not possible and we decide to change the language, then it may be better (if possible) to make a change which fixes whatever prevented us from implementing such a library. Only if we can't do that would I resort to adding individual features to the language.
For example, Scheme has really good pattern matching, despite it not being part of the language. In fact, there are so many pattern matching libraries for Scheme that there are now attempts to standardise their APIs https://srfi.schemers.org/srfi-200/srfi-200.html
My first thought whenever someone proposes changing a language is "could that be done with a library instead?". If it's possible then I think libraries are preferable to language changes; if it's not possible and we decide to change the language, then it may be better (if possible) to make a change which fixes whatever prevented us from implementing such a library. Only if we can't do that would I resort to adding individual features to the language.
For example, Scheme has really good pattern matching, despite it not being part of the language. In fact, there are so many pattern matching libraries for Scheme that there are now attempts to standardise their APIs https://srfi.schemers.org/srfi-200/srfi-200.html
In scheme, or most other Lisps for that matter, you extend the language by writing a library, as the language extension tools are built-in and quite easy to use (macros).
Very few other languages give you that kind of power, and certainly not python.
And packaging!
This has been true for many, many years with "old-style classes".
I agree completely.
I hate how different reading C++0x and C++17 is. I hate reading a "C++ style guide" for a new project.
I already feel that pretty soon, "Python Style Guide" will be a necessary evil...
I hate how different reading C++0x and C++17 is. I hate reading a "C++ style guide" for a new project.
I already feel that pretty soon, "Python Style Guide" will be a necessary evil...
Really? I program Python for over a decade now and compared to other language it was always the one where style somewhat was the smallest issue with that language (compared to the C, C++, C#, Javascript, Java I have seen).
Might be that the whitespace indentations focus the mind. Also: PEPs already act as style guides. And if you are worried to do it wrong, just run it through a code formatter like black. It really isn't that much of a problem IMO.
Might be that the whitespace indentations focus the mind. Also: PEPs already act as style guides. And if you are worried to do it wrong, just run it through a code formatter like black. It really isn't that much of a problem IMO.
I think grandparent is talking about something else. Large C++ codebases often have a guide that says which parts of the language to use and not to use. That can be useful if you have a team with people with different levels of expertise in the language, or expertise in different parts of the language. C++ has grown to be a language with a lot of different parts to choose from.
And it looks like Python is heading in the same direction. It's not difficult to imagine a future where some Python projects are going to want such guides too.
And it looks like Python is heading in the same direction. It's not difficult to imagine a future where some Python projects are going to want such guides too.
Thanks for the clarification.
grandparent?
The parent comment of the parent of my comment, and by extension the writer of that comment.
Hahaha between this, data classes, and the new optional type checkers Python and Ruby are slowly realizing they should have been ML all along.
I'm quite pleased and also laughing my ass off. Vindication!
I'm quite pleased and also laughing my ass off. Vindication!
Prof. Wirth, when designing the Oberon language, adopted this heuristic: a new feature could only be added to the compiler if it reduced the time it took to compile itself.
Now this doesn't quite apply to cPython, of course, and even PyPy is written in RPython, not Python 3, (correct me if I'm wrong on that, eh?) so it doesn't quite apply to that either.
Every time newfangled geld like this comes down the Py3 turnpike I think of all the other Python implementations that either have to implement it also or get left behind: PyPy, Cython, Jython, Brython, Skulpt, Stackless, Pyjs, IronPython, PythonNet, MicroPython, Nuitka, ShedSkin, Transcrypt, etc., not to mention tools like Snakefood or PySonar or MyPy.
(I'll shut up now, you don't want to hear it. I read the tutorial on the new matching and by the end I found myself chanting under my breath "Cthulhu fhtagn! Tekeli-li! Tekeli-li!")
Now this doesn't quite apply to cPython, of course, and even PyPy is written in RPython, not Python 3, (correct me if I'm wrong on that, eh?) so it doesn't quite apply to that either.
Every time newfangled geld like this comes down the Py3 turnpike I think of all the other Python implementations that either have to implement it also or get left behind: PyPy, Cython, Jython, Brython, Skulpt, Stackless, Pyjs, IronPython, PythonNet, MicroPython, Nuitka, ShedSkin, Transcrypt, etc., not to mention tools like Snakefood or PySonar or MyPy.
(I'll shut up now, you don't want to hear it. I read the tutorial on the new matching and by the end I found myself chanting under my breath "Cthulhu fhtagn! Tekeli-li! Tekeli-li!")
Does anyone use Oberon?
"There are dozens of us!"
Yep. You can even get a very nice dev system based on it that's commercially supported: https://astrobe.com/
> Astrobe is a complete integrated embedded software rapid development system running on Windows. It includes a multi-file, fully featured, syntax-aware programmer's editor, native code compiler, linker and builder, program uploader and a serial terminal. Use it to develop reliable software to run on the powerful 32-bit ARM Cortex-M3, Cortex-M4 and Cortex-M7 families of microcontrollers.
You can try out the original Oberon OS workstation in your browser here: https://schierlm.github.io/OberonEmulator/ thanks to a JS emulator for the RISC chip!
Yep. You can even get a very nice dev system based on it that's commercially supported: https://astrobe.com/
> Astrobe is a complete integrated embedded software rapid development system running on Windows. It includes a multi-file, fully featured, syntax-aware programmer's editor, native code compiler, linker and builder, program uploader and a serial terminal. Use it to develop reliable software to run on the powerful 32-bit ARM Cortex-M3, Cortex-M4 and Cortex-M7 families of microcontrollers.
You can try out the original Oberon OS workstation in your browser here: https://schierlm.github.io/OberonEmulator/ thanks to a JS emulator for the RISC chip!
It is too late, but I think they should have adopted the ML family approach of uniform pattern matching (in function clauses and parameters as well). Uniformity is a big deal.
Ok, Scala or Rust got a chance. Partial functions (clauses), currying and patterns complement each other remarkably and it is well-researched and formalized for decades.
Ok, Scala or Rust got a chance. Partial functions (clauses), currying and patterns complement each other remarkably and it is well-researched and formalized for decades.
I’ve followed these things over the years. Had many discussions with people, including a few Python luminaries, but I honestly never expected this day would come. I fell in love with pattern matching when I learned Erlang and Elixir and it’s bugged me ever since that I didn’t have any built in equivalent in Python, until now!
Python 2 had pattern matching on tuples in function arguments; I was quite annoyed, coming from Erlang, when I discovered that was dropped in 3.0, in part because "no one uses it".
This is not truly Erlang style pattern matching. And Python can never have that either because of its design.
> There should be one-- and preferably only one --obvious way to do it.
Unless we keep adding more ways to do the same thing of course.
Unless we keep adding more ways to do the same thing of course.
There have always been 100 ways to do everything in python. People just constantly spout the zen thing to look smart.
There have been multiple ways to do the same thing but that shouldn't be an excuse to keep adding features that don't contribute anything meaningful apart from expanding the syntax.
The zen thing is cited so much because it's often inconsistent the language itself.
The zen thing is cited so much because it's often inconsistent the language itself.
I’m writing a TypeScript package for pattern matching by using generator functions.
https://github.com/RoyalIcing/yieldpattern
It’s amazing to me how much pattern matching, well, matches how I think. A conditional if statement means I have to explicitly read properties off an object or inspect an array to check for what I want. Whereas pattern matching means asking ‘I want something that looks like this’ and do the work for me.
https://github.com/RoyalIcing/yieldpattern
It’s amazing to me how much pattern matching, well, matches how I think. A conditional if statement means I have to explicitly read properties off an object or inspect an array to check for what I want. Whereas pattern matching means asking ‘I want something that looks like this’ and do the work for me.
This looks cool but I want dictionary unpacking (or "destructuring") more than pattern matching, hard to live without it after JS:
>>> {a, b, z} = {"a": 1, "b": 2, "c": 3, "z": 4}
>>> print(a, b, z)
1 2 4
# to get object attrs we can just do
>>> {someval} = SomeObject(someval=42).__dict__()
>>> print(someval)
42Technically I think this feature would do unpacking for you, just more verbosely, like so:
it's more verbose but lets you easily add a default case to catch and handle when a key you expected to be in the dict isn't or the object doesn't actually match the object you expect.
d = {"a": 1, "b": 2, "c": 3, "z": 4}
match d:
case {"a": a, "b": b, "z": z}:
print(a, b, z)
o = SomeObject(someval=42)
match o:
case SomeObject(someval=someval):
print(someval)
it's more verbose but lets you easily add a default case to catch and handle when a key you expected to be in the dict isn't or the object doesn't actually match the object you expect.
`case _:`?
I definitely would have gone for `else:`.
`_` is already used by convention for variables you don't care about, but it's kind mixing things to use it for this too.
I definitely would have gone for `else:`.
`_` is already used by convention for variables you don't care about, but it's kind mixing things to use it for this too.
Not at all since "case _:" is assigning to a variable you do not care about.
If it's actually assigning, then I'm happy!
The tutorials made it look like `_` is a special case, not just what you would expect putting any arbitrary name there to match no matter what.
The tutorials made it look like `_` is a special case, not just what you would expect putting any arbitrary name there to match no matter what.
So the same idea as pattern matching in Scala?
If it can be used as switch case on steroids then I am all for it.
Why Python never got switch case is an interesting question.
If it can be used as switch case on steroids then I am all for it.
Why Python never got switch case is an interesting question.
Switch is a strange relic of C. According to https://en.wikipedia.org/wiki/Switch_statement#Fallthrough the fallthrough behaviour is due to it being implemented as a thin veneer over computed GOTOs. It seems to have infected a surprising number of languages, probably due to a mixture of C's popularity, cargo-culting and Stockholm syndrome.
I'm very glad Python never got C-style switch.
As for Scala, its pattern-matching (among other things!) ultimately comes from ML in the 1970s (its modern descendents being StandardML and Ocaml, and to a lesser extent Scala, Haskell and Rust).
I'm very glad Python never got C-style switch.
As for Scala, its pattern-matching (among other things!) ultimately comes from ML in the 1970s (its modern descendents being StandardML and Ocaml, and to a lesser extent Scala, Haskell and Rust).
> There should be one-- and preferably only one --obvious way to do it.
I honestly don't see what this adds over if statements. You save typing `match ==`? I guess indent block is visually more clear than series of ifs/elifs? That's kinda already solved with pep8 styling of separating logic blocks with newlines.
I honestly don't see what this adds over if statements. You save typing `match ==`? I guess indent block is visually more clear than series of ifs/elifs? That's kinda already solved with pep8 styling of separating logic blocks with newlines.
match response.json()["status"]:
case "500":
raise DeadServer()
case "404":
raise BanServer()
vs
status = response.json()["status"]
if status == "500":
raise DeadServer()
elif status == "404":
raise BanServer()
class Client:
def download(url):
with open_connection() as session:
while True:
response = session.get(url)
match response.json()["status"]:
case "500":
raise BadServer()
case "200":
return response.json()
Pattern matching isn't just about having a switch statement, but about what you can do in the 'case' part of the statement: match complex data and extract variables.
A good example of how pattern matching simplifies code is given in the 'Patterns and Functional Style' section of PEP 635 [1]:
Another example is matching tuples or sequences: the equivalent if statement requires checking the length of the sequence, then extracting the variables, etc. The 'case' statement does all of that in one line.
In other words, as one of my university professors would put it: 'there's nothing that can't be written using plain ifs, adds and jumps, but that doesn't mean we all want to write in assembly with just 3 instructions'.
[1] https://www.python.org/dev/peps/pep-0635/#patterns-and-funct...
A good example of how pattern matching simplifies code is given in the 'Patterns and Functional Style' section of PEP 635 [1]:
match json_pet:
case {"type": "cat", "name": name, "pattern": pattern}:
return Cat(name, pattern)
case {"type": "dog", "name": name, "breed": breed}:
return Dog(name, breed)
case _:
raise ValueError("Not a suitable pet")
if 'type' in json_pet:
if json_pet['type'] == 'cat' and 'pattern' in json_cat:
return Cat(json_pet['name'], json_pet['pattern'])
elif json_pet['type'] == 'dog' and 'breed' in json_pet:
return Dog(json_pet['name'], json_pet['breed'])
raise ValueError("Not a suitable pet")
Another example is matching tuples or sequences: the equivalent if statement requires checking the length of the sequence, then extracting the variables, etc. The 'case' statement does all of that in one line.
In other words, as one of my university professors would put it: 'there's nothing that can't be written using plain ifs, adds and jumps, but that doesn't mean we all want to write in assembly with just 3 instructions'.
[1] https://www.python.org/dev/peps/pep-0635/#patterns-and-funct...
Thanks for the clarification though your example is unpythonic. Here's refactored to be more pythonic and I'd argue it's more readable than pattern match alternative:
pet = json_pet.get('type')
if pet == 'cat':
return Cat(json_pet['name'], json_pet['pattern'])
if pet == 'dog':
return Dog(json_pet['name'], json_pet['breed'])
The difference is the error handling. Suppose the json includes a dog with no breed specified.
As danohuiginn clarified, the reason I'm adding the checks is because assuming there's such a key as 'breed' in the json_pet dictionary can (and will!) lead to KeyError exceptions.
Seems like an issue of data flow then. You usually don't work with dynamic random content and sanitize/clean up your data before working with it.
Even if you were using pre-processed data, pattern matching helps to make the code easier to read. I guess I can think of clearer examples in other languages, like Rust, that supported tagged unions:
// Given this enum
enum NumOrStr {
Num(i32),
Str(string)
}
// That allows you to define values like this
let x = NumOrStr::Num(10);
let y = NumOrStr::Str("hello");
// And some random code that produces a value z that is of type NumOrStr
let z: NumOrStr = ...
match z {
Num(50) => println!("Got my favorite number: fifty!"),
Num(n) => println!('Got {}', n),
String("hello") => say_hello(),
String(s) => do_something_with_string(s)
_ => println!("Default case, x = {:?}", x),
}Matching literal pattern isn't the only thing it adds here.
I think the capturing part is the (more) useful one and can save you more typing than your example.
I think the capturing part is the (more) useful one and can save you more typing than your example.
Well this is some exciting news! algebraic-data-types is my most commonly imported "optional" pip, after the stuff that's strictly necessary for my domain (ie: PyTorch, matplotlib, and such). I guess combined with dataclasses, I can basically finally write ADTs properly in "native" Python.
Ah, nuclear-powered "switch" statement. Good. Like it.
(Yes, I know it isn't "switch"...and yet...)
What remains to be seen is what the execution-time hit might be when compared to other approaches.
(Yes, I know it isn't "switch"...and yet...)
What remains to be seen is what the execution-time hit might be when compared to other approaches.
Does anyone have a usecase where they think this new syntax shines? I'm not really seeing anywhere in my projects that I'd use this, but it seems to be well-loved in Elixir?
They have a dedicated PEP for motivation: https://www.python.org/dev/peps/pep-0635/
Adding pattern matching to a language as big as Python is not only good for Python, but for smaller languages with the feature, as it introduces people to the concept. Imo.
Using _ might be problematic for Django devs, who usually use it for translation, e.g.
from django.utils.translation import gettext as _I'm no Pythonista but a lack of pattern matching is always the first thing I miss when switching back and forth from Scala.
Pattern matching is what I missed most when switching from Swift to Python a year ago. Really happy to see this day come!
Makes me wonder whether to invest time in python or rust going forward.
Different applications but still...my time is limited
Different applications but still...my time is limited
Great. I just hope we won't have to wait 30 more years for immutable (single-assign) values to arrive.
You can test out pattern matching now if you compile Python.
Now I just want multi-lined anonymous functions.
Now I just want multi-lined anonymous functions.
I absolutely love it. Pattern matching is one of my favourite things about ML type languages and I'm glad that it's gradually making its way into other languages.
I don't share the concerns about simplicity. It is easy to read in my opinion and adds a lot of expressiveness in a relatively straight forward way.
I don't share the concerns about simplicity. It is easy to read in my opinion and adds a lot of expressiveness in a relatively straight forward way.
And here I was thinking python was finally adding built-in, native, regex matching to the language, the one last reason I still use perl sometimes because of the quasi java-level verbosity and heavy-handedness of the re python package.
But no, it's just about adding switch to python ... frankly, who cares.
But no, it's just about adding switch to python ... frankly, who cares.
What do you mean with built-in native? Isn't `import re` kinda built-in? It is not an external dependency. So you could even run something like this from the terminal and get 1234 replaced by 2021:
echo "1234foo" | python3 -c 'import re, sys; t=sys.stdin.read(); print(re.sub(r"(\d{4})", "2021", t))'
echo "1234foo" | python3 -c 'import re, sys; t=sys.stdin.read(); print(re.sub(r"(\d{4})", "2021", t))'
In perl:
print($1) if(/^hello (w[^d]+d)/);
Do the same in python and pipe the code to wc -c
Oh, and note the absence of any "import" statement or quotes in the perl code.
Python is not has heavy-handed as C++ for regex parsing, but not by much.
print($1) if(/^hello (w[^d]+d)/);
Do the same in python and pipe the code to wc -c
Oh, and note the absence of any "import" statement or quotes in the perl code.
Python is not has heavy-handed as C++ for regex parsing, but not by much.
I believe they mean adding a literal syntax specifically for regular expressions, ala perl and ruby, so you could write something like:
print(/\d{4}/.sub("2021"))
The backslash makes that example a little ugly, but maybe a delimiter other than "/" would work better. It's too bad that r"" is already taken for raw strings.
print(/\d{4}/.sub("2021"))
The backslash makes that example a little ugly, but maybe a delimiter other than "/" would work better. It's too bad that r"" is already taken for raw strings.
Could be x"" for regeX or p"" for Pattern
This was my train of thought as well.
I think I'm ambivalent towards this. Handling different types of inputs like that is kind of fun. I don't really expect to need to know this one looking at coworkers code in the wild.
I think I'm ambivalent towards this. Handling different types of inputs like that is kind of fun. I don't really expect to need to know this one looking at coworkers code in the wild.
wonderful! hopefully this addition helps people to wake to the reality that python is a convoluted , unprincipled, unintelligent sluggish steaming pile of turd of a language . Python and javascript need to die.
[deleted]
things like this and f-strings make me want to use python more and more. if only i could have a bit better OOP syntax.
What’s wrong with the OOP syntax?
Syntax for writing constructors, getters/setters and information hiding is lacking.
This makes my year
I am so excited for this.
I love Python, but as of 2016, I was missing:
0. Pattern matching 1. Easy support for types & type-checking (thanks mypy + annotations) 2. Better string formatting (yay f-strings)
And the nice part is that these are all relatively opt-in features. So excited to be able to using pattern matching as part of the language!
I love Python, but as of 2016, I was missing:
0. Pattern matching 1. Easy support for types & type-checking (thanks mypy + annotations) 2. Better string formatting (yay f-strings)
And the nice part is that these are all relatively opt-in features. So excited to be able to using pattern matching as part of the language!
...
I'll take my down votes. Please move on opinionated HN-ers.
I'll take my down votes. Please move on opinionated HN-ers.
By that same logic wouldn’t
def f(x):
[deleted]
[deleted]
[deleted]
So, regardless of one's feelings on this particular feature, I think we can see what the loss of a BDFL means: feature bloat. It was once one of the core tenets of python that there be one, best way to do a given thing, because that meant that when you read _another_ programmer's code, you could usually understand it easily. This is as opposed to languages like Perl, javascript, or R, which for all their great traits are each essentially several different languages masquerading as a one.
It's probably inevitable, once you do not have a single person to say no to new features for things that can already be done, that you end up with feature bloat. So, no need to rant about it. But it does effectively demonstrate what the impact of a BDFL is, and what is missing once you don't have one (or at any rate he's not really D any more).
It's probably inevitable, once you do not have a single person to say no to new features for things that can already be done, that you end up with feature bloat. So, no need to rant about it. But it does effectively demonstrate what the impact of a BDFL is, and what is missing once you don't have one (or at any rate he's not really D any more).
Did you notice that Guido was one of the authors of the pattern matching PEP?
ha! Nope!
Direct links to the PEPs themselves, to perhaps save others a little time. Although the LWN write up linked in TFA is a very nice introduction to the topic and its big discussion in the community.
Accepted:
Specification https://www.python.org/dev/peps/pep-0634/
Motivation and Rationale https://www.python.org/dev/peps/pep-0635/
Tutorial https://www.python.org/dev/peps/pep-0636/
Rejected:
Unused variable syntax https://www.python.org/dev/peps/pep-0640/
Explicit Pattern Syntax for Structural Pattern Matching https://www.python.org/dev/peps/pep-0642/
¹ https://news.ycombinator.com/item?id=26073700