The lack of a case against Python 3(blog.lerner.co.il)
blog.lerner.co.il
The lack of a case against Python 3
http://blog.lerner.co.il/case-python-3/
155 comments
Actually, I think the strictness of python3 with regards to bytes/strings is particularly useful for people who speak English as their first language. python3 will force them into writing software that can work with international strings whereas python2 code written by an English speaking person would in many cases just be broken for the international users.
And believe me, as an "international" person, this is profoundly annoying and has trained me in avoiding using the non-ASCII letters of my language which of course "castrates" the written language somewhat (i.e. When people use non-ASCII letters for folder names I think they are just asking for trouble - how normal is that?)
I love how upfront python3 is with the difference between raw bytes and strings. I work a lot with python and in particular I do a lot of work with serializing and deserializing stuff from/to binary blobs. Due to platform issues that I hope will change in the next year or so I can't use python3 currently but I keep all my code compatible with both and all tests should run with python2 and python3. Yes, I am firmly in the python3 camp :)
pycharm inspection definitely helps with keeping code compatible with 2 and 3.
And believe me, as an "international" person, this is profoundly annoying and has trained me in avoiding using the non-ASCII letters of my language which of course "castrates" the written language somewhat (i.e. When people use non-ASCII letters for folder names I think they are just asking for trouble - how normal is that?)
I love how upfront python3 is with the difference between raw bytes and strings. I work a lot with python and in particular I do a lot of work with serializing and deserializing stuff from/to binary blobs. Due to platform issues that I hope will change in the next year or so I can't use python3 currently but I keep all my code compatible with both and all tests should run with python2 and python3. Yes, I am firmly in the python3 camp :)
pycharm inspection definitely helps with keeping code compatible with 2 and 3.
> When people use non-ASCII letters for folder names I think they are just asking for trouble
Well they are
> how normal is that?
Not as normal as it ought to be.
Folder names (and file names) are identifiers. Identifiers should identify. If you allow unicode characters, then you can have two character sequences that look the same but are actually distinct. This is confusing at best and at worst (in URLs) could facilitate fraud.
Well they are
> how normal is that?
Not as normal as it ought to be.
Folder names (and file names) are identifiers. Identifiers should identify. If you allow unicode characters, then you can have two character sequences that look the same but are actually distinct. This is confusing at best and at worst (in URLs) could facilitate fraud.
> If you allow unicode characters, then you can have two character sequences that look the same but are actually distinct.
That would be a software bug. If you want to compare unicode strings, you need to normalize them first following the rules laid out in the standard. https://en.wikipedia.org/wiki/Unicode_equivalence
git for example fails that test. (try creating a repo with a file named 'ü' and check it out both on a mac and a linux system)
There are more issues about glyphs that are a distinct character but look the same in a given font, but what's your proposal? All people transliterate everything to ascii? Display punycode in URLs?
That would be a software bug. If you want to compare unicode strings, you need to normalize them first following the rules laid out in the standard. https://en.wikipedia.org/wiki/Unicode_equivalence
git for example fails that test. (try creating a repo with a file named 'ü' and check it out both on a mac and a linux system)
There are more issues about glyphs that are a distinct character but look the same in a given font, but what's your proposal? All people transliterate everything to ascii? Display punycode in URLs?
> Identifiers should identify
Except that, without Unicode, you now have to use a slighly restricted subset of many European languages and cannot use Cyrillic, Chinese etc at all. So people who are not operating in an English environment are limited in their ability to clearly identify things.
Except that, without Unicode, you now have to use a slighly restricted subset of many European languages and cannot use Cyrillic, Chinese etc at all. So people who are not operating in an English environment are limited in their ability to clearly identify things.
[deleted]
"then you can have two character sequences that look the same but are actually distinct."
If you accept that argument, you should also remove either 1 or l, O or 0, S or 5 etc.
If you accept that argument, you should also remove either 1 or l, O or 0, S or 5 etc.
> Rough estimate 10% of the world speak English as a first language
ASCII isn't even good enough for that 10% – British people need £, which isn't in ASCII – not to mention things like em dashes, en dashes, and curly quotation marks that word processors tend to add.
ASCII isn't even good enough for that 10% – British people need £, which isn't in ASCII – not to mention things like em dashes, en dashes, and curly quotation marks that word processors tend to add.
They're in latin-1 though, which is a byte-per-character encoding.
I don't think anyone who deals with character encodings should encourage latin-1 in 2016 instead of utf8. Fair enough, your old C code will work, but everyone else will hate you.
Except, of course, if you deal with Windows...
Except, of course, if you deal with Windows...
Kids these days also use emoji.
Emoji have been part of Unicode since 1993 (e.g. ️)
I loved python2, but now that I'm in a company where everything is python3 I probably save a few days a year of not mucking about with Unicode BS. It's wonderful. There may be other advantages but even if it were JUST "python2 without horrible Unicode idiosyncrasies" it would be worth it.
> I doubt that only 10% of people are dealing with unicode. Rough estimate 10% of the world speak English as a first language and almost all other languages are written with (at least some) non-ascii characters.
I believe he meant people that have to actually deal with complications of unicode/bytestring themselves.
For most uses, you just handle "strings", and don't really care if they're unicode or not. Only lower-level libraries/apps will ever care if these strings are unicode or not.
I believe he meant people that have to actually deal with complications of unicode/bytestring themselves.
For most uses, you just handle "strings", and don't really care if they're unicode or not. Only lower-level libraries/apps will ever care if these strings are unicode or not.
Yup, that's what I meant!
> I doubt that only 10% of people are dealing with unicode.
I will use a single character to express my absolute agreement:
Edit: I guess even Lisp developers have to consider Unicode.
I will use a single character to express my absolute agreement:
Edit: I guess even Lisp developers have to consider Unicode.
True.
Even if your Lisp supports unicode, it does not make a web framework/application support unicode just so.
Disclaimer: I've added UTF-8 support to a Lisp-based web framework, on top of a Lisp implementation supporting Unicode.
Even if your Lisp supports unicode, it does not make a web framework/application support unicode just so.
Disclaimer: I've added UTF-8 support to a Lisp-based web framework, on top of a Lisp implementation supporting Unicode.
> Edit: I guess even Lisp developers have to consider Unicode.
Why wouldn't they?
Why wouldn't they?
To explain/spoil the joke: HN does not support most of Unicode and is written in a Lisp variant.
Thanks, I clearly didn't get it :-)
Does anyone know what underlying differences there are in Python 3 unicode handling versus Go's rune system. I have found the latter far easier to work with when dealing with text characters in other languages.
This idea that some developers don't need to deal with unicode, really needs to die.
If you're creating a web app and don't support unicode input, you're doing it wrong. You probably can't accurately store the names of your customers, for starters.
If you're creating a web app and don't support unicode input, you're doing it wrong. You probably can't accurately store the names of your customers, for starters.
OP here. I teach Python to Fortune 100 companies, more or less every day.
You're totally right that people who deal with outside-world data need Unicode. Anything that deals with users MUST deal with Unicode.
But a huge proportion of the people I work with -- in Israel, China, and Europe -- couldn't care less about Unicode. That's because they're processing logfiles, dealing with test automation and system administration. For them, Unicode is important a very small proportion of the time.
So I stand by my assertion that a small proportion of developers need to worry about Unicode. But I also agree with you that Unicode is absolutely crucial in the modern world, and that a language lacking good Unicode support isn't going to get very far.
You're totally right that people who deal with outside-world data need Unicode. Anything that deals with users MUST deal with Unicode.
But a huge proportion of the people I work with -- in Israel, China, and Europe -- couldn't care less about Unicode. That's because they're processing logfiles, dealing with test automation and system administration. For them, Unicode is important a very small proportion of the time.
So I stand by my assertion that a small proportion of developers need to worry about Unicode. But I also agree with you that Unicode is absolutely crucial in the modern world, and that a language lacking good Unicode support isn't going to get very far.
> That's because they're processing logfiles, dealing with test automation and system administration ... So I stand by my assertion that a small proportion of developers need to worry about Unicode.
You come to this conclusion based on your dealings with a small subset of developers within a small subset of companies?
I think many other developers in the fortune 500 companies outside of the subset you deal with would laugh at that conclusion, then go back to developing apps that deal with people/locales all over the world.
You come to this conclusion based on your dealings with a small subset of developers within a small subset of companies?
I think many other developers in the fortune 500 companies outside of the subset you deal with would laugh at that conclusion, then go back to developing apps that deal with people/locales all over the world.
This also describes my entire team. And the team before that. There are at least as many developers out there who don't write user facing code as those who do.
When I do run across Unicode, it's just a matter of passing it through, which is usually pretty easy to do.
When I do run across Unicode, it's just a matter of passing it through, which is usually pretty easy to do.
Fair enough. My sample is from lots of companies, but a small subset of tasks there.
Just because your users are dealing with logfiles, their users should still be able to use č ƒ ø or whatever unicode character in the filename or url - which will consequently end up in a log file.
True story: I once broke build automation by including non-ASCII text in the commit message. Just because something is internal doesn't mean that it won't have Unicode.
Most logs are actually bytestrings. If they include user-controlled strings, they will likely have to be parsed as binary blobs to accommodate that.
Many logs are localised - and contain unicode.
There's a difference though. Does the log format itself in some unicode encoding, or does it just embed some utf-X encoded strings? If you just embed some user input and assume it will decode properly, you're likely to run into issues at some point when someone provides explicitly-broken encoding.
Pro tip: Use non-ascii characters in your test data. Instead of John Smith, try Helmut Müller. This will make sure your code is unicode safe, and also make sure that external dependencies (hey, our spreadsheet library silently fails when a ü is in the file name!) also work correctly with unicode.
German Umlauts are not a good test vector, because various SBCS have them as well (latin-1 and others), and can slip through systems not correctly processing Unicode.
Use CJK instead, eg. 나윤선, which will just explode into an obvious gooey mess if the processing pipeline isn't handling Unicode correctly somewhere.
Use CJK instead, eg. 나윤선, which will just explode into an obvious gooey mess if the processing pipeline isn't handling Unicode correctly somewhere.
Also, you can litter your test data with all the fancy emoji you can type in a modern OS these days, those are almost always instantly recognizable when correct versus incorrect and test a fun number of things given:
1) Most (but not all) emoji exist way out in the Astral Plane (a good stress test for UTF-8/UTF-16 handling and little/big endian issues)
2) Emoji can contain "exotic" sequences like Zero Width Joiners (this can be very important to avoid codepoint counting issues because a ZWJ sequence of codepoints is a very different emoji if the codepoints are incorrectly reversed or spliced)
3) Emoji can test your font display stack for inefficiencies in system font fallback (and support for modern typography in general)
1) Most (but not all) emoji exist way out in the Astral Plane (a good stress test for UTF-8/UTF-16 handling and little/big endian issues)
2) Emoji can contain "exotic" sequences like Zero Width Joiners (this can be very important to avoid codepoint counting issues because a ZWJ sequence of codepoints is a very different emoji if the codepoints are incorrectly reversed or spliced)
3) Emoji can test your font display stack for inefficiencies in system font fallback (and support for modern typography in general)
I use a site like http://lingojam.com/FancyTextGenerator to generate 𝖚𝖘𝖊𝖋𝖚𝖑 𝖙𝖊𝖘𝖙 𝖉𝖆𝖙𝖆.
The only downside is that it has taught me that my favourite debugger is not unicode safe :(
The only downside is that it has taught me that my favourite debugger is not unicode safe :(
I like to use "Tokyo" in kanji, since I can remember what the characters look like: 東京
Is that a good test?
Is that a good test?
Many programmers don't need to deal with Unicode simply because they aren't dealing with natural language text in the first place. Programs that need to support Unicode input are precisely the ones that have natural language text input.
I do primarily numerical work with a little bit of embedded mixed in, it's been closing in on three years since I encountered anything involving non-ascii characters.
yes, in webapps you almost always import from future unicode literals in py2 apps.
People still stuck in python2 and arguing that python2 is better than python3 is weird.
I get it, more features does not mean better and newer does not mean better but seriously, just do the jump to python3. It's not that much work and the language is much nicer to work with.
I've built a library with python2/3 support and do most of my work in python3. Maybe my web dev background biases me somehow, but I can't remember the last time I had problems with a major library not supporting python3.
To me this seems more like tradition/age/generation -related. Those who started back in python1/2 are the ones with problems moving to python3, while young people who started with python 3.x are already running 3.5/3.6.
ps. like the author said, there will always be large corporations stuck in very old techs - that doesn't mean the rest of the world too has to use Cobol or MUMPS.
I get it, more features does not mean better and newer does not mean better but seriously, just do the jump to python3. It's not that much work and the language is much nicer to work with.
I've built a library with python2/3 support and do most of my work in python3. Maybe my web dev background biases me somehow, but I can't remember the last time I had problems with a major library not supporting python3.
To me this seems more like tradition/age/generation -related. Those who started back in python1/2 are the ones with problems moving to python3, while young people who started with python 3.x are already running 3.5/3.6.
ps. like the author said, there will always be large corporations stuck in very old techs - that doesn't mean the rest of the world too has to use Cobol or MUMPS.
> but seriously, just do the jump to python3.
Or jump to another language.
Here's my problem: I don't program unless I get paid, and I simply cannot sell this crazy idea: "hey, you know all that software you paid me to write? how about you pay me again so you can have it in python3?"
It seems like if I want my programs to run for more than ten years I shouldn't be using Python.
Or jump to another language.
Here's my problem: I don't program unless I get paid, and I simply cannot sell this crazy idea: "hey, you know all that software you paid me to write? how about you pay me again so you can have it in python3?"
It seems like if I want my programs to run for more than ten years I shouldn't be using Python.
Sounds like you're pitching a complete rewrite, which absolutely isn't the case.
Six works great and is simple to push through for side-by-side compatibility. That combined with `from __future__` for Py2.7 allows most codebases to work with small changes in a side-by-side manner.
Six works great and is simple to push through for side-by-side compatibility. That combined with `from __future__` for Py2.7 allows most codebases to work with small changes in a side-by-side manner.
IMO that's the wrong approach. The correct one would be: "Hey, you know all the software you want to pay me to write? It will be in Python 3 now".
The other, complementary correct approach is "Hey, you know all the software we paid you to write? We'll pay you to port it to Python 3 because we don't like running two different interpreters".
And with tools and decent Unicode handling etc. in the original version, the update will be done inexpensively and on time.
I consider Python 3 a huge benefit for a similar reason: I can avoid or charge $$$ for people still using 2.6 (e.g. on ancient RHEL). There's always somebody. 2.6 is pretty sucky, not to mention unsupported by the PSF.
If you support 2.7, some "clever" manager or client will say that 2.6 support surely isn't so unreasonable. It is. 2.7 is legacy only now IMO.
If you support 2.7, some "clever" manager or client will say that 2.6 support surely isn't so unreasonable. It is. 2.7 is legacy only now IMO.
I'm happily using and deploying Python 3.6 on CentOS 6 using pyenv. You don't have to use the system Python unless there's a policy requiring you to do so.
I agree, and EPEL works fine on RHEL, too. But it wouldn't be enterprise without some BS rules. Hence why compatibility "issues" are such a blessing for detecting enterprise BS early on.
You can get a more recent Python from Software Collections, too:
https://www.softwarecollections.org/en/scls/?search=python
Up to Python 3.5 and 3.6 will probably follow shortly after release.
This is blessed and QA'd by Red Hat themselves, so it's as good as it gets.
"My distro only has Python 2" has to be one of the worst excuses not to use Python 3 :-)
Obvious exception are systems tools (did you know that Ansible modules have to compatible with Python 2.4 since that's what CentOS 5 uses?). But those are the absolute minority.
https://www.softwarecollections.org/en/scls/?search=python
Up to Python 3.5 and 3.6 will probably follow shortly after release.
This is blessed and QA'd by Red Hat themselves, so it's as good as it gets.
"My distro only has Python 2" has to be one of the worst excuses not to use Python 3 :-)
Obvious exception are systems tools (did you know that Ansible modules have to compatible with Python 2.4 since that's what CentOS 5 uses?). But those are the absolute minority.
I'm not going to work for free.
You are not reading what the person you are replying to is saying.
I said I cannot invoice the customer "convert perfectly good codebase to python3", and I said I'm not working for free.
The only other option I can see you might be suggesting is to defraud my customer by converting that codebase to python3 but say I was doing something else.
I don't want the reputation of being one of those people.
The only other option I can see you might be suggesting is to defraud my customer by converting that codebase to python3 but say I was doing something else.
I don't want the reputation of being one of those people.
I believe the path being suggested is: write new software in Python 3, or at least in a 3-friendly way.
That's easy to say, but Python3 isn't good enough:
* Python3 is slower than Python2
* Twisted still doesn't work in Python3
* Division changed.
And so on. I'm not improving Python3 since I don't get any return on it, so I'll simply use a different language for new projects.
However you might want to reconsider: Other people parsed it the same way -- and one person even thought it was a good thing that they got to charge again.
* Python3 is slower than Python2
* Twisted still doesn't work in Python3
* Division changed.
And so on. I'm not improving Python3 since I don't get any return on it, so I'll simply use a different language for new projects.
However you might want to reconsider: Other people parsed it the same way -- and one person even thought it was a good thing that they got to charge again.
Python 3 is not slower than Python 2, you've already been corrected elsewhere in the comments and disregarded it. Python 3.0 was slower than Python 2, but we're up to 3.6 now.
> Division changed.
For the better. I don't get you.
> Division changed.
For the better. I don't get you.
> you've already been corrected elsewhere in the comments and disregarded it
Link please?
Edit: Do you mean this other person[1] who has a performance complaint?
[1]: https://news.ycombinator.com/item?id=13062068
Do you really think being aggressive and bullish is going to convince people to stay with Python? Honestly, I'm considering using screenshots of this conversation as to why my new customer shouldn't invest in Python: Immature and insecure community, ignorant of the problems that companies face.
Link please?
Edit: Do you mean this other person[1] who has a performance complaint?
[1]: https://news.ycombinator.com/item?id=13062068
Do you really think being aggressive and bullish is going to convince people to stay with Python? Honestly, I'm considering using screenshots of this conversation as to why my new customer shouldn't invest in Python: Immature and insecure community, ignorant of the problems that companies face.
So to recap: You hop into this conversation claiming that you can't upgrade to Python 3 because you're not being paid for it. Someone else tells you that the correct approach is to think of it as being unnecessary to port old software. After you double down on your "I'm not working for free!" strawman, I point out that you're not reading what people are saying to you. You triple-down on your assumption and when you finally realize your mistake, you complain about division changing.
And when I point out that that's nonsense, you call me "aggressive and bullish".
Uh, yeah, you wanna share this conversation around be my guest. What I'm getting is you care a lot less about the "problems companies face" than about getting paid.
And when I point out that that's nonsense, you call me "aggressive and bullish".
Uh, yeah, you wanna share this conversation around be my guest. What I'm getting is you care a lot less about the "problems companies face" than about getting paid.
> So to recap: You hop into this conversation claiming that you can't upgrade to Python 3 because you're not being paid for it.
No, I can't upgrade to Python3 because there is no upgrade path.
> Someone else tells you that the correct approach is to think of it as being unnecessary to port old software.
Because there's no upgrade path.
Python2 ends at some point in the near future, and if they want to be extended they will need to be rewritten.
> After you double down on your "I'm not working for free!" strawman, I point out that you're not reading what people are saying to you.
And I respond by saying: yes they are actually saying it. Here's an example:
I consider Python 3 a huge benefit for a similar reason: I can avoid or charge $$$ for people still using 2.6
and:
I do hope other language communities have learned from the Python2 Python3 fiasco. Dealing with no new features in a version with millions of lines of code already written, enough syntactical and other issues that require large underlying changes, etc.
and:
I believe the path being suggested is: write new software in Python 3, or at least in a 3-friendly way.
Which really was you saying, don't worry about software that's already been written. It's dead. Write new software instead!
> You triple-down on your assumption and when you finally realize your mistake, you complain about division changing (and all the libraries that don't work, and the performance issue). And when I point out that that's nonsense, you call me "aggressive and bullish".
You have a choice here: You can try to understand that I'm frustrated by my position, and that other people are frustrated by this position, and instead of trying to denigrate my position, try to find an actual solution for it.
Or you can simply wave your hands, claim there's no solution, and watch all the python2 programmers decide between python3 and anything else.
> What I'm getting is you care a lot less about the "problems companies face" than about getting paid.
Listen: You don't have to work for free. Companies that would fail unless you work for free will probably fail anyway. It means they aren't thinking about what they actually need software for.
We have substantial number of C programs that exist for twenty-five years or more, and we barely have insubstantial python programs that can last for more than a couple. If you don't understand how it's not in anyone's best interests to keep rewriting business functions over and over again, then you really shouldn't be in this conversation, because you clearly think this is about something else.
No, I can't upgrade to Python3 because there is no upgrade path.
> Someone else tells you that the correct approach is to think of it as being unnecessary to port old software.
Because there's no upgrade path.
Python2 ends at some point in the near future, and if they want to be extended they will need to be rewritten.
> After you double down on your "I'm not working for free!" strawman, I point out that you're not reading what people are saying to you.
And I respond by saying: yes they are actually saying it. Here's an example:
I consider Python 3 a huge benefit for a similar reason: I can avoid or charge $$$ for people still using 2.6
and:
I do hope other language communities have learned from the Python2 Python3 fiasco. Dealing with no new features in a version with millions of lines of code already written, enough syntactical and other issues that require large underlying changes, etc.
and:
I believe the path being suggested is: write new software in Python 3, or at least in a 3-friendly way.
Which really was you saying, don't worry about software that's already been written. It's dead. Write new software instead!
> You triple-down on your assumption and when you finally realize your mistake, you complain about division changing (and all the libraries that don't work, and the performance issue). And when I point out that that's nonsense, you call me "aggressive and bullish".
You have a choice here: You can try to understand that I'm frustrated by my position, and that other people are frustrated by this position, and instead of trying to denigrate my position, try to find an actual solution for it.
Or you can simply wave your hands, claim there's no solution, and watch all the python2 programmers decide between python3 and anything else.
> What I'm getting is you care a lot less about the "problems companies face" than about getting paid.
Listen: You don't have to work for free. Companies that would fail unless you work for free will probably fail anyway. It means they aren't thinking about what they actually need software for.
We have substantial number of C programs that exist for twenty-five years or more, and we barely have insubstantial python programs that can last for more than a couple. If you don't understand how it's not in anyone's best interests to keep rewriting business functions over and over again, then you really shouldn't be in this conversation, because you clearly think this is about something else.
Be careful with that, you don't exactly come off smelling like roses in this conversation...
So you really want to bank your career on a petty argument that started because you failed to read properly, then continued because you can't admit you were wrong?
Good luck in your future endeavors...
Good luck in your future endeavors...
Where exactly was I wrong?
I said[1] there's a point where existing programs can't be extended. Python2 programmers have to decide whether they're choosing python3 or something else.
And then there's this surreal suggestion that I recommend replacing python2 programs with python3 programs[2] (even after I said I can't convince anyone to do it), while simultaneously not replacing python2 programs with python3 programs[3].
I'm trying to give you guys the benefit of the doubt, but I don't even think you're trying to have a conversation, but justify your decision that you've already made to support python3.
[1]: https://news.ycombinator.com/item?id=13062079
[2]: https://news.ycombinator.com/item?id=13062349
[3]: https://news.ycombinator.com/item?id=13062151
I said[1] there's a point where existing programs can't be extended. Python2 programmers have to decide whether they're choosing python3 or something else.
And then there's this surreal suggestion that I recommend replacing python2 programs with python3 programs[2] (even after I said I can't convince anyone to do it), while simultaneously not replacing python2 programs with python3 programs[3].
I'm trying to give you guys the benefit of the doubt, but I don't even think you're trying to have a conversation, but justify your decision that you've already made to support python3.
[1]: https://news.ycombinator.com/item?id=13062079
[2]: https://news.ycombinator.com/item?id=13062349
[3]: https://news.ycombinator.com/item?id=13062151
If you write new software in Python 3, why does it matter if division changed?
First, it presupposes I'm going to write new software in Python 3 rather than justifying why I should write new software in python 3.
Second, it matters because I'm thinking about python4 and python5. I've already been through this and got burned moving from python1.5 to python2, and python3 has been very uncomfortable thus far.
Thirdly, it matters because I work in a lot of mature (multi-decade) codebases, and figuring out what division is supposed to mean twenty five years into development is disconcerting.
Then when I'm simply not convinced python is a good investment anymore, this quippy attitude here is extremely off-putting.
Do you really want to know why I don't like python as much as I used to? Or are you trying to justify your decision to further invest?
Second, it matters because I'm thinking about python4 and python5. I've already been through this and got burned moving from python1.5 to python2, and python3 has been very uncomfortable thus far.
Thirdly, it matters because I work in a lot of mature (multi-decade) codebases, and figuring out what division is supposed to mean twenty five years into development is disconcerting.
Then when I'm simply not convinced python is a good investment anymore, this quippy attitude here is extremely off-putting.
Do you really want to know why I don't like python as much as I used to? Or are you trying to justify your decision to further invest?
> First, it presupposes I'm going to write new software in Python 3 rather than justifying why I should write new software in python 3.
It does, because that’s what you were responding to:
> I believe the path being suggested is: write new software in Python 3, or at least in a 3-friendly way.
I assumed your list was a list of reasons not to write new software in Python 3, and found division oddly out of place.
It does, because that’s what you were responding to:
> I believe the path being suggested is: write new software in Python 3, or at least in a 3-friendly way.
I assumed your list was a list of reasons not to write new software in Python 3, and found division oddly out of place.
> It does, because that’s what you were responding to: I believe the path being suggested is: write new software in Python 3, or at least in a 3-friendly way.
You're mistaken. Start at the top of the thread again[1].
The question is whether I (or anyone else) should even bother with Python3.
[1]: https://news.ycombinator.com/item?id=13061570
You're mistaken. Start at the top of the thread again[1].
The question is whether I (or anyone else) should even bother with Python3.
[1]: https://news.ycombinator.com/item?id=13061570
Another language? Do you mean that Ruby, or Go, or Rust, or Javascript are less likely to force you to another jump in the next 20 years?
I dislike the new print in py3, but still the python community do take things seriously and are not switch horses every 6 months. I also do Javascript and, well, I'd love the community there to hold on something and actually dig it for more than 6 months.
I dislike the new print in py3, but still the python community do take things seriously and are not switch horses every 6 months. I also do Javascript and, well, I'd love the community there to hold on something and actually dig it for more than 6 months.
Ruby is actually a good comparison: It has the same kind of switchover, ruby 1.8 didn't support unicode while ruby 1.9 and newer does. The changes happened roughly at the same time as the py2/3 switchover, but happened much much faster. No incompatible syntax changes happened, no semi-automatic translation between 1.8 and 1.9. A lot of libraries worked (and still do) flawlessly on 1.8 and 1.9, many - including rails - offered compatibility relatively early since ruby 1.9 offered significant benefits over 1.8. There were websites and tools listing the compatibility of your favorite gems[1].
So yes, it is definitely possible to make such changes more gradual and effective both by providing the technical means and by assembling sufficient community support.
[1] https://github.com/seattlerb/rubygems-isit19
So yes, it is definitely possible to make such changes more gradual and effective both by providing the technical means and by assembling sufficient community support.
[1] https://github.com/seattlerb/rubygems-isit19
Javascript in particular? Yes, it's less likely to force a jump. It has to stay compatible with the web as it exists today and has existed for the last 20 years. Idiomatic JS will certainly change, but that doesn't force old code to change.
Ruby and Go live on the server side, so do not have nearly as harsh a backwards compatibility constraint.
Come to think of it, I wonder if JS's backcompat story is precisely what has resulted in the intense rate of churn in the libraries and frameworks written in it. Maybe if the language forced periodic change, then that would both reduce the frameworks' evolutionary pressure to change, and increase frameworks' evolutionary pressure to paper over JS's changes. As it is now, the architecture seems to be that you have an unchanging language layer so you put all of the change into the next layer up. (Well, that and the client-side delivery and execution means that framework updates are far far easier than with any server-side language/framework.)
Ruby and Go live on the server side, so do not have nearly as harsh a backwards compatibility constraint.
Come to think of it, I wonder if JS's backcompat story is precisely what has resulted in the intense rate of churn in the libraries and frameworks written in it. Maybe if the language forced periodic change, then that would both reduce the frameworks' evolutionary pressure to change, and increase frameworks' evolutionary pressure to paper over JS's changes. As it is now, the architecture seems to be that you have an unchanging language layer so you put all of the change into the next layer up. (Well, that and the client-side delivery and execution means that framework updates are far far easier than with any server-side language/framework.)
>It seems like if I want my programs to run for more than ten years I shouldn't be using Python
I do hope other language communities have learned from the Python2 Python3 fiasco. Dealing with no new features in a version with millions of lines of code already written, enough syntactical and other issues that require large underlying changes, etc.
I do hope other language communities have learned from the Python2 Python3 fiasco. Dealing with no new features in a version with millions of lines of code already written, enough syntactical and other issues that require large underlying changes, etc.
> Those who started back in python1/2 are the ones with problems moving to python3, while young people who started with python 3.x are already running 3.5/3.6.
I'm not sure this argument works in your favour. You appear to be saying that the people who like Python 3 are those who have no experience of anything else, whereas the people who have experienced other things are reluctant to use it. You'd have to give at least a little credence to the worries of the experienced people.
I'm not sure this argument works in your favour. You appear to be saying that the people who like Python 3 are those who have no experience of anything else, whereas the people who have experienced other things are reluctant to use it. You'd have to give at least a little credence to the worries of the experienced people.
Except I know plenty of experienced Python 2 users (like myself) who moved to 3.x and like it. The point being made, I believe, is that the only reason to prefer 2 is if you are locked into it for historical reasons.
The differences are small, and all the arguments against 3 are really just 'breaking changes suck' - yes, they do, but it turns out that we aren't perfect and language designs aren't perfect - you either make breaking changes or live with bad designs forever.
Python made those breaking changes - the alternative just means that in X years we'll move to an entirely different language that does things better instead of moving to Python 3, and at least having an upgrade path.
The differences are small, and all the arguments against 3 are really just 'breaking changes suck' - yes, they do, but it turns out that we aren't perfect and language designs aren't perfect - you either make breaking changes or live with bad designs forever.
Python made those breaking changes - the alternative just means that in X years we'll move to an entirely different language that does things better instead of moving to Python 3, and at least having an upgrade path.
The other way to look at this is those experienced in 2 are suffering from baby duck syndrome.
Similar phenomena are almost universally a case with new things in IT, CS and probably life. Takes older generation to "die out" for a society as a hole to accept new way.
Similar phenomena are almost universally a case with new things in IT, CS and probably life. Takes older generation to "die out" for a society as a hole to accept new way.
> who have no experience of anything else
...or who have no burden of the past to bear
Don't take me wrong though - I'm not trying to say experience is bad. Just saying it alone is not sufficient.
Don't take me wrong though - I'm not trying to say experience is bad. Just saying it alone is not sufficient.
The biggest problem with Unicode in Python2 was not the UnicodeDecodeError/UnicodeEncodeError, it was the latent UnicodeDecodeError/UnicodeEncodeError. As in, you had some code which works perfectly fine, both in your machine and in production, until someone somewhere types a word with a ç character and your code blows up with one of those two exceptions.
With Python3, the same code will blow up early, even if there was no non-ASCII character for that string in your test set, and will get fixed before going into production.
With Python3, the same code will blow up early, even if there was no non-ASCII character for that string in your test set, and will get fixed before going into production.
That was the main reason I migrated: processing everything internally in Unicode and then encoding/decoding as needed for the outside world creates a well defined 'place-where-unicode-shit-might-bork' border in your project that allows you to not only detect unicode errors earlier, as you stated, but also keep tabs on this 'border'.... eventually you don't even have unicodeerrors in your codebase anymore, because in 3.x this pattern emerges in your head and imho it's a really good thing, it is making your think properly how your code interacts with other entities.
FTR I can't remember ever seeing a UnicodeError in Python 3 that was caused by an ill-formed program. I only saw it when someone put binary shit somewhere were only text should be (eg. that flat text file with the page title should contain text, not an .exe)
Also the distinction between binary and text is the only sane way to handle it. Everything else is simply ill-formed.
Also the distinction between binary and text is the only sane way to handle it. Everything else is simply ill-formed.
I find surreal how much people underestimate how big is the population that deals with those unicodeerrors. It's SO MANY FUCKING LANGUAGES AND SO MANY PEOPLE. The majority of the world population writes using non-ASCII characters!
It's a HUGE problem in Python 2. Python 3 was needed. If you don't think the Unicode change was needed and you actually work with texts in any form, you should also stop developing "for web scale" before fixing this. Because you're actually focusing on a small percentage of the world population on your software.
For me Zed's argument sounds like "This is America, speak English". Specially with his quotes around "international".
It's a HUGE problem in Python 2. Python 3 was needed. If you don't think the Unicode change was needed and you actually work with texts in any form, you should also stop developing "for web scale" before fixing this. Because you're actually focusing on a small percentage of the world population on your software.
For me Zed's argument sounds like "This is America, speak English". Specially with his quotes around "international".
His argument should rather be written like "This is America, you are not allowed even to write your last name normally" - then it sounds stupid/irrational/funny enough.
which seems to be the case in switzerland: If you want to apply for swiss citizenship, your name needs to be compatible with ISO-8859-15 because the central registry does not support unicode: https://www.dasmagazin.ch/2016/09/02/ic-bin-kein-schweizer/ (sorry, german only, which seems oddly apt)
Üክጎርöዕቿ የዪöጌረቿጠነ ልዪቿ öሀቿዪዪäፕቿዕ.
Yea, use Python2, don't use Unicode, but please add information to a registration form like:
"due to our programmers' limitations please don't enter your first name or last name as your write it, use only ASCII characters, otherwise our servers may blow up".
PS) This is what I had to work on last year. Happy US programmers who know nothing except English. And suddenly the software blew up, just because a client wanted to enter his name.
PS) This is what I had to work on last year. Happy US programmers who know nothing except English. And suddenly the software blew up, just because a client wanted to enter his name.
There's also this article [0], which sort of tears Zed's article apart bit by bit. It's honestly a fun read because of the aggressive tone (which I find deserved), and it's also informative.
edit: there was a HN discussion about it as well. [1]
[0] https://eev.ee/blog/2016/11/23/a-rebuttal-for-python-3/
[1] https://news.ycombinator.com/item?id=13027481
edit: there was a HN discussion about it as well. [1]
[0] https://eev.ee/blog/2016/11/23/a-rebuttal-for-python-3/
[1] https://news.ycombinator.com/item?id=13027481
I don't have a problem with Python 3's unicode support, and I don't really understand why people do. I think it works fine, you just want to think of b'' as byte arrays instead of strings. If you're ever carrying a bytes around and passing it into string handling functions, you're Doing It Wrong. Honestly, I can't find anything wrong with Python 3's Unicode support. It's great.
Exactly. Its clear distinction between (unicode) strings and (binary) data is Python 3's biggest plus, imho. No more randomly popping up errors because something is trying to handle the binary data in your str as string; instead you'll get a nice type error. I don't understand how people do _not_ find that terribly annoying with Python 2. (seriously, tell me)
Gut feeling: there is a large intersection between the "Python 3 Unicode sucks" and "Every operating system ought to handle only bytes and never think of encoding, and file names and paths are just bytes". I have words for the latter attitude, but they're not HN-compatible.
I have nothing against Python 3, but right now Python 2.7 is more than good enough for all my current requirements, and is used by every client I work with that uses Python.
At some point that might start applying to Python 3, in which case I will use Python 3. In the meantime I stay out of religious programming wars and use the best tool for the job. For me, right now, that is Python 2.7.
At some point that might start applying to Python 3, in which case I will use Python 3. In the meantime I stay out of religious programming wars and use the best tool for the job. For me, right now, that is Python 2.7.
> Python 2.7 is more than good enough for all my current requirements
> use the best tool for the job. For me, right now, that is Python 2.7.
Sorry, but these two statements seem at odd with each other. I absolutely agree with the first principle, and I agree that in many cases 2.7 is more than good enough; however, in nearly all cases I have encountered (and I have, a lot!) Python 3 is notably better, which means that it is not "the best tool for the job".
> use the best tool for the job. For me, right now, that is Python 2.7.
Sorry, but these two statements seem at odd with each other. I absolutely agree with the first principle, and I agree that in many cases 2.7 is more than good enough; however, in nearly all cases I have encountered (and I have, a lot!) Python 3 is notably better, which means that it is not "the best tool for the job".
Everyone has different needs. For your needs (for most people's needs), Py3 is better. For his needs, Py2.7 is better. It's not hard to understand.
If everything he encounters, for example, is in 2.7, then that's probably the best tool for the job. If I'm commuting to work in the morning, I want a nice sedan with an automatic transmission and fuel injection because it's nice and modern and easy to use. If I'm in rural Africa though, I probably want a truck with a manual transmission and a carburetor. Sure it's outdated, but if I need a mechanic to work on it or parts that will fit it, every other car on the road there is just as outdated as mine.
If everything he encounters, for example, is in 2.7, then that's probably the best tool for the job. If I'm commuting to work in the morning, I want a nice sedan with an automatic transmission and fuel injection because it's nice and modern and easy to use. If I'm in rural Africa though, I probably want a truck with a manual transmission and a carburetor. Sure it's outdated, but if I need a mechanic to work on it or parts that will fit it, every other car on the road there is just as outdated as mine.
> If everything he encounters, for example, is in 2.7, then that's probably the best tool for the job.
I disagree. Your example describes two completely different solutions to different scenarios; but Python 3 is not a different solution, it's a subset. It has everything that 2.7 has, only improved -- which means that for each case that Py 2.7 can handle, Py 3 can handle it just as easily, or better.
My comment was simply about the phrase "the best tool for the job" -- Python 2.7 is in many cases good enough, and there is no point in switching an existing codebase if it means much more work than it brings benefits. However, when starting something new, with the goal of using "the best tool for the job", Python 3 is -- in my experience at least -- the clear winner.
I disagree. Your example describes two completely different solutions to different scenarios; but Python 3 is not a different solution, it's a subset. It has everything that 2.7 has, only improved -- which means that for each case that Py 2.7 can handle, Py 3 can handle it just as easily, or better.
My comment was simply about the phrase "the best tool for the job" -- Python 2.7 is in many cases good enough, and there is no point in switching an existing codebase if it means much more work than it brings benefits. However, when starting something new, with the goal of using "the best tool for the job", Python 3 is -- in my experience at least -- the clear winner.
serious question: why I see really almost no complains in other languages when they introduce breaking changes? is the migration handled differently or the communities are less vocal?
Serious answer (seriously): because Python 3 didn't break enough stuff.
For all the complaints, Python 3 was incredibly conservative about making changes that would completely break existing code. String handling was basically the big shift, and if you were writing good code to begin with wasn't super hard for most problem domains to adjust to. Clearing up some cruft in the way the standard library was organized wasn't hard to deal with either. And there wasn't a lot of new syntax in 3.0/3.1 and quite a bit of it was backported into 2.6/2.7 to make the transition easier.
And then there was a bit of a moratorium on language changes in 3.x to give people time to catch up from their 2.x codebases.
The result of which is a lot of people who didn't think Python 3 was a big enough deal to be worth putting in the effort to migrate, when the 3.x series started releasing. And now many of them are finally noticing all the cool stuff that's in 3.x now (3.6 is about to release), and noticing that their favorite tools and libraries have switched from "we support 2 and 3" to "we're dropping support for 2", and realizing they didn't take advantage of all the time they had to port stuff, and are angry and feeling left out and trying to find complaints to level at Python 3 in the hope that enough people will agree and just magically end it, port all the new features back to Python 2, and let them keep going as-is.
For all the complaints, Python 3 was incredibly conservative about making changes that would completely break existing code. String handling was basically the big shift, and if you were writing good code to begin with wasn't super hard for most problem domains to adjust to. Clearing up some cruft in the way the standard library was organized wasn't hard to deal with either. And there wasn't a lot of new syntax in 3.0/3.1 and quite a bit of it was backported into 2.6/2.7 to make the transition easier.
And then there was a bit of a moratorium on language changes in 3.x to give people time to catch up from their 2.x codebases.
The result of which is a lot of people who didn't think Python 3 was a big enough deal to be worth putting in the effort to migrate, when the 3.x series started releasing. And now many of them are finally noticing all the cool stuff that's in 3.x now (3.6 is about to release), and noticing that their favorite tools and libraries have switched from "we support 2 and 3" to "we're dropping support for 2", and realizing they didn't take advantage of all the time they had to port stuff, and are angry and feeling left out and trying to find complaints to level at Python 3 in the hope that enough people will agree and just magically end it, port all the new features back to Python 2, and let them keep going as-is.
> Serious answer (seriously): because Python 3 didn't break enough stuff.
I could also be because there was another choice (which is what you also said). Ruby 1.8 -> 1.9 was annoying, but you weren't going to keep using 1.8 given e.g. the speed improvements of 1.9. (Also, who adds breaking changes in minor version numbers?)
There's a great talk about changing the culture around Python3 at Facebook[0], I highly recommend it. Technically, it isn't so hard. Culturally, it can be pretty hard. Calling Python2 "Legacy Python" helped at my $work. My boss agreed that continuing 2.x development wasn't a good investment for the future, given the minor technical challanges involved with moving over a few years.
[0] https://www.youtube.com/watch?v=nRtp9NgtXiA
I could also be because there was another choice (which is what you also said). Ruby 1.8 -> 1.9 was annoying, but you weren't going to keep using 1.8 given e.g. the speed improvements of 1.9. (Also, who adds breaking changes in minor version numbers?)
There's a great talk about changing the culture around Python3 at Facebook[0], I highly recommend it. Technically, it isn't so hard. Culturally, it can be pretty hard. Calling Python2 "Legacy Python" helped at my $work. My boss agreed that continuing 2.x development wasn't a good investment for the future, given the minor technical challanges involved with moving over a few years.
[0] https://www.youtube.com/watch?v=nRtp9NgtXiA
As an end-user, not a python programmer, my problem is that python lacks an idiomatic way to specify which version of python a script should run with. (Or, if it does have one, a lot of programs don't bother using it!)
When Arch switched from /usr/bin/python being python 2 to being python 3, buckets of python scripts failed. When downloading third-party python programs, the only way I know to know which version of python to use is to try one and when try the other when it complains about syntax errors.
With a perl-style "use" declaration, the runtime can immediately tell you when you're using the wrong version, or even, in principle, switch to the correct runtime if it's installed.
Hashbangs should refer to a version: e.g. "#!/usr/bin/env python2". Github shows approx. 3M results for "/usr/bin/env python", but less than 200k combined for version-specific hashbangs.
When Arch switched from /usr/bin/python being python 2 to being python 3, buckets of python scripts failed. When downloading third-party python programs, the only way I know to know which version of python to use is to try one and when try the other when it complains about syntax errors.
With a perl-style "use" declaration, the runtime can immediately tell you when you're using the wrong version, or even, in principle, switch to the correct runtime if it's installed.
Hashbangs should refer to a version: e.g. "#!/usr/bin/env python2". Github shows approx. 3M results for "/usr/bin/env python", but less than 200k combined for version-specific hashbangs.
> Hashbangs should refer to a version: e.g. "#!/usr/bin/env python2". Github shows approx. 3M results for "/usr/bin/env python", but less than 200k combined for version-specific hashbangs.
For loose scripts this is the way, "env python" is simply wrong. Always use "..env python2" or "..env python3"; it works everywhere, and where the correct version is not installed it gives a relatively helpful error message (something like "No such file or directory: "python2" ").
For loose scripts this is the way, "env python" is simply wrong. Always use "..env python2" or "..env python3"; it works everywhere, and where the correct version is not installed it gives a relatively helpful error message (something like "No such file or directory: "python2" ").
Either they have a much smaller user base to begin with or they're offering something really enticing in exchange for the pain. The problem isn't that migrating from python2 to python3 is super hard, it's that people don't see any immediate benefits for doing so. Had python3 come with something really enticing (like a significant performance increase or a better parallelism story), a lot more people would have not only accepted a lot more breakages, but also switched years ago.
It's hard to remember now but the problem wasn't so much any changes to the language. It was that the C-api changed. Since most of the important libraries have C-components, this meant that it took a long time for the libraries to become available on python3. Now, many of the libraries have shifted so we're on to the next problem which is that many companies have legacy code bases or very conservative update cycles which is keeping people stuck on python2 (both are the case for me).
Of course, there will always be those who find problems with any change and they're normally the most vocal.
Personally, I've struggled with various unicode issues in my code over the last few years and python3 might have saved me from all that. Even so, for a long time python3 didn't seem to offer enough of an advantage to make the switch although I have ported some of my code across which was relatively easy.
The hardest thing for me is that python3 has banned implicit relative imports within packages. Unfortunately I've tended to mix modules and scripts freely in my projects (scientific simulations) and so I'll need to restructure to port some of my projects.
Of course, there will always be those who find problems with any change and they're normally the most vocal.
Personally, I've struggled with various unicode issues in my code over the last few years and python3 might have saved me from all that. Even so, for a long time python3 didn't seem to offer enough of an advantage to make the switch although I have ported some of my code across which was relatively easy.
The hardest thing for me is that python3 has banned implicit relative imports within packages. Unfortunately I've tended to mix modules and scripts freely in my projects (scientific simulations) and so I'll need to restructure to port some of my projects.
Most languages simply don't make gratuitous breaking changes to things people rely on in most of their programming. The only other cases I can think of in a major language would be 1) Visual Basic, where Visual Basic .NET is really a fundamentally different language that happens to share some syntax conventions, and so got the moniker "Visual Fred" in the VB6 community, which was pretty pissed about the whole thing... but it is also true that other than in branding, Microsoft never really pretended Visual Fred was anything more than a way to make old-time Visual Basic developers a little more comfortable being forced to write C#-ish code... and 2) Perl 6, which left copious development effort and active maintainers on the runtime engine for Perl 5, which continued to receive reasonable updates (so no one was stranded, like they were with Python).
Here's a non-example: Ruby. It was quite possible and not even that difficult to write code that ran on both Ruby 1.8 and Ruby 1.9 that had correct behavior for both systems, despite all of the pain people noted about the transition with Unicode just trying to use vanilla 1.8 code on 1.9... but the transition from Python 2.7 to Python 3.0 required an all-or-nothing approach for the numerous changes which could not be "from __future__ import"ed in 2.7: Python 3.0 almost seemed to go out of its way to screw with developers, renaming "str" to "bytes" and "unicode" to "str" without providing any aliases which could be used across the different runtimes and simultaneously removing the u"" prefix syntax while remapping "" (so there was no cross-version way to even say "this represents text). Even later versions of Python still didn't try to help with exceptions: the syntax simply changed and the only way to write code that worked on both was to catch exceptions without a name and then dig into sys.exc_info() to pull the value as part of the except block.
Meanwhile, the Python developers handled the bootstrap horribly: they have been giving the finger to people for using Python 2 ever since they started working on Python 3, putting 2.7 in what was essentially an "emergency maintenance only" mode, causing the runtime to stagnate for five years while they marched from 3.0 (which was nearly unusable) to 3.5 (which was where Python 3 started to look reasonable) and there was very little library uptake; so it isn't like higher-level code could transition even if it wanted to: application developers were stuck waiting on libraries and library developers were stuck waiting on the language itself to become more reasonable, and in that five year gap many people just started jumping ship to entirely different languages--such as Go, Rust, Clojure, and Elixir--as it isn't like they really had a choice (as again: they were blocked on people who were blocked on other people).
Here's a non-example: Ruby. It was quite possible and not even that difficult to write code that ran on both Ruby 1.8 and Ruby 1.9 that had correct behavior for both systems, despite all of the pain people noted about the transition with Unicode just trying to use vanilla 1.8 code on 1.9... but the transition from Python 2.7 to Python 3.0 required an all-or-nothing approach for the numerous changes which could not be "from __future__ import"ed in 2.7: Python 3.0 almost seemed to go out of its way to screw with developers, renaming "str" to "bytes" and "unicode" to "str" without providing any aliases which could be used across the different runtimes and simultaneously removing the u"" prefix syntax while remapping "" (so there was no cross-version way to even say "this represents text). Even later versions of Python still didn't try to help with exceptions: the syntax simply changed and the only way to write code that worked on both was to catch exceptions without a name and then dig into sys.exc_info() to pull the value as part of the except block.
Meanwhile, the Python developers handled the bootstrap horribly: they have been giving the finger to people for using Python 2 ever since they started working on Python 3, putting 2.7 in what was essentially an "emergency maintenance only" mode, causing the runtime to stagnate for five years while they marched from 3.0 (which was nearly unusable) to 3.5 (which was where Python 3 started to look reasonable) and there was very little library uptake; so it isn't like higher-level code could transition even if it wanted to: application developers were stuck waiting on libraries and library developers were stuck waiting on the language itself to become more reasonable, and in that five year gap many people just started jumping ship to entirely different languages--such as Go, Rust, Clojure, and Elixir--as it isn't like they really had a choice (as again: they were blocked on people who were blocked on other people).
The main reason that Python Unicode is hard is because of the disastrously confusingly named "encode" and "decode". It's a constant question which direction you are going - encoding or decoding, huh, what?
Python 3 Unicode would be MUCH easier if encoding/decoding was more intuitive, memorable and obvious.
Still an unfixed problem. Not even an acknowledged problem.
Python 3 Unicode would be MUCH easier if encoding/decoding was more intuitive, memorable and obvious.
Still an unfixed problem. Not even an acknowledged problem.
`encode` converts text to an encoded form. `decode` converts an encoded form to text. This was confusing in Python 2 because you could encode bytes into… more bytes. What? Python 3 only lets you encode text and decode bytes, as it should, and removes the confusion.
Yeah that explanation really ironed out any confusion.
I think you’d probably get used to it after using it a couple of times/being familiar with what the types represent. Text is a sequence of characters, but you can encode it into a sequence of bytes when necessary. Give it a try!
Also, it’s probably worth mentioning that you rarely need to use `encode` or `decode` in Python 3. Most text can just stay text. If you make a mistake, the stronger types are there to help you notice it.
Also, it’s probably worth mentioning that you rarely need to use `encode` or `decode` in Python 3. Most text can just stay text. If you make a mistake, the stronger types are there to help you notice it.
In python 3 you can use:
b1 = b'bytes'
s = str(b1, 'utf-8')
b2 = bytes(s, 'utf-8')
There should be one obvious way to do it.
How would you design the API?
He's got a point. I'd probably use something like to_bytes and to_str to disambiguate.
encode/decode were likely not changed from Py2 to make porting easier. Otherwise it would likely have been to_bytes and str.from_bytes - much like "int" in Py3 with to_bytes and int.from_bytes.
Qt has QString::fromUtf8 which takes a QByteArray, and QString.toUtf8 which gives a QByteArray. (And similar for a few other encodings.) That was always obvious enough for me.
http://doc.qt.io/qt-5/qstring.html
http://doc.qt.io/qt-5/qstring.html
Like I asked baq, say I don't know the encoding ahead of time, but get it dynamically (like in the declaration of an XML file). How would I decode the string?
I would try utf-8, and otherwise just panic ("invalid input: string is not utf-8 encoded").
Obviously it depends on the context, but I think parsers should be strict, otherwise ambiguities leak into your system.
Obviously it depends on the context, but I think parsers should be strict, otherwise ambiguities leak into your system.
I'm not saying "try multiple encodings till you get something". I'm asking about the API when the encoding is not static, but provided at runtime, like in XML files, eg:
<?xml version="1.0" encoding="ISO-8859-1"?>You would have the QByteArray first (e.g. read from a file or socket), parse the <?xml?>, thus find the encoding, convert the rest to QString with that, and parse the rest from the QString.
My question is how does one "convert the rest to QString with that", in terms of actual code?
I'd copy directly from Rust.
Say I get a file and the name of the encoding it's in (like an XML file with a declaration), how I decode it in Rust?
[deleted]
Explicitly, obviously and unambiguously.
That's a wishlist, not a design. Do you suppose they tried to make it implicit, non-obvious and ambiguous?
I was a die-hard 2.x Pythonista, and as Python 3 started to reach 3.[4/5] releases, in my view it started to get really good in terms of new (and actually useful, not just bloated) features. I for instance love the way you can properly type everything, and that equates to proper interfacing, expectancy between object interactions and also (ofc!), better and smarter IDEs.
Also, at the 3.5 release, most "big" libraries and frameworks were already supporting Py3.x in a stable manner, so more and more the point about being at 2.x "because I have this huge library support" got fragile in my head, and I knew what that meant: "Someday, I will actually have to migrate!"
I started getting worried that the 2.x ways were too entrenched in me and that I'd never be able to migrate, that my Python (2.x) skills would become something as forgotten as my former self (childhood) Turbo Pascal antics and etc. Nonsense!
Here's what happened: I was asked to work on the backend of a new website that was aiming for a fully "portable" (desktop, mobile, and whatever comes next) experience. They wanted me to write an API for it, and I started studying what framework I could use that was simple enough for an API (flask!) and for sane backend storage definition (sqlalchemy!)
- Flask on 3.x -> The more I searched about flask on 3.x, the more I found how it was really stable and production-ready. Also, using 3.x would be really nice on the whole Unicode thing, as I would not be able to make the same string/bytestring mixing assumptions I used to do so carelessly in 2.x (until something blew up...) I would actually have to enforce and handle proper Unicode throughout the api.
- SQLAlchemy on 3.x -> Really, the same. Production-ready, used throughout.
These two convinced me to try to get the initial API study on 3.x, and oh boy, besides stumbling here and there with some changed syntax (mostly, simple print()'s during early stages), migrating to 3.x went real smooth!
As I advanced in my 3.x studies, I started to see benefits in typing everything properly, specially between layers/borders, and it meant less typing and more intelligent autocomplete for my pycharm.
Seriously, If you find yourself struggling or anxious about migrating to 3.x, just find yourself a new project and start doing it on 3.x in the most non-pretentious manner. You'll find yourself just as 'at home' as you felt in 2.x, but you'll still feel something changed for the better...you won't feel strange more than 3 days I promise you :DD
Also, at the 3.5 release, most "big" libraries and frameworks were already supporting Py3.x in a stable manner, so more and more the point about being at 2.x "because I have this huge library support" got fragile in my head, and I knew what that meant: "Someday, I will actually have to migrate!"
I started getting worried that the 2.x ways were too entrenched in me and that I'd never be able to migrate, that my Python (2.x) skills would become something as forgotten as my former self (childhood) Turbo Pascal antics and etc. Nonsense!
Here's what happened: I was asked to work on the backend of a new website that was aiming for a fully "portable" (desktop, mobile, and whatever comes next) experience. They wanted me to write an API for it, and I started studying what framework I could use that was simple enough for an API (flask!) and for sane backend storage definition (sqlalchemy!)
- Flask on 3.x -> The more I searched about flask on 3.x, the more I found how it was really stable and production-ready. Also, using 3.x would be really nice on the whole Unicode thing, as I would not be able to make the same string/bytestring mixing assumptions I used to do so carelessly in 2.x (until something blew up...) I would actually have to enforce and handle proper Unicode throughout the api.
- SQLAlchemy on 3.x -> Really, the same. Production-ready, used throughout.
These two convinced me to try to get the initial API study on 3.x, and oh boy, besides stumbling here and there with some changed syntax (mostly, simple print()'s during early stages), migrating to 3.x went real smooth!
As I advanced in my 3.x studies, I started to see benefits in typing everything properly, specially between layers/borders, and it meant less typing and more intelligent autocomplete for my pycharm.
Seriously, If you find yourself struggling or anxious about migrating to 3.x, just find yourself a new project and start doing it on 3.x in the most non-pretentious manner. You'll find yourself just as 'at home' as you felt in 2.x, but you'll still feel something changed for the better...you won't feel strange more than 3 days I promise you :DD
My biggest issue is that Python 3, from an outsiders perspective, does not look at all like it's about unicode. Python 3 could very well have been done in a way that just fixed the back end implementation of python 2 (implement a string as an extension of a list) and then most things would have probably worked fine. Just handle all the unicode stuff in the back end and treat them the same exact way as Python 2 treated strings.
That's not what happened and instead we had a huge change to one of the fundamental data structures of any programming language and a lot of changes that pretty much no one wanted.
Packages were moved, things were renamed, etc. These are not changes that should be made. Make a fix, don't make me rewrite my code because someone else said "Eh, I like this name better". It's useless to do so and helps no one. There are now swaths of community-written documentation that are useless because every package name in the post has changed for seemingly no reason.
The issue is simple: it could have been handled so much better.
That's not what happened and instead we had a huge change to one of the fundamental data structures of any programming language and a lot of changes that pretty much no one wanted.
Packages were moved, things were renamed, etc. These are not changes that should be made. Make a fix, don't make me rewrite my code because someone else said "Eh, I like this name better". It's useless to do so and helps no one. There are now swaths of community-written documentation that are useless because every package name in the post has changed for seemingly no reason.
The issue is simple: it could have been handled so much better.
I would like to understand: what was the key mistake in the transistion to Python 3 that generated this huge divide and this ultra-annoying situation for the people that still have to use Python today?
What can be learned from that Python desaster to avoid such a catastrophy with some other programming language?
I really would like to read some analysis about what went wrong.
Thank you very much for your attention!
What can be learned from that Python desaster to avoid such a catastrophy with some other programming language?
I really would like to read some analysis about what went wrong.
Thank you very much for your attention!
Personally I think one big mistake was that it took literally years between python 3.0 coming out and numpy and django having a release that supported python 3. I remember python 3 coming out, being interested, looking at it for just long enough to realize that numpy wasn't supported and then not looking at it any more. Then I kind of lost interest in the intervening years and just kept working with python 2 which ended up getting back ports of many python 3 features.
I think that if the teams had somehow managed to coordinate a more or less simultaneous release then much of that early momentum wouldn't have been lost.
I think that if the teams had somehow managed to coordinate a more or less simultaneous release then much of that early momentum wouldn't have been lost.
the key mistake in the transistion to Python 3 that generated this huge divide
Here's how Aaron Swartz (and probably others as well) thinks the transition to Python 3 should have happened: http://www.aaronsw.com/weblog/python3
Here's how Aaron Swartz (and probably others as well) thinks the transition to Python 3 should have happened: http://www.aaronsw.com/weblog/python3
Most people who had bigger Python projects simply could not upgrade because they had a whole bunch of dependencies that did not work under Python3 and that could also not easily be replaced by some other library.
On a big python project at work we only very recently have lost our last dependency on a Python2 library and it took some dedicated effort to track and replace these dependencies. And upgrading all of our own code to be Python3 compatible will be a whole other can of worms (even though we were careful to write our code in a forwards-compatible way using future imports etc.).
On a big python project at work we only very recently have lost our last dependency on a Python2 library and it took some dedicated effort to track and replace these dependencies. And upgrading all of our own code to be Python3 compatible will be a whole other can of worms (even though we were careful to write our code in a forwards-compatible way using future imports etc.).
Hen and egg problem.
End applications did not want to upgrade because no third party libs were available.
Third party libs did not want to upgrade because no applications would use them anyway.
The correct way to handle such a transition is a proper bridge to import old libraries, then applications dare to upgrade without missing out on libraries which later leads to a bigger demand on third party libraries.
Another big reason is simply plain old stupid FUD.
End applications did not want to upgrade because no third party libs were available.
Third party libs did not want to upgrade because no applications would use them anyway.
The correct way to handle such a transition is a proper bridge to import old libraries, then applications dare to upgrade without missing out on libraries which later leads to a bigger demand on third party libraries.
Another big reason is simply plain old stupid FUD.
The big thing is not so much the changes that you need to make for Python 3, but supporting both from the same code base. Combined with the fact that this is often very necessary and will continue to be necessary for quite a while (LTS distros on Python 2.6 or 2.7), people think "why not just use the minimum?". If you have good support from things like six and good testing then it's possible, but it certainly adds a development tax.
Python 3 has been adding more features (especially concurrency-related) that don't seem to be getting back-ports, or at least the libraries that support them in Python 2 lack the ergonomics of nice syntax sugar. Eventually there'll be a tipping point of all major LTS releases on 3.x and enough carrots in Python 3 that none of the little things will matter any more.
Python 3 has been adding more features (especially concurrency-related) that don't seem to be getting back-ports, or at least the libraries that support them in Python 2 lack the ergonomics of nice syntax sugar. Eventually there'll be a tipping point of all major LTS releases on 3.x and enough carrots in Python 3 that none of the little things will matter any more.
Since we keep having these discussions every now and then, here's a link to one of them from a few weeks ago:
https://news.ycombinator.com/item?id=12930082
https://news.ycombinator.com/item?id=12930082
To me, the essay misses Shaw's point. One way to divide out the invested positions of the Python community and see his position for what it is, is to look back at when Shaw similarly pissed off the Ruby community (as distinct from the time he pissed off the Ruby-on-Rails community).
This essay is among the most memorable pieces I have read on Hacker News: https://web.archive.org/web/20120821234313/http://learncodet...
This essay is among the most memorable pieces I have read on Hacker News: https://web.archive.org/web/20120821234313/http://learncodet...
What do you think his point is, then?
Having read Shaw's recent essay [0]:
1. Because the target audience of Learn Python the Hardway is very new beginners and the pedagogy is programmed learning [1][2], Python 3 is a poor choice for the target audience of Learn Python the Hard Way.
2. Shaw is more concerned with the welfare of the book's intended audience than adherence to the dictates of Python's leading figures.
3. More subtly, that sales figures indicate a disconnect between the dictates of Python's leading figures and the interests of his students.
From my outside position, it seems Shaw has a legitimate stake in his decision and people like this essay's author have a stake that boils down to creating drama.
[0]: https://zedshaw.com/2016/11/24/the-end-of-coder-influence/
[1]: https://en.wikipedia.org/wiki/Programmed_learning
[2]: http://hanselminutes.com/407/learning-code-the-hard-way-with...
1. Because the target audience of Learn Python the Hardway is very new beginners and the pedagogy is programmed learning [1][2], Python 3 is a poor choice for the target audience of Learn Python the Hard Way.
2. Shaw is more concerned with the welfare of the book's intended audience than adherence to the dictates of Python's leading figures.
3. More subtly, that sales figures indicate a disconnect between the dictates of Python's leading figures and the interests of his students.
From my outside position, it seems Shaw has a legitimate stake in his decision and people like this essay's author have a stake that boils down to creating drama.
[0]: https://zedshaw.com/2016/11/24/the-end-of-coder-influence/
[1]: https://en.wikipedia.org/wiki/Programmed_learning
[2]: http://hanselminutes.com/407/learning-code-the-hard-way-with...
A noble goal, but his Python 3 post is still nonsense [0].
[0]: https://eev.ee/blog/2016/11/23/a-rebuttal-for-python-3/
[0]: https://eev.ee/blog/2016/11/23/a-rebuttal-for-python-3/
[deleted]
I've built two (soon to be commercialized) web apps in Django over roughly a year. I don't bother to ensure Python 2 compatibility. I refuse to litter my code with unneeded imports and make my code my code uglier/harder to understand.
Maybe I have the luxury of not targeting enterprises with legacy codebases. I genuinely feel sorry for people who have to go out of their way to solve compatilibility problems rather than work on more interesting problems.
Edit: typo
Maybe I have the luxury of not targeting enterprises with legacy codebases. I genuinely feel sorry for people who have to go out of their way to solve compatilibility problems rather than work on more interesting problems.
Edit: typo
For new projects Python 3 has to compete not against Python 2 but also against other scripting languages, especially NodeJs.
I'm unconvinced by the argument that big shops won't migrate by 2020. There is a statistic (which may be false and unproven, but I think is roughly correct) that says all code gets rewritten every 3 years. ie if you make Python 3 available to new code now, then by 2020, your code will be rewritten in Python 3 by natural attrition. That being said, its of course not that simple. However, the investment now in making core libraries compatible and deployed for both languages is likely to be much cheaper than a mass switch when Python 2 gives up the ghost.
One of the companies where I teach just (in 2016) upgraded one of their core systems to Python 2.7. That's after years in which it was written in Python 2.4. Yes, 2.4, the same version my great-grandparents ran back in the Old Country.
So yes, things are rewritten. And things are upgraded. But at a glacier's pace.
2020 is coming soon -- in about three years. There's no way that companies will upgrade some of these systems to Python 3 during that time. Some things, sure -- but there's a lot of legacy code out there, and I don't see them spending the time and money to upgrade something that already works.
So yes, things are rewritten. And things are upgraded. But at a glacier's pace.
2020 is coming soon -- in about three years. There's no way that companies will upgrade some of these systems to Python 3 during that time. Some things, sure -- but there's a lot of legacy code out there, and I don't see them spending the time and money to upgrade something that already works.
Python 2.4 was, until fairly recently, still supported in a RHEL LTS.
Enterprise being stuck with old versions isn't new. But it's a radically different world.
Enterprise being stuck with old versions isn't new. But it's a radically different world.
> But again, the population that will be affected is the 10% who deal with Unicode.
I bet a lot more than 10% of the user base uses emoji. And even more will.
I bet a lot more than 10% of the user base uses emoji. And even more will.
As I said in another Python 3 HN thread, developers from cultures whose languages can be fully satisfied with the Latin-1 charset (which is the absolute majority of the developed world) just don't get how important and essential it is to have a proper (pervasive and implicit) implementation of Unicode at the language level.
> (which is the absolute majority of the developed world)
I'm a German writing web apps using Django, and I constantly got UnicodeDecodeErrors when mingling with older browsers, uploads from Windows users, copy-pasted content from word processors etc.
The problem isn't (only) whether Latin-1 has all the characters, it's what encoding gets thrown at your software.
Python 3 was a blessing for handling user-provided input.
I'm a German writing web apps using Django, and I constantly got UnicodeDecodeErrors when mingling with older browsers, uploads from Windows users, copy-pasted content from word processors etc.
The problem isn't (only) whether Latin-1 has all the characters, it's what encoding gets thrown at your software.
Python 3 was a blessing for handling user-provided input.
[deleted]
A sweeping generalization made even less true lately with emoji.
If you can't handle Unicode fairly well, you can't make an app that is international ... the world is a big place.
I run into unicode issues on almost every project.
I run into unicode issues on almost every project.
I can make a case against 3: it's even slower than 2.
3.0 was, yes, because some significant new bits weren't yet implemented in C.
Python 3.3 caught it back up to, and started to exceed, 2.7 on benchmarks. And the gap is only going to grow as the 3.x series improves and the 2.x series lies stagnant.
Python 3.3 caught it back up to, and started to exceed, 2.7 on benchmarks. And the gap is only going to grow as the 3.x series improves and the 2.x series lies stagnant.
I doubt that only 10% of people are dealing with unicode. Rough estimate 10% of the world speak English as a first language and almost all other languages are written with (at least some) non-ascii characters.
In my experience (I'm from the 90% part of the world), people new to programming and python stumble really early into some (to them) hard to debug (UnicodeDecodeError: ... can't decode byte) errors in python 2. Then it's time to explain the whole strings are not bytes, unicode, encoding and so on thing. In python 3 that's also needed, perhaps even earlier, but bugs are much easier to fix and programs fail on the first run, not when you finally enter some non-ascii characters.
In my mind, python 3 clearly is superior to python 2, and would be even if nothing but the unicode support changed.