Writing Python like Rust (2020)(oatzy.github.io)
oatzy.github.io
Writing Python like Rust (2020)
https://oatzy.github.io/2020/05/10/writing-python-like-rust.html
176 comments
My current role is at a company with a few hundred thousand lines of un-typed, undocumented, mostly un-tested Python. I get a small allowance of "tech debt busting" time, and it's absurd how many bugs I find just by trying to suss out the types on a given code path. It's maddening work.
My experience of porting Javascript code to Typescript (or calling into Javascript from Typescript and figuring out the correct types to pass) is very similar.
Thousands of files all become filled with very obvious bugs once you enable a basic type checker, which then helps brand Typescript as "too much effort" by those who normally write Javascript because now they need to deal with their buggy code. Luckily you can just explicitly cast all of your types without checking and hope that nobody notices!
When you ask people to get their typing right, someone will use buzzwords like "rapid prototyping" and remind you that features are more important than fixing the existing product as long as nobody complains about the bugs, and that's that. Also, Typescript is hard, apparently.
Untyped legacy code sucks. It's one of the reasons I detest working with languages like Javascript, Python, and PHP. At least Python and PHP have types these days, making Javascript objectively inferior.
Thousands of files all become filled with very obvious bugs once you enable a basic type checker, which then helps brand Typescript as "too much effort" by those who normally write Javascript because now they need to deal with their buggy code. Luckily you can just explicitly cast all of your types without checking and hope that nobody notices!
When you ask people to get their typing right, someone will use buzzwords like "rapid prototyping" and remind you that features are more important than fixing the existing product as long as nobody complains about the bugs, and that's that. Also, Typescript is hard, apparently.
Untyped legacy code sucks. It's one of the reasons I detest working with languages like Javascript, Python, and PHP. At least Python and PHP have types these days, making Javascript objectively inferior.
The other way around you have the "it compiles, so I don't need tests" crowd in statically-typed languages.
I am currently firefighting for a customer who has a medium Sized application in c#. Started around 2019 so not ancient, but every programming, architecture and technology choice was wrong.
Sure it compiles. But often it crashes without rhyme or reason and for some reason, customers hate it.
First thing I did was to get it from .net 4.8 to .net 7 and enable all available errors checkers and style linters.
The other howled, but their builds stopped because they could no longer check-in crap. That helped. But fixing this is more work than what this would have needed if they had done it right the first time.
Funny examples: reinvanted rabbitmw badly using raw TCP and XML. Better hope your network is good, If the messages arrive at the same time, they are merged and the app crashes. So there are lots of time.sleep(50) calls to avoid that. Why is that app slow?
The iot devices can only address 254 states. The UI for that allows ulong devices. It's never checked, but if the vast fails, the app just crashes.
Sure it compiles. But often it crashes without rhyme or reason and for some reason, customers hate it.
First thing I did was to get it from .net 4.8 to .net 7 and enable all available errors checkers and style linters.
The other howled, but their builds stopped because they could no longer check-in crap. That helped. But fixing this is more work than what this would have needed if they had done it right the first time.
Funny examples: reinvanted rabbitmw badly using raw TCP and XML. Better hope your network is good, If the messages arrive at the same time, they are merged and the app crashes. So there are lots of time.sleep(50) calls to avoid that. Why is that app slow?
The iot devices can only address 254 states. The UI for that allows ulong devices. It's never checked, but if the vast fails, the app just crashes.
> First thing I did was to get it from .net 4.8 to .net 7
This might not have been a great idea, I'm fairly surprised nobody stopped you before you went through with it.
4.8 is the "final" version of .NET Framework. It's pretty much the longest "Long Term Support" offer Microsoft will ever give for a platform, realistically they'll be supporting it into 2030 and beyond. I'd not start any new projects in it, but upgrading away from it probably wouldn't be the first thing I'd do on a new project.
From .NET5, the framework moved to fundamentally different approach. It's effectively a different framework with the same NET branding on it. Most organisations who decided to move from 4.X to 6.X went through a near total rewrite because the paradigm shift is so massive, lots of organisations decided to stick with the pre-5 versions. The architecture and technology choices probably make a lot more sense in the pre-5 version of .NET, and it's unsurprising that your colleagues are frustrated with this decision, irrespective of the quality of their code.
On top of the move between .NET paradigms, you've moved away from a long-term-support version. NET7 is an 18month short-term-support interim release between two Long Term Support 36month versions .NET6 and .NET8. By moving to 7, you've trapped the company into an early & rapid upgrade path to .NET8 as soon as it comes out in Nov 2023, rather than giving the full 12 month buffer you'd have got if you moved to .NET6, which is in Long Term Support until Nov 2024. You moved the application off of a rock-solid framework with near infinite long-term-support, onto a framework that'll be deprecated before in four months, and out of support entirely in ten.
If I were you, I'd probably not be bringing this decision up in any performance reviews or interviews at new companies.
This might not have been a great idea, I'm fairly surprised nobody stopped you before you went through with it.
4.8 is the "final" version of .NET Framework. It's pretty much the longest "Long Term Support" offer Microsoft will ever give for a platform, realistically they'll be supporting it into 2030 and beyond. I'd not start any new projects in it, but upgrading away from it probably wouldn't be the first thing I'd do on a new project.
From .NET5, the framework moved to fundamentally different approach. It's effectively a different framework with the same NET branding on it. Most organisations who decided to move from 4.X to 6.X went through a near total rewrite because the paradigm shift is so massive, lots of organisations decided to stick with the pre-5 versions. The architecture and technology choices probably make a lot more sense in the pre-5 version of .NET, and it's unsurprising that your colleagues are frustrated with this decision, irrespective of the quality of their code.
On top of the move between .NET paradigms, you've moved away from a long-term-support version. NET7 is an 18month short-term-support interim release between two Long Term Support 36month versions .NET6 and .NET8. By moving to 7, you've trapped the company into an early & rapid upgrade path to .NET8 as soon as it comes out in Nov 2023, rather than giving the full 12 month buffer you'd have got if you moved to .NET6, which is in Long Term Support until Nov 2024. You moved the application off of a rock-solid framework with near infinite long-term-support, onto a framework that'll be deprecated before in four months, and out of support entirely in ten.
If I were you, I'd probably not be bringing this decision up in any performance reviews or interviews at new companies.
I honestly do not get what's your problem.
We moved from 4.8 to 7 because a) the better compiler warnings helped to reduce the amount of shit people where able to check-in AND highlight problems in the existing codebase (beyond "well, it looks like written by the lowest bidder offshore guy drunken on Friday night") and provide management with metrics they could understand (two warnings per LOC is part of your problem). B) it deepened our hiring pool, because it's easier to find people willing to work with current technologies than legacy. C) it allowed us to pull in various modern libraries (BLe especially) which made whole swathes of swamp code irrelevant. D) allowed us to establish a sane Ci/CD Pipeline more easily which we needed to reject bad stuff. F) we ship this product every few months, so not being on lts is not really a problem.
We moved from 4.8 to 7 because a) the better compiler warnings helped to reduce the amount of shit people where able to check-in AND highlight problems in the existing codebase (beyond "well, it looks like written by the lowest bidder offshore guy drunken on Friday night") and provide management with metrics they could understand (two warnings per LOC is part of your problem). B) it deepened our hiring pool, because it's easier to find people willing to work with current technologies than legacy. C) it allowed us to pull in various modern libraries (BLe especially) which made whole swathes of swamp code irrelevant. D) allowed us to establish a sane Ci/CD Pipeline more easily which we needed to reject bad stuff. F) we ship this product every few months, so not being on lts is not really a problem.
"It compiles so I don't need tests" is IMO more likely to be seen in relation to Haskell or Rust. Now with C# (and other popular languages) you may encounter big applications that don't have any kind of automated tests, but I don't think people are proudly declaring that compile=OK among this crowd.
I did not encounter this before also.
But this crowd believed it obviously. The chief perpetrators already left when I got there, so I could not ask directly. Zero tests, zero documentation and a coding style like somebody tried to write pure C in c#.
But this crowd believed it obviously. The chief perpetrators already left when I got there, so I could not ask directly. Zero tests, zero documentation and a coding style like somebody tried to write pure C in c#.
Oh dear, that sounds quite painful!
I like fire Fighting jobs, you always see some new insanity.
I actually get what you mean. There's something kinda rewarding about reducing chaos in a tangled system like that. At a certain point (as long as you're careful) you kind of can't go wrong, every little improvement incrementally makes the world a little bit better and the application a bit easier and more predictable to work with.
Are there many people like that?
In actuality, writing software that is some quality is so hard that one needs all help one can get. One absolutely needs type checking and is just not professional without. One absolutely needs automated tests and is just not professional without. One absolutely needs manual/exploratory testing and is just not professional without.
In actuality, writing software that is some quality is so hard that one needs all help one can get. One absolutely needs type checking and is just not professional without. One absolutely needs automated tests and is just not professional without. One absolutely needs manual/exploratory testing and is just not professional without.
Despite being unprofessional smelly amateurs, dynamically-typed scripting languages delivered and continue to deliver amazing amounts of business value. Which is really the only thing a business cares about, as professionals know.
I mean, you can do good work with Visual Basic and an overworked Excel spreadsheet, but it will probably be worth your while to find some alternatives.
Just because something works, doesn't mean we can't improve it, and just because we can deliver value with one tool, doesn't mean that we couldn't be delivering more value with other tools.
Just because something works, doesn't mean we can't improve it, and just because we can deliver value with one tool, doesn't mean that we couldn't be delivering more value with other tools.
In my experience this depends a lot on the organizations. If the organization's expectation is that sometimes, no tests would be ok, then quickly these people emerge who think that writing tests slows them down.
If culture is that tests are necessary, then these people hide and write their tests (or actually learn that test-writing makes sense).
I am in an organization currently that values testing overall, but being in a regulated industry its almost a little over-done, the result of that is, that for prototypes, tools, etc. the rules are weakened, leading to this weird situation where "of course we write tests, but this was just our prototype, that runs in production, for 3 years" seems to work in some departments...
If culture is that tests are necessary, then these people hide and write their tests (or actually learn that test-writing makes sense).
I am in an organization currently that values testing overall, but being in a regulated industry its almost a little over-done, the result of that is, that for prototypes, tools, etc. the rules are weakened, leading to this weird situation where "of course we write tests, but this was just our prototype, that runs in production, for 3 years" seems to work in some departments...
Yes, absolutely. Or they insist on only integration tests that take hours to run and are horrible to maintain.
This might sort of work if everything from the domain gets a disjoint type, but stringly-typed code is a lot more common and the type system doesn’t know what would make sense.
You always need tests... No its buts or maybes
Not being typed in a untyped/weakly typed language like JS, is NOT a bug.
Just because something is 'legacy' doesn't mean it's crap. It cauld be well thought out and easily maintained.
I've been dealing with a straightforward legacy api in JS. Trying to port it to typescript, the hard part is typing classes with dynamic propertied based on a schema. Has worked great in production for years. Well tested. I have to rewrite it to make it workin typescript apparently. going from about 10 line files to 50+ just for typings. Im sure i'm doing something wrong, but i've asked and it seems my code is wrong? despite having worked great for me all this time.... Yes i'm salty
Just because something is 'legacy' doesn't mean it's crap. It cauld be well thought out and easily maintained.
I've been dealing with a straightforward legacy api in JS. Trying to port it to typescript, the hard part is typing classes with dynamic propertied based on a schema. Has worked great in production for years. Well tested. I have to rewrite it to make it workin typescript apparently. going from about 10 line files to 50+ just for typings. Im sure i'm doing something wrong, but i've asked and it seems my code is wrong? despite having worked great for me all this time.... Yes i'm salty
You can use weak typing and dynamic programming to accomplish some impressive feats, just like you can write bug-free C code without undefined behaviour. It's difficult, especially when the code base grows over a couple of years, but it can be done.
That said, some people see TypeScript as "Javascript, but with types in the definitions", but it's more than that. Sure, the core of it is that you're adding types to your code, but with things like KeyOf<>, Partial<>, Pick<> and other such utilities the TypeScript system is a lot more powerful than that of many statically typed languages.
Typing dynamic JS with TS can be tough because complex dynamic typing requires a lot of assumptions that often are half-truths from a programming language point of view. "When x.foo is a Number then x.bar.baz is a String with at most two characters" can be expressed in Typescript but it's very verbose and requires some digging through the Typescript docs to get right.
Describing types does add more lines of code, but that's not necessarily a bad thing. What you're doing is taking the implicit assumptions based on your dynamically typed program and expressing them in a way that can be verified. Assuming you don't invent completely new types in every single function, the initial typing workload is high but it becomes a lot easier over time, especially if you use your Partial/Pick/keyofs well.
What I found to be helpful is that the ?. operator has been added to Javascript quite some time ago (legacy JS doesn't normally use it) which means you can define properties that don't exist in the beginning but are set down the line to be of type `foo: string | undefined` rather than invent implementations of interfaces for every modification step of the program, using it through `bar.foo?.baz` rather than `bar.foo.baz`, and maintain the exact same assertions your program had before.
Alternatively, using `!` when you're absolutely 100% sure some parameter has been filled in (`bar.foo!.baz!`) can help convince the transpiler of assumptions that can't easily be expressed in code.
That said, some people see TypeScript as "Javascript, but with types in the definitions", but it's more than that. Sure, the core of it is that you're adding types to your code, but with things like KeyOf<>, Partial<>, Pick<> and other such utilities the TypeScript system is a lot more powerful than that of many statically typed languages.
Typing dynamic JS with TS can be tough because complex dynamic typing requires a lot of assumptions that often are half-truths from a programming language point of view. "When x.foo is a Number then x.bar.baz is a String with at most two characters" can be expressed in Typescript but it's very verbose and requires some digging through the Typescript docs to get right.
Describing types does add more lines of code, but that's not necessarily a bad thing. What you're doing is taking the implicit assumptions based on your dynamically typed program and expressing them in a way that can be verified. Assuming you don't invent completely new types in every single function, the initial typing workload is high but it becomes a lot easier over time, especially if you use your Partial/Pick/keyofs well.
What I found to be helpful is that the ?. operator has been added to Javascript quite some time ago (legacy JS doesn't normally use it) which means you can define properties that don't exist in the beginning but are set down the line to be of type `foo: string | undefined` rather than invent implementations of interfaces for every modification step of the program, using it through `bar.foo?.baz` rather than `bar.foo.baz`, and maintain the exact same assertions your program had before.
Alternatively, using `!` when you're absolutely 100% sure some parameter has been filled in (`bar.foo!.baz!`) can help convince the transpiler of assumptions that can't easily be expressed in code.
"having type annotations" and "having types" are two very different things. The people who say TS is too hard/not worth are most definitely not adding type annotations to Python/PHP code.
This might be dependent on the size of your codebase, but it can be useful to start by running checks locally, using the static analysis to find bugs, then submitting the fixes independent of the static analysis. That way, by the time you propose enabling static analysis on all commits, it doesn’t scare people away.
Granted, this has the potential of the opposite problem, convincing detractors that static analysis didn’t find much, but that can be avoided by keeping a list of the bugs that were resolved ahead of time.
Granted, this has the potential of the opposite problem, convincing detractors that static analysis didn’t find much, but that can be avoided by keeping a list of the bugs that were resolved ahead of time.
> Also, Typescript is hard, apparently.
For people who know nothing but JavaScript and dynamic typing, it could be.
For people who know nothing but JavaScript and dynamic typing, it could be.
This is why I can't help but mentally roll my eyes when people say that static typing makes programming slower.
It makes writing the happy path slower, but is writing the happy path _programming?_
By the time you have a program which is complete and (at least mostly) bug free, the static typing has saved you hundreds of hours.
It makes writing the happy path slower, but is writing the happy path _programming?_
By the time you have a program which is complete and (at least mostly) bug free, the static typing has saved you hundreds of hours.
> It makes writing the happy path slower, but is writing the happy path _programming?_
The happy path is just fine for many cases.
Your home isn’t built to withstand a determined attacker with a tank. But we live in a happy world and a lot of stuff exists because people can cut corners.
Good managers/programmers know where they can be sloppy and where they have to be super careful.
The beauty of technical debt is that it only has to be repaid in case of success.
The happy path is just fine for many cases.
Your home isn’t built to withstand a determined attacker with a tank. But we live in a happy world and a lot of stuff exists because people can cut corners.
Good managers/programmers know where they can be sloppy and where they have to be super careful.
The beauty of technical debt is that it only has to be repaid in case of success.
This is an interesting example. There are plenty of rules that must be followed when building a home. Equating type checking to a home being able to withstand a tank is a poor example IMO due to the potential of the scenario actually occurring. You’re going to have type checking errors. You’re never going to have someone coming at your home with a tank. A much better example is a significant amount of rain or wind. You might have that storm once every few years but you’ll be glad you built it to withstand it when it comes.
I’m being generous tbh. In my experience, though I work on pretty critical software, the unhappy path is a fairly critical path and must have some intentional barriers built before being shipped to customers.
I’m being generous tbh. In my experience, though I work on pretty critical software, the unhappy path is a fairly critical path and must have some intentional barriers built before being shipped to customers.
The dynamic type checking analogy of house building is eyeballing support dimensions and then rebuilding walls when they fall down. Fine for a shed, but if it's a building I'm living in I would like a structural engineer to do some calculations to check it's ok before I use it.
Maybe that's the difference between programming and software engineering! Engineers care if their stuff breaks.
Maybe that's the difference between programming and software engineering! Engineers care if their stuff breaks.
Software Engineers that took a proper engineering degree care as well.
"If a builder builds a house for someone, and does not construct it properly, and the house which he built falls in and kills its owner, then that builder shall be put to death."
-- Hammurabi's Code, Babylon 1755–1750 BC
Modern construction law isn't as brutal, still liabilities will be questioned if corners were cut, and brought to court.
In many cases, programming is still at the level of driving chariots with steam engines.
-- Hammurabi's Code, Babylon 1755–1750 BC
Modern construction law isn't as brutal, still liabilities will be questioned if corners were cut, and brought to court.
In many cases, programming is still at the level of driving chariots with steam engines.
Debugging a happy path can quickly become a very unhappy exercise if there are no typechecks. Sure types can slow down things now and then, but if it catches just a few bugs it may already be worth it.
Funnily enough, languages with sufficiently advanced type systems can focus almost exclusively on the happy path thanks to functor/applicative/monad.
--
It seems that this comment is controversial and I'm not sure why. I'm referring to the ability to do something like
--
It seems that this comment is controversial and I'm not sure why. I'm referring to the ability to do something like
h <=< g <=< f
What does that do?
It's more general than this, but in this example it's a chain of three functions, f, g, h, each of which can fail in the same way. The precise failure behavior is encoded in the types themselves, so there is no need to check for it explicitly in the code. If `f` doesn't work, then `g` won't run, and the failure will be handled as expected by the functions' return types. Maybe they'll return an error message or maybe they'll be a "null" sort of value; the types will tell you, but the chaining syntax itself will always be the same.
It's far more common to see this chaining done with `do` blocks but I wanted to keep it all on one line.
This lends itself to a clean and optimistic sort of programming. You just write what you want to happen, chained within the broader context of some exception that might occur. So I agree with the parent that static typing makes things faster and easier, but I would argue that most unhappy paths don't require unique attention, and that it's language smell to make you care about them.
It's far more common to see this chaining done with `do` blocks but I wanted to keep it all on one line.
This lends itself to a clean and optimistic sort of programming. You just write what you want to happen, chained within the broader context of some exception that might occur. So I agree with the parent that static typing makes things faster and easier, but I would argue that most unhappy paths don't require unique attention, and that it's language smell to make you care about them.
Thanks! It's hard to imagine how this would work in a complex example that has multiple error paths, but I understand what you mean in the case of same types/one error case. Railway Oriented Programming and all that.
plus modern statically typed languages have type inference so I don't buy the argument that the happy path is faster in dynamically typed languages.
> It makes writing the happy path slower, but is writing the happy path _programming?_
I don't know if it's necessarily bad programmers. Think about this analogy, there's a blacksmith that can make swords but if you want it done well, it's gonna take more time/money. That same blacksmith can choose to take on customers that insist that he make the sword quicker and cheaper (and probably shittier).
He's not gonna be the one using the sword and he can make it clear up front about the tradeoffs. If it shatters in use and the customer gets maimed, oh well, they were warned. Sometimes the market is flooded with people who just want cheap swords made fast and the blacksmith needs to pay the bills too right?
He's not gonna be the one using the sword and he can make it clear up front about the tradeoffs. If it shatters in use and the customer gets maimed, oh well, they were warned. Sometimes the market is flooded with people who just want cheap swords made fast and the blacksmith needs to pay the bills too right?
One problem with this mentality is that customers need to search more and more to find blacksmiths that provide anything other than cheap swords. They may reasonably conclude that this is just the nature of swords, and that there’s no such thing as a sword that withstands more than one or two blows. Eventually, even people who want good swords accept that they won’t be able to find a blacksmith who makes them, and settle for cheap swords instead.
In effect, https://en.wikipedia.org/wiki/The_Market_for_Lemons
In effect, https://en.wikipedia.org/wiki/The_Market_for_Lemons
That's a fair point, but I don't think it's a problem with the blacksmith per se. You can't really expect every blacksmith to hold themselves to an arbitrary high standard and turn down work when they're possibly one or two missed payments away from financial ruin.
I would say if their society can grant them the safety to turn down work for cheap swords at their discretion without the risk of having their life ruined through a down period that is lacking high-quality work orders, then maybe you might have a case for also holding the blacksmiths accountable to a higher standard of work.
Accepting and taking on the risks associated with highly selective work orders has it's own intrinsic value and should be rewarded somehow. If it's not, then why would anyone do it?
I would say if their society can grant them the safety to turn down work for cheap swords at their discretion without the risk of having their life ruined through a down period that is lacking high-quality work orders, then maybe you might have a case for also holding the blacksmiths accountable to a higher standard of work.
Accepting and taking on the risks associated with highly selective work orders has it's own intrinsic value and should be rewarded somehow. If it's not, then why would anyone do it?
> This is why I can't help but mentally roll my eyes when people say that static typing makes programming slower.
No sane person says that. Also, static typing is not a thing (or at least, what you call "static typing" doesn't mean anything).
Type checking can be static. And in this context it means that it happens before the program is run, so it doesn't affect the speed of the program.
Your conjectures about how static typing (but probably actually checking) saves time developing is also nonsense. Static checking is helpful for analyzing programs, of course, but isn't the primary driver behind program quality, not even the second most important one. But, most importantly, quality typically only adversely affects the time to market. Working towards more correct programs will make the whole process slower and more expensive. Perhaps, at some very bottom of the quality pyramid you can find examples of the opposite -- when the program is so bad that it actually negatively impacts time to market and that metric can be improved by marginal effort in quality control... but if you are out of the swamp, this doesn't work anymore, and any increase in quality will negatively impact ttm, cost, number of employees needed etc.
No sane person says that. Also, static typing is not a thing (or at least, what you call "static typing" doesn't mean anything).
Type checking can be static. And in this context it means that it happens before the program is run, so it doesn't affect the speed of the program.
Your conjectures about how static typing (but probably actually checking) saves time developing is also nonsense. Static checking is helpful for analyzing programs, of course, but isn't the primary driver behind program quality, not even the second most important one. But, most importantly, quality typically only adversely affects the time to market. Working towards more correct programs will make the whole process slower and more expensive. Perhaps, at some very bottom of the quality pyramid you can find examples of the opposite -- when the program is so bad that it actually negatively impacts time to market and that metric can be improved by marginal effort in quality control... but if you are out of the swamp, this doesn't work anymore, and any increase in quality will negatively impact ttm, cost, number of employees needed etc.
I think writing the happy path can be a very important part to programming, yes. I know some programmers who don't need to see immediate results when writing their software, but me personally? I can't really do without it. It helps keep my drive up being able to prototype things quickly, and helps shape the software I am writing, too. Low level languages feel unwieldy to me. Things that are relatively easy in languages like Python get more complex, but I suppose it's also the lack of experience with such languages. Where do I start, though, when doing anything feels so slow to me? :) Difficult problem for sure...
It depends on what you're doing and also your subjective opinions. If you are writing software with a decent amount of lines and it's even somewhat important, you obviously want the typing benefits. On the other hand, if you're prototyping something personal and don't care about minor type errors in the slightest, it's nice and quick to do the job with javascript or python 3.
Are the 'bugs' actually triggered in the real-world usage of the code, or are they just 'potential bugs' which would be triggered given specific inputs that don't happen in the current usage of the code?
Might seem like nitpicking, but the 'probability of a bug being triggered in the future' is an important metric for figuring out if large scale refactorings are actually worth the risk of introducing new bugs.
Disclaimer: I'm coming from the static typing hemisphere, but adding type information to a codebase that was originally written with runtime-duck-typing in mind sounds like it might actually make maintainability of the code worse - that's at least my experience after trying to do exactly this in a medium-sized Python codebase - some of the 'typings' ended up so complex and weird that they completely dominated the code and turned what was very simple code at the beginning into an unreadable mess (looking almost like C++ template code). I eventually rolled back the changes. Basically, if I wanted a statically typed language, I wouldn't have chosen Python in the first place for this type of project.
Might seem like nitpicking, but the 'probability of a bug being triggered in the future' is an important metric for figuring out if large scale refactorings are actually worth the risk of introducing new bugs.
Disclaimer: I'm coming from the static typing hemisphere, but adding type information to a codebase that was originally written with runtime-duck-typing in mind sounds like it might actually make maintainability of the code worse - that's at least my experience after trying to do exactly this in a medium-sized Python codebase - some of the 'typings' ended up so complex and weird that they completely dominated the code and turned what was very simple code at the beginning into an unreadable mess (looking almost like C++ template code). I eventually rolled back the changes. Basically, if I wanted a statically typed language, I wouldn't have chosen Python in the first place for this type of project.
Can't speak for the parent commenter but their situation felt familiar. I'm in a new role where I'm contributing to a very large Python codebase (first commit from over a decade ago) and whilst there's a lot of more recent code written with type hints, they're not enforced and they're not always reliable or consistent. On my second contribution, I introduced a bug that pyright immediately spotted after I upped the checks from basic to strict. Our Sentry reports a lot of non-critical bugs that are being triggered in production that in many cases are _already_ highlighted by the type checker. It's just a big enough codebase that it's not practical to attempt to fix them all.
If I were starting a project today in Python, I would definitely try setting up my tooling and processes to enforce type checking to at least some extent. Fortunately, we have a lot of guardrails that recover from the typical exceptions that occur in those cases, but as I experienced first hand, it's very easy to introduce new bugs when relying only on my intuition for those difficult-to-spot cases. Tooling definitely helps there.
My previous role had me writing TypeScript and Rust so there's always the temptation, or bias, to advocate for strict typing everywhere, but I'm conscious that it's neither practical nor feasible in large codebases with hundreds of contributors (most of whom may be more accustomed to dynamically typed languages).
If I were starting a project today in Python, I would definitely try setting up my tooling and processes to enforce type checking to at least some extent. Fortunately, we have a lot of guardrails that recover from the typical exceptions that occur in those cases, but as I experienced first hand, it's very easy to introduce new bugs when relying only on my intuition for those difficult-to-spot cases. Tooling definitely helps there.
My previous role had me writing TypeScript and Rust so there's always the temptation, or bias, to advocate for strict typing everywhere, but I'm conscious that it's neither practical nor feasible in large codebases with hundreds of contributors (most of whom may be more accustomed to dynamically typed languages).
That's more of a cultural thing. Because Python is easy to get into, and you can be productive very fast, a lot of people with little experience in programming start to build things that are out of their reach.
They just don't know it, and the people hiring them don't either.
It's not a new phenomenon, we got the same when Java arrived, then PHP, etc.
Every time something makes things easier, you get a wave of beginners set to create things they are not qualified to do. And because you now need software for everything, leaders also get into positions where are not qualify to make some decisions about it. You pair unqualified leaders with unqualified devs, and you get this.
And it's now easier than ever to be in this position, also you are more pressured than ever to get into this position.
I've worked on plenty of big Python systems myself, it's not the tech per se (it has the usual pros and cons game like every language), but Python is the language used "when you don't know programming".
Which is ironic, because when I started to use it in Python 2.4, only passionate devs used it because it was niche, and we had the opposite effect.
They just don't know it, and the people hiring them don't either.
It's not a new phenomenon, we got the same when Java arrived, then PHP, etc.
Every time something makes things easier, you get a wave of beginners set to create things they are not qualified to do. And because you now need software for everything, leaders also get into positions where are not qualify to make some decisions about it. You pair unqualified leaders with unqualified devs, and you get this.
And it's now easier than ever to be in this position, also you are more pressured than ever to get into this position.
I've worked on plenty of big Python systems myself, it's not the tech per se (it has the usual pros and cons game like every language), but Python is the language used "when you don't know programming".
Which is ironic, because when I started to use it in Python 2.4, only passionate devs used it because it was niche, and we had the opposite effect.
> Every time something makes things easier, you get a wave of beginners set to create things they are not qualified to do. And because you now need software for everything, leaders also get into positions where are not qualify to make some decisions about it. You pair unqualified leaders with unqualified devs, and you get this.
How easy it is to call other people not qualified because they have different opinions.
I've seen more projects failed because the only thing the devs cared about was an abstract code correctness, which made even basic PRs getting polished for weeks. I don't think you need to worry too much about types or tests if you are just doing a prototoype to validate ideas or your user base is 0.
How easy it is to call other people not qualified because they have different opinions.
I've seen more projects failed because the only thing the devs cared about was an abstract code correctness, which made even basic PRs getting polished for weeks. I don't think you need to worry too much about types or tests if you are just doing a prototoype to validate ideas or your user base is 0.
It is BASIC all over again.
Bringing a non-trivial code base up to even half-decent standards can be a monumental task, and because such code bases are inherently overly complex and risky to change there's only so much you can do without breaking production constantly. It's a lose-lose proposition, which is why I'd never want to be "the programmer" at some place like that, no matter the pay.
It is no problem to write buggy unmaintainable code in a strongly typed language. It is also perfectly possible to write clean and maintainable code in a duck type language.
Ah memories... I once had 100K Python 2.x codebase to maintain. No docs, no tests and then being required to add features. Had to use the debugger[1] to make sense of the codebase and types, formats etc. Sheer madness and
such a waste of time...
To this day, I'm baffled by the dynamic language folks who cannot get their head around how strictness/rigor (via a good expressive type system) actually makes maintenance easier and more importantly: cheaper.
[1] https://github.com/inducer/pudb
To this day, I'm baffled by the dynamic language folks who cannot get their head around how strictness/rigor (via a good expressive type system) actually makes maintenance easier and more importantly: cheaper.
[1] https://github.com/inducer/pudb
django has 500K lines of python and it's considered well written and stable, so it's possible to do python with large code base I think. oddly, django does not seem to be even using type hints, new features such as dataclasses was not found from the source base either, still it just works well somehow.
That's because a great deal of Django was written before type hints or even dataclasses existed.
And type hints aren't really that mature yet. If you look at the `typing` module's documentation in Python 3.11, there's 23 occurrences of the string "Changed in version". And the Python 3.12 version of the same document adds 9 more. Type hinting in Python is constantly evolving.
That's the main reason we haven't adopted it at my place. We do have types of args and returns in docstrings though and it is enough for IDEs to do checks and suggestions from those types.
I was curious:
SLOC Directory SLOC-by-Language (Sorted)
232099 tests python=232099
102470 django python=102470
424 docs python=424
158 scripts python=142,sh=16
21 top_dir python=21
0 extras (none)
0 js_tests (none)
Totals grouped by language (dominant language first):
python: 335156 (100.00%)
sh: 16 (0.00%)
Please credit this data as "generated using David A. Wheeler's 'SLOCCount'."Wow, 2.3x for tests/src ratio. I remember some recommendation for Java to have around 0.9x ratio, otherwise too much tests maintenance.
Depends on the project of course, e.g. SQLite has an insane ratio.
Depends on the project of course, e.g. SQLite has an insane ratio.
Could also be that Java's typing already accounts for some part of what would need to be tests in Python code...
Exactly, that's what I meant. E.g.
if very_rare_condition:
raise MyRareException
I've always wondered if we shouldn't be pushing more for type inference with flow analysis. I consider static type checking a must, but the voices opposing type annotations have some valid points: it pollutes the source code, slows down prototyping, and only catches a limited class of errors.
A static checker with global type inference could be a middle ground. Pylance/Pyright already does that for most VS Code users. A more advanced form of this could track types that are too tedious for humans, like list emptiness, bounds, string alphabet, functions that can raise exceptions, idempotence, side effects, etc.
Those types can then be shown in the IDE, according to user preference, like docs or source control metadata.
I have a feeling that with enough caching this could be done with ok performance, but I hear that useful error reporting is an unsolved problem.
A static checker with global type inference could be a middle ground. Pylance/Pyright already does that for most VS Code users. A more advanced form of this could track types that are too tedious for humans, like list emptiness, bounds, string alphabet, functions that can raise exceptions, idempotence, side effects, etc.
Those types can then be shown in the IDE, according to user preference, like docs or source control metadata.
I have a feeling that with enough caching this could be done with ok performance, but I hear that useful error reporting is an unsolved problem.
It's funny how the pendulum swings. 10-15 years ago sticking your neck our and advocating for static types was like painting a target on your back. Everything was about dynamic languages, Ruby was the coolest of cool, and people who waved the placard of high level types, type inference, etc. from the ML/Haskell cool were seen on the whole as impractical academic ivory tower types, cloistered around Lambda The Ultimate...
When I advocated for the importance of static typing for software robustness the response I got from many other industry people I knew was that this was the job of intensive unit testing. That the supposed loss of rapid turnaround that came with static types wasn't worth it, and that test-first-design solved this problem.
Now Typescript and Rust are common place, and we have Pythonistas annotating their programs with types. I'm happy, but maybe a bit bitter about the past :-)
When I advocated for the importance of static typing for software robustness the response I got from many other industry people I knew was that this was the job of intensive unit testing. That the supposed loss of rapid turnaround that came with static types wasn't worth it, and that test-first-design solved this problem.
Now Typescript and Rust are common place, and we have Pythonistas annotating their programs with types. I'm happy, but maybe a bit bitter about the past :-)
That sounds impressive but I feel like this might be a misunderstanding of Python.
You can break almost any python code by passing in the wrong type. The code throwing runtime type-errors IS the type checking.
The danger is only if the code happily operates on the wrong types and returns incorrect results.
You can break almost any python code by passing in the wrong type. The code throwing runtime type-errors IS the type checking.
The danger is only if the code happily operates on the wrong types and returns incorrect results.
https://github.com/beartype/beartype
I wish more people started using Beartype, it makes Python bearable
I wish more people started using Beartype, it makes Python bearable
I work at a company that has a fucking nightmare of a C# project. It's total hell to work on. I don't think the language makes much of a difference.
> trying to suss out the types on a given code path.
Are there any tools for that? If not could one be made? Sounds almost like an advanced linter.
Are there any tools for that? If not could one be made? Sounds almost like an advanced linter.
So much for all those "you don't need static typing" / "dynamic typing is better" arguments.
Are you implying people writing code should have some training in how computers actually work? Heretic!
Just imagine how much more bugs you add...
Also, imagine how much more productive you would've been if you instead of doing what you are doing you'd do something worthwhile, like, design the program or organize the testing better.
I've heard so many nonsense claims like yours both in proprietary projects and in open-source ones, and not a single time did the code base became better. In most cases a small(-er) pile of garbage became a bigger pile of garbage.
Also, imagine how much more productive you would've been if you instead of doing what you are doing you'd do something worthwhile, like, design the program or organize the testing better.
I've heard so many nonsense claims like yours both in proprietary projects and in open-source ones, and not a single time did the code base became better. In most cases a small(-er) pile of garbage became a bigger pile of garbage.
> What I like about the Rust (and to a lesser extent, Go) approach to error handling is, you know up front if and what errors a function might return. And additionally, the compiler forces you to at least consider how you’re going to deal with it.
> By comparison, the python approach to dealing with exceptions is quite chunky
> try: > function() > except SomeException: > # etc
> I think this, in fact, encourages just letting exceptions bubble up, rather than fussing about with handling them.
Letting exceptions bubble up is usually the right thing to do. Handle them centrally, e.g. in middleware for a web application.
> By comparison, the python approach to dealing with exceptions is quite chunky
> try: > function() > except SomeException: > # etc
> I think this, in fact, encourages just letting exceptions bubble up, rather than fussing about with handling them.
Letting exceptions bubble up is usually the right thing to do. Handle them centrally, e.g. in middleware for a web application.
Agree. I spent quite a lot of effort at a previous role trying to convince junior developers to stop using except pretty much everywhere and just let the program explode, because that's really what you want most of the time. The problem is that sometimes in Python exceptions can be used for control flow, or for things that are reasonably common. In that case, it's very useful to know ahead of time what errors might be returned.
> The problem is that sometimes in Python exceptions can be used for control flow, or for things that are reasonably common.
The way that an iterator signals that it is done iterating is by raising StopIteration ( https://docs.python.org/3/library/exceptions.html#StopIterat... ).
I don't know if it's still the case in Python 3, but in 2 that exception was core to how for loops and list comprehensions worked.
I don't know if it's still the case in Python 3, but in 2 that exception was core to how for loops and list comprehensions worked.
This is still the case.
I think any type of parsing benefits from some happy-path/unhappy-path support in the language, because parsing essentially is about figuring out whether your input conforms to some format and then translating it into a more useful representation. And it often works in a nested way, where you parse sub-parts of your input and then combine those parts later. If you don’t use some form of exceptions/result types/CPS, you end up with lots of code like
Here, exceptions for control flow can help, because instead of the check, you just try to extract, and if it doesn’t work, you try the next version. (However, result types usually work better for this, in programming languages that support them.)
And as the last point, in my experience, many programming problems are handled cleanest when treating them as the problem of parsing some form of input (not necessarily a string) into some form of output. So this comes up more often than one might first think.
[Also, Python uses exceptions for some types of control flow internally. For example, StopIteration is thrown by Iterators for the (rather unexceptional) case that the Iterator is finished producing output. Admittedly, you can almost always just use a for loop, except when you want access to the StopIteration’s value at the end.]
if first part of input_data has format a:
extract first part of output from input_data
Here, exceptions for control flow can help, because instead of the check, you just try to extract, and if it doesn’t work, you try the next version. (However, result types usually work better for this, in programming languages that support them.)
And as the last point, in my experience, many programming problems are handled cleanest when treating them as the problem of parsing some form of input (not necessarily a string) into some form of output. So this comes up more often than one might first think.
[Also, Python uses exceptions for some types of control flow internally. For example, StopIteration is thrown by Iterators for the (rather unexceptional) case that the Iterator is finished producing output. Admittedly, you can almost always just use a for loop, except when you want access to the StopIteration’s value at the end.]
Checking if a string represents an integer. There is a great SO post[0] with various alternatives, which all feel worse.
[0] https://stackoverflow.com/questions/1265665/how-can-i-check-...
[0] https://stackoverflow.com/questions/1265665/how-can-i-check-...
> The problem is that sometimes in Python exceptions can be used for control flow, or for things that are reasonably common.
Using exceptions for control flow across function boundaries seems a recipe for confusion, unless very clearly documented.
I'm confused — how would one use exceptions if not for control flow?
In general exceptions are error handling, for "exceptional" situations. For example suppose we're supposed to ensure the water flow is at least X litres per second, but now X is NaN, that's nonsense, raise an exception.
You are correct that unlike a Result sum type, or an error code, or numerous other error solutions the exceptions inherently also change control flow.
In Python though it's usual to (ab)use exceptions for other flows, for example to indicate that we're done, lets just throw a Done exception.
The distinction is why Rust reifies control flow as the ControlFlow type, https://doc.rust-lang.org/core/ops/enum.ControlFlow.html -- so we have shared vocabulary to distinguish "This failed w/ error X" Result::Err(X) from "We should stop now, result Y" ControlFlow::Break(Y) even though sometimes it's true that we're stopping because of an error so we end up transforming Result::Err(X) into ControlFlow::Break(X). Doing that transformation is what the Try trait, the operator ? does today it calls Try::branch(result) and if that's ControlFlow::Break(X) your function exits with X, if it's ControlFlow::Continue(C) then the operator's result is C.
You are correct that unlike a Result sum type, or an error code, or numerous other error solutions the exceptions inherently also change control flow.
In Python though it's usual to (ab)use exceptions for other flows, for example to indicate that we're done, lets just throw a Done exception.
The distinction is why Rust reifies control flow as the ControlFlow type, https://doc.rust-lang.org/core/ops/enum.ControlFlow.html -- so we have shared vocabulary to distinguish "This failed w/ error X" Result::Err(X) from "We should stop now, result Y" ControlFlow::Break(Y) even though sometimes it's true that we're stopping because of an error so we end up transforming Result::Err(X) into ControlFlow::Break(X). Doing that transformation is what the Try trait, the operator ? does today it calls Try::branch(result) and if that's ControlFlow::Break(X) your function exits with X, if it's ControlFlow::Continue(C) then the operator's result is C.
With Rust & Go style error handling you generally still do bubble up errors and handle them centrally. But you don't always want to do that. Sometimes you want to handle errors locally and e.g. retry things, or try something else in case of an error or ignore some errors etc. You also should add context (`raise ConfigParseError() from int_parse_error`) which approximately 0% of people do with exceptions.
Handling errors locally is difficult to do with exception handling because it's impossible to tell what errors a function might produce (if any!). It's also more verbose (especially compared to Rust) which just makes it annoying.
Checked exceptions are the usual answer to this but they're so tedious most people don't bother with them. C++ even dropped support for them from the language.
Rust's error handling is much better than exceptions in almost every way.
Handling errors locally is difficult to do with exception handling because it's impossible to tell what errors a function might produce (if any!). It's also more verbose (especially compared to Rust) which just makes it annoying.
Checked exceptions are the usual answer to this but they're so tedious most people don't bother with them. C++ even dropped support for them from the language.
Rust's error handling is much better than exceptions in almost every way.
Why do you think nobody wraps exceptions? Everyone did in every project in every language I've worked on.
Every rest API client library will wrap the http library errors which will wrap network errors.
Languages even directly support the notion of keeping a reference to the original exception.
If anything, people do it too much.
Every rest API client library will wrap the http library errors which will wrap network errors.
Languages even directly support the notion of keeping a reference to the original exception.
If anything, people do it too much.
There's always that edge case, say "FileReadTimedOut" presented as a UncaughtServerError that no developer ever knew the existence of, let alone thought to wrap it (in, say, the LogoStorage).
With Rust this situation is impossible. Unexpected errors are made impossible by the type checker, a dev must handle it. Sure, a dev may say "meh, I'll just .into() (or wrap, if you want) any error that file-io gives into NotFound" but she does so knowingly and actively.
With Rust this situation is impossible. Unexpected errors are made impossible by the type checker, a dev must handle it. Sure, a dev may say "meh, I'll just .into() (or wrap, if you want) any error that file-io gives into NotFound" but she does so knowingly and actively.
A good fraction of bugs and exceptions I’ve seen in real python code bases is because the code itself didn’t handle the None edge cases at the exact moment they get created, and hence fail in a different place and let the debugging engineer scratching their head.
If you write typed python code that lets you avoid this while writing the bad code at the beginning itself, that’s a win. Actual runtime errors caused due to invalid user input should of course throw appropriate exceptions deep in the code and bubble up and be handled in middleware.
If you write typed python code that lets you avoid this while writing the bad code at the beginning itself, that’s a win. Actual runtime errors caused due to invalid user input should of course throw appropriate exceptions deep in the code and bubble up and be handled in middleware.
Handle errors as soon as you can is my stance. If a low-level function raises, but its immediate caller knows how to handle that specific exception, handle it (for example, turn a zero division error into None, if it makes sense for the domain).
Only bubble up if your handling consists of reraising anyway and no business logic can be applied (which might be the majority of cases).
Only bubble up if your handling consists of reraising anyway and no business logic can be applied (which might be the majority of cases).
This is interestingly divergent from the advice given in “Philosophy of Software Design” by John O. which is “exceptions are most useful when they're thrown the furthest”
Funny you’re mentioning that exact book. I almost put it as the source in my original comment, but maybe I remember wrong.
What I remember is the “deep interfaces” advice, which is great. It encourages small surfaces but deep implementations. Hiding the potentially large list of raisable exceptions helps in keeping an interface or API small. Errors are handled transparently and the user only needs to deal with those they know how to handle. What good is it to raise a deep error that by the time it reaches the top levels (like a web servers middleware loop), it can no longer be handled reasonably and has to be thrown? That’s why, if you can handle it, do so ASAP.
That’s what I remember from that book. Again, perhaps I’m misremembering.
What I remember is the “deep interfaces” advice, which is great. It encourages small surfaces but deep implementations. Hiding the potentially large list of raisable exceptions helps in keeping an interface or API small. Errors are handled transparently and the user only needs to deal with those they know how to handle. What good is it to raise a deep error that by the time it reaches the top levels (like a web servers middleware loop), it can no longer be handled reasonably and has to be thrown? That’s why, if you can handle it, do so ASAP.
That’s what I remember from that book. Again, perhaps I’m misremembering.
Note the comment you reply adds "this might be the majority of cases".
There are exceptions that are a normal result of some function call, and they can be handled by the caller.
But most exceptions are of the kind that you want to pass to some general error reporting thing far away.
There are exceptions that are a normal result of some function call, and they can be handled by the caller.
But most exceptions are of the kind that you want to pass to some general error reporting thing far away.
I don’t think the pieces of advice contradict each other. If exceptions are handled at the lowest scope possible, and exceptions are most useful when the rise above many scopes, then exceptions are most useful when the scope that encounters the error is very deep compared to the scope that can handle the error.
This agrees with my intuition, that error codes are most fragile when they must be checked and re-raised across many scopes.
This agrees with my intuition, that error codes are most fragile when they must be checked and re-raised across many scopes.
I like to say that the whole point of exceptions is not to handle them -- except when you know exactly which exception can happen and how to handle it. Everything else should be allowed to bubble up and handle on the highest level, whether it's by returning an error HTTP code, showing an error message to the user, or simply logging it at the right level.
This is why the try: block should never contain more than one single line, with the except block(s) limited only to the errors that can be expected from that line and that you explicitly want to handle at that point. Trying multiple lines and handling generic levels is a perfect way to get errors that are difficult to track.
This is why the try: block should never contain more than one single line, with the except block(s) limited only to the errors that can be expected from that line and that you explicitly want to handle at that point. Trying multiple lines and handling generic levels is a perfect way to get errors that are difficult to track.
Usually? Yea there is a lot of stuff that you handle "centrally", but there is also a lot of stuff that you should handle locally. E.g. "try this then that". Or stuff that you try, just log or save the error, and continue.
Exceptions are obviously great for bubbling up, but just awful for handling locally. I don't know what it is: maybe it's the horrible try/catch control flow that breaks everything up, or the fact that all functions can "return" exceptions apart from the ones in the signature (if any), so you're never quite sure what happens or what to do.
Exceptions are obviously great for bubbling up, but just awful for handling locally. I don't know what it is: maybe it's the horrible try/catch control flow that breaks everything up, or the fact that all functions can "return" exceptions apart from the ones in the signature (if any), so you're never quite sure what happens or what to do.
> Handle them centrally, e.g. in middleware for a web application.
So, for a Django app is it better to use their built-in middleware framework or something third-party that sits outside of Django?
https://docs.djangoproject.com/en/4.2/topics/http/middleware...
So, for a Django app is it better to use their built-in middleware framework or something third-party that sits outside of Django?
https://docs.djangoproject.com/en/4.2/topics/http/middleware...
Depends on what you mean by "sits outside of". If an exception reaches the response code in a web application, it's going to be converted to an HTTP error code and sent. You want to be thinking about what you want your API (or website) to expose to the user and work back from there.
> Unlike in a strongly typed language, the python type annotations aren’t enforced.
Python is a strongly typed language but type checks happen at runtime rather than ahead of time [0]. The author is correct that the type annotations aren't enforced.
[0] https://wiki.python.org/moin/Why%20is%20Python%20a%20dynamic...
Python is a strongly typed language but type checks happen at runtime rather than ahead of time [0]. The author is correct that the type annotations aren't enforced.
[0] https://wiki.python.org/moin/Why%20is%20Python%20a%20dynamic...
I'm not taking those arguments seriously anymore. Here's from the linked doc:
> In a weakly typed language a compiler / interpreter will sometimes change the type of a variable
Well,
Even if you take the "dynamic but strong" argument, it only holds for a few specific operations. A function like this cannot be considered strongly typed:
People have been desperately wanting to claim that Python has strong typing, but it not having weak typing doesn't make it strong. They're not binary opposites.
> In a weakly typed language a compiler / interpreter will sometimes change the type of a variable
Well,
a = 1
a = "a"
a = [1, a]
Even if you take the "dynamic but strong" argument, it only holds for a few specific operations. A function like this cannot be considered strongly typed:
def f(x, y):
return x + y
People have been desperately wanting to claim that Python has strong typing, but it not having weak typing doesn't make it strong. They're not binary opposites.
Your `f` function is just polymorphic.
Would you also consider the following Haskell function to not be strongly typed?
It's just that in Haskell's case, this checking is done at compile-time and in python's case at runtime.
Would you also consider the following Haskell function to not be strongly typed?
add a b = a + b
It's just that in Haskell's case, this checking is done at compile-time and in python's case at runtime.
But, to push it a bit further: how does Haskell deal with [1, 2] + ["a", {"x": 1}]? I think it can only do that if it knows the type of both. In Python, you don't declare those types. It's just an array. The result is just another array. It's as if the arguments to the operator get cast to [Any], where Any is the union of all possible types, which is then also the result type.
Or, if you want to look at it in another way: every function, class member and variable is super-hyper-extra-polymorphic, with the types limited by one or two sanity checks at runtime for a handful of operations that happen to apply to them. If my f() would avoid the addition operator sometimes, e.g. like this:
It may not be weak, but I'm not sure calling it "strong typing" makes sense.
Or, if you want to look at it in another way: every function, class member and variable is super-hyper-extra-polymorphic, with the types limited by one or two sanity checks at runtime for a handful of operations that happen to apply to them. If my f() would avoid the addition operator sometimes, e.g. like this:
def f(x, y):
if somepredicate(x):
return y
return x + y
It may not be weak, but I'm not sure calling it "strong typing" makes sense.
It is strongly typed as 1 + "1" doesn't work. This has nothing to do with knowing the types beforehand and disallowing the execution.
There is a difference between type _conversion_ and changing the type of a variable. In your example, there's no conversion performed. "a" is dynamically typed, it's type changes with each assignment. The interpreter does not coerce the type at any step.
Your definition of "f" is no different than this in C++: template<typename T, typename U> auto f(T x, U y) -> decltype(x + y) { return x + y; }
"f" is duck typed, effectively. It works fine if operator+ is well defined for "x" and "y", otherwise it fails. The difference, though is a compile time error in C++ vs a runtime error in Python (maybe a linter error with appropriate type hints).
Your definition of "f" is no different than this in C++: template<typename T, typename U> auto f(T x, U y) -> decltype(x + y) { return x + y; }
"f" is duck typed, effectively. It works fine if operator+ is well defined for "x" and "y", otherwise it fails. The difference, though is a compile time error in C++ vs a runtime error in Python (maybe a linter error with appropriate type hints).
I don't think there are generally agreed upon definitions of strong vs. weak, but I think a useful definition is about changing the type _while referring to the same data_ as weakness. In your Python example there, I think we can say it's dynamically + strongly typed because you're assigning the name `a` to three different objects with different types, not changing the interpretation of any actual data in memory. C would be statically + weakly typed, because you can cast the same exact data to whatever type by using `void*`. Rust is static + strong, and I can't really think of a dynamic + weak language. Probably that would be a nightmare to work in.
You're right, there are none. The link from the GP shows several.
> changing the type _while referring to the same data_ as weakness
Python performs arithmetic and comparison on ints mixed with floats. In other languages, you have to cast those first. It does array subscription on a string. You could even see an expression like 3 * "x" as a type change.
> I can't really think of a dynamic + weak language
It depends on what you call a type, I guess. Is [1, 2] a different type than ["a", "b"]? You can't tell in Python, because there is no type declaration. That makes comparison with polymorphism moot. You can think that the type of b in a + b must be "whatever a's plus operator accepts", but I wouldn't call one or two sanity checks before adding strong typing.
> changing the type _while referring to the same data_ as weakness
Python performs arithmetic and comparison on ints mixed with floats. In other languages, you have to cast those first. It does array subscription on a string. You could even see an expression like 3 * "x" as a type change.
> I can't really think of a dynamic + weak language
It depends on what you call a type, I guess. Is [1, 2] a different type than ["a", "b"]? You can't tell in Python, because there is no type declaration. That makes comparison with polymorphism moot. You can think that the type of b in a + b must be "whatever a's plus operator accepts", but I wouldn't call one or two sanity checks before adding strong typing.
> I can't really think of a dynamic + weak language.
Javascript comes to mind. https://www.destroyallsoftware.com/talks/wat
Javascript comes to mind. https://www.destroyallsoftware.com/talks/wat
I would phrase it like this:
Not only is Python dynamically typed, it is strongly dynamically typed.
Not only is Python dynamically typed, it is strongly dynamically typed.
[deleted]
> Of course, if you’re the sort of person who deals with errors in rust by just slapping ? on anything that might fail, then you’re more or less just doing exceptions
Well, no, not really. You typically still explicitly enumerate how your code may fail to use that notation (unless it’s a small utility where you just anyhow it), you also have the fallible values clearly denoted as such, and not everything is fallible. And last, but not least, every time I slap a ‘?’ I tend to quickly evaluate what the failure case is, and if it’s something I can’t do anything about at the moment and should just bubble up, or is it something I can actually handle.
So, no, not the same thing as exceptions at all!
Well, no, not really. You typically still explicitly enumerate how your code may fail to use that notation (unless it’s a small utility where you just anyhow it), you also have the fallible values clearly denoted as such, and not everything is fallible. And last, but not least, every time I slap a ‘?’ I tend to quickly evaluate what the failure case is, and if it’s something I can’t do anything about at the moment and should just bubble up, or is it something I can actually handle.
So, no, not the same thing as exceptions at all!
And also, returning errors by value does not introduce hidden control flow!
In python, you have no way of knowing what(if any) exceptions that a function may throw, so unless you catch everything you have no way of knowing where execution will end up after calling a function.
Not to mention the intentionally(!) thrown exceptions for iterators. In my book, exceptions are goto by another name.
Not to mention the intentionally(!) thrown exceptions for iterators. In my book, exceptions are goto by another name.
Exceptions are goto in disguise to some extent, as it’s forward-jumping only. An important limitation to retain structured programming in this case.
I could show you 20 functions in stdlib itself and popular libraries that return an error but don't mention under what circumstances. They have an internal error type that is a catchall and this isn't exactly an upgrade over exceptions.
Type annotations do not rust make. And lack of types does not Python make.
As someone who has learned to love both languages as tools for the job, I want to write Python like Python and rust like rust.
As someone who has learned to love both languages as tools for the job, I want to write Python like Python and rust like rust.
I want to write correct and maintainable code, and techniques like these make that easier for me. I've found that if it's hard to type hint my code, there's probably a better way to design it.
I find that approaching tasks from the type-first perspective can really help illuminate the problem. I can't imagine trying to code without having both types at the forefront of my mind and all of the IDE's helpful type-related features.
There's little value in maintainable code when the task at hand is to quickly run some statistics and drop it right away. Similar to a Bash script, or, to a lesser extent -- R. But the speed of writing is paramount.
Software development has never been about building correct systems that you see so many here care about. There might be a few contexts where that is the main value being optimized for, but the vast majority of other it is not. I think a lot of smart people are in the wrong industry and it causes them pain. I see so many overworked and stressed out people and it is almost always internal reality causing it.
http://www.laputan.org/mud/
http://www.laputan.org/mud/
Similar to why I like unit-testing. Writing code in a testable way usually straightens out what would have been a thornier first-pass interface. Even if the test never fails, it made me be explicit about my assumptions.
> Type annotations do not rust make. And lack of types does not Python make.
That is never implied. See the first example in the article.
That is never implied. See the first example in the article.
(2020).
Things have arguably become even nicer (although slightly more divergent between the two) since then: Python's `Optional[T]` can now be written as `T | None`, and the core container types can now be annotated directly (e.g. `List[T]` becomes `list[T]`).
Combined via pyO3[1], Python and Rust are a real joy to write together.
[1]: https://github.com/PyO3/pyo3
Things have arguably become even nicer (although slightly more divergent between the two) since then: Python's `Optional[T]` can now be written as `T | None`, and the core container types can now be annotated directly (e.g. `List[T]` becomes `list[T]`).
Combined via pyO3[1], Python and Rust are a real joy to write together.
[1]: https://github.com/PyO3/pyo3
I don’t think the new way to write None is an improvement in this case. I use the pipe as a replacement for Union (of which Optional is a special case, but oddly, there is prefer to use Optional; it reads nicely).
I'm not that deep into type annotations in Python, but I'm not a fan of type-only structures, or mixing concrete with type aliases. That is, `list[T]` is a concrete list, but `Optional[T]` is either a plain T or a plain None. There is no class called Optional.
I prefer the new way, to me it's explicit about the type of Object that can be passed in.
Didn't notice the date, I was confused about the comment about pattern matching
While I respect people that prefer programming without explicit types, I can’t understand it.
I use to think the same, then I tried Elixir. It made me realize the language ecosystem can outweigh cons as big as not having types.
The Elixir ecosystem is top notch. Phoenix, Ecto, Livebook, Mix, Hex, Nx, Axon, Scholar, Explorer, Broadway, and Rustler are libraries and tools that make it hard to use anything else even when the language doesn't have types. I never enjoyed programming in a language more than Elixir.
Tho, they are currently are in the process of researching a type system for Elixir.
The Elixir ecosystem is top notch. Phoenix, Ecto, Livebook, Mix, Hex, Nx, Axon, Scholar, Explorer, Broadway, and Rustler are libraries and tools that make it hard to use anything else even when the language doesn't have types. I never enjoyed programming in a language more than Elixir.
Tho, they are currently are in the process of researching a type system for Elixir.
Elixir is functional and immutable first, (and structs) - that alleviates somewhat the pain of not having a strong type system. I really like the architecture of OTP, but I still find myself using Elixir in anger.
Especially, if I'm trying to bring and old(er) codebase (Elixir 1.9) up to the newest release... and upgrading packages, checking if things are likely to brake etc.
I constantly find myself realising that I cannot trust the compiler, which means more defensive coding, more tests, more time spent...
As an example, upgraded postgrex. The return tuple of one its functions had changed from 2 elements to 3 elements. Did the compiler caught it? No. Had I had dialyzer type spec-s on my private function, I would at least gotten a warning. Thus, morale of the story: always type spec even your private functions , especially if they use 3rd party code.
Especially, if I'm trying to bring and old(er) codebase (Elixir 1.9) up to the newest release... and upgrading packages, checking if things are likely to brake etc.
I constantly find myself realising that I cannot trust the compiler, which means more defensive coding, more tests, more time spent...
As an example, upgraded postgrex. The return tuple of one its functions had changed from 2 elements to 3 elements. Did the compiler caught it? No. Had I had dialyzer type spec-s on my private function, I would at least gotten a warning. Thus, morale of the story: always type spec even your private functions , especially if they use 3rd party code.
{:ok, x} | {:err, y} is my favorite pattern. You can make sure that the caller handles both cases, the ok case and the error case. What you put in y is up to you. If you have dependencies you can decide the bubble up error or return something that you want. We have a bigger system in Elixir and it was trivial to write the correct implementation for exponential backoff to interact with the outside word (calling lots of external services that sometimes time out). With Python this is not as trivial because Guido hates pattern matching and you have catch exceptions and do if else a lot. With Elixir I see immediately while developing if I left out a case. With Python I see it in production when we crashed.
Types are really just a way of applying certain tests to part of your code.
In dynamic languages the typing is just moved to the test suite. We can see that with the way type hints are enforced in Python and Ruby - by running a program that takes those hints and essentially runs a bunch of tests.
If you look at the code of Terraform, for example, you'll see that it spends a lot of time trying to get around the limitations of Go typing.
The rise of dynamic languages in the early 2000s was a direct result of the perceived limitations of the type systems of Java, C++, etc. and the ability to use a different way of constraining code (Test driven development, behaviour driven development).
Now it is swinging back the other way with the implementation of more flexible type systems.
Before too long the limitations of those type systems will start to grate and the pendulum will swing back to dynamic languages again - based upon some new innovations there.
Get used to the back and forth in IT. The static/dynamic cycle is but one of many. (Centralisation/decentralisation is the other main one). You'll find lots of work in IT is reimplementing existing stuff in the current fashionable paradigm.
In dynamic languages the typing is just moved to the test suite. We can see that with the way type hints are enforced in Python and Ruby - by running a program that takes those hints and essentially runs a bunch of tests.
If you look at the code of Terraform, for example, you'll see that it spends a lot of time trying to get around the limitations of Go typing.
The rise of dynamic languages in the early 2000s was a direct result of the perceived limitations of the type systems of Java, C++, etc. and the ability to use a different way of constraining code (Test driven development, behaviour driven development).
Now it is swinging back the other way with the implementation of more flexible type systems.
Before too long the limitations of those type systems will start to grate and the pendulum will swing back to dynamic languages again - based upon some new innovations there.
Get used to the back and forth in IT. The static/dynamic cycle is but one of many. (Centralisation/decentralisation is the other main one). You'll find lots of work in IT is reimplementing existing stuff in the current fashionable paradigm.
No modern type systems are powerful and flexible enough that I don't believe the pendulum will swing back toward dynamic languages, at least to the same degree. Especially with dependent typing becoming more of a thing.
Then again perhaps LLM programming will become the new thing. ;)
Then again perhaps LLM programming will become the new thing. ;)
Types replace some tests and introduce a lot of ceremony.
If this a worthwhile tradeoff depends.
E.g. I wrote c# and Python for a living. Especially in the mid 2010s python was much better for the Microservices I wrote because they are so small that you could unit test them exhaustively. The lack of ceremony and small domain model made development super easy. C# just felt clunky and limiting. You spend lots of time wrangling types which could have been figured out by the compiler.
Nowadays c# has gotten a lot of features that reduce type ceremony and extend their usefulness (nuallibility checks, better tuples etc) so it feels much more ergonomic (unlike java where the culture still goes off on needless verbosity) . Python also got types which I use a lot, it helps to document it for other developers, ides and spot bugs.
E.g. I wrote c# and Python for a living. Especially in the mid 2010s python was much better for the Microservices I wrote because they are so small that you could unit test them exhaustively. The lack of ceremony and small domain model made development super easy. C# just felt clunky and limiting. You spend lots of time wrangling types which could have been figured out by the compiler.
Nowadays c# has gotten a lot of features that reduce type ceremony and extend their usefulness (nuallibility checks, better tuples etc) so it feels much more ergonomic (unlike java where the culture still goes off on needless verbosity) . Python also got types which I use a lot, it helps to document it for other developers, ides and spot bugs.
I'm one of those people. :) Well, it's not so much that I prefer no explicit types, I just dislike how most languages require you to specify them, and to do so at the least opportune time, so I end up preferring languages that don't require them at all. (Ideally, the language would let you start with none and then layer them in as the design gels.)
FWIW, there are definitely ways to do it "right" in languages like Python, but it's a slightly different approach to dev than in other languages, so people create a mess if they don't shift, or if they are just untrained/sloppy/lazy/whatever. But when you have a team of people doing it right, it's really quite nice, even as the codebase grows large.
But if even just one person on the team can't be trusted to do it that way, you're probably better off in some other language.
FWIW, there are definitely ways to do it "right" in languages like Python, but it's a slightly different approach to dev than in other languages, so people create a mess if they don't shift, or if they are just untrained/sloppy/lazy/whatever. But when you have a team of people doing it right, it's really quite nice, even as the codebase grows large.
But if even just one person on the team can't be trusted to do it that way, you're probably better off in some other language.
What do you think about e.g. Roc having whole-program type inference that always works and always infers the most general types?
In isolation, I love the idea. I don't have a strong sense of how the tradeoffs would affect things in practice though.
Perhaps they are not a big deal, but if they are too restrictive, then maybe the next best thing would be to have anything go (like in Python or Javascript), but the compiler guides you to where it's having trouble inferring things.
Perhaps they are not a big deal, but if they are too restrictive, then maybe the next best thing would be to have anything go (like in Python or Javascript), but the compiler guides you to where it's having trouble inferring things.
Many programmers don't get a choice in the language they have to use. Or they don't get a choice in the language they learn first. In either case, it is best to steel-man the case for the language you're using at any given time. Ideally, you should do this once for a language with explicit types (e.g. C++/Rust), once for a language with dynamic/implicit types (e.g. JS, Python3) and maybe once for a gradual typing system like TypeScript.
Then you can appreciate which tool to use and when, rather than getting distracted by strange religious notions about how "this language is better than that one". It also prevents you from forcing conventions the language doesn't align well with, and that if you're going to _try_ to do so - you should do so gradually and very carefully.
Then you can appreciate which tool to use and when, rather than getting distracted by strange religious notions about how "this language is better than that one". It also prevents you from forcing conventions the language doesn't align well with, and that if you're going to _try_ to do so - you should do so gradually and very carefully.
This was something that was in vogue in the mid / late 2000s, then Haskell became the cool kid language and types are back.
The thing that drives me crazy about Python typing is that I need to import stuff to make full use of the type system. Like I get that the typing is option because that would be a huge language overhaul, but it feels weird that so many tools get hidden inside libraries (standing or otherwise).
The thing that drives me crazy about Python typing is that I need to import stuff to make full use of the type system. Like I get that the typing is option because that would be a huge language overhaul, but it feels weird that so many tools get hidden inside libraries (standing or otherwise).
AFAIK, you only need to import the actual types that you're using (`from typing import List, Iterable, Optional, ...`). What else is there?
In newer Python versions, you don't need most of those imports either:
foo: list[str] = ["a"]
bar: int | None = 1But why do we have to import any? They should be part of the core language by now. The lack of boldness of Python core developers is sometimes very annoying.
Most of it is part of the core language now.
Yep the only one I've found myself importing as of late is Any.
[deleted]
I can make two observations that explain some of it.
Other aspects of a language can matter more than the types. Python, for example, was a well constructed language in many ways that make up for much of its disadvantage on the typing side.
Many commonly used statically typed languages have poor type systems. They will both fail to detect some important errors (like null pointers) and get in the way when writing more generic code.
Other aspects of a language can matter more than the types. Python, for example, was a well constructed language in many ways that make up for much of its disadvantage on the typing side.
Many commonly used statically typed languages have poor type systems. They will both fail to detect some important errors (like null pointers) and get in the way when writing more generic code.
For me it is that I learned to program in dynamically typed languages and have used them for decades. I am interested in using types, but it is not enforced at my work, I am not given time to learn how to use them at work, and I’m too busy at home to learn it on my own.
I find it helps to think about it like this:
Whether you specify them or not, the data moving around your application _has_ types.
Strings, ints, booleans - they're all there. You're just keeping that information in your head while you work.
By writing them in the code, you're simply offloading all of that and saying "I trust the computer to validate this better than I can." Which is exactly the kind of thing we should be trusting computers to do. They won't forget what type something is 5 function calls deep and they won't get tired from lack of sleep.
I know I prefer having the computer tell me I made a mistake while I'm developing something rather than a user emailing me after I've shipped it.
Whether you specify them or not, the data moving around your application _has_ types.
Strings, ints, booleans - they're all there. You're just keeping that information in your head while you work.
By writing them in the code, you're simply offloading all of that and saying "I trust the computer to validate this better than I can." Which is exactly the kind of thing we should be trusting computers to do. They won't forget what type something is 5 function calls deep and they won't get tired from lack of sleep.
I know I prefer having the computer tell me I made a mistake while I'm developing something rather than a user emailing me after I've shipped it.
Sure, I just have no idea how to write them, what the syntax is, where to write them, etc. Since my work doesn’t care to give me time to find out, and I don’t have time to do it on my own, it isn’t going to happen until it becomes a priority at work.
Type hints are an absolute must for me in modern Python, starting with one really simple reason: All the tooling can use them.
That means:
Oh, and to everyone using LLM in their editing environment: They understand type hints as well, and can make good use of them.
And for when I am feeling especially lazy: I have a keybind that sends the current code block to the LLM asking it to add type hints and a docstring ;-)
That means:
- The static checker / linter
- My LSP server
- Documentation Generators
- Testing frameworks
Oh, and to everyone using LLM in their editing environment: They understand type hints as well, and can make good use of them.
And for when I am feeling especially lazy: I have a keybind that sends the current code block to the LLM asking it to add type hints and a docstring ;-)
There is also improving support with tools like cython.
"In spite of the topic of this blog, I think you should always aim to write code which is idiomatic to the language you’re using. If only so it’s accessible to whoever has to maintain your code after you’re gone."
Come on. This approach is almost always used to justify shitty practices just because they are idiomatic (read: the language has a severe limitation in the area)
This is a really good example - if you can use a library with proper option types or other constructs which reduce bugs - go ahead! Similar with C# and libraries like language-ext - zero idiomacity but in some cases, can replace hundreds of lines of code while preventing obvious bugs.
Whoever has to maintain it afterwards should be grateful about being exposed to different styles and language idioms instead of complaining that he's been taken out of his comfort zone, in my opinion.
Come on. This approach is almost always used to justify shitty practices just because they are idiomatic (read: the language has a severe limitation in the area)
This is a really good example - if you can use a library with proper option types or other constructs which reduce bugs - go ahead! Similar with C# and libraries like language-ext - zero idiomacity but in some cases, can replace hundreds of lines of code while preventing obvious bugs.
Whoever has to maintain it afterwards should be grateful about being exposed to different styles and language idioms instead of complaining that he's been taken out of his comfort zone, in my opinion.
Conversely, your approach has been used to justify the most abhorrent programming practices even perpetrated on mankind. What you get are not carefully chosen abstractions, but custom mini-languages with no specification masquerading as the original language by redefining basic primitives, a dependency hell to arbitrary and non maintained weekend projects someone dumped on github, which would have been easy to write in terms of the standard library - if only the original author knew its capabilities - but now they must be reverse-engineered from context etc.
The maintainer is not ready for a paradigm shift because they will not own your program in perpetuity. Even you yourself ran away and abandoned it, so how can you expect someone else to love it more? No, the maintainer has a limited time and attention span to work on a specific bug or feature, and your code is either maintainable or it's not.
The maintainer is not ready for a paradigm shift because they will not own your program in perpetuity. Even you yourself ran away and abandoned it, so how can you expect someone else to love it more? No, the maintainer has a limited time and attention span to work on a specific bug or feature, and your code is either maintainable or it's not.
I think you assume that the choice is between "good, idiomatic code" and "non-idiomatic, horrible, abandoned mess".
Most of what you said are problems with simply not documenting code and writing code well enough; being idiomatic slightly helps in that by making it more familiar, but that's just a masking of the fact that the program is written horribly with no documentation.
Libraries like language-ext are absolutely not "badly chosen abstractions" or "dependency hell".
Also, "your code is either maintainable or it's not." is heavily subjective. Maintainability is often also used to criticise stylistic choices like variable naming, tabs vs. spaces, brace placement and other things which might be non-standard but don't detract from readability at all. If it did, there wouldn't be so many different standards across languages.
Most of what you said are problems with simply not documenting code and writing code well enough; being idiomatic slightly helps in that by making it more familiar, but that's just a masking of the fact that the program is written horribly with no documentation.
Libraries like language-ext are absolutely not "badly chosen abstractions" or "dependency hell".
Also, "your code is either maintainable or it's not." is heavily subjective. Maintainability is often also used to criticise stylistic choices like variable naming, tabs vs. spaces, brace placement and other things which might be non-standard but don't detract from readability at all. If it did, there wouldn't be so many different standards across languages.
The basic idea is that, keeping all other things constant, the code with the least surprises and paradigm shifts is the more maintainable one, because almost all code innovations have a learning curve that pays back the more you use the concept.
The maintainer is by definition on the losing end of that tradeoff, because they have to learn lots of things about your code and have limited opportunities to apply that learning. So poor code without surprises is much better than poor code that tries to take you outside of your comfort zone.
The maintainer is by definition on the losing end of that tradeoff, because they have to learn lots of things about your code and have limited opportunities to apply that learning. So poor code without surprises is much better than poor code that tries to take you outside of your comfort zone.
But it also raises the average software quality by educating the maintainer. If all you do is write Python the same way your entire life, your code quality won't be that great. This is a fairly small sacrifice for a greater societal gain.
Python itself was designed with the intent of pressuring programmers to write Python in one specific way. The problems that crop up in large Python codebases are probably the result of using Python outside of its intended scope of scripting and toy programs.
Yes. But people do use (or more like, misuse) Python for different things such as large systems. In cases like that, others still scream about "pythonicity" and "idiomaticness" just for the sake of language policing, even if those guidelines make zero sense.
I had the misfortune of using Python for.... larger systems than it was intended for, and the code reviewer basically screamed at me for daring to use staticmethods instead of free-floating functions. Even though it was an internal implementation detail and it was exactly the point that the function itself shouldn't be discoverable if you import the file. But convention and being pythonic trumps everything...
So yes, I think that people who criticise others for not writing "idiomatic" code just want to nitpick most of the time.
I had the misfortune of using Python for.... larger systems than it was intended for, and the code reviewer basically screamed at me for daring to use staticmethods instead of free-floating functions. Even though it was an internal implementation detail and it was exactly the point that the function itself shouldn't be discoverable if you import the file. But convention and being pythonic trumps everything...
So yes, I think that people who criticise others for not writing "idiomatic" code just want to nitpick most of the time.
There was another one of these posts floating around recently https://kobzol.github.io/rust/python/2023/05/20/writing-pyth...
My personal experience is that while this would be a really good way of getting people to write quality Python code, a lot of python users (data scientists) are not software engineers and don't have the discipline to do this without very strict PR reviews
My personal experience is that while this would be a really good way of getting people to write quality Python code, a lot of python users (data scientists) are not software engineers and don't have the discipline to do this without very strict PR reviews
NewType and TypeGuard are pretty close to a trait. I really want this approach to be awesome but it’s actually ironically seemingly more work to write python like rust than to just write rust. I say this because if you write python like rust then you have to re-invent and unit test and e2e test a bunch of stuff you get for free in rust.
I do think Immutables.Map + NewType + TypeGuard is enough for some awesome category theory stuff, but it’s a bit harder with Pydantic if you want to handle nested stuff (I guess I just need to make more data classes) and then another issue is you have to write custom code to check if a thing implements a given trait. Then you have to use that.
Making sure all of that basic stuff you get for free in rust works correctly in your Python implementation is a bit of work! Worth it? unclear, still testing, I do love me some duck typing!
I do think Immutables.Map + NewType + TypeGuard is enough for some awesome category theory stuff, but it’s a bit harder with Pydantic if you want to handle nested stuff (I guess I just need to make more data classes) and then another issue is you have to write custom code to check if a thing implements a given trait. Then you have to use that.
Making sure all of that basic stuff you get for free in rust works correctly in your Python implementation is a bit of work! Worth it? unclear, still testing, I do love me some duck typing!
> At this point, you’re maybe wondering why we didn’t just write the project in Rust?
Err, aren’t Python and Rust designed to write completely different kinds of programs?
Python => scripting, data stuff, unsafe and slow, quick’n’dirty PoCs
Rust => compiled, system stuff, robust and fast, production work.
If the author is balancing the 2, they probably should have used C# !
Err, aren’t Python and Rust designed to write completely different kinds of programs?
Python => scripting, data stuff, unsafe and slow, quick’n’dirty PoCs
Rust => compiled, system stuff, robust and fast, production work.
If the author is balancing the 2, they probably should have used C# !
Designed with different goals absolutely, they have very different use cases that they individually accel at.
This however does not stop people from trying to use their one preferred language fer beyond it's focus area.
IMO one of the major turning points in a developers career is when they learn that finding the right tool for the problem is always the game you are playing. But this almost always takes learning this lesson the hard way by pushing a language one know very well past its breakpoint.
This however does not stop people from trying to use their one preferred language fer beyond it's focus area.
IMO one of the major turning points in a developers career is when they learn that finding the right tool for the problem is always the game you are playing. But this almost always takes learning this lesson the hard way by pushing a language one know very well past its breakpoint.
I'm there. I write Ruby, JavaScript, Python, Java, Bash, and Rust for work.
Rust is surprisingly good at many tasks I would otherwise pick "the proper" tool for. A web service, a quick frontend in wasm (leptos), a hackish commandline tool, or, indeed, that low-level/embedded thing that Rust was "meant" for.
Rust is surprisingly good at many tasks I would otherwise pick "the proper" tool for. A web service, a quick frontend in wasm (leptos), a hackish commandline tool, or, indeed, that low-level/embedded thing that Rust was "meant" for.
C# really is an absolute gem! Don't know why more people don't give it a try.
I'd rather write rust like python
Im hoping Mojo supports essentially that.
Sounds like Nim.
> Of course, one advantage Python has over Rust in this case is being able to give fields default values.
https://doc.rust-lang.org/std/default/trait.Default.html
https://doc.rust-lang.org/std/default/trait.Default.html
A few weeks ago I tweeted this [1]:
Frankly, writing Python code in 2023 with types annotations, dataclasses, structural pattern matching, and an editor (Kate ♥) that supports LSP with the mypy plugin for pylsp installed is just as cool as writing OCaml code!
The addition of the `match` construct since Python 3.10 is indeed a huge additional improvement that is called for in the linked post which dates back from before it was a thing (from the blogpost: “One thing I miss from Rust is its ‘richer’ enums, and pattern matching”).
[1], in french https://twitter.com/p4bl0/status/1667300422426476544
Frankly, writing Python code in 2023 with types annotations, dataclasses, structural pattern matching, and an editor (Kate ♥) that supports LSP with the mypy plugin for pylsp installed is just as cool as writing OCaml code!
The addition of the `match` construct since Python 3.10 is indeed a huge additional improvement that is called for in the linked post which dates back from before it was a thing (from the blogpost: “One thing I miss from Rust is its ‘richer’ enums, and pattern matching”).
[1], in french https://twitter.com/p4bl0/status/1667300422426476544
Convincing my colleagues to use type hints in Python has been very easy. The benefits are obvious and immediate. To the point where I wonder why there was even a time when it was acceptable not to have them.
Wasn't this posted here less than a month ago?
OP has too much free time on their hands and is doing something worthless, but that somehow attracts a lot of "me too" comments. Guess this only indicates how bad the average professional or an aspiring professional is in this field...
OP has too much free time on their hands and is doing something worthless, but that somehow attracts a lot of "me too" comments. Guess this only indicates how bad the average professional or an aspiring professional is in this field...
Very similar article a month ago: https://news.ycombinator.com/item?id=36018621
I didn't know about Protocol! How does this differ from ABC?
You need to explicitly inherit from an ABC to be considered an instance of it. With Protocols you just need to match the interface, duck-typing style.
(You can also do this with ABCs by writing custom __subclasscheck__ and __instancecheck__ functions, but those can't be statically checked, they only work at runtime)
AFAIK protocols actually use ABCs internally(?) Just in a more constrained form that static type checkers can understand.
(You can also do this with ABCs by writing custom __subclasscheck__ and __instancecheck__ functions, but those can't be statically checked, they only work at runtime)
AFAIK protocols actually use ABCs internally(?) Just in a more constrained form that static type checkers can understand.
Trying to write Objects in Python is already bad enough, I can't imagine trying to force any other kind of complex crap through that language. If you want something that isn't Python, your best bet is not using Python.