Whenever: Typed and DST-safe datetimes for Python(github.com)
github.com
Whenever: Typed and DST-safe datetimes for Python
https://github.com/ariebovenberg/whenever
150 comments
https://dev.arie.bovenberg.net/blog/python-datetime-pitfalls... highlighted yet another thing that I hadn't previously considered and makes me plug <https://infiniteundo.com/post/25326999628/falsehoods-program...> <https://news.ycombinator.com/item?id=4128208>
Discussed on HN back then:
https://news.ycombinator.com/item?id=39417231 (147 comments)
https://news.ycombinator.com/item?id=39417231 (147 comments)
I am a seasoned programmer but whenever I deal with datetime objects I do my best with unit tests and then just hope none of these “edge” cases apply to us. Meaning: I have no idea really how it works under the hood.
Now at least there’s an LLM that might spot a bug every now and then so that’s nice.
Now at least there’s an LLM that might spot a bug every now and then so that’s nice.
[deleted]
Excellent read.
Ah nice it solves the Liskov violation that the standard library has. In the standard library, dates can be compared with <, and datetimes are dates. But compare a datetime with a date with <, and you get an error. This drove me nuts at work recently.
I wonder what benefits this choice has that outweigh the risks of this behavior.
I wonder what benefits this choice has that outweigh the risks of this behavior.
What do you expect? There are so many ways to handle this behvaiour it's pretty obvious why this is not allowed. Do you take datetime.date and then compare? Do you assume all dates are datetimes at midnight?
I’ve tried Arrow, Delorean, and Pendulum, plus the stdlib datetime of course, and settled on Whenever. It fits what I actually do with datetimes better, plus it seems more actively maintained. With the others I always seem to have a nagging feeling in the back of my mind that I am missing a whole load of edge cases. With Pendulum that seems more baked into the API.
Sounds like we need an industry/language-wide test suite to check these many date/time/calendar libraries against. Like the browser acid tests, though focused to baseline functionality only.
https://en.wikipedia.org/wiki/Acid3
I like this new lib (Thank You) but the name unfortunately implies the opposite of what it is. "Whenever" sounds like you don't care, but you'd only be using this if you did care! Also Shakira, haha. Hmm, pedantic is taken. Timely, precise, punctual, meticulous, ahorita, pronto, etc. I like that temporal name.
Finally, none of these links mention immutability, but it should be mentioned at the top.
https://en.wikipedia.org/wiki/Acid3
I like this new lib (Thank You) but the name unfortunately implies the opposite of what it is. "Whenever" sounds like you don't care, but you'd only be using this if you did care! Also Shakira, haha. Hmm, pedantic is taken. Timely, precise, punctual, meticulous, ahorita, pronto, etc. I like that temporal name.
Finally, none of these links mention immutability, but it should be mentioned at the top.
Without the slightest sense of irony, I actually strongly suspect such a test suite would only be valid at one moment in time, since the timezone legislation is almost continuously in flux. That's why <https://www.iana.org/time-zones> and its friend <https://www.oracle.com/java/technologies/javase/tzupdater-re...> exist. As if to illustrate my point, the latest update was 2025-03-22, presumably nuking any such conformance test from Mar 21st
It would have to take the real world into account, no? Additionally it could test various timezone definition permutations without necessarily being dependent on a real one.
In that case, you'd have unit tests that confirm behaviors like compatibility or failure of some operations between types and integrations tests which pull an up to date DB of rules and tests against that.
Am I the only one to stick with the std lib, read the docs and changelogs carefully, and implement functions I really need the way my application makes use of them?
I learned the hard way, that dependencies kill projects.
Not saying this isn't great, thanks for creating it! It does have its use cases, of course.
I learned the hard way, that dependencies kill projects.
Not saying this isn't great, thanks for creating it! It does have its use cases, of course.
> Am I the only one to stick with the std lib, read the docs and changelogs carefully
I work in healthcare. If I have a choice between "reading docs/changelogs carefully, implementing functions", and "adding an extra dependency", I'm taking the dependency every single time.
I don't want footguns in my code, I don't want code I have to write and test myself, and I don't want to have to become an expert in a domain before I can write something that serves my purpose.
For the datetime library, specifically, I'm switching to whenever for everything, because I've been bitten by conversions and naive/aware datetime confusion too many times.
I work in healthcare. If I have a choice between "reading docs/changelogs carefully, implementing functions", and "adding an extra dependency", I'm taking the dependency every single time.
I don't want footguns in my code, I don't want code I have to write and test myself, and I don't want to have to become an expert in a domain before I can write something that serves my purpose.
For the datetime library, specifically, I'm switching to whenever for everything, because I've been bitten by conversions and naive/aware datetime confusion too many times.
My hope is that a lib like this one or similar could rally mindshare and become integrated as the new standard, and adopted by the wider developer community. In near term, it comes down to trade-offs. I see no decision that works for all use cases. Dependencies introduce ticking time bombs, stdlibs should be correct and intuitive, but at least when not they are usually well tested and maintained, but when stdlib don't meet urgent production needs you have to do something.
Link to Tom Scott & Computerphile from 10y ago on tz madness. https://www.youtube.com/watch?v=-5wpm-gesOY
Link to Tom Scott & Computerphile from 10y ago on tz madness. https://www.youtube.com/watch?v=-5wpm-gesOY
Is there any part of the Python standard library written in Rust? I would see that as a big impediment to having Whenever adopted as standard.
I don’t think so, but they do offer a pure-Python version as well.
https://whenever.readthedocs.io/en/latest/faq.html#how-can-i...
https://whenever.readthedocs.io/en/latest/faq.html#how-can-i...
True, but does it suffer in efficiency? I wish they had included the pure Python version in their benchmark.
[deleted]
It's basically what happened in Java. Everyone used jodatime, and they took great inspiration from that when making the new standard time api for java 8.
> I work in healthcare. If I have a choice between "reading docs/changelogs carefully, implementing functions", and "adding an extra dependency", I'm taking the dependency every single time.
This kinda sums up the sorry state of software engineering. People can't even be bothered to read docs but will just blindly install a package just because someone was able to package it and upload it to PyPI.
Taking on a dependency does not relieve you of reading docs, but it also adds a further burden as you now need to trust the code. The stdlib is much more heavily tested and documented than any 3rd party library will be.
This kinda sums up the sorry state of software engineering. People can't even be bothered to read docs but will just blindly install a package just because someone was able to package it and upload it to PyPI.
Taking on a dependency does not relieve you of reading docs, but it also adds a further burden as you now need to trust the code. The stdlib is much more heavily tested and documented than any 3rd party library will be.
The stdlib datetime module is in more of a sorry state than certain third party libraries and is full of footguns, as you, a read-the-docs-and-changelogs-carefully person, can surely tell from the myriad deprecations and warnings added over the years, not to mention the confusing params like tz and tzinfo. Heavily tested doesn’t mean shit when the semantics of the API is fundamentally flawed and can’t be updated.
> Taking on a dependency does not relieve you of reading docs, but it also adds a further burden as you now need to trust the code. The stdlib is much more heavily tested and documented than any 3rd party library will be.
Sure, but the opposite applies as well. Sticking with the flawed stdlib means you are trusting that every single future developer is as careful in reading all the docs as you are - even when it's someone reviewing that random trivial-looking patch which is secretly hiding a massive footgun. A junior developer submitted a five-line patch touching datetime? Better schedule several hours for a proper analysis!
Or you can write your own wrapper code, of course. Which will almost certainly have worse documentation and testing than a popular third-party library.
External libraries aren't evil. When chosen properly, they relieve you of burdens. You shouldn't grab any random "leftpad"-like dependency like they are going out of fashion, but something as complicated as timezone handling is best left to code written by domain experts - which means using a library.
Sure, but the opposite applies as well. Sticking with the flawed stdlib means you are trusting that every single future developer is as careful in reading all the docs as you are - even when it's someone reviewing that random trivial-looking patch which is secretly hiding a massive footgun. A junior developer submitted a five-line patch touching datetime? Better schedule several hours for a proper analysis!
Or you can write your own wrapper code, of course. Which will almost certainly have worse documentation and testing than a popular third-party library.
External libraries aren't evil. When chosen properly, they relieve you of burdens. You shouldn't grab any random "leftpad"-like dependency like they are going out of fashion, but something as complicated as timezone handling is best left to code written by domain experts - which means using a library.
> The stdlib is much more heavily tested and documented than any 3rd party library will be.
You initially said you write your own code instead of using libraries, I replied to that, and now it's that you use the stdlib instead of libraries. I won't argue against shifting goalposts.
You initially said you write your own code instead of using libraries, I replied to that, and now it's that you use the stdlib instead of libraries. I won't argue against shifting goalposts.
The poster who you just replied to is not the same person who started the thread. I am yet another person.
In addition, the original post begins with, "Am I the only one to stick with the std lib". The goalposts are stable.
In addition, the original post begins with, "Am I the only one to stick with the std lib". The goalposts are stable.
Again, the library in the article removes some stdlib footguns. The footguns are there, and if you use the stdlib, you need to remove (or avoid) them yourself. There's no free lunch, and you'll either need to remove them yourself (and test that code), avoid them (and constantly remember to not hit them), or use another library.
It's not a choice between "using a dependency" or "using something in the stdlib", where all other code remains the same, otherwise there would be no point to writing a library, as it would offer nothing over `datetime`.
It's not a choice between "using a dependency" or "using something in the stdlib", where all other code remains the same, otherwise there would be no point to writing a library, as it would offer nothing over `datetime`.
That’s why I’m a software developer though, because I learned the footguns in the standard library. I use it before third party libraries whenever I can. I can’t understand the fear, just learn your job.
If your approach to footguns is "just learn your job", I'm sorry, but you aren't a very good software developer.
[deleted]
Did OP blindly install the package?
Or do it with sight?
Or do it with sight?
> People can't even be bothered to read docs but will just blindly install a package just because someone was able to package it and upload it to PyPI.
That's a straw man argument. No one said "blindly". You can very well carefully consider the pros and cons of adding a dependency and arrive at the conclusion that it makes sense. Many PyPI packages are in the Debian stable repositories, you could use that as an additional barrier as well.
That's a straw man argument. No one said "blindly". You can very well carefully consider the pros and cons of adding a dependency and arrive at the conclusion that it makes sense. Many PyPI packages are in the Debian stable repositories, you could use that as an additional barrier as well.
To be fair, comment^^ actually said
I think it matters a lot how much one can trust a 3rd party library, how much it is used, how much it is maintained etc. Moreover, it also matters how central and important this is to what you are actually doing, for example if the datetimes I come across are pretty much specific and with limited scope, I would probably care about reading docs less than if I am reading data with datetimes that may be totally wild. Furthermore, there are some libraries that are just considered the standard in some fields. If I use python I call pandas to read csvs, I am obviously not gonna write my own csv parser. It will also make your code more readable for others that already know and use the same libraries if they are so standard. But that's probably a very small minority of libraries that people actually use in certain languages.
choice between "reading docs/changelogs carefully, implementing functions", and "adding an extra dependency"
I think it matters a lot how much one can trust a 3rd party library, how much it is used, how much it is maintained etc. Moreover, it also matters how central and important this is to what you are actually doing, for example if the datetimes I come across are pretty much specific and with limited scope, I would probably care about reading docs less than if I am reading data with datetimes that may be totally wild. Furthermore, there are some libraries that are just considered the standard in some fields. If I use python I call pandas to read csvs, I am obviously not gonna write my own csv parser. It will also make your code more readable for others that already know and use the same libraries if they are so standard. But that's probably a very small minority of libraries that people actually use in certain languages.
> If I use python I call pandas to read csvs
So it's you that isn't just using the built in csv parser in this project I inherited. Come back and repent.
So it's you that isn't just using the built in csv parser in this project I inherited. Come back and repent.
How could it possibly be true that libraries that are so commonly used that you consider them “the standard” are so infrequently used as to be a “very small minority” of libraries people actually use?
I am not sure I understand the comment. I used pandas as an example of sth I frequently encounter when dealing with python code dealing with data frames. Is it not commonly used? What’s your argument here, I am genuinely asking.
I'm trying to square two aspects of your comment:
> Some libraries are so common that basically everybody uses them, to the point that they're considered the standard and they don't count when I raise complaints about people pulling in dependencies
> "But that's probably a very small minority of libraries that people actually use in certain languages."
It seems like by definition, the majority of libraries that people are actually using are going to be the popular ones that everybody is using.
> Some libraries are so common that basically everybody uses them, to the point that they're considered the standard and they don't count when I raise complaints about people pulling in dependencies
> "But that's probably a very small minority of libraries that people actually use in certain languages."
It seems like by definition, the majority of libraries that people are actually using are going to be the popular ones that everybody is using.
I get where you’re coming from. There’s a price you pay though eventually. You’ll have to thoroughly vet all your dependencies for malicious code at some point. Otherwise how do you have any clue what you’re running?
There are so many footguns in the datetime lib.
That's why I use a Flake8 plugin to prohibit especially egregious footguns.
https://github.com/jkittner/flake8-ban-utcnow
That's why I use a Flake8 plugin to prohibit especially egregious footguns.
https://github.com/jkittner/flake8-ban-utcnow
Honestly yeah who in tarnation created that function and called it utcnow
These things are really frustrating
These things are really frustrating
Yeah, utcnow is completely broken. They should have fixed it when they created datetime.timezone.utc, but they didn't. The recommendation is to use datetime.datetime.now(datetime.timezone.utc) instead. utcnow should be deprecated and eventually removed.
You are a sad minority, IME. I’m right there with you. I extended the uuid library to generate UUIDv7, based off of the RFC. It’s pretty easy to implement, as it turns out. Overruled, because “we don’t want to have to maintain additional code.” As if the ABI for bitshifts is going to change?!
There’s an out of sight, out of mind mentality with dependencies.
As long as there is a conscious decision to build or ‘buy’, it’s fine. I think some people can be a little too careless with adding dependencies though, not realising they can have an equal if not greater maintenance burden.
As long as there is a conscious decision to build or ‘buy’, it’s fine. I think some people can be a little too careless with adding dependencies though, not realising they can have an equal if not greater maintenance burden.
this is a great idea if you want to slow down your project. most projects start with few rules and “best practices” like this. everyone is free to pull in dependencies as needed. because they are needed. but then once the project grows larger, those who have been around longer want to reverse course and gatekeep dependencies. but this is the opposite of what helped the project grow initially. and later contributors have a harder time making similar progress because they have to fight to add basic libraries. ensuring that efficiency per engineer goes down
I think this is fairly unrealistic. Does all your datetime manipulation involve proper use of the fold parameter as indicated in the article?
Are you saying you never pull in dependencies? Why stop there, why not re-implement the std lib as well? Surely there is a sensible middle ground: If you only need a small part of a dependency, consider implementing it. If you make heavy use of a dependency and want to benefit of years if not decades of dedicated developers testing and maturing its code, with a large community who has already stepped in all pitfalls you might step into and collectively encountered all the edge cases, just use the dependency.
Creating from scratch also creates hidden debt, it's just moved onto yourself. Especially when working with dates and timezones.
I cannot imagine having the spare time to invest in building date/time foundations and maintaining them through changes to DST timing and country/time zone changes.
The only crazier idea I can think of is implementing character encoding conversions myself.
The only crazier idea I can think of is implementing character encoding conversions myself.
Seriously.
Nobody needs a package for "left-pad", which is the most infamous example.
But there are a lot of areas where it's not a good use of your time to reinvent the wheel. There's no moral virtue in writing everything yourself. And popular packages have gone through a more bug-finding and bug-fixing than your personal code probably ever will.
Nobody needs a package for "left-pad", which is the most infamous example.
But there are a lot of areas where it's not a good use of your time to reinvent the wheel. There's no moral virtue in writing everything yourself. And popular packages have gone through a more bug-finding and bug-fixing than your personal code probably ever will.
A library that goes "poof" when you need to upgrade it is also a hidden debt
A member of staff who goes “poof” as the only one who understands your wrapper library that has a critical bug is a more common kind of hidden debt.
There is also a distinction to be made between "technical" and "political" dependencies. Technical dependencies usually track some spec or common consensus, and political dependencies are e.g. timedate-library. Political dependencies are almost like business logic, because they have to track changing political decision made all over the world (or be local to some country).
Timedates are hard, and units may require even harder historical/present "political" tracking as how they are defined, and I would never want to maintain this kind of dependency: https://github.com/ryantenney/gnu-units/blob/master/units.da...
And what comes to timedate problems, I try to keep it simple if the project allows: store and operate with UTC timestamps everywhere and only temporarily convert to to local time (DSTs applied, if such) when displaying it in user-facing UI. This functionality/understanding can be locked into own 20-line microlibrary-dependency, which forces its responsible person to understand country's timezone and when e.g. DST changes and where/how/who decides DST changes and what APIs is used to get UTC time (and of course, its dependencies, e.g. NTP, and its dependencies. e.g. unperturbed ground-state hyperfine transition frequency of the caesium-133 atom, which result is then combined with the Pope Gregory XIII's Gregorian calendar, which is a type of solar calendar mixed with religious event for fixing the time, which which is then finally corrected by rewinding/forwarding "the clock" because its too bright or dark for the politicians).
Timedates are hard, and units may require even harder historical/present "political" tracking as how they are defined, and I would never want to maintain this kind of dependency: https://github.com/ryantenney/gnu-units/blob/master/units.da...
And what comes to timedate problems, I try to keep it simple if the project allows: store and operate with UTC timestamps everywhere and only temporarily convert to to local time (DSTs applied, if such) when displaying it in user-facing UI. This functionality/understanding can be locked into own 20-line microlibrary-dependency, which forces its responsible person to understand country's timezone and when e.g. DST changes and where/how/who decides DST changes and what APIs is used to get UTC time (and of course, its dependencies, e.g. NTP, and its dependencies. e.g. unperturbed ground-state hyperfine transition frequency of the caesium-133 atom, which result is then combined with the Pope Gregory XIII's Gregorian calendar, which is a type of solar calendar mixed with religious event for fixing the time, which which is then finally corrected by rewinding/forwarding "the clock" because its too bright or dark for the politicians).
A service goes poof, a library either slowly deteriorates or breaks just as easily as a self-written one (if an underlying platform breaks it for some reason). The self-written one is maintained by 1 person, the other is used by 100+ people who could jump in a collaborate on its fixing.
I would still rather using a library for dates, a million times so.
I would still rather using a library for dates, a million times so.
> a library either slowly deteriorates or breaks just as easily as a self-written one
Yes, I agree with this
> The self-written one is maintained by 1 person, the other is used by 100+ people who could jump in a collaborate on its fixing.
Libraries that have 100 people collaborating on it are very few
Most likely you'll have to vendor it and fix whatever issues you have.
Even worse when it's a dependency of a dependency and you also use it, so, let's say a dependency of yours bumps the version but this breaks your code. (Not sure this breaks only in python or in js as well, but it's possibly that it does)
Yes, I agree with this
> The self-written one is maintained by 1 person, the other is used by 100+ people who could jump in a collaborate on its fixing.
Libraries that have 100 people collaborating on it are very few
Most likely you'll have to vendor it and fix whatever issues you have.
Even worse when it's a dependency of a dependency and you also use it, so, let's say a dependency of yours bumps the version but this breaks your code. (Not sure this breaks only in python or in js as well, but it's possibly that it does)
Honest question: why is it so common for software developers to not upgrade their dependencies on regular intervals?
I can’t for the life of me figure out why. If you update everything incrementally you bypass the upgrade version problem when you’re so far behind that so much has changed that it becomes an overwhelming burden.
I think frozen dependencies are a big anti pattern, and places where I work that regularly update their deps tended to have better software practices generally
I can’t for the life of me figure out why. If you update everything incrementally you bypass the upgrade version problem when you’re so far behind that so much has changed that it becomes an overwhelming burden.
I think frozen dependencies are a big anti pattern, and places where I work that regularly update their deps tended to have better software practices generally
Because time spent updating dependencies won't contribute to adding new features.
Besides, any update risks breaking stuff. Not freezing dependencies isn't an option, because that means any commit can cause breakage in a completely unrelated part of the codebase, in a way which can be extremely confusing to debug. And you don't really want to install the very newest versions either, better wait a week or two for someone else to run into bugs and release a .1 version.
The sweet spot is somewhere in the middle: update often enough to avoid updates becoming a massive burden, stick with fixed versions between updates. I reckon it's best to just schedule some dedicated time for it every month or two.
Besides, any update risks breaking stuff. Not freezing dependencies isn't an option, because that means any commit can cause breakage in a completely unrelated part of the codebase, in a way which can be extremely confusing to debug. And you don't really want to install the very newest versions either, better wait a week or two for someone else to run into bugs and release a .1 version.
The sweet spot is somewhere in the middle: update often enough to avoid updates becoming a massive burden, stick with fixed versions between updates. I reckon it's best to just schedule some dedicated time for it every month or two.
It typically takes me maybe an hour to update my dependencies? I run type checks and e2e tests as part of that to have a relatively high degree of confidence that nothing has been broken. Also splitting the change into multiple steps (i.e. do minor/patch upgrades first, check nothing's broken, run a major upgrade, check nothing's broken, etc) means it's fairly to see where something is causing problems and needs to be handled more carefully.
I do this typically every couple of weeks, and it takes up almost no time at all in comparison to time spent on other work. Someone needs to review the eventual PR created, but that's also typically fairly easy. NPM makes this all very easy to do. In Python I've used tools like PDM or uv to handle dependencies similarly.
I do this typically every couple of weeks, and it takes up almost no time at all in comparison to time spent on other work. Someone needs to review the eventual PR created, but that's also typically fairly easy. NPM makes this all very easy to do. In Python I've used tools like PDM or uv to handle dependencies similarly.
Yes, in python only with modern tools this is somewhat feasible
And you still have upgrades that break interfaces and such
And you still have upgrades that break interfaces and such
That’s why we do it on an interval and not when it’s first available. Avoids bleeding edge bugs but prevents the overwhelming factor of being years out of date. It typically gets handed out at the end of the month or there about.
Being judicious in selecting dependencies goes a long way too. Not always easy but certainly worth the time
Being judicious in selecting dependencies goes a long way too. Not always easy but certainly worth the time
I'm glad to hear that all libraries you use honor semver religiously. For the rest of us stuck here on planet Earth it's rolling the dice what, exactly, is going to happen going from 1.7.12 to 1.7.13 for any random dep. The only way to find out is to try it and then unfuck things when it spits out some transitive error message because it turns out that package-A and package-F hate each other but only on Thursdays with a full moon
They don’t all follow semver perfectly or in some cases at all, but we have an interval
when we do upgrades.
Our test suite is comprehensive and will catch most breakages automatically. The key to success is robust testing, as it cuts the manual footprint significantly.
This does mean we are quite judicious with selecting dependencies.
It isn’t all that complicated when everyone is following best practices most of the time I have found[0]
It still leavings me wondering in a lot of cases
[0]: perhaps this is the real heart of the issue is best practices are systematically ignored. I’ve worked at places like that and it’s no wonder they grind through folks
Our test suite is comprehensive and will catch most breakages automatically. The key to success is robust testing, as it cuts the manual footprint significantly.
This does mean we are quite judicious with selecting dependencies.
It isn’t all that complicated when everyone is following best practices most of the time I have found[0]
It still leavings me wondering in a lot of cases
[0]: perhaps this is the real heart of the issue is best practices are systematically ignored. I’ve worked at places like that and it’s no wonder they grind through folks
I mean this seriously: congratulations on working someplace that has such rigorous requirements that you are able to write a comprehensive test suite that doesn't get completely rewritten every month or so. I guess because I enjoy working in startups, the cost I have to pay is that "requirements" are spelt "fever dream" and the "acceptance criteria" field in Jira is often just "." (because it's a required field, donchano)
Functions that you have to document, test and maintain of course. You do that, right? And all the people in your team, they do that and will keep doing that once you leave, right? And they all understand the business domain and all the pitfalls that come with it and have the skill, time, and resources to take care of it, right?
And this for every single problem: time, text, maths, network, parsing, formatting, validating, authenticating...
And this for every single problem: time, text, maths, network, parsing, formatting, validating, authenticating...
Curious about examples of projects being killed by dependencies.
While I've never seen a project killed by dependencies, I've certainly seen projects stuck on treadmill of constant dependency updates.
You know, they import 5 libraries, each of which imports 5 more libraries, each of which imports 5 more libraries, and suddenly they're buried in 'critical' updates because there's a denial-of-service bug in the date parser used by the yaml parser used by the configuration library used by the logging library used by the application.
You know, they import 5 libraries, each of which imports 5 more libraries, each of which imports 5 more libraries, and suddenly they're buried in 'critical' updates because there's a denial-of-service bug in the date parser used by the yaml parser used by the configuration library used by the logging library used by the application.
Not killed IME but bloated and dragged down by tech debt.
E.g the JS project that uses the stdlib Date API, and pulls in moment.js, and also uses date-fns.
Or the one that pulls in bits and pieces of lodash, ramda, and other functional libraries.
And maybe it uses native fetch and axios depending on the current phase of the moon.
They don’t die but time is wasted in code review trying to understand if there is any kind of deliberate approach behind the scattershot application of packages with duplicated purposes.
(picking on JS is perhaps unfair but it’s probably the most egregious example of dependency hell)
E.g the JS project that uses the stdlib Date API, and pulls in moment.js, and also uses date-fns.
Or the one that pulls in bits and pieces of lodash, ramda, and other functional libraries.
And maybe it uses native fetch and axios depending on the current phase of the moon.
They don’t die but time is wasted in code review trying to understand if there is any kind of deliberate approach behind the scattershot application of packages with duplicated purposes.
(picking on JS is perhaps unfair but it’s probably the most egregious example of dependency hell)
The Date example is possibly an even better example of why dependencies that get hard problems right are so important. If Python's datetime library is bad, Date is truly terrible. Every time I have used it I have regretted it long term (including when using it in combination with date-fns in the hope that that makes it usable). And in the end, trying to keep things simple with Date has caused more technical debt than using a sensible dependency.
Some problems simply require using the right tools. They aren't necessarily hard, but they will be if you try to hammer a nail in with a screwdriver. The Date API, and to a certain extent Python's datetime module, are screwdrivers for a nail-shaped problem.
The rest of your example seem to have more to do with bad dependency practices than using dependencies in the first place. If you are going to include a dependency, think about it, consider whether it's worth it, document that decision, and then consistently use that dependency. Just because you've seen projects use dependencies poorly doesn't mean dependents are bad by themselves.
Some problems simply require using the right tools. They aren't necessarily hard, but they will be if you try to hammer a nail in with a screwdriver. The Date API, and to a certain extent Python's datetime module, are screwdrivers for a nail-shaped problem.
The rest of your example seem to have more to do with bad dependency practices than using dependencies in the first place. If you are going to include a dependency, think about it, consider whether it's worth it, document that decision, and then consistently use that dependency. Just because you've seen projects use dependencies poorly doesn't mean dependents are bad by themselves.
Fortunately, JS is getting Temporal, which will be great. No such thing on the horizon for Python yet.
Well, there's libraries like this one, which is the point here.
And even in JS, Temporal won't be available broadly for a good while yet (it will probably be rolling out in Firefox in a couple of months' time, but in Safari it's still behind a feature flag, and I don't think there's even a feature-flagged implementation for Chrome yet). In the meantime, it makes sense to use a polyfill — again a library.
By all means choose your dependencies wisely, but the point I'm trying to make is that very often a sensible use of dependencies will reduce your technical debt, and attempting to use bad approaches to complex topics just because they're built into the standard library will cause so many problems down the line.
And even in JS, Temporal won't be available broadly for a good while yet (it will probably be rolling out in Firefox in a couple of months' time, but in Safari it's still behind a feature flag, and I don't think there's even a feature-flagged implementation for Chrome yet). In the meantime, it makes sense to use a polyfill — again a library.
By all means choose your dependencies wisely, but the point I'm trying to make is that very often a sensible use of dependencies will reduce your technical debt, and attempting to use bad approaches to complex topics just because they're built into the standard library will cause so many problems down the line.
I fully agree with that. Just a bit disappointed that Python is not even considering to fix/replace it's problematic datetime library (as far as I know). Excellent third-party library is good, but an excellent standard library is even better!
Maybe this is the first step toward that future.
In Temporal's case, it takes significant inspiration from several existing datetime libraries in Javascript. Those were all stepping stones (among others) that led to Temporal.
In Temporal's case, it takes significant inspiration from several existing datetime libraries in Javascript. Those were all stepping stones (among others) that led to Temporal.
I used a bad example with the Date API, except that, say, using moment or date-fns and sticking to it would be better than jumbling them up. If you’re going to do a migration from one to the other then it has to be committed to.
Dependencies themselves can’t be good or bad. They only acquire that property in the context of the dependent codebase. And a codebase that you can’t spin up from scratch or easily grok because of its approach (or lack thereof) to dependencies is almost certainly on the bad end of the spectrum.
I’ve seen this destroy some python codebases internally. In fact, my choice of a JS example was unfair because it becomes an absolute shitshow in python once you have dependencies and transitive dependencies with very tight version constraints.
Dependencies themselves can’t be good or bad. They only acquire that property in the context of the dependent codebase. And a codebase that you can’t spin up from scratch or easily grok because of its approach (or lack thereof) to dependencies is almost certainly on the bad end of the spectrum.
I’ve seen this destroy some python codebases internally. In fact, my choice of a JS example was unfair because it becomes an absolute shitshow in python once you have dependencies and transitive dependencies with very tight version constraints.
I'm not the creator, the credit for that goes to Arie Bovenberg. I just wanted to show this to people.
As others stated, there are many rough edges and footguns in the stdlib.
BUT ... in my (and yours apparently) opinion, it's a matter of knowing those edges/guns, and work with them.
Like you, I also prefer to create my own code around those instead of bringing in some library that brings in their own foot guns and possibly sub-dependencies and and and...
So your projects end up with their own "lib" of scattered time functions, possibly with new small bugs. I'd then rather have a proper well-tested and maintained library.
The "lib" one needs for date time functions is as good as non-existent.
But yeah, you will have to create tests for the codepath, instead of relying on the tests the library maintainers create.
But yeah, you will have to create tests for the codepath, instead of relying on the tests the library maintainers create.
Given the list of issues (most of which I had never heard of) that the library author describes with datetime, I think I trust his tests over the ones I’d write.
Given how many tests I see fail on 29th February at the companies I’ve been at, I don’t trust my colleagues’ tests either!
Given how many tests I see fail on 29th February at the companies I’ve been at, I don’t trust my colleagues’ tests either!
In a previous job where we had complete hell with an internal datetime lib under ambiguous ownership, a colleague joked that the original authors had taken early retirement when they turned 2,147,483,647 years old.
> available in Rust or pure Python.
Hard pass. The complexity of having to use binary packages or build things is not worth the performance benefit. The pure-Python version requires building from source and passing special flags, so it is not possible to specify it in requirements.txt.
Hard pass. The complexity of having to use binary packages or build things is not worth the performance benefit. The pure-Python version requires building from source and passing special flags, so it is not possible to specify it in requirements.txt.
That seems like an easy fix, they could release it as `whenever[pure]`. It would probably take less time to write up the issue than to write your comment.
Extras only affect dependencies, you can’t have different codebases for them.
An issue was closed as not planned: https://github.com/ariebovenberg/whenever/issues/158
An issue was closed as not planned: https://github.com/ariebovenberg/whenever/issues/158
Author here. To summarize the long discussion in the issue:
1. I _love_ pure Python packages. Not everybody should be forced to use Rust. I want installing pure-Python to be as easy as possible
2. Having separate names on PyPi (with or without extras) creates confusion for libraries depending on whenever: should they depend on whenever-py or whenever-rust? If one “overwrites” the other, this adds confusion.
3. Most users expect to “pip install whenever” and start using the “fast” version
For me, points (3) and (2) weigh heavy enough to make (1) slightly more cumbersome.
But: Maybe I’ve missed something. Have a read in the above issue or add your 2 cents.
edit: formatting
1. I _love_ pure Python packages. Not everybody should be forced to use Rust. I want installing pure-Python to be as easy as possible
2. Having separate names on PyPi (with or without extras) creates confusion for libraries depending on whenever: should they depend on whenever-py or whenever-rust? If one “overwrites” the other, this adds confusion.
3. Most users expect to “pip install whenever” and start using the “fast” version
For me, points (3) and (2) weigh heavy enough to make (1) slightly more cumbersome.
But: Maybe I’ve missed something. Have a read in the above issue or add your 2 cents.
edit: formatting
1. Is the Rust version really faster enough to warrant the extra complication? Is it faster in real-life code, not only in synthetic benchmarks?
2. I would expect `pip install whenever` to give me the pure-Python version, and `pip install whenever-rust` to give me the Rust extras. Both packages can be installed at the same time; the pure-Python package detects and uses the Rust implementation if present.
2. I would expect `pip install whenever` to give me the pure-Python version, and `pip install whenever-rust` to give me the Rust extras. Both packages can be installed at the same time; the pure-Python package detects and uses the Rust implementation if present.
[deleted]
Maybe something like psycopg[binary], would it be possible to separate the rust binaries as an extra which would be imported if it exists (i.e. When installed with whenever[binary]) otherwise the vanilla python version is used. Not sure of the specifics tho, just throwing out an idea
Ah, so you are not using pyQT, numpy, any database driver, pillow or anything using cryptography, then?
For the libraries you listed, the benefits of using a native library are much larger, since they’re wrapping a well-known library that is known to be secure and fully-featured, or since the performance benefits are actually visible in any significant code snippet. But here, there is no Rust library to wrap, and I doubt the performance of a date-time library would have any effect on the performance of virtually all applications (maybe except for calendar apps).
datetime handling can absolutely be a hot spot, especially if you're parsing or formatting them. Even for relatively simple things like "parse a huge csv file with dates into dataclasses".
In particular, default implementation of datetime in cpython is a C module (with a fallback to pure python one) https://github.com/python/cpython/blob/main/Modules/_datetim...
Not saying it's necessarily justified in case of this library, but if they want to compete with stdlib datetime in terms of performance, some parts will need to be compiled.
In particular, default implementation of datetime in cpython is a C module (with a fallback to pure python one) https://github.com/python/cpython/blob/main/Modules/_datetim...
Not saying it's necessarily justified in case of this library, but if they want to compete with stdlib datetime in terms of performance, some parts will need to be compiled.
> The pure-Python version requires building from source and passing special flags, so it is not possible to specify it in requirements.txt.
You can put any flags in requirements.txt, including -r[equiring] another txt etc.
Your point may apply to modern pyproject.toml tooling though, or at least that it wouldn't be simply another entry in the dependencies array.
You can put any flags in requirements.txt, including -r[equiring] another txt etc.
Your point may apply to modern pyproject.toml tooling though, or at least that it wouldn't be simply another entry in the dependencies array.
The special flags are environment variables, you can’t pass that in requirements.txt: https://whenever.readthedocs.io/en/latest/faq.html#how-can-i...
A big revelation for me in solving so much timezone insanity came from realising that timezones should be expressed as locations rather than zones.
Avoid general terms like "Pacific Standard Time" and stick to location-specific ones like: "Vancouver/Canada". The latter is how people expect their time to work, and correctly handles whatever quirky choices jurisdictions choose to do with their time.
Avoid general terms like "Pacific Standard Time" and stick to location-specific ones like: "Vancouver/Canada". The latter is how people expect their time to work, and correctly handles whatever quirky choices jurisdictions choose to do with their time.
In my experience, all worthy date/time libraries use time zone IDs from the "tz database". Ref: https://en.wikipedia.org/wiki/Tz_database
Searching the list here: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
I cannot find an entry for "Pacific Standard Time" nor "Vancouver/Canada", but I can see: "America/Vancouver".
Searching the list here: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
I cannot find an entry for "Pacific Standard Time" nor "Vancouver/Canada", but I can see: "America/Vancouver".
The rule of thumb is: Use UTC to record when things happened (e.g. logging), use local time + timezone name (e.g. `Europe/London`) to schedule things for the future (e.g. meetings).
Funny it doesn’t add comparison to date times in pandas, which is probably used to handle more dates than any of the others.
Pandas uses stdlib or numpy for it seems.
> If performance isn't your top priority, a pure Python version is available as well.
Then it would have been nice to see the benchmarks of the pure Python implementation as well. What if it's worse than arrow?
Then it would have been nice to see the benchmarks of the pure Python implementation as well. What if it's worse than arrow?
Author here. It's answered briefly in the FAQ
> In casual benchmarks, the pure-Python version is about 10x slower than the Rust version, making it 5x slower than the standard library but still (in general) faster than Pendulum and Arrow.
"(in general)" here since the speed compares differently per operation, while the Rust version is faster across the board. That said, there's no operation that is _significantly_ (or unnecessarily) slower than Arrow or Pendulum.
edit: I'm considering adding comparison to the pure Python version once I get the time for a more expanded "benchmarks" page in the docs
> In casual benchmarks, the pure-Python version is about 10x slower than the Rust version, making it 5x slower than the standard library but still (in general) faster than Pendulum and Arrow.
"(in general)" here since the speed compares differently per operation, while the Rust version is faster across the board. That said, there's no operation that is _significantly_ (or unnecessarily) slower than Arrow or Pendulum.
edit: I'm considering adding comparison to the pure Python version once I get the time for a more expanded "benchmarks" page in the docs
Thank you. My apologies for not reading the FAQ. Also thank you for sharing your library.
No problem. After all, the readme does go from mentioning Pure Python directly to showing a benchmark graph where it's curiously absent
Does someone know when these performance issues matter? My understanding is that datetime is a shortlived object, you wouldn't want thousands of datetime objects all over the codebase.
Almost all of the time UTC is enough, if I need to filter/bucket/aggregate by some range, I can reach for datetime with tz for these filter/bucket/aggregate criteria, convert them to UTC and on continues `int` comparison.
I'd imagine all of the cases handled by Whenever are mostly when datetime is a long lived object, which I don't see a need for at all.
I use it purely for allowing tz input from client, convert to UTC immediately when it arrives, or, if I really need the tz, then save it separately, which is rare (one example is calendar, where tz should be stored, although probably not even next to every UTC but at the user level, another is workforce scheduling, where 8am-4pm or 8pm-4am can mean different things for different locations -- but this is no longer datetime, it's purely time in a timezone).
Almost all of the time UTC is enough, if I need to filter/bucket/aggregate by some range, I can reach for datetime with tz for these filter/bucket/aggregate criteria, convert them to UTC and on continues `int` comparison.
I'd imagine all of the cases handled by Whenever are mostly when datetime is a long lived object, which I don't see a need for at all.
I use it purely for allowing tz input from client, convert to UTC immediately when it arrives, or, if I really need the tz, then save it separately, which is rare (one example is calendar, where tz should be stored, although probably not even next to every UTC but at the user level, another is workforce scheduling, where 8am-4pm or 8pm-4am can mean different things for different locations -- but this is no longer datetime, it's purely time in a timezone).
In my experience it's for calendar-related stuff. You need to store things permanently with the timezone, especially for recurring events. You don't want your scheduled lunch to move from 12 to 1 because it's DST.
And so anything server-related with calendars will be making tons of these conversions constantly. And you can't cache things long-term in UTC because the conversions of future events can change, when countries change DST etc.
And so anything server-related with calendars will be making tons of these conversions constantly. And you can't cache things long-term in UTC because the conversions of future events can change, when countries change DST etc.
But lunch is 12 in time, not in date. You have to decide, with short lived datetime what the desired outcome is for today.
So you would not store that in UTC but just in time.
But yes, I’m ignoring the standard of calendar formats , maybe they are simpler .
I read through the article listing all the weirdness of other datetime libraries and I’d say many were covering cases where you behave that timezoned datetime is long lived .
One case even pointed out datetime construction with an impossible hour.
So you would not store that in UTC but just in time.
But yes, I’m ignoring the standard of calendar formats , maybe they are simpler .
I read through the article listing all the weirdness of other datetime libraries and I’d say many were covering cases where you behave that timezoned datetime is long lived .
One case even pointed out datetime construction with an impossible hour.
No, my weekly lunch is at 12 every 7 days across a variety of dates. The number of hours between each lunch changes due to DST.
I’m not sure I understand your disagreement and then expressing what appears to be exactly what they said; that an appointment happens at a specific time of day, not as a recurrent datetime that needs to shift with DST.
I don't understand what you don't understand?
If you have a recurring lunch, it's always at the same local time interval, but not the same UTC interval, because of DST. Calculating with it requires datetimes, not just times or UTC from the start, contrary to who I was responding to. What is unclear about that?
If you have a recurring lunch, it's always at the same local time interval, but not the same UTC interval, because of DST. Calculating with it requires datetimes, not just times or UTC from the start, contrary to who I was responding to. What is unclear about that?
Lunch as a recurring event in this particular setting is defined as a daily (recurrence type enum) event occurring at 12 (time).
You’ve also stated you want to ignore the timezone and display 12 in whatever tz.
So if my interface is all events between start_utc and end_utc I will construct local datetime and can convert it to UTC and send it to frontend.
The problem with hours that don’t exist in a tz/DST needs to be dealt with separately and given Whenever raises an error and datetime does not is good. In the article that’s one of few that applies, others only exist if you have massive amounts of long lived tz datetime objects.
Yet again, no need for long lived datetime. The problem you picked is time + occurrence.
Imagine a work shift from 23-8 , DST might turn it into less or more than 9 hours. Library does not help as you have to explicitly decide what to do. To avoid the issue then you’d reinterpret user input as start time and duration. When constructing hours, you’d shortly reach out to datetime that is localized. This is again not a datetime problem as work shift is just time.
Showing data on a chart in localized time, one has to explicitly decide what to do with hours that don’t exist or are 2 hours long (visually). Having long lived tz datetime does not help.
You’ve also stated you want to ignore the timezone and display 12 in whatever tz.
So if my interface is all events between start_utc and end_utc I will construct local datetime and can convert it to UTC and send it to frontend.
The problem with hours that don’t exist in a tz/DST needs to be dealt with separately and given Whenever raises an error and datetime does not is good. In the article that’s one of few that applies, others only exist if you have massive amounts of long lived tz datetime objects.
Yet again, no need for long lived datetime. The problem you picked is time + occurrence.
Imagine a work shift from 23-8 , DST might turn it into less or more than 9 hours. Library does not help as you have to explicitly decide what to do. To avoid the issue then you’d reinterpret user input as start time and duration. When constructing hours, you’d shortly reach out to datetime that is localized. This is again not a datetime problem as work shift is just time.
Showing data on a chart in localized time, one has to explicitly decide what to do with hours that don’t exist or are 2 hours long (visually). Having long lived tz datetime does not help.
I genuinely don't understand what you're talking about.
> You’ve also stated you want to ignore the timezone and display 12 in whatever tz.
No I didn't. I literally said "You need to store things permanently with the timezone". Obviously lunch at 12 belongs to a specific timezone.
> Yet again, no need for long lived datetime.
You keep saying this. I don't know where you're getting it from. You need to store these things in databases. If my calendar goes 2 years in the future, these datetimes live for 2 years at least.
It sounds like you're arguing that datetime libraries don't need to exist. I'm genuinely baffled. That feels as nonsensical to me as saying that floating point numbers don't need to exist. Long-lived datetimes are a commonly required thing in anything regarding calendars or scheduling. If you don't believe there is a genuine use case for this, I don't know what to tell you.
I was trying initially to provide you with a basic answer to what seemed to be a genuine question, but I can't help you anymore.
> You’ve also stated you want to ignore the timezone and display 12 in whatever tz.
No I didn't. I literally said "You need to store things permanently with the timezone". Obviously lunch at 12 belongs to a specific timezone.
> Yet again, no need for long lived datetime.
You keep saying this. I don't know where you're getting it from. You need to store these things in databases. If my calendar goes 2 years in the future, these datetimes live for 2 years at least.
It sounds like you're arguing that datetime libraries don't need to exist. I'm genuinely baffled. That feels as nonsensical to me as saying that floating point numbers don't need to exist. Long-lived datetimes are a commonly required thing in anything regarding calendars or scheduling. If you don't believe there is a genuine use case for this, I don't know what to tell you.
I was trying initially to provide you with a basic answer to what seemed to be a genuine question, but I can't help you anymore.
> You don't want your scheduled lunch to move from 12 to 1 because it's DST.
Yes, therefore you won't store lunch as `datetime` you will store it as `time` and on "render" you will make sure it is 12th hour for localized datetime. Look at RFC 5545. If you move from Japan to Romania, you'll again on render (with stored 12 as `time`) display lunch at 12th hour. The only thing one stores is daily occurrence and 12th hour, with careful interpretation what is a boundary of a day and what is 12th hour. `datetime` does not help, similar to how it does not help with DST issues for workshifts (which are again `time`).
> If my calendar goes 2 years in the future, these datetimes live for 2 years at least.
Why would you materialize recurring events, if you move from Vietnam to Spain, how will all those lunches become Spain lunches? Just recompute the whole future? Sounds like a mistake of storing long-lived objects that should have been replaced with computation with short-lived ones.
> It sounds like you're arguing that datetime libraries don't need to exist.
Steelman me. 2 individuals tried to expand to include you into discussion, you seem to go in the other direction every time.
I am not implying that. I'm implying that I don't understand the performance implications, given that I'd expect to work with efficient UTC most of the time, with only a constant amount of stuff going from datetime-with-tz to UTC. I cannot imagine a case where you'd abundantly store datetime-with-tz, it sounds like a mistake.
My own impression is that majority opinion is that going from datetime-with-tz to UTC and then doing comparisons, filtering, bucketing, aggregating is somehow flawed? If I need 24 hour buckets, I can do it through UTC, if I need "daily" buckets, I can still do it through UTC -- with short-lived datetime-with-tz to UTC computation, yet UI will still have to deal with 23/25 hour days, `datetime` does not solve that problem. I understand monthly buckets, still easy to do through UTC with short-lived conversion. (go from datetime-with-tz to UTC that's used to filter/group entries in db)
Yes, therefore you won't store lunch as `datetime` you will store it as `time` and on "render" you will make sure it is 12th hour for localized datetime. Look at RFC 5545. If you move from Japan to Romania, you'll again on render (with stored 12 as `time`) display lunch at 12th hour. The only thing one stores is daily occurrence and 12th hour, with careful interpretation what is a boundary of a day and what is 12th hour. `datetime` does not help, similar to how it does not help with DST issues for workshifts (which are again `time`).
> If my calendar goes 2 years in the future, these datetimes live for 2 years at least.
Why would you materialize recurring events, if you move from Vietnam to Spain, how will all those lunches become Spain lunches? Just recompute the whole future? Sounds like a mistake of storing long-lived objects that should have been replaced with computation with short-lived ones.
> It sounds like you're arguing that datetime libraries don't need to exist.
Steelman me. 2 individuals tried to expand to include you into discussion, you seem to go in the other direction every time.
I am not implying that. I'm implying that I don't understand the performance implications, given that I'd expect to work with efficient UTC most of the time, with only a constant amount of stuff going from datetime-with-tz to UTC. I cannot imagine a case where you'd abundantly store datetime-with-tz, it sounds like a mistake.
My own impression is that majority opinion is that going from datetime-with-tz to UTC and then doing comparisons, filtering, bucketing, aggregating is somehow flawed? If I need 24 hour buckets, I can do it through UTC, if I need "daily" buckets, I can still do it through UTC -- with short-lived datetime-with-tz to UTC computation, yet UI will still have to deal with 23/25 hour days, `datetime` does not solve that problem. I understand monthly buckets, still easy to do through UTC with short-lived conversion. (go from datetime-with-tz to UTC that's used to filter/group entries in db)
You seem to be deliberately misunderstanding me, so I don't know what to tell you. Again, I tried to help explain it to you, but you are refusing to acknowledge that people really do store events as future datetimes attached to a timezone, not just as local times, and I don't know why you continue to insist otherwise.
Good luck.
Good luck.
I did not insist people do not store future datetimes with offsets. I wondered if it's a mistake. People who store future datetimes with tz obviously have to parse all of them and would need performance, but my opening statement was questioning when this is actually necessary, as it looked rare to me.
RFC 5545 introduces floating time to -- I assume -- avoid this silliness, so that your lunch is actually forced to be rendered at 12 regardless of tz.
RFC 5545 introduces floating time to -- I assume -- avoid this silliness, so that your lunch is actually forced to be rendered at 12 regardless of tz.
A tangent, but I hope the world gets its shit together and gets rid of DST.
I am currently enjoying DST-free life in Japan, and feel that people around the world deserve to get this much respect from their own official clocks.
I am currently enjoying DST-free life in Japan, and feel that people around the world deserve to get this much respect from their own official clocks.
Almost everyone wants to get rid of the twice annual clock changes but are nearly evenly divided on if DST should be permanent or cease to exist. It's a strange artifact of wanting clock noon to be the midpoint of the workday but also wanting to maximize the hours of daylight after work.
Who wants clock noon to be the midpoint of the workday? The canonical working hours are 9am to 5pm [Parton 1980] whose midpoint is at 1pm. Many people work earlier and/or later, but my impression is that it's pretty unusual to have the midpoint at noon.
(Schools tend to have earlier times. It's not so unusual for a school's workday to have its midpoint at about noon, I think.)
(Schools tend to have earlier times. It's not so unusual for a school's workday to have its midpoint at about noon, I think.)
Clock noon already isn't the midpoint of the working day (9-5) for many people. I don't think anyone cares about it being when the sun is at its highest in the sky either. This isn't even something most people know, and it's not even true unless you live on the Greenwich meridian or n*15 degrees east or west. What matters is how early we have to get up in winter vs early we have to go to bed in the summer.
I've always been in favour of keeping the clocks at non-DST all year, but now I have a new proposal: keep them at DST and just hibernate in the winter. Work an hour or two less in the winter when it's miserable.
I've always been in favour of keeping the clocks at non-DST all year, but now I have a new proposal: keep them at DST and just hibernate in the winter. Work an hour or two less in the winter when it's miserable.
>but also wanting to maximize the hours of daylight after work.
Maybe adjust the work schedule to e.g. start at 8 instead of 9?
Rather than mess with the actual clock.
Maybe adjust the work schedule to e.g. start at 8 instead of 9?
Rather than mess with the actual clock.
This is a coordination problem. DST was the solution many societies found.
Shouldn't people wanting to maximize the hours of daylight after work work night shifts?
Where do I find a software engineering job that has night shifts?
Remote job for a company in a different time zone
I would wish for that as well, but it’s unlikely to happen. In the EU for example, some countries would be on the losing side, either by getting “bad” hours or by having to move to a different time zone than their neighbor, which has significant economic consequences. Such countries won’t agree to a DST abolishment that disadvantages them.
And for program code, it wouldn’t really help as long as it’s still expected to be able to correctly handle dates in the past.
And for program code, it wouldn’t really help as long as it’s still expected to be able to correctly handle dates in the past.
I don’t understand how eliminating DST would impact economics of neighboring countries… today they both change clocks, tomorrow they don’t. What changes?
It’s not eliminating DST as such, it’s switching to a different time zone than their neighbor. Eliminating DST means that either the East-most or West-most country would get unfavorable hours, either in summer or winter. They can mitigate that by changing their time zone, but now that means they’ll be in a different time zone than their neighbor, which they previously shared a time zone with. For example Spain could switch to a different time zone than France, to get less extreme hours of days compared to what is currently Central European Time, but having a time difference would have economic costs between Spain and the rest of CET land. (Arguably, both Spain and France should switch to UTC, based on their proximity to the Greenwich meridian, but a time difference between France and Germany would be even worse.)
I’m not sure how real those costs would be beyond a transitional period, but that’s the discussions that have been going on. It’s a political risk, and nobody wants to be on the losing side of such a change.
I’m not sure how real those costs would be beyond a transitional period, but that’s the discussions that have been going on. It’s a political risk, and nobody wants to be on the losing side of such a change.
I like that the type names are the same as in Java (java.time package). Great work!
I go for Arrow when I want anything beyond the basics. This looks pretty interesting, not really because of the greater coverage in edge cases, but because while it has a Rustified mode, a pure Python mode is also available. If I do use whenever, I don't have to worry about having something else or falling back to datetime if I want better datetime handling in a project on my phone, or in some other environment where the Rust toolchain is non-existent or problematic. Kudos.
Looks amazing. I had to deal with time handling in my very first programming job 25 years ago, and lousy handling has been a pet peeve of mine ever since.
Dates and HTTP requests are the two things I always manipulate through libraries (no matter the language, except maybe for timestamps). It is so much simpler that way.
I am an amateur dev, though, so maybe someone who masters the language will be better off using the raw standard libraries.
I am an amateur dev, though, so maybe someone who masters the language will be better off using the raw standard libraries.
Honestly, no. There are times when you want to get low level but, when you do, you need to commit to learning that domain as well as the problem domain you’re being paid to solve. If those are disjoint, well, have fun!
I've read the link and the GitHub readme page.
I'm sure I'm in the top 1% of software devs for the most number of timestamps parsed. [1]
DST is not a problem in Python. It's parsing string timestamps. All libraries are bad, including this one, except Pandas. Pandas does great at DST too btw.
And I'm not shilling for Pandas either. I'm a Polars user who helicopters Pandas in whenever there's a timestamp that needs to be parsed.
Pandas has great defaults. Here's string timestamps I expect to be paesed by default. I'm willing to pass timezone in case of naive timestamps:
* All ISO 8601 formats and all its weird mutant children that differ by a tiny bit.
* 2025-05-01 (parsed not as date, but as timestamp)
* 2025-05-01 00:00:00 (or 00.0 or 00.000 or 0.000000 etc)
* 2025-05-01 00:00:00z (or uppercase Z or 00.0z or 00.000z or 0.000000z)
* 2025-05-01 00:00:00+02:00 (I don't need this converted to some time zone. Store offset if you must or convert to UTC. It should be comparable to other non naive timestamps).
* 2025-03-30 02:30:00+02:00 (This is a non existent timestamp wrt European DST but a legitimate timestamp in timestamp representation, therefore it should be allowed unless I specify CET or Europe/Berlin whatever)
* There's other timestamps formats that are non standard but are obvious. Allow for a Boolean parameter called accept_sensible_string_parsing and then parse the following:
Disclaimer: I'm on the phone and on the couch so I wasn't able to test the lib for its string parsing before posting this comment.
I'm sure I'm in the top 1% of software devs for the most number of timestamps parsed. [1]
DST is not a problem in Python. It's parsing string timestamps. All libraries are bad, including this one, except Pandas. Pandas does great at DST too btw.
And I'm not shilling for Pandas either. I'm a Polars user who helicopters Pandas in whenever there's a timestamp that needs to be parsed.
Pandas has great defaults. Here's string timestamps I expect to be paesed by default. I'm willing to pass timezone in case of naive timestamps:
* All ISO 8601 formats and all its weird mutant children that differ by a tiny bit.
* 2025-05-01 (parsed not as date, but as timestamp)
* 2025-05-01 00:00:00 (or 00.0 or 00.000 or 0.000000 etc)
* 2025-05-01 00:00:00z (or uppercase Z or 00.0z or 00.000z or 0.000000z)
* 2025-05-01 00:00:00+02:00 (I don't need this converted to some time zone. Store offset if you must or convert to UTC. It should be comparable to other non naive timestamps).
* 2025-03-30 02:30:00+02:00 (This is a non existent timestamp wrt European DST but a legitimate timestamp in timestamp representation, therefore it should be allowed unless I specify CET or Europe/Berlin whatever)
* There's other timestamps formats that are non standard but are obvious. Allow for a Boolean parameter called accept_sensible_string_parsing and then parse the following:
\* 2025-05-01 00:00 (HH:mm format)
\* 2025-05-01 00:00+01:00 (HH:mm format)
Disclaimer: I'm on the phone and on the couch so I wasn't able to test the lib for its string parsing before posting this comment.
Author here. It's indeed a hard problem to parse "All ISO 8601 formats and all its weird mutant children that differ by a tiny bit."
Since the ISO standard is so expansive, every library needs to decide for itself what to support. The ISO standard allows all sorts of weird things, like 2-digit years, fractional months, disallowing -00:00 offset, ordinal days, etc.
Javascript's big datetime redesign (Temporal) has an interesting overview of the decisions they made [1]. Whenever is currently undergoing an expansion of ISO support as well, if you'd like to chime in [2].
[1] https://tc39.es/proposal-temporal/#sec-temporal-iso8601gramm... [2] https://github.com/ariebovenberg/whenever/issues/204#issueco...
Javascript's big datetime redesign (Temporal) has an interesting overview of the decisions they made [1]. Whenever is currently undergoing an expansion of ISO support as well, if you'd like to chime in [2].
[1] https://tc39.es/proposal-temporal/#sec-temporal-iso8601gramm... [2] https://github.com/ariebovenberg/whenever/issues/204#issueco...
Thanks for the reply and apologies for the general cynicism. It's not lost on me that it's people like you that build tools that make the work tick. I'm just a loud potential customer and I'm just forwarding the frustration that I have with my own customers onto you :)
Your customers are software devs like me. When we're in control of generating timestamps, we know we must use standard ISO formatting.
However, what do I do when my customers give me access to an S3 bucket with 1 billion timestamps in an arbitrary (yet decipherable) format?
In the GitHub issue you seem to have undergone an evolution from purity to pragmatism. I support this 100%.
What I've also noticed is that you seem to try to find grounding or motivation for "where to draw the line" from what's already been done in Temporal or Python stdlib etc. This is where I'd like to challenge your intuitions and ask you instead to open the flood gates and accept any format that is theoretically sensible under ISO format.
Why? The damage has already been done. Any format you can think of, already exists out there. You just haven't realized it yet.
You know who has accepted this? Pandas devs (I assume, I don't them). The following are legitimate timestamps under Pandas (22.2.x):
* 2025-03-30T (nope, not a typo)
* 2025-03-30T01 (HH)
* 2025-03-30 01 (same as above)
* 2025-03-30 01 (two or more spaces is also acceptable)
In my opinion Pandas doesn't go far enough. Here's an example from real customer data I've seen in the past that Pandas doesn't parse.
* 2025-03-30+00:00 (this is very sensible in my opinion. Unless there's a deeper theoretical regex pattern conflicts with other parts of the ISO format)
Here's an example that isn't decipherable under a flexible ISO interpretation and shouldn't be supported.
* 2025-30-03 (theoretically you can infer that 30 is a day, and 03 is month. BUT you shouldn't accept this. Pandas used to allow such things. I believe they no longer do)
I understand writing these flexible regexes or if-else statements will hurt your benchmarks and will be painful to maintain. Maybe release them under an new call like `parse_best_effort` (or even `youre_welcome`) and document pitfalls and performance degradation. Trust me, I'd rather use a reliable generic but slow parser than spend hours writing a write a god awful regex that I will only use once (I've spent literal weeks writing regexes and fixes in the last decade).
Pandas has been around since 2012 dealing with customer data. They have seen it all and you can learn a lot from them. ISOs and RFCs when it comes to timestamps don't mean squat. If possible try to make Whenever useful rather than fast or pure. I'd rather use a slimmer faster alternative to pandas for parsing Timestamps if one is available but there aren't any at the moment.
If time permits I'll try to compile a non exhaustive list of real world timestamp formats and post in the issue.
Thank you for your work!
P.S. seeing BurntSushi in the GitHub issue gives me imposter syndrome :)
Your customers are software devs like me. When we're in control of generating timestamps, we know we must use standard ISO formatting.
However, what do I do when my customers give me access to an S3 bucket with 1 billion timestamps in an arbitrary (yet decipherable) format?
In the GitHub issue you seem to have undergone an evolution from purity to pragmatism. I support this 100%.
What I've also noticed is that you seem to try to find grounding or motivation for "where to draw the line" from what's already been done in Temporal or Python stdlib etc. This is where I'd like to challenge your intuitions and ask you instead to open the flood gates and accept any format that is theoretically sensible under ISO format.
Why? The damage has already been done. Any format you can think of, already exists out there. You just haven't realized it yet.
You know who has accepted this? Pandas devs (I assume, I don't them). The following are legitimate timestamps under Pandas (22.2.x):
* 2025-03-30T (nope, not a typo)
* 2025-03-30T01 (HH)
* 2025-03-30 01 (same as above)
* 2025-03-30 01 (two or more spaces is also acceptable)
In my opinion Pandas doesn't go far enough. Here's an example from real customer data I've seen in the past that Pandas doesn't parse.
* 2025-03-30+00:00 (this is very sensible in my opinion. Unless there's a deeper theoretical regex pattern conflicts with other parts of the ISO format)
Here's an example that isn't decipherable under a flexible ISO interpretation and shouldn't be supported.
* 2025-30-03 (theoretically you can infer that 30 is a day, and 03 is month. BUT you shouldn't accept this. Pandas used to allow such things. I believe they no longer do)
I understand writing these flexible regexes or if-else statements will hurt your benchmarks and will be painful to maintain. Maybe release them under an new call like `parse_best_effort` (or even `youre_welcome`) and document pitfalls and performance degradation. Trust me, I'd rather use a reliable generic but slow parser than spend hours writing a write a god awful regex that I will only use once (I've spent literal weeks writing regexes and fixes in the last decade).
Pandas has been around since 2012 dealing with customer data. They have seen it all and you can learn a lot from them. ISOs and RFCs when it comes to timestamps don't mean squat. If possible try to make Whenever useful rather than fast or pure. I'd rather use a slimmer faster alternative to pandas for parsing Timestamps if one is available but there aren't any at the moment.
If time permits I'll try to compile a non exhaustive list of real world timestamp formats and post in the issue.
Thank you for your work!
P.S. seeing BurntSushi in the GitHub issue gives me imposter syndrome :)
Because you pinged me... Jiff also generally follows in Temporal's footsteps here. Your broader point of supporting things beyond the specs (ISO 8601, RFC 3339, RFC 9557, RFC 2822 and so on) has already been absorbed into the Temporal ISO 8601 extensions. And that's what Jiff supports (and presumably, whenever, although I don't know enough about whenever to be absolutely precise in what it supports). So I think the philosophical point has already been conceded by the Temporal project itself. What's left, it seems, is a measure of degree. How far do you go in supporting oddball formats?
I honestly do not know the answer to that question myself. But I wouldn't necessarily look to Pandas as the shining beacon on a hill here. Not because Pandas is doing anything wrong per se, but because it's a totally different domain and use case. On the one hand, you have a general purpose library that needs to consider all of its users for all general purpose datetime use cases. On the other hand, you have a data scienc-y library designed for trying to slurp up and make sense of messy data at scale. There may be things that make sense in the latter that don't in the former.
In particular, a major gap in your reasoning, from what I can see, is that constraints beget better error reporting. I don't know how to precisely weigh error reporting versus flexible parsing, but there ought to be some deliberation there. The more flexible your format, the harder it is to give good error messages when you get invalid data.
Moreover, "flexible parsing" doesn't actually have to be in the datetime library. The task of flexible parsing is not, in and of itself, overtly challenging. It's a tedious task that can be build on top of the foundation of a good datetime library. I grant that this is a bit of a cop-out, but it's part of the calculus when designing ecosystem libraries like this.
Speaking for me personally (in the context of Jiff), something I wouldn't mind so much is adding a dedicated "flexible" parsing mode that one can opt into. But I don't think I'd want to make it the default.
I honestly do not know the answer to that question myself. But I wouldn't necessarily look to Pandas as the shining beacon on a hill here. Not because Pandas is doing anything wrong per se, but because it's a totally different domain and use case. On the one hand, you have a general purpose library that needs to consider all of its users for all general purpose datetime use cases. On the other hand, you have a data scienc-y library designed for trying to slurp up and make sense of messy data at scale. There may be things that make sense in the latter that don't in the former.
In particular, a major gap in your reasoning, from what I can see, is that constraints beget better error reporting. I don't know how to precisely weigh error reporting versus flexible parsing, but there ought to be some deliberation there. The more flexible your format, the harder it is to give good error messages when you get invalid data.
Moreover, "flexible parsing" doesn't actually have to be in the datetime library. The task of flexible parsing is not, in and of itself, overtly challenging. It's a tedious task that can be build on top of the foundation of a good datetime library. I grant that this is a bit of a cop-out, but it's part of the calculus when designing ecosystem libraries like this.
Speaking for me personally (in the context of Jiff), something I wouldn't mind so much is adding a dedicated "flexible" parsing mode that one can opt into. But I don't think I'd want to make it the default.
I really hoped this was about https://www.dangermouse.net/esoteric/whenever.html
I am still trying to cope with pytz and dateutils
Reading this post and comment section makes me shake my head. This looks like a near clone of Java JSR-310 (new date/time APIs), which was headed by the original author of Joda Time (Stephen Colebourne). Java 8 (and JSR-310) was released in 2014 -- 11 years ago(!). Amazingly, Python has suffered with their date/time libs this whole time with very little concerted effort to create new date/time APIs in the standard library. It's pathetic. I know I will be downvoted for this post, but I don't care. The Python standard library has so many of these awful weaknesses that other languages handle better. Except for machine learning R&D, I never recommend to use Python for any enterprise project except trivial ones. You are walking into a double trap of (1) weak types and (2) weak standard library.
https://dev.arie.bovenberg.net/blog/python-datetime-pitfalls...