Hexadecimal clock counting down to 2038 problem(retr0.id)
retr0.id
Hexadecimal clock counting down to 2038 problem
https://retr0.id/stuff/2038/
136 comments
Should dates be strings?
Yes absolutely. Dates are too complicated, requiring to much context, with too much to go wrong, for any non human readable format. At least with strings other systems have half a chance of parsing them correctly, or at least parsing them in a transparent manner.
Or to put it another way, they cannot be trusted to engineers to get it right every time. Let's face it, even something as simple as floating point numbers have serious issues when implemented in computer systems (so another candidate for strings quite frankly - notable that JSON is strings all the way through).
Or to put it another way, they cannot be trusted to engineers to get it right every time. Let's face it, even something as simple as floating point numbers have serious issues when implemented in computer systems (so another candidate for strings quite frankly - notable that JSON is strings all the way through).
It’s not just a matter of trust. Locale specific datetimes in the future are simply not computable to a point on a number line in the general case. Some locales have date changes that must be determined by the local government every single year. Israel is an example of such a case.
Unix timestamp (actual reference with easy math) + timezone give you absolute time to work with, such that you can work out the correct presentation for the timezone even if the timezone has undergone multiple revisions in the intervening years.
There's still a bunch of problems (timezone of event origin? Event reporting origin? Server? Client? Client + presentational overlay?) and confusing things to work through (including relativistic problems) but the consistency of the reference point is valuable. Nobody is going to confuse what timezone you are using as your point of reference when you use Unix epoch. Just use a 64-bit number to hold it.
There's still a bunch of problems (timezone of event origin? Event reporting origin? Server? Client? Client + presentational overlay?) and confusing things to work through (including relativistic problems) but the consistency of the reference point is valuable. Nobody is going to confuse what timezone you are using as your point of reference when you use Unix epoch. Just use a 64-bit number to hold it.
The suggested approach simply doesn’t work, because human calendars simply don’t work that way.
If the state of Indiana decides this year to change when they set their clocks forward and back, and you stored your appointment datetime as a timestamp, when do you show up for an appointment? At the new 930 AM after statutory date adjustment, or the now incorrect time stamp in your database? Will the DMV respect your claim that you had an appointment at epoch timestamp 1726651800, or will they say sorry buddy the clock on the wall says 9:30 AM, you missed your window? Are your taxes next year due at 1713173400, or are they due at midnight April 15th?
This is NOT a UI issue. This is an issue where the timestamp you calculated in the past literally does not mean the same calendar date in the future as it does now.
And if your answer is to recalculate the timestamp based on locale changes, what was the point of storing the timestamp? (I hope you stored an audit trail of the locale as well as the timestamp, otherwise you can't even recompute the timestamp). The timestamp is never the ground truth, the human readable string is the ground truth.
The timestamp for a particular human date time is not generally a computable function, because human governments change how dates are reckoned frequently, and the human government, not your database are the ground truth.
If the state of Indiana decides this year to change when they set their clocks forward and back, and you stored your appointment datetime as a timestamp, when do you show up for an appointment? At the new 930 AM after statutory date adjustment, or the now incorrect time stamp in your database? Will the DMV respect your claim that you had an appointment at epoch timestamp 1726651800, or will they say sorry buddy the clock on the wall says 9:30 AM, you missed your window? Are your taxes next year due at 1713173400, or are they due at midnight April 15th?
This is NOT a UI issue. This is an issue where the timestamp you calculated in the past literally does not mean the same calendar date in the future as it does now.
And if your answer is to recalculate the timestamp based on locale changes, what was the point of storing the timestamp? (I hope you stored an audit trail of the locale as well as the timestamp, otherwise you can't even recompute the timestamp). The timestamp is never the ground truth, the human readable string is the ground truth.
The timestamp for a particular human date time is not generally a computable function, because human governments change how dates are reckoned frequently, and the human government, not your database are the ground truth.
Very good point, which is a reminder that you have different "types" of dates that you may want to store.
Went storing the time at which an event happened (e.g. for a system log) then a UTC or UNIX time works, and it will be transformed at display time depending on the user's timezone.
When storing a future time at which to do something, it should be stored with its target timezone so you can always make sure it happens at the right moment. For this iso datetime representation makes a lot of sense.
Went storing the time at which an event happened (e.g. for a system log) then a UTC or UNIX time works, and it will be transformed at display time depending on the user's timezone.
When storing a future time at which to do something, it should be stored with its target timezone so you can always make sure it happens at the right moment. For this iso datetime representation makes a lot of sense.
That is exactly was I was referring to with needing to know whether it was an origination stamp or what, ie the context of the stamp. That problem holds regardless of the format you store the stamp in, because even if there is a colloquial understanding of how to interpret a time change (1030 on Thursday is still 1030 on Thursday) it doesn't hold for every context (5 hrs from now is still 5 hours from now, except if when you said 5 hours from now you meant clock time)
> If the state of Indiana decides this year to change when they set their clocks forward and back
On the other hand, the state of Indiana could just as easily solve most of our time issues by passing a law declaring a day to be exactly 24 hours ;)
https://en.wikipedia.org/wiki/Indiana_Pi_Bill
On the other hand, the state of Indiana could just as easily solve most of our time issues by passing a law declaring a day to be exactly 24 hours ;)
https://en.wikipedia.org/wiki/Indiana_Pi_Bill
> CSV
ISO8601 / RFC3339 for life.
[deleted]
Not really.
I mean, sure, serialize them as ISO8601 strings, but, no, dates are fundamentally not strings, and that shouldn't be their basic internal representation in most sane systems.
In CSV, though, everything is strings. But the question there is what string format.
I mean, sure, serialize them as ISO8601 strings, but, no, dates are fundamentally not strings, and that shouldn't be their basic internal representation in most sane systems.
In CSV, though, everything is strings. But the question there is what string format.
the question is what format. what you're really doing is storing a slash-separated CSV of integers with only one entry, and storing a set of integers is what many old systems already did.
the real solution is to store 64-bit seconds (or perhaps 96-bit microseconds, if you need precision) since some epoch, like jan 1st 1970. it's a simple integer that monotonically increases with a defined starting point. you can't get any more simple or reliable than that.
the unix time stamp was a wonderful solution, they just chose to use too few bits for how long their system would end up living. 64-bit integer seconds will last something like 42x the age of the universe, so that ought to cover things.
the real solution is to store 64-bit seconds (or perhaps 96-bit microseconds, if you need precision) since some epoch, like jan 1st 1970. it's a simple integer that monotonically increases with a defined starting point. you can't get any more simple or reliable than that.
the unix time stamp was a wonderful solution, they just chose to use too few bits for how long their system would end up living. 64-bit integer seconds will last something like 42x the age of the universe, so that ought to cover things.
Dates / timestamps fail to capture an 'event'. At best they are a partial event specification with the assumption that they are converted correctly to an assumed increment along a single time scale.
Event specification is a very complex and politically entwined idea. This involves not just a frame of reference for the time, but also a location within which that time should make sense. Otherwise it is impossible to derive the correct rule-set for evaluating the human interpreted value of that moment.
An alternative which might make sense is to force people to specify which ruleset they desire with the offset value within said ruleset.
Sadly, my preferred representation of time isn't expressed in either of the popular standards (RFC 3339 or ISO-8601). https://ijmacd.github.io/rfc3339-iso8601/
RFC 3339 gets closet to my preference with: 2023-09-17 23:51:41Z
However these are examples of what I'd prefer to see:
Time would be expressed numerically 'large endian first' format, with zero or one punctuation element between units, preferably expressed with individual units matching the most correct (easiest for a human to read what they expect) value at a glance. This means - (or no separation) for dates, and : (or no separation) for HMS. (Space) or _ (Underscore) are preferable for between units. The timezone 'name' follows the smallest unit of time, which is itself followed by the reference city. Implicitly events in the past were accurate at the time they were encoded, while events in the future might need re-validation and conversion if the rules have changed. Though it would be even smarter for the human important metadata (a string, probably) of recognizing the future event(s) be stored rather than a precise moment which may be incorrect.
Event specification is a very complex and politically entwined idea. This involves not just a frame of reference for the time, but also a location within which that time should make sense. Otherwise it is impossible to derive the correct rule-set for evaluating the human interpreted value of that moment.
An alternative which might make sense is to force people to specify which ruleset they desire with the offset value within said ruleset.
Sadly, my preferred representation of time isn't expressed in either of the popular standards (RFC 3339 or ISO-8601). https://ijmacd.github.io/rfc3339-iso8601/
RFC 3339 gets closet to my preference with: 2023-09-17 23:51:41Z
However these are examples of what I'd prefer to see:
* 2023-09-17 23:51:41 Z UTC
* 2023-09-17 16:51:41 PST8PDT USA/WA/Seattle
* 2023-09-17 16:51:41 PST8PDT Canada/British Columbia/Vancouver
Time would be expressed numerically 'large endian first' format, with zero or one punctuation element between units, preferably expressed with individual units matching the most correct (easiest for a human to read what they expect) value at a glance. This means - (or no separation) for dates, and : (or no separation) for HMS. (Space) or _ (Underscore) are preferable for between units. The timezone 'name' follows the smallest unit of time, which is itself followed by the reference city. Implicitly events in the past were accurate at the time they were encoded, while events in the future might need re-validation and conversion if the rules have changed. Though it would be even smarter for the human important metadata (a string, probably) of recognizing the future event(s) be stored rather than a precise moment which may be incorrect.
Locale and time are two separate pieces of data. Why store them together? Why not unix time and locale as two separate data points, and then mapping them to a human readable value is a UI concern?
They are not, though it may be implicit in your thoughts that E.G. UTC or TAI or some other standard is associated with a single scalar value which represents some time along that frame of reference.
However not all times are generated within such references. Most frequently (when sampling human facing outputs) times are printed within some other reference frame. As I tried to elaborate that usually relates to political realms, and there even cities may not be a sufficiently precise specification. (Offhand E.G. would timestamps from east and west Berlin while it was divided differ? I've not memorized that trivia, but it is one example of a politically divided city.)
Further, not so many decades ago timezones along the western Americas coast happened to match, but have since diverged at times during the year due to political changes in timezone observations. It is not inconceivable that actions at a state, county, city, or other zoning level (E.G. Provences for Canada) might further change the meaning of a human friendly representation of an event (time in a place).
However not all times are generated within such references. Most frequently (when sampling human facing outputs) times are printed within some other reference frame. As I tried to elaborate that usually relates to political realms, and there even cities may not be a sufficiently precise specification. (Offhand E.G. would timestamps from east and west Berlin while it was divided differ? I've not memorized that trivia, but it is one example of a politically divided city.)
Further, not so many decades ago timezones along the western Americas coast happened to match, but have since diverged at times during the year due to political changes in timezone observations. It is not inconceivable that actions at a state, county, city, or other zoning level (E.G. Provences for Canada) might further change the meaning of a human friendly representation of an event (time in a place).
That makes sense, but really we are not going to be able to guarantee full accuracy of a timestamp against future changes in timezone boundaries. I guess if you had a GPS coordinate, that could be done, but it would be a monumental task for little benefit. Which is why IMO time data should be stored in unix, because then we can at least guarantee that we have an accurate ordering of events, at least within the precision of the timestamp.
Would rather convert to unix at the time the event is recorded since we will be able to convert to the unix standard from whatever locale. If we store timestamps with a locale then we have to worry about the exact type of locale changes that you mentioned, and there is no guarantee that we will even know how to translate the locale that was stored at the time the TS was created. Sure, unix could change too (e.g. leap seconds), but I would be much more confident that time libraries will handle such changes
Would rather convert to unix at the time the event is recorded since we will be able to convert to the unix standard from whatever locale. If we store timestamps with a locale then we have to worry about the exact type of locale changes that you mentioned, and there is no guarantee that we will even know how to translate the locale that was stored at the time the TS was created. Sure, unix could change too (e.g. leap seconds), but I would be much more confident that time libraries will handle such changes
You're going to need to answer https://news.ycombinator.com/item?id=37551163 before you continue with any more of this.
Otherwise this isn't going to be a meaningful discussion.
Otherwise this isn't going to be a meaningful discussion.
you're stretching the definition of "event" pretty far there. if you're not referring to an atomic-clock-precise instant in time, you're going to need to elaborate as to what your definition of an "event" is and explain why location/culture/politics are important.
at any rate, if you even want to have a hope of keeping track of any of that stuff, you're going to need a rigorous and atomically-precise physical standard to actually map it all to. look at the tz database. it's a set of mappings that ultimately rely on such a standard of atomic-backed time.
at any rate, if you even want to have a hope of keeping track of any of that stuff, you're going to need a rigorous and atomically-precise physical standard to actually map it all to. look at the tz database. it's a set of mappings that ultimately rely on such a standard of atomic-backed time.
> it's a simple integer that monotonically increases
It isn't. See leap seconds.
Using a string would allow you to distinguish UTC from UT1. Only UT1 is monotonic.
It isn't. See leap seconds.
Using a string would allow you to distinguish UTC from UT1. Only UT1 is monotonic.
I specifically did not say "UTC" at all. Read my actual comment.
https://en.wikipedia.org/wiki/Unix_time
"Unix time is a date and time representation widely used in computing. It measures time by the number of seconds that have elapsed since 00:00:00 UTC on 1 January 1970, the Unix epoch, without adjustments made due to leap seconds."
Unix time does not include leap seconds or other adjustments.
https://en.wikipedia.org/wiki/Unix_time
"Unix time is a date and time representation widely used in computing. It measures time by the number of seconds that have elapsed since 00:00:00 UTC on 1 January 1970, the Unix epoch, without adjustments made due to leap seconds."
Unix time does not include leap seconds or other adjustments.
scrubbed
[Edit] From Wiki, your code is subtly mistaken:
> When dealing with periods that do not encompass a UTC leap second, the difference between two Unix time numbers is equal to the duration in seconds of the period between the corresponding points in time. This is a common computational technique. However, where leap seconds occur, such calculations give the wrong answer. In applications where this level of accuracy is required, it is necessary to consult a table of leap seconds when dealing with Unix times, and it is often preferable to use a different time encoding that does not suffer from this problem.
[Edit] From Wiki, your code is subtly mistaken:
> When dealing with periods that do not encompass a UTC leap second, the difference between two Unix time numbers is equal to the duration in seconds of the period between the corresponding points in time. This is a common computational technique. However, where leap seconds occur, such calculations give the wrong answer. In applications where this level of accuracy is required, it is necessary to consult a table of leap seconds when dealing with Unix times, and it is often preferable to use a different time encoding that does not suffer from this problem.
That only matters if you care about taking leap seconds into account. If you want atomic clock time, not mushy political calendar time, then an integer number of ticks suffices.
Would you prefer if I used the article's term of "precise time"?
At any rate, it's a lot better to use precise time timestamps, as a single integer of seconds/miliseconds/vibrations of a caesium atom>, than a dumb "01/01/99" string of integers that rolls over and breaks at Y2K. (with associated fun parsing bugs)
Would you prefer if I used the article's term of "precise time"?
At any rate, it's a lot better to use precise time timestamps, as a single integer of seconds/miliseconds/vibrations of a caesium atom>, than a dumb "01/01/99" string of integers that rolls over and breaks at Y2K. (with associated fun parsing bugs)
> At any rate, it's a lot better to use precise time timestamps, as a single integer of seconds/miliseconds/vibrations of a caesium atom>
No, it is not always better. If you want to store when a contract is going to end you don't want care how many milliseconds are between now and then. You care that it happens exactly at midnight of the last day of the year. Not an hour earlier or later. Even when politicians intervene between now and then.
No, it is not always better. If you want to store when a contract is going to end you don't want care how many milliseconds are between now and then. You care that it happens exactly at midnight of the last day of the year. Not an hour earlier or later. Even when politicians intervene between now and then.
Wrong problem. None of what you're talking about is even remotely relevant.
If you can't establish accurate precise time, you have no ability to handle all that political crap layered on top. You're trying to solve a top-of-stack problem in the bottom-most layer.
Look at the tz database, the way in which the problem you think needs to be solved is actually handled. It implements all your mushy ever-changing political end-of-day end-of-week crap as a table of conversions to precise-time seconds.
If you can't establish accurate precise time, you have no ability to handle all that political crap layered on top. You're trying to solve a top-of-stack problem in the bottom-most layer.
Look at the tz database, the way in which the problem you think needs to be solved is actually handled. It implements all your mushy ever-changing political end-of-day end-of-week crap as a table of conversions to precise-time seconds.
In theory yes, in practice no. When you store a future contract end date in seconds/milliseconds/microseconds since some epoch. You will do that with tz database version x. A few year later some admin updates the tz database. Now you have to remember if you have to reencode your timestamps or not. To do that correctly, you need to store the version of the tz database that was used, translate it using that version to a "human-readable" date and translate it back to the the timestamp with the latest version of tz. If you don't do that you will experience silent data corruption.
[deleted]
Everyone - especially those in my own team - told me that dates needed to be human readable strings in my APIs and messages, but I ignored them and used Java time stamps (Unix, but milliseconds) instead.
Lo, nobody complained about the readability because most time problems are relative to other events, rather than to a specific clock time. It’s so much easier to work out that two events occurred a few milliseconds apart when you don’t need to take segmentation (ie: ymdhms.s) into account. And using an integer obviates time zone and daylight savings corner cases. You can simply subtract two numbers and get useful information. And it’s much easier to do the math in your head and in scripts.
For those relatively rare times that you do need to know the clock time, there are CLI tools (ie, date -s) and websites galore.
Of course my use case was not everyone’s use case. But I ignored the blanket recommendation to use human readable time, and despite the naysayers, life became much easier for me and my team. I doubt that I would go back to string timestamps for any purpose.
Honestly, human time is for humans, it’s a presentation problem. Computer time is for computing.
Lo, nobody complained about the readability because most time problems are relative to other events, rather than to a specific clock time. It’s so much easier to work out that two events occurred a few milliseconds apart when you don’t need to take segmentation (ie: ymdhms.s) into account. And using an integer obviates time zone and daylight savings corner cases. You can simply subtract two numbers and get useful information. And it’s much easier to do the math in your head and in scripts.
For those relatively rare times that you do need to know the clock time, there are CLI tools (ie, date -s) and websites galore.
Of course my use case was not everyone’s use case. But I ignored the blanket recommendation to use human readable time, and despite the naysayers, life became much easier for me and my team. I doubt that I would go back to string timestamps for any purpose.
Honestly, human time is for humans, it’s a presentation problem. Computer time is for computing.
Leap seconds and UTC vs UT1? Even in your use case, this would matter.
But yeah, I take your point.
But yeah, I take your point.
> It’s so much easier to work out that two events occurred a few milliseconds apart when you don’t need to take segmentation (ie: ymdhms.s) into account.
IF you know they occurred a few milliseconds apart, integers are marginally easier.
IF you know they occurred a few milliseconds apart, integers are marginally easier.
1695029423495
1695029423502
2023-09-18T09:30:23.495Z
2023-09-18T09:30:24.502Z
1695029423495
1695029483502
2023-09-18T09:30:23.495Z
2023-09-18T09:30:25.502Z
that’s the argument I kept hearing, but in practice, it’s trivial.
I personally find that it's much more difficult to compute the number of seconds between two different ISO timestamps without help, and that’s my point.
1695029423495
1695029483502
2023-09-18T09:30:23.495Z
2023-09-18T09:30:25.502Z
I personally find that it's much more difficult to compute the number of seconds between two different ISO timestamps without help, and that’s my point.
Dates should be integer seconds since epoch. Timezones are essentially a UI concept.
The date value should always be stored and passed around as an int — and then only transformed into a user-readable string on the client-side at render-time, only when it's necessary for a human to read it. It's the job of frontend code to take in the date as input (still as an int), detect the reader's timezone, and render accordingly.
For some systems, you may want more precision, in which case millis (or nanos) are reasonable, as long as you choose a big enough int type to hold them.
Once humanity goes from 1 planets to N planets, the whole thing will become a mess again thanks to relativity, and software engineers of the future will have to deal with figuring out how to convert an Earth timestamp to a Martian timestamp — and what it even means for two timestamps to be "equal" then.
But that's next century's problem. ;)
The date value should always be stored and passed around as an int — and then only transformed into a user-readable string on the client-side at render-time, only when it's necessary for a human to read it. It's the job of frontend code to take in the date as input (still as an int), detect the reader's timezone, and render accordingly.
For some systems, you may want more precision, in which case millis (or nanos) are reasonable, as long as you choose a big enough int type to hold them.
Once humanity goes from 1 planets to N planets, the whole thing will become a mess again thanks to relativity, and software engineers of the future will have to deal with figuring out how to convert an Earth timestamp to a Martian timestamp — and what it even means for two timestamps to be "equal" then.
But that's next century's problem. ;)
Not sure what you mean by passing the timezone as int.
Edited a few words and punctuation marks to make my intent clearer; hope that helps. The idea is to deal with the timezone at render-time, not to pass it around.
When you store the time of a future event (like a reminder for 1 year from now), and integer since epoch does not work.
Your event is initially in the user's timezone (e.g. 10am 20th of June 2024 in France), if you store it as a number then you are fixing the exact second at which it will happen. However, if the timezone in question has some changes you didn't anticipate (new law voted, no more summer time), then you will either be triggering your event 1h before (e.g. 9am instead of 10am) which is probably not what your user wanted, or you'll have to recompute all your timestamps to take that change into account.
Instead, you could store rhe event with the target timezone, and let your datetime library that is up to date figure out next year.
Your event is initially in the user's timezone (e.g. 10am 20th of June 2024 in France), if you store it as a number then you are fixing the exact second at which it will happen. However, if the timezone in question has some changes you didn't anticipate (new law voted, no more summer time), then you will either be triggering your event 1h before (e.g. 9am instead of 10am) which is probably not what your user wanted, or you'll have to recompute all your timestamps to take that change into account.
Instead, you could store rhe event with the target timezone, and let your datetime library that is up to date figure out next year.
> However, if the timezone in question has some changes you didn't anticipate (new law voted, no more summer time), then you will either be triggering your event 1h before (e.g. 9am instead of 10am) which is probably not what your user wanted, or you'll have to recompute all your timestamps to take that change into account.
This is a really interesting scenario that I hadn't considered at all. You're totally right; this approach breaks down there.
Off the top of my head, it looks like this scenario would require manual action either way, though. Either it's an update of the packaged datetime library to incorporate this new regulation, or it's a database migration to update the timestamps for affected users.
This is a really interesting scenario that I hadn't considered at all. You're totally right; this approach breaks down there.
Off the top of my head, it looks like this scenario would require manual action either way, though. Either it's an update of the packaged datetime library to incorporate this new regulation, or it's a database migration to update the timestamps for affected users.
The same goes for "times this business is open" to store "9am-4pm on weekdays" you probably want to store almost exactly that string, rather than some ints. Generally speaking, dates stored as "ints" are mostly good for timestamps of events that have already happened (or even more precisely events that your application generated/logged live). Other use cases may be more rooted in the "human" dates than in absolute time
> Timezones are essentially a UI concept.
Sometimes they are, sometimes they are not.
For example...date+time of birth. The legal date (timezone) is a significant part of this event's time.
> passed around as an int
We're talking about a CSV. CSV don't have ints.
You can use base-10 encoding of the Unix timestamp, or the ISO 8601.
Sometimes they are, sometimes they are not.
For example...date+time of birth. The legal date (timezone) is a significant part of this event's time.
> passed around as an int
We're talking about a CSV. CSV don't have ints.
You can use base-10 encoding of the Unix timestamp, or the ISO 8601.
Yes to ALL of that.
I would add that a UI might prompt for a date as well, which of course would be entered using a local time input element based on the the users time zone. But that local time would immediately be converted to an int and passed to an API.
I would add that a UI might prompt for a date as well, which of course would be entered using a local time input element based on the the users time zone. But that local time would immediately be converted to an int and passed to an API.
Depends what they're for, to say 1000 hours from this point in time, no, to say at 10am in your local time zone in 100 days, yes.
If it's in the past, no
If it's in the past, no
There have been a few times at work where someone says “people use two digit years, can’t we just store two digit years?” To which I reply: “that’s literally the y2k problem”, usually gets us back to four-digit years.
Personally I use four-digit years even when I'm writing the date on some food that I'm putting in the fridge -- somethng that needs to be thrown away before the end of the week let alone the end of the century -- but at the same time I'm thinking to myself what a pedant I am.
Let me applaud that! Small redundancies like these in writing lessen ambiguity and cognitive load. Although "2023" is longer than "23" it is more readily apparent as a year when e.g. looking/scanning for dates in the fridge. You might be saving your future brain some precious energy in writing full years instead of shortened ones. (Only half joking, the same could presumably be achieved by a well layed out label with ample white space and good typography.)
We used to have to write the date at the top of every work page in school and draw a margin strictly with a ruler. At the time naturally I hated the arbritrariness of it but I really appreciate it as an adult. It feels a lot easier to be confident and break away impostor syndrome when you work with little rituals like this. Similar to how I was taught to write with all caps on engineering documents for legibility. It makes very little actual difference, it just feels professional, and I think inspires me to take a little more pride and care in other areas.
>> "It only takes 23 years to forget history."
I dont think this means people have forgot history, rather, I think people make tradeoffs. You trade off size/space/ease for long-term disruption. Most developers making these decisions now will be close to a 100yo in 2199. It will not be their problem, or even their childrens' problem. Possibly their grand-children's problem.
Alternatively, I could argue that the life of most systems is short enough that 80yrs is too long a horizon to worry about the problem.
I dont think this means people have forgot history, rather, I think people make tradeoffs. You trade off size/space/ease for long-term disruption. Most developers making these decisions now will be close to a 100yo in 2199. It will not be their problem, or even their childrens' problem. Possibly their grand-children's problem.
Alternatively, I could argue that the life of most systems is short enough that 80yrs is too long a horizon to worry about the problem.
> Most developers making these decisions now will be close to a 100yo in 2199.
We're giving people too much responsibility too young.
We're giving people too much responsibility too young.
We’re definitely giving negative seventy year-olds too much responsibility.
On the other hand, the ability to communicate with people several decades before they're born should have promising implications for the in-much-higher-demand ability to communicate with people after they've died.
This is what happens when tech companies start looking down on anyone over the age of 30.
Do people under 30 care about an extra 2 bytes in a date object?
Probably, as a lot of people from the current TikTok generation might lack the attention span to read more than two digits, or rather four digits till the end. Embedding some random game's gameplay and audio into your financial export to keep them engaged would increase the size by a couple of magnitudes, too.
"There are 5 differences between these two spreadsheets. Can you find them all?"
Sure, but the cost of 2 bytes is a lot different now than it was in 1950 (or 1970, or whatever). Store the full year.
Pretty cool! Although it's not obvious, without reading, that the issue falls at clock-bottom, which is counter to analog clock convention, both for 12 hour and 24 hour clocks. I would encourage some coloring, shading, or a larger mark.
Logically, reaching for a justification, an explanation, sure, it's reasonable that the MSB flips halfway around, but again it differs from the analog clock reading conventions.
v2 (+/v2.html) which he posted on another thread, does make it clearer
https://retr0.id/stuff/2038/v2.html
Other thread of his blog domain
https://www.da.vidbuchanan.co.uk/blog/unix-clock.html
https://news.ycombinator.com/item?id=37492371
The color separation is also a bit large, mainly 2-3 and 3-4, while 1-2 are similar but distinct.
Logically, reaching for a justification, an explanation, sure, it's reasonable that the MSB flips halfway around, but again it differs from the analog clock reading conventions.
v2 (+/v2.html) which he posted on another thread, does make it clearer
https://retr0.id/stuff/2038/v2.html
Other thread of his blog domain
https://www.da.vidbuchanan.co.uk/blog/unix-clock.html
https://news.ycombinator.com/item?id=37492371
The color separation is also a bit large, mainly 2-3 and 3-4, while 1-2 are similar but distinct.
If you like to have the Unix epoch in your prompt, you can do something like this:
If you wanted your own countdown, you could easily do `2147483647 - EPOCHSECONDS` and display that time instead.
EDIT: and if you want to mimic that oh-so-cool clock, you could do:
function prompt_epoch() {
printf -v COMMA_EPOCH "%'d" ${EPOCHSECONDS}
p10k segment -f 66 -t ${COMMA_EPOCH}
}
If you wanted your own countdown, you could easily do `2147483647 - EPOCHSECONDS` and display that time instead.
EDIT: and if you want to mimic that oh-so-cool clock, you could do:
printf '%x\n' $EPOCHSECONDS
> EPOCHSECONDS relies on `zmodload zsh/datetime`
Is there any advantage to using that module instead of invoking
Is there any advantage to using that module instead of invoking
date +%s
Yes, it doesn't fork a separate process. I doubt I could really tell the difference in prompt responsiveness, but I liked learning about the printf -v COMMA_EPOCH method.
This is displayed whnever I open a new terminal:
Welcome to workstation - Today is Sunday, September 17th 2023
There have been 53 years and 272 days in the unix epoch!
There are 14 years and 127 days until the 32-bit time apocalypse
Welcome to workstation - Today is Sunday, September 17th 2023
There have been 53 years and 272 days in the unix epoch!
There are 14 years and 127 days until the 32-bit time apocalypse
Prepare yourself...Discussed four days ago (15 comments): https://news.ycombinator.com/item?id=37492371
This makes me nostalgic for the mid-2000s when all the geeks had binary watches
binary watches are pure vanity. no one wore binary watches because they were useful, they weren't useful at all, unless your goal was socializing.
people wore binary wristwatches because they wanted to be seen wearing binary wristwatches, that's it.
to me, activities like this are the absolute epitome of waste and vanity. (both are bad, if I'm not being clear, here.)
but, I also think that it's perfectly ok for people to like whatever they want to like, as long as it doesn't hurt anyone, so I guess it's like "I don't like what you're saying, but you absolutely have the right to say it."
people wore binary wristwatches because they wanted to be seen wearing binary wristwatches, that's it.
to me, activities like this are the absolute epitome of waste and vanity. (both are bad, if I'm not being clear, here.)
but, I also think that it's perfectly ok for people to like whatever they want to like, as long as it doesn't hurt anyone, so I guess it's like "I don't like what you're saying, but you absolutely have the right to say it."
I definitely knew at least one engineer who learned to read their binary watch. The goal was absolutely social. It's a conversation-starter, for a crowd that, let's face it, could use more practice having conversations.
You're right that it's wasteful—those watches are mostly e-waste at this point. But so are all the electronics we've bought in the past 20 years. I'm typing on future e-waste right now!
You're right that it's wasteful—those watches are mostly e-waste at this point. But so are all the electronics we've bought in the past 20 years. I'm typing on future e-waste right now!
Why didn’t past generations consider these ramifications for future generations? 2038 must have seemed so far away, but now it’s so close I bet even in 2038 and beyond people will still be reading this comment. Maybe they will be cursing us for the tech debt they inherited!
Because the hardware didn’t support it. We knew in 1999 that the 2038 problem existed, for example, but our choices were a) stop representing dates as seconds since the Unix epoch, and introduce who knows how many bugs in every imaginable piece of software, or b) kick the can down the road until 2038, when presumably everything would support 64-bit integers. b) was the obvious choice then, and it still looks like the right choice to me in hindsight.
64 bit ints (long long) have been used on 32 bit platforms since mid-80s (ILP32LL)
>Why didn’t past generations consider these ramifications for future generations? 2000 must have seemed so far away, but now it’s so close I bet even in 2000 and beyond people will still be reading this comment. Maybe they will be cursing us for the tech debt they inherited!
~Not enough people in 1985, I guess. They seem more concerned with leap years than anything. https://groups.google.com/g/net.bugs/c/ZGlqGwNaq3I
~Not enough people in 1985, I guess. They seem more concerned with leap years than anything. https://groups.google.com/g/net.bugs/c/ZGlqGwNaq3I
I suppose they figured, quite reasonably, that things written in the 60's would be updated by 2038 and to address this problem was premature optimisation. In the 2000s 64-bit went mainstream which gave us a solid ~30 years to tackle this issue. 32-bit time was introduced in a time where there was a very real argument for it in terms of performance, simplicity, and cost.
I'm not even sure if in hindsight the wrong decision was made.
I'm not even sure if in hindsight the wrong decision was made.
Why do you assume they didn’t consider it? Memory was too scarce and too expensive to do it.
Unix time is 32 bits, so I guess Unix time was invented a bit later, but around 1970, when Unix was written, RAM cost around $700k/megabyte, or, ballpark, $1/byte (https://jcmit.net/memoryprice.htm)
Even if you had infinite memory, you couldn’t use much of it in your computer. The PDP-7 that Unix started life on topped out at 144kB, the PDP-11 that it was soon ported to at 288kB (64k 36-bit words), and many installs wouldn’t have that.
Unix time is 32 bits, so I guess Unix time was invented a bit later, but around 1970, when Unix was written, RAM cost around $700k/megabyte, or, ballpark, $1/byte (https://jcmit.net/memoryprice.htm)
Even if you had infinite memory, you couldn’t use much of it in your computer. The PDP-7 that Unix started life on topped out at 144kB, the PDP-11 that it was soon ported to at 288kB (64k 36-bit words), and many installs wouldn’t have that.
How much code have you written that you expect to still be in use 68 years from now?
Only the really bad code has a hope of lasting that long.
See: climate change, American social security administration, preservation of ancient manuscripts and ruins, etc...
But at least Jeff bezos is putting his fortune into an immortal clock in the desert (I'm being sarcastic but the desert clock is actually real and supposedly a gift to future generations as if measuring time wasn't a problem that stone-age civilizations all over the world were able to solve independently).
But at least Jeff bezos is putting his fortune into an immortal clock in the desert (I'm being sarcastic but the desert clock is actually real and supposedly a gift to future generations as if measuring time wasn't a problem that stone-age civilizations all over the world were able to solve independently).
For someone adult in 2000, 2038 might be retirement age, so they are probably investing in their pension and paying off the mortgage assuming 2038 is coming, but then at work it is like "wont be here when that happens!"
[deleted]
I love that hexadecimal-analog clock.
I know, right? I really want one for my wall.
Other interesting time storage issues, including the Y2038 problem.
https://en.m.wikipedia.org/wiki/Time_formatting_and_storage_...
https://en.m.wikipedia.org/wiki/Time_formatting_and_storage_...
Can't we fix this once and for all using variable length integers?
https://en.wikipedia.org/wiki/Variable-length_quantity
This way the counter can scale up to any number of bits.
https://en.wikipedia.org/wiki/Variable-length_quantity
This way the counter can scale up to any number of bits.
That would add a lot of complexity to structs that want to have a static size.
i’d just pick a huge number and move on.
i’d just pick a huge number and move on.
That's unnecessarily complicated.
A 64-bit signed Unix time fixes this for 25x longer than the remaining lifetime of the sun.
A 64-bit signed Unix time fixes this for 25x longer than the remaining lifetime of the sun.
Our descendants around alpha centauri are going to have so much legacy code to upgrade.
Unless you want to keep time to nanosecond granularity, like in a typical modern trading system. Then you have only +/- 292 years.
What if we have picosecond precision timers in the future? Femtosecond precision timers? Who knows, right?
True, but here we're taking about Unix time, which is seconds since 1970 UTC.
AFAIK, most systems with absolute nanosecond precision use two 64-bit ints.
AFAIK, most systems with absolute nanosecond precision use two 64-bit ints.
In my experience unix time in units finer than seconds is more common than not.
The NYSE stock exchange uses nanos since Unix epoch as a single 64bit int.
I've never seen someone waste a whole second 64bits
The NYSE stock exchange uses nanos since Unix epoch as a single 64bit int.
I've never seen someone waste a whole second 64bits
Ok, then in those cases use 128 bits.
Unix time is by definition "seconds since Unix epoch."
It common to have other measurements, e.g. JS uses milliseconds since epoch.
But technically that isn't "Unix time"
It common to have other measurements, e.g. JS uses milliseconds since epoch.
But technically that isn't "Unix time"
If you want to deal with date/time properly, then you will need a separate field for nanoseconds anyways, in order to deal with leap seconds; in such a case the number of nanoseconds can exceed one billion. In such a case the nanoseconds can fit in 32-bits and you don't need 64-bits; only the seconds is 64-bits and nanoseconds is 32-bits.
What are you talking about. A single 64 bit int is plenty for nanoseconds since epoch
It is already fixed once and for all by using i64, unless you care about dates hundreds of billions of years in the future or past.
Or unless you use more precise clocks. Nanosecond resolution cuts those billions down to hundreds of years. We're probably gonna get picosecond or femtosecond resolution clocks someday. Then we're gonna have yet another one of these threads and someone's gonna tell me 128 bits will fix it forever.
Those are already different types, though. What type it makes sense to use for seconds isn’t necessarily the same as what type it makes sense to use for other time units. I agree that using 128 bits makes sense for nanoseconds if for some reason you care about nanosecond-precision timestamps merely hundreds of years in the future or past.
I find it hard to believe we will ever have femtoseconds-resolution timers or ever even care what femtosecond an event happened at. But even if we do, incidentally, 128 bits is still fine. i128 femtoseconds will overflow in approximately the year 1000000000000000000000000.
So, I don’t think there’s any practically useful time unit for which we need variable length integers and can’t just use i128.
I find it hard to believe we will ever have femtoseconds-resolution timers or ever even care what femtosecond an event happened at. But even if we do, incidentally, 128 bits is still fine. i128 femtoseconds will overflow in approximately the year 1000000000000000000000000.
So, I don’t think there’s any practically useful time unit for which we need variable length integers and can’t just use i128.
> I find it hard to believe we will ever [...] care what femtosecond an event happened at
GNSS satellites require precise clocks to accurately calculate ranges. Signals travel at the speed of light so small imprecisions result in large errors: timing differences of one millisecond produce about 300 kilometers of error in distance measurements. It seems GPS satellites have clocks accurate to within 40 nanoseconds. One nanosecond gives around 30 cm of error, 40 nanoseconds would give about 12 meters.
Picosecond resolution would bring those errors down to the millimeter and micrometer ranges. Femtosecond resolution would bring the errors down to nanometers.
GNSS satellites require precise clocks to accurately calculate ranges. Signals travel at the speed of light so small imprecisions result in large errors: timing differences of one millisecond produce about 300 kilometers of error in distance measurements. It seems GPS satellites have clocks accurate to within 40 nanoseconds. One nanosecond gives around 30 cm of error, 40 nanoseconds would give about 12 meters.
Picosecond resolution would bring those errors down to the millimeter and micrometer ranges. Femtosecond resolution would bring the errors down to nanometers.
Okay, fair enough. But i128 is still plenty for femtoseconds.
Keep in mind that going from 64 to 128 bits squares the available range. The number of femtoseconds is merely a million times the number of nanoseconds, whereas the number of numbers representable by an i128 is 2^64 times the number representable by an i64.
Keep in mind that going from 64 to 128 bits squares the available range. The number of femtoseconds is merely a million times the number of nanoseconds, whereas the number of numbers representable by an i128 is 2^64 times the number representable by an i64.
> Picosecond resolution would bring those errors down to the millimeter and micrometer ranges. Femtosecond resolution would bring the errors down to nanometers.
And nobody can measure the signal that precisely.
And nobody can measure the signal that precisely.
I'm not knowledgeable enough about signals to know whether it's impossible or not. Maybe it's not possible now but one day it will be.
Mainframe software will be the most painful to fix.
Obligatory: Could we use LLMs?
In case you missed it, that’s IBM’s plan (or at least their marketing team’s plan)!
https://arstechnica.com/information-technology/2023/08/ibms-...
https://arstechnica.com/information-technology/2023/08/ibms-...
Also obligatory: calendar based cryptocurrency that measures time on the blockchain and also incidentally makes its creator rich by providing him with the largest initial allocation of currency.
LLMs can suggest migrated code but you still have to review it and test it. And you have to review not only what it migrates but also what it does not.
So it does not save you much time.
So it does not save you much time.
Would any “real” mainframe software (not Linux on z) use Unix epoch dates though?
I’d have thought mainframe dates were mostly binary coded decimal in EBCDIC and already futureproofed during the y2k mania.
I’d have thought mainframe dates were mostly binary coded decimal in EBCDIC and already futureproofed during the y2k mania.
That is not the real 2038 problem.
But cool clock, really love it!
But cool clock, really love it!
What is the real 2038 problem?
Why a signed int?
time_t has historically been a typedef to a 32-bit signed integer, so the signedness is baked into a lot of legacy software still running today.
Afaik this is so that you can go back in time before epoch (less than 0)
Then why not start the epoch however far back they thought they need to go back? Maybe a naive question.
Whether you make 0 be in 1902 and use an unsigned type, or make 0 be in 1970 and use a signed type, the range of representable dates is the same.
Sure. But (and what I should have added) I don't think they would need to go back that far.
But you're right, you can get the same results with both.
But you're right, you can get the same results with both.
As others have pointed out in the thread, it was already not enough to represent the dates of birth of plenty of people who were still alive in 1970.
Good point. :)
You may need an arbitrary distance back in time, so really any epoch is as good as any other. So if you're going to pick arbitrarily, zero around where most numbers are going to be when you set the standard seems to make sense.
Was the negative sign commonly used for delta times?
The epoch was set very close to the start of UNIX (it started development in 1969, before epoch 0). Negative epoch dates are quite common, e.g. many people's birthdays.
In 1970, I suppose there are many people whose birthdate is before 1901-12-13?
0 UNIX epoch is 1970-01-01. So everyone between 1901 and 1970 have negative birthdays in UNIX time. E.g. Brian Keringhan's birthday is -881107200 in UNIX time.
Converting to unsigned int only changes the first digit (6 becomes e) but I think `e5075ea9` is a little easier to reason about.
[deleted]
time_t is now 64 bits.[1] The change probably happened around 2014.[2]
[1] https://gcc.godbolt.org/z/Wv99v4W6E
[2] https://lwn.net/Articles/598408/
[1] https://gcc.godbolt.org/z/Wv99v4W6E
[2] https://lwn.net/Articles/598408/
Using a signed int allows you to represent times before 1970. This is not just theoretical; it’s done in the wild. Materialize, the data warehouse software I work on, used to have a bug causing it to fail on negative Unix times and we found out when somebody tried to load in a database with human dates of birth.
2018-July-22 was the last day "that wouldn't work for all people" excuse was true.
Chiyo Miyako was born on 1901-May-02
Chiyo Miyako was born on 1901-May-02
Yeah, but now when you try to load in a database of documents (e.g. legal, blueprints, etc...), the date the document was created crashes everything instead.
Why hex though?
2 hex characters represent 1 byte (8bits) so it lines up nicely if you’re looking at where memory gets saved, and will align nicely with big rollovers since it’s a power of 2 radix. FFFFFFFE => FFFFFFFF => 00000000, rather than 4294967294 => 4294967295 => 0 or the equivalent formatted as a date string. Feels better for a count down thing.
Plus bonus points for, when mentioning how 64 bit ints are the solution here:
FF FF FF FF (u32)
=
00 00 00 00 FF FF FF FF (u64)
makes it pretty clear what the trade off in memory usage is vs range gained.
Plus bonus points for, when mentioning how 64 bit ints are the solution here:
FF FF FF FF (u32)
=
00 00 00 00 FF FF FF FF (u64)
makes it pretty clear what the trade off in memory usage is vs range gained.
It only takes 23 years to forget history.