Unix time is bad and needs replacement, not UTC(z.vandillen.dev)
z.vandillen.dev
Unix time is bad and needs replacement, not UTC
https://z.vandillen.dev/2022/11/21/unix-time-bad/
95 comments
I expect you’ll find that a single spot on earth can have multiple values of the current time, depending on who you ask.
For example, India, Pakistan, and China are each are in their own time zone, and (https://en.wikipedia.org/wiki/Jammu_and_Kashmir_(union_terri...) “Jammu and Kashmir is a region administered by India as a union territory and consists of the southern portion of the larger Kashmir region, which has been the subject of a dispute between India and Pakistan since 1947, and between India and China since 1962”
⇒ I expect there are events that are recorded to have happened at three different times, depending on who you ask.
https://en.wikipedia.org/wiki/List_of_territorial_disputes has many other territorial disputes, including ones between Canada and the USA (https://en.wikipedia.org/wiki/List_of_areas_disputed_by_Cana...), but most of them won’t be across time zones.
Also, there are other calendars in use than the Gregorian one. Most confusingly, some groups still use the Julian Calender.
For example, India, Pakistan, and China are each are in their own time zone, and (https://en.wikipedia.org/wiki/Jammu_and_Kashmir_(union_terri...) “Jammu and Kashmir is a region administered by India as a union territory and consists of the southern portion of the larger Kashmir region, which has been the subject of a dispute between India and Pakistan since 1947, and between India and China since 1962”
⇒ I expect there are events that are recorded to have happened at three different times, depending on who you ask.
https://en.wikipedia.org/wiki/List_of_territorial_disputes has many other territorial disputes, including ones between Canada and the USA (https://en.wikipedia.org/wiki/List_of_areas_disputed_by_Cana...), but most of them won’t be across time zones.
Also, there are other calendars in use than the Gregorian one. Most confusingly, some groups still use the Julian Calender.
True, and thats why it's important to have both the political time centre with a TZ database city, as well as coordinates as an options. They are both different things.
TZ database city covers the political time based on a political centre, and is future proof for that use case.
Coordinates cover changing political centres. If a country border moves but you need to specify a time that will update with any boundary changes this is the only way. The coordinates also don't have to match the location of the event, they are specifying the "human time" reference point.
TZ database city covers the political time based on a political centre, and is future proof for that use case.
Coordinates cover changing political centres. If a country border moves but you need to specify a time that will update with any boundary changes this is the only way. The coordinates also don't have to match the location of the event, they are specifying the "human time" reference point.
The problem with coordinates is that they don't necessarily cover changing political centres. The opinion of the true legal time will differ based on political calculations. In a territory that is internationally recognised as Ukraine, annexed by Russia but under the contested control of Ukraine, how does the coordinates help? They probably don't. You'll have to know whether that particular time was from the Ukrainian perspective or the Russian perspective. In most cases that will be easier to code by a tzid than geographical coordinates.
Another example is Xinjiang province, where in some towns everything is in local time, in some towns everything is in legal time, in some towns it depends on if you're Han or indigenous, and in some towns it depends on what you're doing (bus timetable vs shop opening hours).
Another example is in a very large but lightly populated part of Australia, where there local community uses a defacto timezone (Australia/Eucla) but the legal timezone is different (Australia/Perth) - it is no use saying that the polls close at 6pm and the general store closes at 6pm and coding them with the same coordinates, because one will refer to a moment 45mins after the other. Other places may have similar variations of opinions where the formal boundary of two timezones isn't really obvious - I have heard that the timezone boundary around Broken Hill is a matter of opinion as much as it is a matter of law.
The other possibility is of course that perhaps in a country like the US where timezone boundaries are relatively unstable over time because they pass through states. It's true that coordinates could be used to deal with it. But it's not obvious that this is such a systematic and regular form of variation that timestamps should use coordinates, especially considering almost any record that relates to time in a place probably already codes the place so an ad hoc migration should generally be possible. (In such a case, I would still like tagging the time with a tzid so you know for a fact whether it is premigration or a postmigration.)
Another example is Xinjiang province, where in some towns everything is in local time, in some towns everything is in legal time, in some towns it depends on if you're Han or indigenous, and in some towns it depends on what you're doing (bus timetable vs shop opening hours).
Another example is in a very large but lightly populated part of Australia, where there local community uses a defacto timezone (Australia/Eucla) but the legal timezone is different (Australia/Perth) - it is no use saying that the polls close at 6pm and the general store closes at 6pm and coding them with the same coordinates, because one will refer to a moment 45mins after the other. Other places may have similar variations of opinions where the formal boundary of two timezones isn't really obvious - I have heard that the timezone boundary around Broken Hill is a matter of opinion as much as it is a matter of law.
The other possibility is of course that perhaps in a country like the US where timezone boundaries are relatively unstable over time because they pass through states. It's true that coordinates could be used to deal with it. But it's not obvious that this is such a systematic and regular form of variation that timestamps should use coordinates, especially considering almost any record that relates to time in a place probably already codes the place so an ad hoc migration should generally be possible. (In such a case, I would still like tagging the time with a tzid so you know for a fact whether it is premigration or a postmigration.)
The engineering solution to this problem isn't to add more parity bits to deal with the political changes, it's to use TAI, and make sure that TAI is well run and consistent.
We will need to encode planetary time origins, and should think about the equivalent, but I don't think that the solution to that will happen until after we need it to happen.
We will need to encode planetary time origins, and should think about the equivalent, but I don't think that the solution to that will happen until after we need it to happen.
Even city names may change depending on who or when you ask, so this does not seem like a maintainable solution.
> I think we need something like this:
> 2022-11-23T08:52:02!Europe/London
There is a IETF standard proposal, which is very closed to getting finalised, that defines a similar format:
It's already in use by the Temporal TC39 proposal, which defines new types for working with temporal data in JavaScript.
ref: https://github.com/tc39/proposal-temporal
> 2022-11-23T08:52:02!Europe/London
There is a IETF standard proposal, which is very closed to getting finalised, that defines a similar format:
2022-11-23T08:52:02+01:00[Europe/London]
It's already in use by the Temporal TC39 proposal, which defines new types for working with temporal data in JavaScript.
ref: https://github.com/tc39/proposal-temporal
It's still a bit of a weird format, because it has an offset and a zone. What does your example mean, given that it has an offset of +1 hour, on a date when in the zone you've specified, the offset is +0?
For more fun, what does 2024-11-23T08:52:02+00:00[Europe/London] mean now, and what does it mean if the UK switches to a +1 hour offset all year round in 2023?
If i understand the proposal correctly, the actual moment in time represented by one of these is given by the time and the offset. The bit in square brackets is just metadata. The spec says:
> This document does not address extensions to the format where the semantic result is no longer a fixed timestamp that is referenced to a (past or future) UTC time.
But that result is precisely why we need this!
For more fun, what does 2024-11-23T08:52:02+00:00[Europe/London] mean now, and what does it mean if the UK switches to a +1 hour offset all year round in 2023?
If i understand the proposal correctly, the actual moment in time represented by one of these is given by the time and the offset. The bit in square brackets is just metadata. The spec says:
> This document does not address extensions to the format where the semantic result is no longer a fixed timestamp that is referenced to a (past or future) UTC time.
But that result is precisely why we need this!
ISO 8601 was designed as a standard format for conveying date/times from the Gregorian calendar. The Gregorian calendar divides time into days, months, years which are units chosen based on Earth rotation around its axis and its orbit around the star Sol.
I don’t see why it would make sense to include points outside of Earth in that system, as I think you’re proposing.
Regarding the other part of your proposal to include locations: ISO 8601 purposely excludes those because converting a location to a UTC offset is a political process that requires lookups for historical changes to that conversion.
I don’t see why it would make sense to include points outside of Earth in that system, as I think you’re proposing.
Regarding the other part of your proposal to include locations: ISO 8601 purposely excludes those because converting a location to a UTC offset is a political process that requires lookups for historical changes to that conversion.
OK, so my moon landing example what slightly wrong, the Astronauts were using ET not UTC, and so we can infer that prodominatly used "human time" at that location on the moon was actually ET (what the astonougts used). And so it should be:
For other planetary bodies, or other locations in the universe the first part of the time stamp could use any other format that could be in use at that location. It doesn't have to be tied to an Earth centric system.
> ISO 8601 purposely excludes those because converting a location to a UTC offset is a political process that requires lookups for historical changes to that conversion
My point is we need a standardised interchange format what actually takes that into account as that is how humans actually operate.
1969-07-20T16:17:00!Moon/0.67416/23.47314
For other planetary bodies, or other locations in the universe the first part of the time stamp could use any other format that could be in use at that location. It doesn't have to be tied to an Earth centric system.
> ISO 8601 purposely excludes those because converting a location to a UTC offset is a political process that requires lookups for historical changes to that conversion
My point is we need a standardised interchange format what actually takes that into account as that is how humans actually operate.
It makes sense that moon dates are in sync with Earth, after all its motion is in sync with Earth too. Mars days are slightly longer than Earth's ones but there are more of them in a Martian year. You want months and years to be local if there are seasons that affect life on that planet. I expect that Mars will have its own months and years but they'll know Earth date for quite a long time, at least until they'll have to depend on Earth for survival.
That reminds me of WH40k, a fantasy universe where time/date was tracked based on when the last contact/sync [1] with earth's time has occured, with similar idea as NTP stratum.
I guess for multiple planets some astronomical recurring event could be used as sync source (say have "tick" be determined by some pulsar and adjusted for doppler shift)
* [1] https://warhammer40k.fandom.com/wiki/Imperial_Dating_System
I guess for multiple planets some astronomical recurring event could be used as sync source (say have "tick" be determined by some pulsar and adjusted for doppler shift)
* [1] https://warhammer40k.fandom.com/wiki/Imperial_Dating_System
Different than the UTC/TAI issue: A limitation of only using UTC timestamps came up when I worked at a social photo sharing company. We needed to define how to store and transport them, and everyone was saying UTC is the answer and I was arguing for local time + TZ and/or location.
The example I used was if New Years pictures were posted by a photographer in say New Zealand that was being viewed by someone in the US. Showing the US local time for that photo doesn't indicate how many minutes before or after midnight the photo was actually taken. For data transfer, ended up using strings for local date/time and location to ensure that nothing tried to interpret and transform unless explicitly intended.
The example I used was if New Years pictures were posted by a photographer in say New Zealand that was being viewed by someone in the US. Showing the US local time for that photo doesn't indicate how many minutes before or after midnight the photo was actually taken. For data transfer, ended up using strings for local date/time and location to ensure that nothing tried to interpret and transform unless explicitly intended.
Given the findings of general relativity we might also wanna include the speed of the object in fractions of lightspeed ; )
Speed relative to what?
That is the obvious catch of keeping time in a universe where everything and their dog moves.
Generally a good Idea, but writing the location-names or even coordinates in the timestring seems like bloat and slightly unreliable as it does not documentate from which point in time and which culture the timestring originates. As it's necessary to use a database anyway, why not just put a timezone-id there?
And you can even maintain multiple versions of the database, if neccessary. Just change !tz to !tz2, !tz3, etc.
2022-11-23T08:52:02!tz:423
And you can even maintain multiple versions of the database, if neccessary. Just change !tz to !tz2, !tz3, etc.
Having multiple time zone databases to sync and manage sounds like even worse idea.
ID, well, it saves a tiny bit of space but honestly I don't think it is worth it if you're already storing info about timezone as text.
If you're storing it in binary form you can convert the name to DB entry on save/display
ID, well, it saves a tiny bit of space but honestly I don't think it is worth it if you're already storing info about timezone as text.
If you're storing it in binary form you can convert the name to DB entry on save/display
My point is, Timezone-Names alone are not completely reliable. They only name the location, but not the political or historical situation of this location.
Take Europe/Berlin for example, the city was once divided in two parts, and is now in a country which is changing timezone twice a year. With a location, you can only communicate to use the rules valid today, not rules which were valid at some point in the past, or which are valid today, but in a different situation/context.
Take Europe/Berlin for example, the city was once divided in two parts, and is now in a country which is changing timezone twice a year. With a location, you can only communicate to use the rules valid today, not rules which were valid at some point in the past, or which are valid today, but in a different situation/context.
And parts of Mexico used to be the same timezone as Los Angeles but then daylight savings time changed and the city you referenced has to change.
There’s no way to look at America/Los Angeles and know if that is California or Oregon, and if Oregon were to give up daylight savings time you’d have a problem.
There’s no way to look at America/Los Angeles and know if that is California or Oregon, and if Oregon were to give up daylight savings time you’d have a problem.
It's even more complicated than that. Whether timestamps should change when timezones change depends on the use case.
"Let's meet at 8am Mar 3 2023" probably shouldn't become "Let's meet at 7am Mar 3 2023" if the timezones shifts.
"Let's meet at 8am Mar 3 2023" probably shouldn't become "Let's meet at 7am Mar 3 2023" if the timezones shifts.
Wellll, that depends. The base case, sure. but meetings are between two or more people. so what do you do when one of the people is in a different timezone from the other, with different dates for daylight savings time crossover? a solution can be reached, but it's not clear that the meeting time shouldn't move.
It's fairly unusual to have a system where people can write timestamps in arbitrary timezones. Most of the time there's a suitable context where a timezone just makes sense
For example, in a booking system the timezone will be associated with the venue or the destination.
In a scheduler for a trading system certain timestamps (start, stop time etc) will be the timezone of the trading venue.
For past events (logging) UTC essentially always makes sense.
For example, in a booking system the timezone will be associated with the venue or the destination.
In a scheduler for a trading system certain timestamps (start, stop time etc) will be the timezone of the trading venue.
For past events (logging) UTC essentially always makes sense.
Yes. Even storing a datetime from the future requires you to choose the correct DST the date is in, instead of the current one.
I wonder how bad of a timeformat would be just a bitfield struct with all of the parts(year/month/day/hour/minute/second/usec) broken out to separate fields. It should fit quite well to 64bits, so you could pass it along as an opaque 64bit integer value.
Is it not irrelevant for most use cases? Leap seconds or whatever. Think of the time of this post. It is stored but how important is the accuracy?
I agree that for many use cases the exact second is irrelevant, like submission times and things likes that. That being said, programmers do seem to expect, for example, that code like `(int)time(NULL) - previous_time` works and accurately gives the difference in time between now and whenever `previous_time` was given its value. This does work, but only 99.99% of the time. Then it breaks 0.01% of the time. For bug-free code it's important that the expectations of the programmer are in line with reality, and the problem with Unix time is that programmers' expectations of the mathematical properties of Unix time and the real mathematical properties of Unix time do not line up. This is what leads to the countless bugs involving leap seconds.
The fact that accuracy is often not important is also crucial to my proposal of legacy Unix time, because legacy Unix time would eventually go out of sync with the rotation of the Earth just like TAI. In almost all cases this would practically be an aesthetic bug where dates in old user interfaces would be a few seconds off from the real date, and therefore not particularly harmful.
The fact that accuracy is often not important is also crucial to my proposal of legacy Unix time, because legacy Unix time would eventually go out of sync with the rotation of the Earth just like TAI. In almost all cases this would practically be an aesthetic bug where dates in old user interfaces would be a few seconds off from the real date, and therefore not particularly harmful.
Timestamps are used for a lot of scheduling tasks that need to be reasonably precise. So an event happening one second earlier than expected, or even twice, could be a problem I suppose. Not exactly impossible to deal with, but I guess lots of people screw it up? Haven't really worked on systems where this _truly_ matters.
Synchronization between computers is hard.
https://www.gnu.org/software/coreutils/manual/coreutils.html...
Could it matter for high-frequency trading? Since these trades occur in milliseconds, I presume trading being off by a second would be considered very damaging.
It is not just very damaging, it also has legal implications. For example MIFID II regulations require at least 100us resolution for trade reporting. In practice PTP and GPS clocks are a requirement.
Scheduling tasks on a system is done using UTC/local time which the system keeps accurate. Unix time is not used for that.
I am not aware of any operating system kernel that schedules tasks or timer expiration using a strict definition of UTC. Unix time is the most common, and local monotonic (non-decreasing) clocks are becoming common for that as well.
Of course there are application level schedulers that use wall clock time in UTC format, and if you set the clock back your task may run twice, which is a more serious real world problem than running twice around a leap second. That is why no one who is not attached to civil time should use UTC or Unix time for short duration timer expiration in particular.
Of course there are application level schedulers that use wall clock time in UTC format, and if you set the clock back your task may run twice, which is a more serious real world problem than running twice around a leap second. That is why no one who is not attached to civil time should use UTC or Unix time for short duration timer expiration in particular.
That was my thought when considering the difficult and impossible tasks of converting to/from UTC-based timestamps. If we accept that UTC is only for human use then we should also accept that humans can't tell the time of day down to the second just based on their circadian rhythm and the Sun.
Given a TAI-based timestamp a hundred years in the future we can be pretty certain in saying "yeah it will be around lunch time" (assuming there is no major changes in the solar system that would probably have such an impact on the lives of every human that nobody would care about this event anyway).
Given a TAI-based timestamp a hundred years in the future we can be pretty certain in saying "yeah it will be around lunch time" (assuming there is no major changes in the solar system that would probably have such an impact on the lives of every human that nobody would care about this event anyway).
But if you follow that argument then you might as well drop leap secnds from UTC (which also drops them from Unix time). Which is what was done. So computers can have nice math and conversions to UTC are not needlessly complicated for little benefit to humans.
For things like mission-critical logging, having time jump around, or even skew, is a real PITA.
Everything is bad and needs replacement :)
Can someone comment on the practical usefulness of having UTC stay strictly synchronized with the rotation of the Earth?
Intuitively it should be synchronized, but giving it more thought it is not entirely obvious why; e.g. my current local time zone is more than half an hour off of local solar time, and that isn't really a practical problem, so if UTC would be, say, five minutes off of actual solar time in Greenwich, what issues does it cause for people? Astronomy would be using specific time systems and/or adjustments anyway.
Intuitively it should be synchronized, but giving it more thought it is not entirely obvious why; e.g. my current local time zone is more than half an hour off of local solar time, and that isn't really a practical problem, so if UTC would be, say, five minutes off of actual solar time in Greenwich, what issues does it cause for people? Astronomy would be using specific time systems and/or adjustments anyway.
I see leap seconds as regulatory capture by astronomers :D Basically it's convenient for them to have civil time (UTC) never more than 0.5s from UT1.
Astronomers don’t care, they already need like 5 different clocks.
Also the allowable variance between UTC and ut1 is 0.9.
Also the allowable variance between UTC and ut1 is 0.9.
Or one clock that can switch between 5 different times.
I didn't necessarily mean 5 different timekeeping devices (though that would probably be more convenient in many cases), rather than an astronomer needs to keep track of multiple different "times" concurrently.
> what issues does it cause for people?
Five minutes - no problem at all.
If the offset rose to 60 minutes, I know people complain bitterly twice a year when we change the clocks by an hour and they lose an hour of light for sports in the evening, or lose an hour of sleep. But with leapseconds taking ~0.5 seconds a year, that won't be a problem for a few thousand years.
Five minutes - no problem at all.
If the offset rose to 60 minutes, I know people complain bitterly twice a year when we change the clocks by an hour and they lose an hour of light for sports in the evening, or lose an hour of sleep. But with leapseconds taking ~0.5 seconds a year, that won't be a problem for a few thousand years.
But that’s caused by changing the clock all the time, the opposite of what PeterisP proposed. When it comes to constant offsets[1], there are countries whose political time zones are shifted from their geographical time zones by 1h (Spain, Iceland) or even 2h (Western China). It doesn’t seem to be causing problems.
[1]: Because allowing the time to drift microscopically from human perception is indistinguishable from not changing it, only that after a sufficiently long time you live in a time zone shifted by a constant, only there was never really a big event to shift it. It’s like getting old - you don’t notice it, until it’s already happened.
[1]: Because allowing the time to drift microscopically from human perception is indistinguishable from not changing it, only that after a sufficiently long time you live in a time zone shifted by a constant, only there was never really a big event to shift it. It’s like getting old - you don’t notice it, until it’s already happened.
Around the time TAI drifts an hour from UT1, the current definition of UTC will require several leap seconds each year to keep in sync. So there isn’t much point saying that we have to keep the current definition of UTC to avoid leap hours, because the current definition of UTC will not work that long. https://www.ucolick.org/%7Esla/leapsecs/dutc.html
60mins offset is fine¹. We have this all the time with timezones. But imagine investigating and outage or scheduling an automated event when you have to make sure and communicate with non-technical folks that your graph / setting etc is off by 5:23 minutes.
The OP proposal does say UTC should be used when representing time to end users, but in practice there is going to be blurry lines.
¹TBH, still confusing enough to introduce quite a bit of friction when comparing graphs etc for outages.
The OP proposal does say UTC should be used when representing time to end users, but in practice there is going to be blurry lines.
¹TBH, still confusing enough to introduce quite a bit of friction when comparing graphs etc for outages.
I don't think there is a practical need for UTC to be synchronized with the rotation of the Earth for a long time, because although the error accumulates, it only accumulates very slowly. That being said, it's unclear how people would solve the problems that will arise once that long time has passed. In any case, although it's probably too late for leap seconds, untying the definition of Unix time from UTC would allow astronomers to decide with more freedom what the future of UTC is going to be.
> That being said, it's unclear how people would solve the problems that will arise once that long time has passed.
I feel like we should be able to easily "bundle" leap seconds into existing leap days on leap years. Make February 29th just some random number of seconds shorter or longer than, say, February 28th. Most people feel February 29ths are weird anyway, few people would probably notice when they have strange amounts of seconds in their final hour. Some software might hate it if Feb 29 23:59:56 rolls over to Mar 1 00:00:00 or much more rarely there's a Feb 29 23:59:62 (or using the existing Unix time "smearing" technique, some Feb 29 you get three or more Feb 29 23:59:59 timestamps in a row), but other than that it would 1) be on a predictable day, and 2) a day that's already a weird every 4 years except every 400 outlier that's already expected to be an edge case.
I feel like we should be able to easily "bundle" leap seconds into existing leap days on leap years. Make February 29th just some random number of seconds shorter or longer than, say, February 28th. Most people feel February 29ths are weird anyway, few people would probably notice when they have strange amounts of seconds in their final hour. Some software might hate it if Feb 29 23:59:56 rolls over to Mar 1 00:00:00 or much more rarely there's a Feb 29 23:59:62 (or using the existing Unix time "smearing" technique, some Feb 29 you get three or more Feb 29 23:59:59 timestamps in a row), but other than that it would 1) be on a predictable day, and 2) a day that's already a weird every 4 years except every 400 outlier that's already expected to be an edge case.
Large cloud providers smear time. A day containing a leap second has the normal number of Unixtime "seconds" but each is 1/86400 too long. The inaccuracy this introduces is relevant to such a tiny tiny percentage of use cases; who already have to deal with the extreme complexities of time at this accuracy that everyone wins.
Large cloud providers are actually exactly the ones who argued that leap seconds should be abolished, so I presume there were enough use cases where this was an issue. I faintly remember one article about the announcement of the voting results where they talked about the issues that smearing could cause but I don't remember exactly what it was, or where I read it.
Voting running after midnight local time (where leap seconds get inserted) and dependent on sub-seconds, with only five leap seconds since 2000 introduced reads like a really interesting case.
I find it difficult to believe there are use cases affected by this; given that NTP normally has an error window of some milliseconds, those use cases would never have worked.
Wrong. The smearing cumulates during a day...
>The exact way Unix time handles leap seconds* is complex and an unnecessary detail for this article, but it will suffice to say that (i) whenever a positive leap second occurs, a timestamp is used twice, and (ii) if a negative leap second were to occur, a timestamp is skipped.
Huh, I always assumed that Unix time just ignores them and ticks forward. I get why e.g. UTC needs them but why would it make sense for Unix time to account for them given how it's defined.
Huh, I always assumed that Unix time just ignores them and ticks forward. I get why e.g. UTC needs them but why would it make sense for Unix time to account for them given how it's defined.
If Unix time ignored them, the conversion between timestamps and UTC would no longer be possible with just arithmetic operations. You'd need a list of leap seconds and take them into account. Now one can argue if this is better or worse than the current solution, but it's not without its tradeoffs.
I'd argue that a very short and rather simple and infrequently changing list of delta seconds (which are only ever applicable around 23:59:59, December 31 of some years) is vastly preferable over any other of the solutions (or rather 'coping strategies' as it were) that are used now, things like Google and Amazon smearing their leap seconds but in ever so slightly incompatible ways.
According to Wikipedia there are 27 leap seconds and obviously there is already a procedure in place to communicate new ones to the OS.
Adding them into the conversion would add O(log(#leap_seconds)) complexity.
I think that continued use of leap seconds is entirely due to inertia.
Adding them into the conversion would add O(log(#leap_seconds)) complexity.
I think that continued use of leap seconds is entirely due to inertia.
Edit: My original comment is incorrect. Please read child comments.
Original comment:
Yes, to convert from UNIX time to UTC, you do need a list of leap seconds.
Original comment:
Yes, to convert from UNIX time to UTC, you do need a list of leap seconds.
Others have already corrected you, but in case it makes it more concrete, here is the implementation of timegm() in musl (the libc used in Alpine Linux and some other Linux distributions). timegm() takes a time in UTC and converts it to Unix time.
http://git.musl-libc.org/cgit/musl/tree/src/time/timegm.c
http://git.musl-libc.org/cgit/musl/tree/src/time/__tm_to_sec...
You'll notice there is no handling or knowledge of leap seconds. (Leap years are handled, of course.)
http://git.musl-libc.org/cgit/musl/tree/src/time/timegm.c
http://git.musl-libc.org/cgit/musl/tree/src/time/__tm_to_sec...
You'll notice there is no handling or knowledge of leap seconds. (Leap years are handled, of course.)
No. You need a list of leap seconds to convert from Unix time to TAI.
It's usually set up to sync to NTP.
Clocks are out of sync all the time, so any code that relies on precise timestamps needs some way to account for out-of-sync clocks anyway.
So if you store two timestamps, you can't rely that the actual time that has passed between storing the two timestamps is equal to their difference, since the clock might have been slow, fast, or might have been synchronised with another clock between the recording of the two timestamps.
I think dealing with leap seconds should work in a similar way as working with other clock synchronisation issues, so I really don't see the point of adding new types of clocks in unix.
So if you store two timestamps, you can't rely that the actual time that has passed between storing the two timestamps is equal to their difference, since the clock might have been slow, fast, or might have been synchronised with another clock between the recording of the two timestamps.
I think dealing with leap seconds should work in a similar way as working with other clock synchronisation issues, so I really don't see the point of adding new types of clocks in unix.
I think the article should not have left out the observation that in case you want to get to local time from UTC, you can only do so with certainty for the past, the present and the near future, because there could always be a change in daylight savings time (DST) regulations for a given locale. In my eyes that makes UTC a tiny little less useful for some purposes. OTOH human schedules should not be affected. Even for a country like [Egypt] which apparently has no issues with changing DST regulations with a mere few days of notice, meeting at 1:00pm in Cairo on some given day in the future will always mean whatever the local time stipulates it to be, come the day.
OTOH I believe that for computers, adopting something like TAI for all timestamps would probably a good thing. Re-adjusting display dates for the past where needed is much less complex wrt leap seconds than it is for time zones and DST. The conceptual simplicity of TAI and the invariants afforded by it come with the slight disadvantage that "the future can not be predicted", but we already cannot predict the future of countries changing time zones and DST dates, so we'd not be any worse off in that regard.
[Egypt](https://en.wikipedia.org/wiki/Daylight_saving_time_in_Egypt)
OTOH I believe that for computers, adopting something like TAI for all timestamps would probably a good thing. Re-adjusting display dates for the past where needed is much less complex wrt leap seconds than it is for time zones and DST. The conceptual simplicity of TAI and the invariants afforded by it come with the slight disadvantage that "the future can not be predicted", but we already cannot predict the future of countries changing time zones and DST dates, so we'd not be any worse off in that regard.
[Egypt](https://en.wikipedia.org/wiki/Daylight_saving_time_in_Egypt)
EDIT: This comment was corrected by numerous replies below and the original comment contained incorrect information. Thanks to all posters for the corrections.
Original comment:
> The simplified definition of Unix time is that Unix time counts the amount of seconds that have passed since the first of January 1970 (which is referred to as “the epoch”, similar to the first of January of 1 AD being the epoch of the Gregorian calendar). This is of course not the complete definition, since it does not take leap seconds into consideration.
I disagree with the statement that UNIX time is not the amount of seconds since the Epoch. Leap seconds never happened, they were added to UTC and UTC only in order to correct it for sunrise and sunset.
If we started an experiment on 23:00 and ended it on 01:00 the next day, but a leap second occured in the mean time, the experiment wasn't running for 7200 seconds, but for 7199 seconds. The leap second had nothing to do with time counting, a stopwatch would not add a second to the reading.
If we started an experiment on Epoch, the UNIX time tells us the amount of seconds this experiment has been running. If we added 86400 leap seconds in between, the experiment would count real time, not our made up time. One day wouldn't be added to the count.
As far as I understand it, that's what UNIX time is, and there's nothing wrong with that. It's a feature.
Please add your opinion.
Original comment:
> The simplified definition of Unix time is that Unix time counts the amount of seconds that have passed since the first of January 1970 (which is referred to as “the epoch”, similar to the first of January of 1 AD being the epoch of the Gregorian calendar). This is of course not the complete definition, since it does not take leap seconds into consideration.
I disagree with the statement that UNIX time is not the amount of seconds since the Epoch. Leap seconds never happened, they were added to UTC and UTC only in order to correct it for sunrise and sunset.
If we started an experiment on 23:00 and ended it on 01:00 the next day, but a leap second occured in the mean time, the experiment wasn't running for 7200 seconds, but for 7199 seconds. The leap second had nothing to do with time counting, a stopwatch would not add a second to the reading.
If we started an experiment on Epoch, the UNIX time tells us the amount of seconds this experiment has been running. If we added 86400 leap seconds in between, the experiment would count real time, not our made up time. One day wouldn't be added to the count.
As far as I understand it, that's what UNIX time is, and there's nothing wrong with that. It's a feature.
Please add your opinion.
Your understanding of Unix time is wrong. In your experiment the difference between the end and start Unix timestamps would be 7200 even though thats off from the experiment duration. That is exactly the problem: Unix time gets adjusted for leap seconds to stay in sync with UTC instead of real time.
> As far as I understand it, that's what UNIX time is
Then you don’t understand Unix time. In the first case it would return 7200, and in the second case it would count “made up time”.
UNIX time is 86400*days since epoch + seconds since midnight. It does not measure “real time”, whatever that is.
> Leap seconds never happened, they were added to UTC and UTC only in order to correct it for sunrise and sunset.
You’re apparently confusing leap seconds and DST. Leap seconds are used to compensate for minute variations in the speed at which the earth rotates (irregularities and an overall slowdown). So they serve to resynchronize atomic time (TAI) and solar time (UT1). This resynchronized version (UT1 ~ 1s) is UTC.
Then you don’t understand Unix time. In the first case it would return 7200, and in the second case it would count “made up time”.
UNIX time is 86400*days since epoch + seconds since midnight. It does not measure “real time”, whatever that is.
> Leap seconds never happened, they were added to UTC and UTC only in order to correct it for sunrise and sunset.
You’re apparently confusing leap seconds and DST. Leap seconds are used to compensate for minute variations in the speed at which the earth rotates (irregularities and an overall slowdown). So they serve to resynchronize atomic time (TAI) and solar time (UT1). This resynchronized version (UT1 ~ 1s) is UTC.
In short, Unix time is based on UTC. If there's a leap second at midnight, you start a stopwatch ten seconds before midnight and stop it ten seconds after, then there is going to be a one second difference between your stopwatch and the difference between the Unix time timestamps of the moments you started and stopped the stopwatch.
For the purposes of a stopwatch/timer, this is exactly what you want. A measurement of time that is exact and does not include leap seconds. Accounting for DST or leap seconds or any other changes to the local timezone for that matter would be disastrous. Imagine you are trying to bake a cake and the timer doesn’t go off for an extra hour due to the clocks getting set back. Now you just have a ruined cake :(
As far as Unix and UTC are concerned, many systems provide the option to smear the system clock over a 24h period (-12h, +12h) centered on the leap second. This avoids the discontinuity and can help avoid errors some incorrectly written programs may not be able to tolerate.
For leap smearing, see: https://developers.google.com/time/smear
As far as Unix and UTC are concerned, many systems provide the option to smear the system clock over a 24h period (-12h, +12h) centered on the leap second. This avoids the discontinuity and can help avoid errors some incorrectly written programs may not be able to tolerate.
For leap smearing, see: https://developers.google.com/time/smear
The author is right. I couldn't believe it when I first learnt the difference between UTC and TAI that Unix time based on UTC. It makes no sense. And now they are talking about "correcting" UTC to make Unix time work? Thus defeating the entire point of UTC?
There's something about time that people seem to find very difficult to reason with. It reminds me of the arguments between whether we should use standard time all year or daylight savings time all year.
There's something about time that people seem to find very difficult to reason with. It reminds me of the arguments between whether we should use standard time all year or daylight savings time all year.
Sorry, no. UNIX time is easy, works fine for most of cases and can be extended easily to the rest of the cases, and a single well-defined compromise approach. Replacing it with three different time representations is dangerous overengineering.
We should all re-read "the rise of worse is better" until enlightenment (https://dreamsongs.com/RiseOfWorseIsBetter.html).
We should all re-read "the rise of worse is better" until enlightenment (https://dreamsongs.com/RiseOfWorseIsBetter.html).
Please don't bite, but why not move the UTC merdian?
why not move the UTC merdian?
You certainly mean to say the UTC nerdian, right? It's defined to cut through the median of all discussions around UTC, TAI, Unix timestamps and leap seconds that occur on HN and I see no reason to move it to another forum.
You certainly mean to say the UTC nerdian, right? It's defined to cut through the median of all discussions around UTC, TAI, Unix timestamps and leap seconds that occur on HN and I see no reason to move it to another forum.
There is an obvious need for time synchronized with the rotation of the Earth so leap seconds, or an equivalent mechanism, is also needed. The question is where to convert between linear time, more suited for computer systems, and Earth-synchronised time.
In terms of computer system design I'd say that the best is usually to convert as late as possible in a presentation layer and to use linear time internally everywhere.
In terms of computer system design I'd say that the best is usually to convert as late as possible in a presentation layer and to use linear time internally everywhere.
[deleted]
[deleted]
As much as I appreciate the topic, I fear this will be one of those 14 to 15 standards transitions. The proposal is complex in that it proposes three time units.
I see a lower-bound case for two: sun/Earth time and universal time. Farmers must be able to express "I will plant seeds at noon on August 29, 2329". They care about how the sun affects Earth. The worker trying to schedule a meeting is the same, because they'll want to guarantee the meeting happens during daylight. As such, I find the obsession over "always use Unix timestamps" a bit weird. Yeah, they're simple, but semantically wonky for some practical things. Everyone else, dealing with durations and time deltas unrelated to geological conditions want TAI.
For the farmer, the problem is the Gregorian calendar is deeply flawed/inaccurate, not that UTC has/had a leap second. Sun and Earth rotations are not aligned, so we cannot expect these to line up. We need year, day of year and time of day. The year is just the "number of solar turns" integer. The first and last day of the year would have to be cut up in unsavoury ways to account for misalignment. The time of day only needs to line up with Earth's rotation, but this means the speed relative TAI will need to shift as Earth is affected/slowed by the universe. (Now, I would suggest making the clock zenith based on something around the poles or leaning of the Earth axis, to end the Greenwich bias: that's pretty damn arbitrary.)
I see a lower-bound case for two: sun/Earth time and universal time. Farmers must be able to express "I will plant seeds at noon on August 29, 2329". They care about how the sun affects Earth. The worker trying to schedule a meeting is the same, because they'll want to guarantee the meeting happens during daylight. As such, I find the obsession over "always use Unix timestamps" a bit weird. Yeah, they're simple, but semantically wonky for some practical things. Everyone else, dealing with durations and time deltas unrelated to geological conditions want TAI.
For the farmer, the problem is the Gregorian calendar is deeply flawed/inaccurate, not that UTC has/had a leap second. Sun and Earth rotations are not aligned, so we cannot expect these to line up. We need year, day of year and time of day. The year is just the "number of solar turns" integer. The first and last day of the year would have to be cut up in unsavoury ways to account for misalignment. The time of day only needs to line up with Earth's rotation, but this means the speed relative TAI will need to shift as Earth is affected/slowed by the universe. (Now, I would suggest making the clock zenith based on something around the poles or leaning of the Earth axis, to end the Greenwich bias: that's pretty damn arbitrary.)
I hope that, when they redefine UTC, they define a new name for it. UTC has been the same timescale since it started; the history of redefinitions of GMT is a horror-story.
I guess what I'm really saying is that UTC-without-LS won't be UTC; rather, we'd declare that the new universal civil-time timescale is to be UTC-without-LS. You could carry on computing old-fashioned UTC after the transition; but one wouldn't generally use UTC to render civil-time for display.
I guess what I'm really saying is that UTC-without-LS won't be UTC; rather, we'd declare that the new universal civil-time timescale is to be UTC-without-LS. You could carry on computing old-fashioned UTC after the transition; but one wouldn't generally use UTC to render civil-time for display.
Leap seconds are determined manually, though, so it seems like there's no practical difference other than UTC's error from solar time going out of spec?
> other than UTC's error from solar time going out of spec?
Well, yeah. Except that if it's not in-spec, it's not UTC; it's something else, and it shouldn't be called UTC. If you don't change the name, you would end up unable to compare two UTC times without also knowing which version of UTC the two times were given in.
Well, yeah. Except that if it's not in-spec, it's not UTC; it's something else, and it shouldn't be called UTC. If you don't change the name, you would end up unable to compare two UTC times without also knowing which version of UTC the two times were given in.
I don't follow. UTC with no more leap second updates isn't a new version. It's the same timebase, just without any future updates to the leap second table. If the folk that maintain UTC decide to stop adding or subtracting leap seconds, it's not a fork in the timebase.
That fork already happened with TAI. TAI was the same as universal time in 1958, and has progressed without any leap seconds. The current offset between TAI and UTC is 37 seconds. We could all agree to switch unix time to TAI, but there would be a 37 second discontinuity.
I currently work on autonomous aircraft, and we use GPS time internally for avionics. The monotonicity is very nice for correctness when developing real time code. Outside of avionics, it's very confusing for people that tend to work in UTC. I've written table based timebase converters a few times now.
That fork already happened with TAI. TAI was the same as universal time in 1958, and has progressed without any leap seconds. The current offset between TAI and UTC is 37 seconds. We could all agree to switch unix time to TAI, but there would be a 37 second discontinuity.
I currently work on autonomous aircraft, and we use GPS time internally for avionics. The monotonicity is very nice for correctness when developing real time code. Outside of avionics, it's very confusing for people that tend to work in UTC. I've written table based timebase converters a few times now.
Since the French-speaking world has started to acknowledge that English is the modern lingua franca, perhaps we could now reach agreement to call the new system UCT for "Universal Coordinated Time".
This is effectively just an agreement to _stop_ redefining UTC every couple of years.
Leap-seconds are a fundamental part of the definition of UTC. Declaring a leap-second doesn't amount to declaring a new timescale. Abolishing the leap-second, however, does declare a new timescale.
If we’re going to change the timestamp let’s do it correctly this time and start counting from 0001-01-01 (like “ticks” in C#) or 0000-01-01 because adding another number (1970) for other people to have to learn about and memorize is bad design and unnecessary overhead and one subgroup of one profession naming a whole time epoch was kind of pretentious. It would also have the added benefit of being hard to confuse a value in this timestamp with a Unix timestamp.
You know, that they used a different calendar back then, right?
> let’s do it correctly this time and start counting from 0001-01-01
[...]
> one subgroup of one profession naming a whole time epoch was kind of pretentious.
I think that the UNIX epoch is bless contentious that the other epoch, as is not tied to a religious event. OS wars not withstanding.
Also the UNIX time was specifically designed for UNIX.
[...]
> one subgroup of one profession naming a whole time epoch was kind of pretentious.
I think that the UNIX epoch is bless contentious that the other epoch, as is not tied to a religious event. OS wars not withstanding.
Also the UNIX time was specifically designed for UNIX.
Not to mention all these are measured inside the gravity well of Earth and Sun, introducing relativistic dilation (of, apparently, about 600 picoseconds per second [this value would have resulted in the first relativistic leap second in Unix Time this October]).
Anyone find it weird that they just abolished it without picking up replacement first ? Feels very "let have future us worry about that problem".
It does feel very unscientific but ultimately the UTC won't get out of sync from rotation for more than a minute in my lifetime (unless desync rate increases I guess) so I don't really care
It does feel very unscientific but ultimately the UTC won't get out of sync from rotation for more than a minute in my lifetime (unless desync rate increases I guess) so I don't really care
These are 'interesting', but ultimately naive ideas, and the headline is presumptuous.
Instead, the world needs agreement. And, wow!, an agreement among countries to simplify time keeping is an achievement!
We do not need a more complex POSIX time spec, but we need to cope with billions of written lines of code from the past that use the existing API. We need that API to be as stable as possible to reduce complexity. We need to see that no changes are needed to existing code.
We need to reduce complexity, e.g., by removing an irregular, unpredictable drift between TAI and UTC.
We do not need to rewrite legacy code to use a shiny new API, but we need to make sure that legacy code will not break, because we do not know, unfortunately, were all that code hides.
Instead, the world needs agreement. And, wow!, an agreement among countries to simplify time keeping is an achievement!
We do not need a more complex POSIX time spec, but we need to cope with billions of written lines of code from the past that use the existing API. We need that API to be as stable as possible to reduce complexity. We need to see that no changes are needed to existing code.
We need to reduce complexity, e.g., by removing an irregular, unpredictable drift between TAI and UTC.
We do not need to rewrite legacy code to use a shiny new API, but we need to make sure that legacy code will not break, because we do not know, unfortunately, were all that code hides.
Thank you for your opinions. The point of my proposal was to keep the API entirely backwards compatible, so rewrites would not be necessary. The important part is the "Legacy Unix Time" part. The other two definitions I provided simply serve that definition. I hope that you will agree that this is not that much more complicated.
Notably I don't disagree with the idea of abolishing leap seconds, but I also don't think it's entirely agreeable to have the world's timekeeping standards changed solely because of engineering needs with regards to Unix time. In my opinion it's somewhat silly to have the timekeeping standard of the world changed because someone at Bell labs made an unfortunate design decision regarding Unix time, rather than changing Unix time itself.
Notably I don't disagree with the idea of abolishing leap seconds, but I also don't think it's entirely agreeable to have the world's timekeeping standards changed solely because of engineering needs with regards to Unix time. In my opinion it's somewhat silly to have the timekeeping standard of the world changed because someone at Bell labs made an unfortunate design decision regarding Unix time, rather than changing Unix time itself.
> I hope that you will agree that this is not that much more complicated.
Sorry, but I do not agree. The current Unix time API is good enough. Especially with the leap seconds gone. We need no more complication by innovation here.
Any extensions inevitable lead to a lot of people starting to use those additional APIs, others trying to fix old code to use new APIs, and a few will start religiously fighting for the use of the new 'right' API. Software that handles time would be more diverse. And you will never live to see that old API gone anyway, because we don't know where it hides, so in the future, we would have to deal with bugs in the (mis)usage of more APIs.
> In my opinion it's somewhat silly to have the timekeeping standard of the world changed because someone at Bell labs made an unfortunate design decision regarding Unix time, rather than changing Unix time itself.
Well, maybe it is silly. But that's really irrelevant. Don't be stubborn to insist on API changes/extensions, because you think a past decision was wrong. You can't change reality anyway: the vastly bigger problem is changing Unix time, because code is written and is running in critical systems.
And because leap seconds were inserted manually anyway, we already have a solution that works: don't insert any more leap second manually.
Also note that the danger was to do something new: remove a second. That is a very dangerous experiment. E.g., the German DCF77 cannot even announce that (I think the UK and US time signals can, IIRC -- I am not sure about Japan and the other time signals). Of course you could start to postulate we need a new time signal format in Germany (and effectively half of Europe) to fix that, but that is just a much more dangerous approach.
Sorry, but I do not agree. The current Unix time API is good enough. Especially with the leap seconds gone. We need no more complication by innovation here.
Any extensions inevitable lead to a lot of people starting to use those additional APIs, others trying to fix old code to use new APIs, and a few will start religiously fighting for the use of the new 'right' API. Software that handles time would be more diverse. And you will never live to see that old API gone anyway, because we don't know where it hides, so in the future, we would have to deal with bugs in the (mis)usage of more APIs.
> In my opinion it's somewhat silly to have the timekeeping standard of the world changed because someone at Bell labs made an unfortunate design decision regarding Unix time, rather than changing Unix time itself.
Well, maybe it is silly. But that's really irrelevant. Don't be stubborn to insist on API changes/extensions, because you think a past decision was wrong. You can't change reality anyway: the vastly bigger problem is changing Unix time, because code is written and is running in critical systems.
And because leap seconds were inserted manually anyway, we already have a solution that works: don't insert any more leap second manually.
Also note that the danger was to do something new: remove a second. That is a very dangerous experiment. E.g., the German DCF77 cannot even announce that (I think the UK and US time signals can, IIRC -- I am not sure about Japan and the other time signals). Of course you could start to postulate we need a new time signal format in Germany (and effectively half of Europe) to fix that, but that is just a much more dangerous approach.
TIL that we can't know for sure the timestamp of now + some time span greater than 6 months.
You cannot know the delta, but the interpretation of a timestamp set 6 months in the future can be unambiguous
E.g. "27th March 2023 19:00 Europe/London" is not ambiguous
E.g. "27th March 2023 19:00 Europe/London" is not ambiguous
I like to think of it as a difference between physical time and human time.
For engineering, science, precise measurement, simulation and such using TAI is clearly the cleaner, most robust solution. But as soon as you have a humans "perception" of time as part of the system a timezone aware timestamp is required. But even then, the most common version of that ISO 8601 is flawed.
ISO 8601 only has a fixed offset, it has no concept of timezones or geolocation. Borders between timezones move, DST windows change or disappear, it has no way to account for that. What we need is an extended ISO 8601 that includes a location rather than offset. Human time is tied to a geographic location in time. I think we need something like this:
using the names from the TZ database: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones
or even coordinates:
we probably even want planetary body:
This is the only way to store "human time" in a future proof way.
Physical moment in time = TAI timestamp
Human perception of a moment in time = Geolocated timestamp