Time Zone Bugs I Ran Into(blog.davidojeda.dev)
blog.davidojeda.dev
Time Zone Bugs I Ran Into
https://blog.davidojeda.dev/4-time-zone-bugs-i-ran-into
67 comments
Yeah, 1 realization I made a long time ago was a day could have 25 hours or 23 hours. And a clever system having a timeline would show, e.g. in the Central European timezone:
1 AM, 2 AM (CET), 2 AM (CEST), 3 AM, 4 AM...
or
1 AM, 2 AM (CEST), 4 AM (CET), 5 AM, ...
1 AM, 2 AM (CET), 2 AM (CEST), 3 AM, 4 AM...
or
1 AM, 2 AM (CEST), 4 AM (CET), 5 AM, ...
According to a commenter above, the norwegian station of Troll[0] is on UCT in summer time but UTC+2 in winter time, so its "day" can have 22~26h.
Then there's the possibility of timezones straight moving around which can make entire days replay or disappear e.g. in Samoa there is no 2011-12-30, the day literally doesn't exist, Samoa local calendar goes straight 2011-12-29 to 2011-12-31, because at midnight the country switched from UTC-11:00 to UTC+13:00.
[0] https://en.wikipedia.org/wiki/Troll_(research_station)
Then there's the possibility of timezones straight moving around which can make entire days replay or disappear e.g. in Samoa there is no 2011-12-30, the day literally doesn't exist, Samoa local calendar goes straight 2011-12-29 to 2011-12-31, because at midnight the country switched from UTC-11:00 to UTC+13:00.
[0] https://en.wikipedia.org/wiki/Troll_(research_station)
There's an intriguing way that railways handle the transition around here (central Europe): one way, trains stop in stations for an hour, the other, they all get an +1 hr delay at next station.
Sorry but do you have a citation on that? Never seen anything like that in Europe.
Not original commenter, but basically he's right, according to the German Rail (in German): https://www.deutschebahn.com/pr-hamburg-de/hintergrund/theme...
If a night loses 1 hour, the train stops for less time at the scheduled stops, or arrives with a delay. If the night gains an hour, the train waits somewhere for an hour. Which sounds like a painful thing, having to travel (or wait) an extra hour because of clocks.
Interestingly if the night loses an hour, the trains are automatically an hour behind schedule...
If a night loses 1 hour, the train stops for less time at the scheduled stops, or arrives with a delay. If the night gains an hour, the train waits somewhere for an hour. Which sounds like a painful thing, having to travel (or wait) an extra hour because of clocks.
Interestingly if the night loses an hour, the trains are automatically an hour behind schedule...
Beeing an hour behind schedule is not that uncommon.....
Other than [OWNRESEARCH], i.e. experienced that a few times? Not really, no.
Its like this in germany
For an absolute event that was observed, UTC is, indeed, correct.
For anything interacting with planning though, the user's needs should be considered. Often what you actually want to store is a "recurrence rule" for a single event, while being sure to include the defining aspects picked, or at least confirmed, by the user.
For anything interacting with planning though, the user's needs should be considered. Often what you actually want to store is a "recurrence rule" for a single event, while being sure to include the defining aspects picked, or at least confirmed, by the user.
> The duplicate/missing hour around DST switching can also be quite annoying.
Oh yes. I used to work for company which produced medical software which continuously recorded patient data. Those duplicated or missing hours were always the worst and error-prone use cases.
Oh yes. I used to work for company which produced medical software which continuously recorded patient data. Those duplicated or missing hours were always the worst and error-prone use cases.
More than one vendor has become rather upset when I've demanded that they handle those situations appropriately. But much of what I do involves automation so I don't want any surprises or in this situation, two known exceptional cases that need to be accounted for. Also I find this to be a code smell, like brown M&Ms in a contract rider.
Don't know if they've fixed it but when I had a Fitbit, they never was able to get these DST change over events handled correctly. What's worse is that their website and their app both handled the change over in different ways, both wrong.
Don't know if they've fixed it but when I had a Fitbit, they never was able to get these DST change over events handled correctly. What's worse is that their website and their app both handled the change over in different ways, both wrong.
What do you mean by handle appropriately here? This is one of the cases where I'd simply store UTC and display the (sometimes ambiguous) local time in the UI.
The problem was that they weren't storing events in UTC but were using local server time instead. They had no explanation for what would happen during the switch from daylight time to standard time but couldn't rule out the possibility of out of order events happening due to it.
> So you if you store local times as UTC, you might not get back the correct value when you convert back after a tzinfo update
That shouldn't be a problem for datetimes in the past. tzdata contains not only the current offset but all historical ones as well.
It might be a problem for /future/ datetimes. If I say the meeting is at 1700 America/New_York and the local civil authority changes the offset in the meantime I probably don't want the meeting to move in local time.
That shouldn't be a problem for datetimes in the past. tzdata contains not only the current offset but all historical ones as well.
It might be a problem for /future/ datetimes. If I say the meeting is at 1700 America/New_York and the local civil authority changes the offset in the meantime I probably don't want the meeting to move in local time.
Unfortunately no, it can be a problem even for past dates. The new java.time and old java.util.TimeZone APIs use different offsets for dates < 1990, which resulted in a difference of 6 minutes and 32 seconds for my dates.
I filed a bug (JDK-8214243) but basically it's a known issue and there's no fix.
I filed a bug (JDK-8214243) but basically it's a known issue and there's no fix.
When migrating from .net's TimeZoneInfo (based on windows timezones) to NodaTime (based on tzdb) we ran into differences in the mid nineties for the German timezone.
Missing hour, ha! The Norwegians have apparently named their Antarctic station Troll for a reason: the TZ Antarctica/Troll uses UTC/UTC+2 - great way to test/break things that need to work with dates.
To really troll software developers they should repeat one hour three times, instead of repeating two hours twice.
I feel like this would also troll everyone in their society.
>What time is it?
2am.
>It’s been 2am for 2 hours!
>What time is it?
2am.
>It’s been 2am for 2 hours!
Appropriate to link to the classic “Falsehoods programmers believe about time” here: https://infiniteundo.com/post/25326999628/falsehoods-program...
Time is one of the easiest ways to find a bug in a backend app. Search for "23:59:59" or any use of local server time and you got yourself a bug.
I work for one of the largest banks in the world and I get to see a lot of code for some mundane backend stuff. For some reasons, people think that the day has always 23 hour, 59 minutes and 59 seconds and that there is a one second delay between one day ending and another starting. Then they are incredulous some trades have vanished.
There was a serious movement to stop data flowing around midnight until I reminded everybody that midnight happens 24 times a day (and not even that, there are fractional timezones, DST, etc. to complicate matters).
I work for one of the largest banks in the world and I get to see a lot of code for some mundane backend stuff. For some reasons, people think that the day has always 23 hour, 59 minutes and 59 seconds and that there is a one second delay between one day ending and another starting. Then they are incredulous some trades have vanished.
There was a serious movement to stop data flowing around midnight until I reminded everybody that midnight happens 24 times a day (and not even that, there are fractional timezones, DST, etc. to complicate matters).
> Search for "23:59:59" or any use of local server time and you got yourself a bug.
I thought this meant to search the logs / database and I was absolutely horrified that maybe I didn't understand how time works.
I think you're talking about hardcoding 23:59:59 as EOD in the source code though, right?
I thought this meant to search the logs / database and I was absolutely horrified that maybe I didn't understand how time works.
I think you're talking about hardcoding 23:59:59 as EOD in the source code though, right?
Yes. 23:59:59 is used when somebody wants to get end of day but doesn't know to take beginning of current day and add 1 day to it. End of today is actually 00:00:00 of tomorrow. For some reason developers don't want to have tomorrow's date in the timestamp when they think about events happening today but this is ok, because it should be used as upper limit (ie. with some kind of less than operator).
This sounds like an Office Space-esque rounding error. I recently wrote some code that requests all entries in a single day. The time query range is 00:00:00 to 23:59:59.999999999. I haven’t checked if nanosecond precision is sufficient for the timestamp source (labview requesting datetime from Windows 7), but I’m under the impression it is. If I was handling something higher stakes like finances, I would tack on a few more 9s.
Well then, let me introduce you to the regularly occurring 23:59:60: https://en.m.wikipedia.org/wiki/Leap_second
No server environment will ever see it. Any sane person will immediately understand this is going to break too much stuff and it is not worth educating thousands of developers.
Everybody just smears it over some period of time.
Everybody just smears it over some period of time.
Some of the cloud hosting NTP environments abolish leaps by smearing them out over a long period. That's one way to help out application code unprepared for them.
More surprising is that a number of expensive, precision, commercial GPS chips do NOT all handle leap seconds the same way. I worked in a shop that used about three different ones for their PPS and clock outputs and we got about three different behaviors from them. Evidently the leap seconds are broadcast in supplementary data and then applied on chip when it produces a UTC output.
More surprising is that a number of expensive, precision, commercial GPS chips do NOT all handle leap seconds the same way. I worked in a shop that used about three different ones for their PPS and clock outputs and we got about three different behaviors from them. Evidently the leap seconds are broadcast in supplementary data and then applied on chip when it produces a UTC output.
Honestly, it's simpler than that. Instead of doing <= for your end time, simply say <.
That is, give me events where time >=30/1/2020 00:00:00 and time < 31/1/2020 00:00:00.
That is, give me events where time >=30/1/2020 00:00:00 and time < 31/1/2020 00:00:00.
Yes, that is correct solution.
You know what the problem is? The developer wants to have events (like trades, transactions, log lines, users signed up, etc.) from a particular day and for some reason they fear/dislike having the next day. For some reason the 31/1/2020 00:00:00 is bad so the will prefer 30/1/2020 23:59:59.
I honestly don't know where this is coming from but it is real and I spend literally tens of hours trying to explain this to people, sometimes with no avail.
Actually, I know where this is coming from. This is mistaken attempt at addressing the last second.
How do you design system that filters stuff by giving range of dates? Go to any reservation system and you will see that to select just today, you need to select from=today and to=today instead of from=today and to=tomorrow.
I think fundamentally, this is mental model that causes people to think about the time not as a continuum but rather in chunks of some unit.
Depending on the type of event they will select best-matching unit (typically year, day, minute or second) and treat it as atomic with the timestamp of beginning of the time duration representing entire duration.
So in those minds, 23:59:59 represents entire last second of the day and there is no problem. 00:00:00 would represent the first second of the next day and that would be a problem.
Unfortunately, this is wrong mental model for things that happen on continuum of time or when somebody needs to block things using different units.
It only really works for things that are really managed in blocks of a given unit. For example, I can understand specifying accounting date range as from 1st to 31st of the month. This is right. Accounting date is no date at all, it is a label for a set of events that through some rules were associated with the label.
You know what the problem is? The developer wants to have events (like trades, transactions, log lines, users signed up, etc.) from a particular day and for some reason they fear/dislike having the next day. For some reason the 31/1/2020 00:00:00 is bad so the will prefer 30/1/2020 23:59:59.
I honestly don't know where this is coming from but it is real and I spend literally tens of hours trying to explain this to people, sometimes with no avail.
Actually, I know where this is coming from. This is mistaken attempt at addressing the last second.
How do you design system that filters stuff by giving range of dates? Go to any reservation system and you will see that to select just today, you need to select from=today and to=today instead of from=today and to=tomorrow.
I think fundamentally, this is mental model that causes people to think about the time not as a continuum but rather in chunks of some unit.
Depending on the type of event they will select best-matching unit (typically year, day, minute or second) and treat it as atomic with the timestamp of beginning of the time duration representing entire duration.
So in those minds, 23:59:59 represents entire last second of the day and there is no problem. 00:00:00 would represent the first second of the next day and that would be a problem.
Unfortunately, this is wrong mental model for things that happen on continuum of time or when somebody needs to block things using different units.
It only really works for things that are really managed in blocks of a given unit. For example, I can understand specifying accounting date range as from 1st to 31st of the month. This is right. Accounting date is no date at all, it is a label for a set of events that through some rules were associated with the label.
Tidy. Thanks. It needs an extra line to properly establish what the next day is, but this is much more robust. For a larger set of reasons related to how data is being stored, the base date variable is a string in format yyyyMMdd.
dateNext = sprintf(‘%s’, datetime(date, ‘format’, ‘uuuuMMdd’) + days(1));I generally store those as midnight of the next day, but figuring out when to display those as 23:59 of the previous day and when to display them as 00:00 of the next day can be tricky.
I don't think 23:59:59 is the same as midnight, that sounds like an error to me.
Users often find half open intervals confusing, so it often makes sense to render an interval as ending one tick before it actually ends (especially when the boundary is at midnight).
We introduced this change after frequent complaints about our invoices when we still rendered a month as "01.03.2020 00:00 - 01.04.2020 00:00", while people are fine with "01.03.2020 00:00 - 31.03.2020 23:59" despite the minor inaccuracy.
We introduced this change after frequent complaints about our invoices when we still rendered a month as "01.03.2020 00:00 - 01.04.2020 00:00", while people are fine with "01.03.2020 00:00 - 31.03.2020 23:59" despite the minor inaccuracy.
On Java: it had been a common sightseeing in the IANA tz mailing list that some complained about TZUpdater [1] not working. It's amusing if not annoying, and the root cause traces back to the intricacy of time zones themselves.
Apparently some countries at some points had (and sometimes, like Ireland [2], have) a "winter time" as opposed to a "summer time". In the other words their standard time takes place in summer and the "winter time" has a negative offset to the standard time. Technically this is not different from a "summer time" in other countries, but C `struct tm` has a `tm_isdst` flag that ideally should be true in winter for those countries.
tzdata made transition to support negative DST offsets in early 2018, but this caused unwanted software problems so the choise was made to support the "rearguard" format that doesn't have negative DST offsets. This was an one-off incident for most cases, but TZUpdater didn't support the more recent (and more correct) "vanguard" format. And yet users tend to feed the vanguard format to TZUpdater and thought tzdata was at fault. This annoyance continued until TZUpdater finally supported the rearguard format in late 2019 (I think).
[1] https://www.oracle.com/java/technologies/javase/tzupdater-re...
[2] https://en.wikipedia.org/wiki/Time_in_the_Republic_of_Irelan...
Apparently some countries at some points had (and sometimes, like Ireland [2], have) a "winter time" as opposed to a "summer time". In the other words their standard time takes place in summer and the "winter time" has a negative offset to the standard time. Technically this is not different from a "summer time" in other countries, but C `struct tm` has a `tm_isdst` flag that ideally should be true in winter for those countries.
tzdata made transition to support negative DST offsets in early 2018, but this caused unwanted software problems so the choise was made to support the "rearguard" format that doesn't have negative DST offsets. This was an one-off incident for most cases, but TZUpdater didn't support the more recent (and more correct) "vanguard" format. And yet users tend to feed the vanguard format to TZUpdater and thought tzdata was at fault. This annoyance continued until TZUpdater finally supported the rearguard format in late 2019 (I think).
[1] https://www.oracle.com/java/technologies/javase/tzupdater-re...
[2] https://en.wikipedia.org/wiki/Time_in_the_Republic_of_Irelan...
This blog page, ostensibly just text and images, made 30 requests for javascript files, and several more to analytics. In all, it made 62 requests over 50 seconds, transferring 1.9Mb - for less than a thousand words of content. Why?
Because it is 1000 words of content with the ability for you to comment on the content, click one of many numerous react options, retweet it, bookmark it, see how long it will take you to read, view other posts with the same tags, for the developer to know who visited, where they visited from, what time they visited, and where else on the site they went and be all presumably configurable as it is not a website the developer made, but built with a blogging platform.
It is a full fledged web app because it is a web app for managing thousands of different blogs and doing dozens of little random features, each of which has their own little bit of JS.
It is a full fledged web app because it is a web app for managing thousands of different blogs and doing dozens of little random features, each of which has their own little bit of JS.
I guess you answered the technical part of the original ”Why?” question, but the subtext of ”How did we arrive at this state?” remains a mystery.
Memory and bandwidtg got cheap. Its probably parkinsons 5th law: a webpage will use as many resources as there are available.
You can do all of those things on Wordpress at a fraction of requests, time taken and bandwidth consumed.
E:
For comparison:
3.5k word article on Wordpress with comments and analytics, scripting enabled: https://i.imgur.com/nTlMxwL.png
TFA, 900 words, scripting enabled: https://i.imgur.com/wWRMPts.png
E:
For comparison:
3.5k word article on Wordpress with comments and analytics, scripting enabled: https://i.imgur.com/nTlMxwL.png
TFA, 900 words, scripting enabled: https://i.imgur.com/wWRMPts.png
It seems to me you have just awoken from 20 year of coma.
Welcome to 2020. A lot has changed. People still use phones to access Internet but not exactly the same way (you may want to get updated on this in the closest Apple store. No, not a store with apples either). Oh, and please, were a mask.
Welcome to 2020. A lot has changed. People still use phones to access Internet but not exactly the same way (you may want to get updated on this in the closest Apple store. No, not a store with apples either). Oh, and please, were a mask.
Just because you can.
The page loaded instantaneously on my 1Gbps link. My phone on 4G loaded it fine too.
It is also very much possible that Internet Explorer 3 is not supported either.
Yes, this is a rant in reply to yours because some people have better things to do than to analyze DevTools outputs to say the 12654 bytes were loaded instead of 12341 over 50 calls instead of 27.
I am personally very, very found of the philosophy of https://motherfuckingwebsite.com/ and truly believe that this is how web sites with information should look like. But the ones that are readable on my devices are great too. And I do not give a shit about the volume because I do not have to.
And yes, this is horribly annoying to those who have 120x50 displays over a 6kbps link with 1 MB of capping.
The page loaded instantaneously on my 1Gbps link. My phone on 4G loaded it fine too.
It is also very much possible that Internet Explorer 3 is not supported either.
Yes, this is a rant in reply to yours because some people have better things to do than to analyze DevTools outputs to say the 12654 bytes were loaded instead of 12341 over 50 calls instead of 27.
I am personally very, very found of the philosophy of https://motherfuckingwebsite.com/ and truly believe that this is how web sites with information should look like. But the ones that are readable on my devices are great too. And I do not give a shit about the volume because I do not have to.
And yes, this is horribly annoying to those who have 120x50 displays over a 6kbps link with 1 MB of capping.
Except, you don't need to look at DevTools. As I've mentioned in response to a sibling comment, Wordpress has existed for ages, can be themed as you want, has comments and powerful analytics, and delivers 3.5x the content in half as much bandwidth and requests at one fifth the time required. And of course, it'll be readable on whatever device you have, with or without snarky hyperbole.
Certainly, but so what? The site loads perfectly on my devices so I am happy. It probably loads the same on many other evergreen-ish devices with a decent bandwidth available.
Like I said - I like minimal web sites but the ones that works are fine as well. This one works, I would have never wondered about the weight without your comment.
If a site starts to load slowly or is not readable, I will just skip it.
Like I said - I like minimal web sites but the ones that works are fine as well. This one works, I would have never wondered about the weight without your comment.
If a site starts to load slowly or is not readable, I will just skip it.
I am on your side: I fix it by disabling javascript.
The only reason I found out was because I'd done that: I've set uBlock to block JS by default, only turning it on for websites that don't work without JS at all. I've gotten used to seeing the purple sign with 10 or 20 blocked requests, but seeing that number hit 30 for a simple blog was... shocking.
Listen blogging isn’t easy so you, the reader, are going to have to pay one way or another.
> the reader, are going to have to pay one way or another.
Many blogs don’t exist as a way to produce an income and absolutely don’t have enough traffic to do so.
Many blogs don’t exist as a way to produce an income and absolutely don’t have enough traffic to do so.
Can’t most of these issues be solved by not using timezones at all on servers?
All data should be UTC; timezones only apply when the data is shown to the user, possibly on the client itself, which likely can automatically display the right time for a UTC date.
All data should be UTC; timezones only apply when the data is shown to the user, possibly on the client itself, which likely can automatically display the right time for a UTC date.
It's a question of knowing your data.
UTC time probably won't always give you the correct result, since timezones and DST aren't fixed.
Say, you want to store the time of your doctor's appointment months in advance.
Here it makes sense to store the time with a timezone - but not the timezone offset. This makes sense because no matter what the government decides about changing DST or moving timezone offsets alltogether - your doctor's appointment is going to be at 0900am at a specific geographic location and since you will physically be there with your watch synchronized to your doctor's.
But say you have an online appointment with your psychotherapist who lives in a completely different timezone (possibly you do not know which one) from you at some specific time in the future.
Here making the appointment via UTC makes sense - to meet up you need a globally synchronized clock.
The hard part: how do you know determine whether you can schedule these two one-hour appointments right after each other and not have a conflict ? I am not sure it's perfectly possible.
UTC time probably won't always give you the correct result, since timezones and DST aren't fixed.
Say, you want to store the time of your doctor's appointment months in advance.
Here it makes sense to store the time with a timezone - but not the timezone offset. This makes sense because no matter what the government decides about changing DST or moving timezone offsets alltogether - your doctor's appointment is going to be at 0900am at a specific geographic location and since you will physically be there with your watch synchronized to your doctor's.
But say you have an online appointment with your psychotherapist who lives in a completely different timezone (possibly you do not know which one) from you at some specific time in the future.
Here making the appointment via UTC makes sense - to meet up you need a globally synchronized clock.
The hard part: how do you know determine whether you can schedule these two one-hour appointments right after each other and not have a conflict ? I am not sure it's perfectly possible.
It would be nice if systems monitored time zone definition changes so you could receive a notification like "Due to a time zone or DST definition change, your appointments now conflict". This has probably not been implemented anywhere.
You would still store both future times with offset. When calculating an offset for a future time you take the future time and the zone together to determine the exact time with offset. DST is predictable.
> DST is predictable
No it isn't.
* Europe postponed planned DST abolition due to COVID19: https://www.timeanddate.com/time/europe/eu-dst.html
* Argentina had a period of deciding each year what the DST dates would be until one year they just decided not to: https://en.m.wikipedia.org/wiki/Daylight_saving_time_in_Arge...
* Morocco and Yukon, Canada made updates to their DST policies earlier this year: https://mm.icann.org/pipermail/tz-announce/2020-April/000058...
No it isn't.
* Europe postponed planned DST abolition due to COVID19: https://www.timeanddate.com/time/europe/eu-dst.html
* Argentina had a period of deciding each year what the DST dates would be until one year they just decided not to: https://en.m.wikipedia.org/wiki/Daylight_saving_time_in_Arge...
* Morocco and Yukon, Canada made updates to their DST policies earlier this year: https://mm.icann.org/pipermail/tz-announce/2020-April/000058...
DST schedules change several times each year, often with little prior notice. They changed in North America in 2007, which caused a lot of pain for software that thinks DST is predictable, such as Microsoft Exchange.
If you’re actually working with dates (e.g. calendaring), storing dates as UTC is a potentially critical loss of information.
Basically, UTC is mostly fine for recording past events (e.g. event logs though even then pre-1970 events can be problematic), but for any future event it’s very risky, because time zones « move around » sometimes with very little heads up (down to weeks if not days for DST) but a user working in a specific time one expects their stuff to be fixed in their time zones’ frame of reference.
But of course this indices the problem of ambiguity around DST switch & other backwards TZ movements.
Basically, UTC is mostly fine for recording past events (e.g. event logs though even then pre-1970 events can be problematic), but for any future event it’s very risky, because time zones « move around » sometimes with very little heads up (down to weeks if not days for DST) but a user working in a specific time one expects their stuff to be fixed in their time zones’ frame of reference.
But of course this indices the problem of ambiguity around DST switch & other backwards TZ movements.
That is usually enough when recording events that happened at a specific point in time. Even then you sometimes want to record the offset in the user's timezone, so you know the relevant local time (e.g. when taking a photo).
But it's not enough for things agreed to happen in a specific time zone (meetings, contracts, subscription billing, etc.) In those cases you have to record the local time and timezone name (plus disambiguation for DST switching) and figure out how to handle changes of the timezone definition (e.g. by updating denormalized UTC representations or offsets).
But it's not enough for things agreed to happen in a specific time zone (meetings, contracts, subscription billing, etc.) In those cases you have to record the local time and timezone name (plus disambiguation for DST switching) and figure out how to handle changes of the timezone definition (e.g. by updating denormalized UTC representations or offsets).
That only works for absolute times. Things like "tomorrow at 9am" are harder since the definition of it depends on the users local time.
The POSIX timezones having - and + reversed once cost me a day of pulling hairs: http://www.pixelbeat.org/docs/linux_timezones/
I have encountered some bugs related to time zone, when using cheap surveillance cameras, for which the time zone/ Daylight Saving Time doesn't always work properly. The displayed time inside the video feed may be off by one hour / also different than the time the notification email.
I have encountered some bugs related to time zone with docker in which the displayed time in the displayed console log output is different from the local time, which is kind of confusing when reporting issues on github. But if your code isn't done properly and have filters based on local time you need remember to pass the correct time zone configuration when shipping with docker.
When reading blog posts / social media / messaging apps you need to be aware of the "responded at xxxx time" usually is local relative to the poster, which may cause confusing anachronism, when trying to know if the message is before or after an event. (If you get the time via an API, it usually is UTC, but when you are scrapping you need to be careful).
Also with DST you always need to check that your alarm clocks have the correct date so they change correctly.
Semi-related, on a dual booting linux/windows, my windows machine after reboot is usually still in the past at the time when it was last shut off, which prevents logging into Fusion360 (without signaling why the server refuse the connection), until my windows get it's time at the right time by synchronizing with the ntp time server.
I have encountered some bugs related to time zone with docker in which the displayed time in the displayed console log output is different from the local time, which is kind of confusing when reporting issues on github. But if your code isn't done properly and have filters based on local time you need remember to pass the correct time zone configuration when shipping with docker.
When reading blog posts / social media / messaging apps you need to be aware of the "responded at xxxx time" usually is local relative to the poster, which may cause confusing anachronism, when trying to know if the message is before or after an event. (If you get the time via an API, it usually is UTC, but when you are scrapping you need to be careful).
Also with DST you always need to check that your alarm clocks have the correct date so they change correctly.
Semi-related, on a dual booting linux/windows, my windows machine after reboot is usually still in the past at the time when it was last shut off, which prevents logging into Fusion360 (without signaling why the server refuse the connection), until my windows get it's time at the right time by synchronizing with the ntp time server.
My favorite(1) way of dealing with timezones is how Fitbit does it: They just ignore the fact that they exist! They will give you accurate minute-by-minute heart rate data for a someone, but what timezone did that occur in? Whatever one the watch was set to when it happened.
What if I live on the East Coast and fly to California and back and what to see how my body reacted to the trip? Well some times will happen twice (and get overwritten), some will never happen, and who knows when the change happened in the middle (probably when you happened to sync your Fitbit, sometime after your phone changed timezones).
(1) To complain about
What if I live on the East Coast and fly to California and back and what to see how my body reacted to the trip? Well some times will happen twice (and get overwritten), some will never happen, and who knows when the change happened in the middle (probably when you happened to sync your Fitbit, sometime after your phone changed timezones).
(1) To complain about
My favourite TZ bug was where thanks to a timezone bug some (but not all) events were showing up a day early after our etl. It turned out that a combination of several seemingly logical steps we had managed to end up in an insane place. If you just want the tldr, dates are not times, don't store dates as times or you can be bitten by timezone bugs. Also dates don't have a timezone, don't store them as something that has a timezone or you can make mistakes.
So:
1)We decided to store all timestamps as UTC. A bunch of analysis we were doing was temporal so it was really important to be able to compare timestamps coming from different source systems (with different native timestamps) accurately.
2)Some events were things occurring on specific dates rather than at a particular time. No problem, dates don't have a timezone. We can't possibly be bitten by a timezone bug on dates.
3)One of our serialization formats didn't have a date type so we decided to store dates as times with a timestamp of midnight (This is ancient Pig-era hadoop so you're probably talking an earlyish avro format but I can't remember). You can always just chop the time off to turn them back to a date and nothing can possibly go wrong.
4)One of the event source systems used Europe/London timezone (ie daylight savings in the summer, GMT in the winter) as the system tz rather than UTC. No big deal right, we're doing consistent UTC conversion so this couldn't possibly hurt anything.
Bingo! Some of the date based events appeared one day earlier than they should. Do you see why?
This was an absolute bastard to debug because it was in the middle of a somewhat flakey data pipeline taht used to take >9hrs to run but basically what was happening is during daylight savings months these events were being converted from 00:00 1 Aug BST (say) to 23:00 31 July UTC, so when serialised once you chopped off the time the event would end up one day early. It wasn't all dates because not all days are during daylight savings and events that had an actual time (not a date that had been turned into a time) were of course fine. Also date events that were recorded on a system that used UTC as its system timezone were also fine.
So:
1)We decided to store all timestamps as UTC. A bunch of analysis we were doing was temporal so it was really important to be able to compare timestamps coming from different source systems (with different native timestamps) accurately.
2)Some events were things occurring on specific dates rather than at a particular time. No problem, dates don't have a timezone. We can't possibly be bitten by a timezone bug on dates.
3)One of our serialization formats didn't have a date type so we decided to store dates as times with a timestamp of midnight (This is ancient Pig-era hadoop so you're probably talking an earlyish avro format but I can't remember). You can always just chop the time off to turn them back to a date and nothing can possibly go wrong.
4)One of the event source systems used Europe/London timezone (ie daylight savings in the summer, GMT in the winter) as the system tz rather than UTC. No big deal right, we're doing consistent UTC conversion so this couldn't possibly hurt anything.
Bingo! Some of the date based events appeared one day earlier than they should. Do you see why?
This was an absolute bastard to debug because it was in the middle of a somewhat flakey data pipeline taht used to take >9hrs to run but basically what was happening is during daylight savings months these events were being converted from 00:00 1 Aug BST (say) to 23:00 31 July UTC, so when serialised once you chopped off the time the event would end up one day early. It wasn't all dates because not all days are during daylight savings and events that had an actual time (not a date that had been turned into a time) were of course fine. Also date events that were recorded on a system that used UTC as its system timezone were also fine.
Many people use datetime data types for dates, some unknowingly, some don’t care and some because they don’t see an alternative. Its quite strange there is rarely language/library support for true, not timezone dependent dates.
JodaTime (and use-310) does in right. Most do it wrong.
> dates are not times, don't store dates as times
They kind of are, or rather they are time intervals. When people say 25th September they most often mean the interval from 00:00:00 on the 25th until 24:00:00 on the 25th (or 23:59:(59|60) if you don't like one moment in time belonging to two days). This is obviously timezone dependend, at all times some timezones are on a different date than other timezones.
They kind of are, or rather they are time intervals. When people say 25th September they most often mean the interval from 00:00:00 on the 25th until 24:00:00 on the 25th (or 23:59:(59|60) if you don't like one moment in time belonging to two days). This is obviously timezone dependend, at all times some timezones are on a different date than other timezones.
Often they don’t. Time-to-business-date is a business or legal rule, not a celestial one. The “day” often ends at 5pm. Trading systems may have eod at 2. If you deposit a cheque into a machine at 5:30pm on Thursday, that is posted Friday. A trade at 3pm is next-day. Heaven forbid you have a global operation.
Yeah this is my experience too. Often there is a time or times associated with a date but they can be difficult/impossible to deduce and many intuitive answers are wrong or riddled with important exceptions.
Always store the authoritative version of a time, which usually is UTC for points in time, but local time for calendar based things.
The duplicate/missing hour around DST switching can also be quite annoying.