Code that breaks 2 or 3 times every month(ngzhian.github.io)
ngzhian.github.io
Code that breaks 2 or 3 times every month
http://ngzhian.github.io/blog/python-dateutil-gotcha.html
19 comments
umm... it's because the date 31st isn't in those months... Simply, they're switching to February <current day right now> 2006, because you did not specify a day.
Sure enough. But is it a sane thing to do for a "convert readable date to date object" function to silently amend the current day-of-month to a otherwise underdpecified date?
I really don't see the use case for that anywhere.
I really don't see the use case for that anywhere.
I would expect to use the first day of a month as a default, just like when time is not specified it should use 00:00 as default. Taking the current time/day as a default sounds off.
Probably hard to fix backwards compatible though...
Probably hard to fix backwards compatible though...
Agreed; I expect date parsers to give me earliest-available value for a field if it's not specified, with the exception being the current year rather than 0 AD or whatever.
Javascript:
Javascript:
new Date("Feb 2006")
> Wed Feb 01 2006 00:00:00 GMT-0800 (Pacific Standard Time)
require 'time'
Time.parse "Feb 2006"
=> 2006-02-01 00:00:00 -0700According to the documentation there's a 'default' parameter, which contains the datetime used to get otherwise unspecified fields. If not given, the default is the current date. Thus to get what you want:
>>> from datetime import datetime
>>> from dateutil import parser
>>> now = datetime.now()
>>> first = datetime(now.year, now.month, 1, 12, 0, 0)
>>> dateutil.parser.parse('February 2006', default=first)
datetime.datetime(2006, 2, 1, 12, 0)[deleted]
Exactly, not sure how much code has been written to take advantage of this fact. Taking day 1 is a much safer option definitely.
Moreover, often this is not what you want. Generally you'll want either "the last past day that had that number" (logging of activity), or "the first future day that has that number" (appointment planning). But having a date that can be randomly in the past or the future sounds a bit strange.
I made an app that generated reports. The queries ran quickly when I tested it directly on the database but when I ran my app it would hang on a particular query. Sometimes.
I started making some small seemingly insignificant changes to my code and suddenly my app generated all the reports quickly. All was well until the next month when I had to generate the same reports and my app failed on the same query. I solved it by reverting the small insignificant change I made the previous month. WTF.
I looked into it and there's a wealth of information on the topic[0]. Apparently, how my code passed the query to the database differed to how I tested my query directly on the database which cause the database to compile the query differently and use different execution plans.
My most recent fix was to add another index which the database suggested. Let's hope it works next month.
[0] http://www.sommarskog.se/query-plan-mysteries.html
I started making some small seemingly insignificant changes to my code and suddenly my app generated all the reports quickly. All was well until the next month when I had to generate the same reports and my app failed on the same query. I solved it by reverting the small insignificant change I made the previous month. WTF.
I looked into it and there's a wealth of information on the topic[0]. Apparently, how my code passed the query to the database differed to how I tested my query directly on the database which cause the database to compile the query differently and use different execution plans.
My most recent fix was to add another index which the database suggested. Let's hope it works next month.
[0] http://www.sommarskog.se/query-plan-mysteries.html
I do a lot of minor performance tuning in mssql, and while I am not any senior professional, I often see parameter sniffing or a cached plan that makes no sense causing an issue when the server thinks its too expensive to create a new plan, and as a result the cardinality estimates are all wrong, the operators are often wrong, and you can see a query that often runs in insignificant time take forever.
As far as I know, the small changes you made just invalidated the plan (as the code wasnt the same) and make the server recompile your query's plan. You can accomplish the same thing by using sp_recompile
Depending on the code, I have fixed this with a few different answers: 1. You can write code that is clearer and simpler for the optimizer, sargeable queries and all that. 2. You can actually (I think in 2012 or later) issue a plan guide so that the query always runs in the way you decide. 3. You can do more hacky things like deciding to recompile the plan each time you run the query, or do a similar thing by assigning all your parameters to local variables, the query optimizer then cant parameter sniff.
You want to learn a little bit about parameter sniffing, because an execution plan that is poor can often be caused by re-using old values to plan the query,
As far as I know, the small changes you made just invalidated the plan (as the code wasnt the same) and make the server recompile your query's plan. You can accomplish the same thing by using sp_recompile
Depending on the code, I have fixed this with a few different answers: 1. You can write code that is clearer and simpler for the optimizer, sargeable queries and all that. 2. You can actually (I think in 2012 or later) issue a plan guide so that the query always runs in the way you decide. 3. You can do more hacky things like deciding to recompile the plan each time you run the query, or do a similar thing by assigning all your parameters to local variables, the query optimizer then cant parameter sniff.
You want to learn a little bit about parameter sniffing, because an execution plan that is poor can often be caused by re-using old values to plan the query,
Oh wow, that is a pretty bothersome trouble. I don't know nearly enough about databases and generating reports to understand this, but that link is saved. Thanks for sharing!
Once I recognized a certain test pass on all days except on Mondays.
The test created models for a few days and tested for a sum of the current week. The dates where all relative (today, yesterday, etc.), so when the test run on a Monday some records where for the last week and that's why the sum wasn't the same.
But I guess your problem is harder to debug.
The test created models for a few days and tested for a sum of the current week. The dates where all relative (today, yesterday, etc.), so when the test run on a Monday some records where for the last week and that's why the sum wasn't the same.
But I guess your problem is harder to debug.
Tests that relied on date (especially involving getting the current date) are usually hard to get right. I had an assignment in school to build a To-do list, and had lots of problems trying to test it. I think I solved it by having methods take 2 dates, initial and relative date, rather than just getting the current date.
Yes, a method is usually much easier to test, if the method gets the current date passed as an argument. The same goes for methods calculating on random values. Passing these to the method enables testing.
This is kind of cool as well: https://bugs.launchpad.net/ubuntu/+source/cupsys/+bug/255161...
I recently had to write a wrapper around this particular library to get it to parse timezones correctly. At least I have a defined datetime form I'm passing in!