This is a great point about making sensible assumptions. Too often I see evidence that people think that data analysis should be devoid of assumptions, and any assumptions completely invalidate the analysis. In reality, almost all analysis will have some assumptions, and the good questions to ask is how plausible are the assumptions and what is the plausible impact of what happens if the assumptions are wrong.
Why should we expect there isn't a correlation? They do mention it in their analysis. At the end of the day, just because there might be a systematic bias in your result doesn't mean there is a systematic bias.
All real-world analysis (especially for observational studies) rests on certain assumptions. It is always true that these assumptions might be wrong, but it is important to think about whether or not the assumption is plausible. It seems plausible that on average, a lawyer who is able to get better outcomes when they don't settle is also able to get better outcomes when they do settle.
Furthermore, even if most cases are settled, the rare cases that do go to trial can have an outsized impact. Usually people settle because a bad judgment is devastating (as well as not wanting to pay legal costs).
It is always easy to criticize a data-driven analysis by saying its assumptions could be wrong. In the real world, all analysis is based on assumptions, some of which you can always claim might not be correct. But you have to really present an argument as to why and by how much the assumption is likely to be wrong, you can't just state that the assumption might be wrong. The assumption that cases which don't settle are not at all indicative of how well a lawyer performs is a very bold claim, much bolder in my mind (admittedly I am not a lawyer) than the claim that there should be some correlation between lawyer performance and results in cases that did not settle. It is possible that lower ranking lawyers settle less often, I'd like to see the data on that.
Furthermore, on average the effects you are mentioning will wash-out, unless there is a systematic bias whereby lower ranking firms and higher ranking firms settle in different manners.
The main thing I want from job descriptions is a salary range. The fact that companies don't post salaries is a strong counter-point to how companies complain about how hard it is to hire software engineers.
The problem is that the term data analyst has come to mean data reporter. Similarly "business analyst" generally involves tasks that are best solved in Excel. The "science" in data science is about testing predictions. But I agree data science is a terrible phrase.
You don't have to be productive all the time. It is important to have some time to relax and have fun. There are far worse things you could be doing than playing too many video games. You can try replacing video games with a more productive activity, but make sure it is something you enjoy doing and don't feel the need to quit gaming entirely.
> Taking papers at face value is really only a problem in science reporting and at (very) sub-par institutions/venues.
> WRT the former, science reporters often grossly misunderstand the paper anyways. All the good reproducible science in the world is of zero help if science reporters are going to bastardize the results beyond recognition anyways...
Science is funded by the public, and done for the public. Good science reporting is very important to ensure that science continues to get funded. Too often scientific papers are written in a way that makes them incomprehensible to anyone outside of the field, whether that is through pressure to use the least amount of words possible or use of technical jargon.
The explanation glosses over a few important details. Gradient boosting works by adding some small weight to the instances the model is incorrectly predicting. The amount of extra weight these instances get is a parameter that is tuned with validation - because this parameter can be 0, if you are doing correct cv gradient boosting trees is usually superior to random forests. You also do need to tune the number of trees you use in gradient boosting or else you will overfit.
Gradient boosting doesn't get nearly enough hype as compared to things like neural nets. The significant majority of winning solutions to Kaggle competitions for a non-image or text-processing dataset will use xgboost to do gradient boosting as part of the ensemble model. Furthermore, it is a really easy method to understand and use while still being state-of-the-art.
This statement is too general. You could of said the same thing about chess, there are chess Grandmasters who devote their lives to studying the game yet computers play chess at a much higher level than any human.
I would say that table is really quite valuable. Kaggle problems come from all types of companies, so it doesn't make sense to say that it is "overfitted patterns that he's adopted in his own realm". With that said, validation on your own dataset will trump general knowledge, so you shouldn't view these parameters as hard and fast rules. But the parameters in that table will provide a useful starting point, and if you stray too far from them that is a warning sign that you might be overfitting.
For image competitions you are right. Neural networks are often in winning teams ensembles, but they require a lot more work than something like xgboost (gradient-boosted decision trees). For a dataset that isn't image processing or NLP, xgboost is in general much more widely used than neural nets. Neural nets suffer from the amount of computing resources and knowledge needed to apply them, though given infinite knowledge and computing power they are probably on par with or better than xgboost. And if you need to analyze an image they are great.
1) It depends heavily on the model. Something like xgboost (gradient boosted decision tree) will handle irrelevant features fairly well, while other models (like linear models, especially without lasso regularization) will have much more trouble. In virtually all cases adding noise will decrease model performance.
2) Same as 1), depends on the model. With good hyper-parameters xgboost can handle correlated features well, while other models may struggle.
3) With a good model (again like xgboost), feature engineering is usually the best use of your time. Removing "bad" labels and "noise" in the data is especially dangerous, as if you are not extremely careful you can make your model worse. If you can identify why the label is "bad" then you can remove or correct it, but you need a reason why you wouldn't have these bad labels on your test dataset. Removing outliers can help your model, but it is risky. In contrast smart feature engineering is low risk and can provide large gains if you see a pattern the model could not see. Feature selection can be important as well, and is generally pretty quick assuming you have good hardware, so you might as well do it, especially if you have some knowledge about which features you expect to be not that useful.
There is no way machine learning will be a necessary skill for software engineering, if that is your motivation I would not spend time learning it. However, if you still want to learn it you should first study statistics, for instance http://www-bcf.usc.edu/~gareth/ISL/.
My advice is Python, but it depends on what your background is and what you want to do. If this is your first language and you have a stats background, R is a solid choice. If you already know another language, R has a lot of flaws that are quite frustrating. Perhaps the worst thing about R is how hard it is to google answers to as opposed to Python.
Interesting. After watching the show Billions, and reading up on how much money hedge fund managers make on fees (seems totally ridiculous), I wonder how common is illegal insider trading for hedge funds? No matter how good your model is, you won't beat someone with information your model doesn't have.
To really understand if companies are biased or not, you also need to know the percent of applicants to these companies who are black. If only 2% of applicants to Google are black, I would expect only 2% of new hires at Google to be black.
The assumption that a white applicant and a black applicant should be roughly equal is a strong prior. I would need to see convincing data to counteract this assumption.