In an era where businesses must adapt to fast-paced environments, real-time automated decision-making has become a cornerstone of operational efficiency and competitive advantage. This capability involves a seamless blend of decision logic, cutting-edge infrastructure, robust data strategies, and precise application handling. Organizations striving for agility and responsiveness must master the art of designing systems that enable instant, data-driven decisions while ensuring consistency, accuracy, and scalability...
Yeah the addition of window functions onto sqlite last year was quite a good feature to enable more data use cases with sqlite.
The exclude clause is part of the "window" configuration which defines which row to process, with it you can for instance define a range between all the previous and next rows and check if your data-point looks abnormal.
"Colab is still a relatively closed environment; at one point, I wanted to add a Python package and couldn't" - quite a big drawback for a data-science tool. CoCalc seems interesting though.
There is no issue hiring juniors, they have a lot to learn and they should be coached through it for sure. That mid level, help coach juniors, is also not an issue. But I would rather hire a junior analyst or a junior engineer than a junior data-scientist. The breadth of what they need to learn before actually delivering value is quite vast.
I can hire an analyst or engineer and quickly they will be able to learn, get good at something and deliver value. After 2 years they can easily grow into a data-scientist with the good base they have had from their previous positions.
Some organization essentially rebrand analysts positions to "junior datascientist", but I believe this make it a mismatch of expectation for graduates getting out of school.
You can have junior position for everything, so long as you revise your expectations.
A lot of firm, rebrand analysts jobs to be junior data-scientists position to essentially attract candidates, so yeah you can be "junior" at it, but it's essentially a different job.
"Junior anything needs a lot of supervision, it's probably the defining trait of a junior." - yes, but I would argue that junior data-scientists requires much more supervision than the same person doing a junior analyst or engineer role.
"The degree should get you to a point where you can actually learn the job."
I think the question there is "how fast"?
"You hire juniors and give them as much work as they can do with supervision." - Agree to a large extent, the main problem there is that their expectations don't usually match what they are ready for.
It's a job you grow into by not working at the job.: Agreed! There are quite a few paths which makes it easy to grow into, such as analyst or data/software engineer. There is however a mismatch of expectation that you can just start as a data-scientist out of school, which represent a steep learning curve and that people are usually not ready for.
Frankly the issue is more that a lot of them are dreaming of a position that they are not ready for, and want that from the get go.
They are generally ill equipped, but do not regard position such as analyst and engineers which would give them the foothold they would need to enter that position after a couple of years.
What they have to learn goes far beyond coding and entails getting a sense for data, the business sense that goes with it, an understanding of how to put models, etl code etc.. into production...
Most is fairly hard to teach in a classroom and rather requires practical experience in a business context, so I think the issue is more of an expectation issue than an educational issue. Although it is partially exacerbated by programs such as "Msc of DataScience", that makes student believe they would be ready for these positions straight after graduation.
One or two try/except is usually fine, but when you are nesting 3+ more try/except with the same error, the code becomes quickly unreadable.
Often you would be better off having a first look at the input to figure out which case you should tackle, decompose the nested try/excepts into separate function or at the very least keep it to two nested levels.
Hey - I am not saying not to hire or train junior people, rather hire them as analyst or data-engineer. There is much less of learning gap in these positions, and from there they should have a path forward to a data-scientist position. This rather than hiring them as junior data-scientist and essentially just paying for their education.
There is always some exception to the rule. Frankly the issue is more the number of topics one needs to learn to actually add value as a data-scientist. Junior data scientists need a hell lot of supervision compared to analysts or data-engineers. Most of what it takes to be a data-scientist is acquired on the job rather than something learned in class. And yes some of that could be checked during the interview, but that would also mean a very low pass rate and a lot of time spent hiring on these roles. I would rather hire these people straight out of college in role where they would have much more chance to excel, and would need less supervision. If they are good enough then give them more modeling tasks.
I think for New-York as an HQ there is quite some better chance for Amazon to build some serious operations. They already have Qusidi in New Jersey, which they might be looking to consolidate.