Sounds like they can do predicate pushdown to the storage nodes now. And...mainly beneficial for medium-to-highly selective scans of mostly uncached tables?
I am a dataologist, even if my title says software something something. Commercial software development is about getting data from point A to point B more than anything else. There's a little bit of logic, a little bit of munging, but almost all of what could be called an algorithm is done by a library or some other system--probably a database. It's maybe once a year that doing something algorithmic, e.g. reduce a regex to a hand-coded FSM, or write an exponentially decaying counter, is actually the best thing I can do to solve the problem at hand.
IOCP doesn't do it[1]. Well, if it does then it's not documented. You can post custom completion packets so at first glance it looks easy to make open/close be async...I think there is probably a good reason why NT won't do that for you.
That's pretty awesome though, that you have to worry about latency of open().
Reminds me of an IBM technical bulletin I found, somehow, about 10 years ago. It advised that if you were moving from this z9 hardware configuration to that one, you would have 30 minutes of downtime. Implying that other changes required no downtime, which I inferred wasn't stated because it wasn't insanely awesome to mainframe guys, adding CPUs and RAM to a running system was just how things worked.
I was in college at the time and looked at job postings for mainframe programmers because I wanted to work on that. Never found one that required less than 5-10 years experience, not then and not when I've checked every couple years after that. Too bad; I quite liked the idea of working in an ecosystem that starts from "let's make this work every single time" instead of "let's make this work well enough to keep customer complaints to a dull roar".
If you're using Spark's built-in scheduler then the cluster manager is a SPOF. Hadoop docs say you can get active/standy ResourceManager, not that I've tried it. Spark can also use k8s and nomad to schedule executors, and those have HA modes as well. I assume Mesos does HA.
You're still boned if the driver dies. I am pretty sure that the driver keeps some important state in RAM so if the node hosting it goes down you have to restart from the beginning, even if the cluster manager restarts the driver.
In addition, a lot of the techniques used to write high-performance Java boil down to "write it like C". Avoid interfaces, avoid polymorphic virtual calls (as you can't avoid virtuals entirely), avoid complex object graphs, avoid allocating as much as possible...it's not nearly as nice as naive Java. Still nicer than C IMO. If your process segfaults you can know for certain that it's a platform bug.
The short excerpt you can read for free says this is more about stuff like the two minutes between clocking out at closing time and locking the door on your way out. It might be wage theft in some sense but isn’t the same thing as requiring hours of unpaid overtime. On the other hand, I suspect the part of the article you can’t read for free is about an employer who tried to cover a substantial amount of work under this exemption and now the court has to ruin things for everybody to prevent abuses.
Why, on the off chance one flies into someone’s open mouth and the surprise triggers a swallowing reflex? I wouldn’t have a lot of sympathy for an adult who ate the homemade pills a guy threw at him and then suffered serious side effects.
Oh my God, people use Slack without changing the "Notify me about" setting to "Direct messages, mentions, and keywords"? I work on a 100% remote team and would rapidly go insane without that.
Deep learning! It does everything really well, especially reducing complex problems to a single number. It's well known that anything which doesn't fit in the reals isn't really real, if you know what I mean.