The linked article says they decided to do CD in 2016 fwiw so that's not inconsistent with what I said.
You reduced the number of patches a lot and also pushed very hard to get us to 3.0 after we sat on 2.6.38 ~forever. Which was very appreciated, btw. We built the whole plan going forward based on this work.
I'm not arguing that anyone should be nice to anyone or not (it's a waste of breath when it comes to Linux). I'm just saying that the benchmarking was thorough and that contemporary 2014 hardware could zero pages fast.
For the peanut gallery more: I worked with both of these guys at Meta on this.
The "servers are only on for a few hours" thing was like never true so I have no idea where that claim is coming from. The web performance test took more than a few hours to run alone and we had way more aggressive soaks for other workloads.
My recollection was that "write zeroes" just became a cheaper operation between '12 and '14.
A fun fact to distract from the awkwardness: a lot of the kernel work done in the early days was exceedingly scrappy. The port mapping stuff for memcached UDP before SO_REUSEPORT for example. FB binaries couldn't even run on vanilla linux a lot of the time. Over the next several years we put a TON of effort in getting as close to mainline as possible and now Meta is one of the biggest drivers of Linux development.
It was definitely a toy, I transliterated from python bytecode (a stack based vm) into bpf. I also wrote the full code gen stack myself (bpf was simpler back then)
But using llvm and not marrying things to cpython implementation makes this approach way better
The cloud business model is to use scale and customer ownership to crush hardware margins to dust. They’re also building their own accelerators to try to cut Nvidia out altogether.
The thing that we need in order for your dream to become a reality is excellent user space frameworks, so I encourage you (and anyone else) to go build one or (better) find one you like and contribute.
> File IO is perhaps the best example of this (at least on Linux). To handle such cases, languages must provide some sort of alternative strategy such as performing the work in a dedicated pool of OS threads.
AIO has existed for a long time. A lot longer than io_uring.
I think the thing that the author misses here is that the majority of IO that happens is actually interrupt driven in the first place, so async io is always going to be the more efficient approach.
The author also misses that scheduling threads efficiently from a kernel context is really hard. Async io also confers a benefit in terms of “data scheduling.” This is more relevant for workloads like memcached.
Age is great. I used the rust crate to write an ftp server that encrypts the files before they hit disk (specific use case is having a drop box for my network scanner) and I love the simplicity and composability it provides.
One feature request: it would be awesome to have paraphrase encryption for age private keys.
I still remember when I came to you with my “attachments shouldn’t live in the mssql database” plan and you said “yeah, probably, but doing it any other way would be a million times harder to maintain.” You were 100% right and I think of it often when I encounter someone who is about to do a similar dumb thing for “the right reasons.”
It's a bit harsh but I always feel like Fog Creek might be the cautionary tale in "what happens if you over hire for capability vs. your requirements?" I think that a less capable team would have never landed on the "let's maintain our own programming language" approach w.r.t. Wasabi.
As an aside, I do think that targeting Mono was the right thing to do for the universe, as it butterfly-effected tedu into writing weird and wonderful technical blog posts for the next ten years :p
I had an internship at Fog Creek and would add that it was probably the most friendly and harmonious place I worked, which made it very reality-show-incompatible (and very 21-year-old-me incompatible, I wasn't asked back lol). Certainly the representation of you as an asshole was ridiculous IMO.
(Since you're answering arbitrary Fog Creek questions) In retrospect, do you think it was a mistake to make kiln hg-centric at first?
Admittedly, I got a B in 410 and only managed to TA 213, but I do work in the space now.
Yes, it’s hard to do research if you start at “let’s design a kernel from scratch,” but you’d never need to do that. You can just hack Linux or a bsd, or even use something like bpf to extend it.
The thing that annoyed me about 410 is that when I switched to Linux I realized that pusha/popa didn’t matter at all. It was a good course for writing reentrant C and learning the very basics of hardware, but the really hard stuff is in the weird dynamics of memory management on NUMA systems and work conserving io, which you can’t get anywhere near if you are starting from scratch.
You reduced the number of patches a lot and also pushed very hard to get us to 3.0 after we sat on 2.6.38 ~forever. Which was very appreciated, btw. We built the whole plan going forward based on this work.
I'm not arguing that anyone should be nice to anyone or not (it's a waste of breath when it comes to Linux). I'm just saying that the benchmarking was thorough and that contemporary 2014 hardware could zero pages fast.