Neural accelerators are easy to use from Metal. They kick in automatically if you do a matmul using Metal Performance Primitives and you use bf16 or smaller (they don't seem to work in fp32).
Yeah I kind of think authors didn't conduct a thorough-enough literature review here. There are well-known relations between number of hash functions you use and the FPR, cache-blocking and register-blocking are classic techniques (Cache-, Hash-, and Space-Efficient Bloom Filters by Putze et. al), and there are even ways of generating patterns from only a single hash function that works well (shamelessly shilling my own blogpost on the topic: https://save-buffer.github.io/bloom_filter.html)
I also find the use of atomics to build the filter confusing here. If you're doing a join, you're presumably doing a batch of hashes, so it'd be much more efficient to partition your Bloom filter, lock the partitions, and do a bulk insertion.
Have you tried SolveSpace? It's easily my favorite open source CAD program. The main things it's missing are shells, fillets, and chamfers. But I've been able to 3D print quite a few parts using it!
Giving it a quick look, seems like they've addressed a lot of the shortcomings of Parquet which is very exciting. In no particular order:
- Parquet metadata is Thrift, but with comments saying "if this field exists, this other field must exist", and no code actually verifying the fact, so I'm pretty sure you could feed it bogus Thrift metadata and crash the reader.
- Parquet metadata must be parsed out, meaning you have to: allocate a buffer, read the metadata bytes, and then dynamically keep allocating a whole bunch of stuff as you parse the metadata bytes, since you don't know the size of the materialized metadata! Too many heap allocations! This file format's Flatbuffers approach seems to solve this as you can interpret Flatbuffer bytes directly.
- The encodings are much more powerful. I think a lot of people in the database community have been saying that we need composable/recursive lightweight encodings for a long time. BtrBlocks was the first such format that was open in my memory, and then FastLanes followed up. Both of these were much better than Parquet by itself, so I'm glad ideas from those two formats are being taken up.
- Parquet did the Dremel record-shredding thing which just made my brain explode and I'm glad they got rid of it. It seemed to needlessly complicate the format with no real benefit.
- Parquet datapages might contain different numbers of rows, so you have to scan the whole ColumnChunk to find the row you want. Here it seems like you can just jump to the DataPage (IOUnit) you want.
- They got rid of the heavyweight compression and just stuck with the Delta/Dictionary/RLE stuff. Heavyweight compression never did anything anyway, and was super annoying to implement, and basically required you to pull in 20 dependencies.
Overall great improvement, I'm looking forward to this taking over the data analytics space.
I figured this was pretty obvious given that MLPs are universal function approximators. A giant MLP could achieve the same results as a transformer. The problem is the scale - we can’t train a big enough MLP. Transformers are a performance optimization, and that’s why they’re useful.
Was it a 2D game engine? I recall before using Zero I used an engine called ProjectFUN, which is similar to what you described. That was around 2013 though, not 2002.
Wow you guys wrote Zero Engine? I used it in summer camps at DigiPen a long time ago, I really enjoyed using it and programming in Zilch (I wrote an AI that played a galaga clone!). This Unity fiasco had me wondering how Zero was doing. Seems like it's going to have a resurgence! Super exciting!
It's been really cool seeing PRQL come to life! I'm not involved with it, but I still remember the very first HN post about a year ago where it was just a proposal. And look at it now! It's really gaining steam, I'm really excited about it!
> To me the article read as if our job was to make every bit of code as fast as possible.
My interpretation was that he thinks the baseline for performance is too low. There are kind of two parts to performance: non-pessimization and optimization. He’s mostly harping on the former point, that you don’t have to go tuning anything. Just by writing code without following the clean code rules you can get a huge speed up for free, without any tuning at all. It’ll be free of obvious overheads, even if it doesn’t do anything special to fully utilize the hardware.