On the contrary, "dark silicon" instead suggests that separating fp32 and int32 (now in GA102/104, fp32 and int32/fp32) data paths at the cost of more die space usage currently makes excellent sense. (See also: tensor cores, ray tracing cores.) Jensen Huang very briefly alluded to this when during the GA102/104 announcement he mentioned the end of Dennard scaling.
Fun fact: Alex Krizhevsky's cuda-convnet was also an early adopter of CHWN tensor layout. Basically, having the batch size N as the major dimension limits you to running batch sizes that are multiples of the warp size (typically 32), but then you also have an easier time of implementing fast kernels for all your neural and tensor ops, including tensor convolutions, without getting nearly as stuck in the weeds of microarchitectural optimizations.
rustc does have a working nvptx target today, though it’s not supported nearly as well as the mainstream cpu targets, and some things you would really want for gpu programming (e.g. shared memory address space) are not currently exposed in the rust language. But kernels written in rust can compile to ptx; you’ll still need to write glue code.
Her short story collections are quite accessible. If you want to dive into the Hainish cycle, _Four Ways to Forgiveness. Or for a mostly Hainish but more diverse set of stories, _The Wind's Twelve Quarters_.
I first learned about calculus from, I think, this book, "Calculus the Easy Way" [1] (or, at least the first chapter or two). It does make use of some algebra, but you're not necessarily limited to the strict progression of pre-algebra -> algebra -> calculus.
> Due to the complexity involved, Cerebras is not only designing the chip, but they also must design the full system. ... The WSE will come in 15U units with a chassis for the WSE and another one for the power and miscellaneous components. The final product is intended to act like any other network-attached accelerator over a 100 GbE.
So it's NIC bandwidth bottlenecked. Though 100 GbE is in the same ballpark as PCIe 3, but last I heard 40-100 GbE were pretty CPU-intensive compared to alternatives.
Cool to see this on HN. I briefly worked there on a closely related project during high school after this paper was published. It was my first real experience doing wet lab work-- learned a bunch, met cool people, was fun. (Also did a lot of pipetting and PCR.)
To expand a bit, as I understand it, Remora gives a type system (and type inference) to APL- and J-style reranking, which is called rank polymorphism in Remora. The key thing is that type inference in Remora is based on Presburger arithmetic (0,+) which is decidable, unlike the stronger but overly powerful (for their purposes) than full Peano arithmetic (0,1,+,*).
The last time I measured ImageNet JPEGs with EXIF orientation metadata, the number of affected images was actually quite small (< 100, out of a dataset of 1.28M).
There are also some duplicates, but altogether it seems fairly "clean."
