Regarding 2x faster than pytorch being a condition for tinygrad to come out of alpha:
Can they/someone else give more details as to what workloads pytorch is more than 2x slower than the hardware provides? Most of the papers use standard components and I assume pytorch is already pretty performant at implementing them at 50+% of extractable performance from typical GPUs.
If they mean more esoteric stuff that requires writing custom kernels to get good performance out of the chips, then that's a different issue.
The expression f(z) = \sum_i 1/(z-\lambda_i) is called Stieltjes transform and is heavily used in random matrix theory and similar expressions are used in other works such as Batson, Spielman and Srivastava. This is all to analyze the behavior of eigenvalues which is exactly what they were trying to understand. I'd be very surprised if Aaronson doesn't know about this.
I think it is not just the library but the huge costs associated with storage, encoding and bandwidth. YouTube has innovated significantly to make it as cheap as possible to run such a service and it is likely that it would take an enormous amount of money for any competitor to replicate it.
(Disclaimer: I work at Google but no connection to YouTube)
Netflix is definitely trying its hardest to get into the Indian market and I do think they are gaining some ground recently. They have picked up a lot of new telugu movies and has basically become essential if you want to stream new movies. They also have mobile only plans at 199Rs but not sure how many such subscriptions they sell but for me at least the 649Rs plan is not too bad even if my parents watch at most 1-2 movies a month on Netflix.
If they are able to get even 2 million telugu households over to subscribe the next few years, that would be a revenue of about $150 million which I think is probably much less than the amount of money they spend on telugu content.
I didn't want to associate this account w/ my real name but now that you mentioned it wasn't right of me to not point that out. I added a disclaimer.
The posted algorithm and the one mentioned in my paper are very similar. It is just that the cumulative sum computation is parallelized in the posted website.
Can they/someone else give more details as to what workloads pytorch is more than 2x slower than the hardware provides? Most of the papers use standard components and I assume pytorch is already pretty performant at implementing them at 50+% of extractable performance from typical GPUs.
If they mean more esoteric stuff that requires writing custom kernels to get good performance out of the chips, then that's a different issue.