> I won’t go into the M/M/1 part, as it doesn’t really end up affecting the Important Thing I’m telling you about. What does matter, however, is that ‘∞’.
I'm really no expert but if I understand it correctly the `M/M` part defines the distributions of the arrival and service processes, so this definitely is important as others have already mentioned in the comments. E.g. a D/D/1 queue where D stands for deterministic arrival shouldn't suffer from this problem.
This doesn't change the interesting fact the article presents but throwing in technical terms without proper explanation is imo bad style. Either don't mention it (would be better in this case) or explain it and why it is relevant.
This is also a common "mistake" unexperienced people make when writing papers. They mention a fact that is somehow related but is completely irrelevant to the topic and the rest of the paper. I don't want to assume anything but to me this often smells like showing-off.
> Forget about performance for a moment, and just write those imbricated loops.
Right, if we forget about performance a three-nested-loop GEMM is certainly better than any other implementation. But that's completely meaningless and you could find such examples for almost any task. This is almost as saying writing hash functions is easy as long as you forget about collisions and avalanche effects.
> When you realize that most deep learning library, "einsum is all you need", and that einsum is just loops and sum, it kind of screams you are doing it the wrong way when using numerical libraries ranging from BLAS to cuda.
That is just not true: It is correct that deep learning tasks mainly reduce to convolutions and GEMM. Convolutions can be mapped to contraction kernels which again can be mapped to GEMM. However, Numpy's einsum is a great example how you shouldn't do it as it's notoriously slow. Also, the main task in general tensor contraction functions is to find an optimal contraction ordering rather than executing GEMM (you can have exponential savings if you choose a good contraction sequence). You can read more about this in e.g. https://arxiv.org/abs/2002.01935.
> Loops can also be automatically differentiated. Loops can also be reordered. Loops can be tiled. There are some techniques (polyhedral loop optimization) to do it automatically. Loops can be parallelized and vectorized.
This is true in theory and while modern compilers get better and better there is still no compiler framework that could, given an arbitrary linear algebra task outperform a manually optimized BLAS or LAPACK library.
Also, you are mentioning just a very few low-level optimization techniques that can be applied while modern compiler research currently focuses on different aspects as implemented in XLA for example.
> Optimizing code, even though it feels quite fun and rewarding to do manually is a job for compilers.
> Write naive algorithms but smart compilers.
Still, don't write bad code and use compilers as an excuse.
> I won’t go into the M/M/1 part, as it doesn’t really end up affecting the Important Thing I’m telling you about. What does matter, however, is that ‘∞’.
I'm really no expert but if I understand it correctly the `M/M` part defines the distributions of the arrival and service processes, so this definitely is important as others have already mentioned in the comments. E.g. a D/D/1 queue where D stands for deterministic arrival shouldn't suffer from this problem.
This doesn't change the interesting fact the article presents but throwing in technical terms without proper explanation is imo bad style. Either don't mention it (would be better in this case) or explain it and why it is relevant.
This is also a common "mistake" unexperienced people make when writing papers. They mention a fact that is somehow related but is completely irrelevant to the topic and the rest of the paper. I don't want to assume anything but to me this often smells like showing-off.