"As many have said GPUs are so fast because they are so efficient for matrix multiplication and convolution, but nobody gave a real explanation for why this is so. The real reason for this is memory bandwidth and not necessarily parallelism."
Sorry but please don't take Eigen (https://eigen.tuxfamily.org) away from me. Can't speak for others, but the scientific code I work on would become unreadable like that.