Ask HN: Did PageRank delay the invention of transformers and modern AI?
1 comments
I doubt. Without the Google's money that PageRank created, others wouldn't have assembled the mix of compute, information, and brains that made the invention of transformers possible.
You need the vast amount of compute to accelerate matrix computation, that came from accelerating distbelief (https://research.google/pubs/large-scale-distributed-deep-ne...) for Search and Ads. Google developed custom ASICs (TPUs) because it would have been too costly to use CPUs and GPUs for their use cases.
You need the world information that came from Search.
You need the money to pay the researchers, and the willingness to do discretionary research that may not be directly applicable to your main products.
Transformers derive from neural machine translation.
You need the vast amount of compute to accelerate matrix computation, that came from accelerating distbelief (https://research.google/pubs/large-scale-distributed-deep-ne...) for Search and Ads. Google developed custom ASICs (TPUs) because it would have been too costly to use CPUs and GPUs for their use cases.
You need the world information that came from Search.
You need the money to pay the researchers, and the willingness to do discretionary research that may not be directly applicable to your main products.
Transformers derive from neural machine translation.
Maybe AI researchers overvalued the importance of circularity?