jumpCastle·2 lata temu·discussAttention was invented because Bengio lab had to be disciplined about a black box (google had more compute)
jumpCastle·2 lata temu·discussAlso the parameters are optimized also with loss of future tokens in the sequence.