jumpCastle·2 yıl önce·discussAttention was invented because Bengio lab had to be disciplined about a black box (google had more compute)
jumpCastle·2 yıl önce·discussAlso the parameters are optimized also with loss of future tokens in the sequence.