Arrows of Time for Large Language Models(arxiv.org)

6 pointsby tianlonghace 2 años3 comments

Arrows of Time for Large Language Models

https://arxiv.org/abs/2401.17505

4 comments

nyoncorehace 2 años

Isn't it obvious that since LLM are trained to predict the next word they do better than to predict the previous one?

frotaurhace 2 años

In the paper it is mentioned that the LLMs predicting the previous token are indeed pre-trained in this way, so it is not true that the difference is obvious.

[deleted]hace 2 años

tianlonghace 2 años

There is a link with entropy creation?