HackerTrans
トップ新着トレンドコメント過去質問紹介求人

rand0mwalk

no profile record

投稿

Are new LLM models trained on old benchmarks?

1 ポイント·投稿者 rand0mwalk·2 年前·0 コメント

What Is Nous Research?

2 ポイント·投稿者 rand0mwalk·2 年前·0 コメント

Systems and Machine Learning and Graphics Topics

3 ポイント·投稿者 rand0mwalk·2 年前·1 コメント

コメント

rand0mwalk
·8 か月前·議論
Tokens start as a special [MASK] token. Then as the diffusion process runs they are "unmasked" i.e. sampled.

So yes, you define a sequence of [MASK] tokens with some length ahead of time.

In practice, if a model wants to write a shorter sequence, it'll just fill the remaining tokens with empty content. If it wants to write a longer sequence, you'll have to identify this and extend the sequence with more [MASK] tokens. This is typically obvious since there's no "end of sequence" token present if the model wants to generate more.
rand0mwalk
·2 年前·議論
Is there an accompanying paper out there?