HackerTrans
TopNewTrendsCommentsPastAskShowJobs

rand0mwalk

no profile record

Submissions

Are new LLM models trained on old benchmarks?

1 points·by rand0mwalk·2 jaar geleden·0 comments

What Is Nous Research?

2 points·by rand0mwalk·2 jaar geleden·0 comments

Systems and Machine Learning and Graphics Topics

3 points·by rand0mwalk·2 jaar geleden·1 comments

comments

rand0mwalk
·8 maanden geleden·discuss
Tokens start as a special [MASK] token. Then as the diffusion process runs they are "unmasked" i.e. sampled.

So yes, you define a sequence of [MASK] tokens with some length ahead of time.

In practice, if a model wants to write a shorter sequence, it'll just fill the remaining tokens with empty content. If it wants to write a longer sequence, you'll have to identify this and extend the sequence with more [MASK] tokens. This is typically obvious since there's no "end of sequence" token present if the model wants to generate more.
rand0mwalk
·2 jaar geleden·discuss
Is there an accompanying paper out there?