I test if qwen3.5-35B-A3B can exit vim when running in a harness that allows it to read the raw terminal buffer and send raw key presses to the terminal. Enjoy!
I'm trying to train a variable resolution ViT using IJEPA. I'm currently topping out at about 30% on imagenet1k after training for 20 epochs (6 hours)
It'd be cool to have some help and feedback. I'm on the right track to getting really killer setup that is super fast to train it needs more evaluations and more tuning. Anyone interested?
i experiment with using a (mostly) unmodified llama model to generate images, by training on the bits from a lossy compression algorithm. It turns out the key is having a decoder which can give 'hints' as conditioning information for the autoregressive model, about what the decoder is going to do with the next token in the stream
Quantization also works as regularization; it stops the neural network from being able to use arbitrarily complex internal rules.
But really it's only really useful if you absolutely need to have a discrete embedding space for some sort of downstream usage. VQVAEs can be difficult to get to converge, they have problems stemming from the approximation of the gradient like codebook collapse
I wrote a short article about jpg and if we could use concepts from how jpg works to make an image autoencoder that has a left-to-right positional bias and variable compression
Basically, existing VAEs are pretty good at compression, but have bad properties like 2D latent position bias and difficulty training on batches of mixed resolutions
So I try something I call DCT-Autoencoder, which takes ideas from JPG to learn compression of patched DCT features of an image