I'm always glad to see more multi-page work in VLM-based OCR. Especially single-pass. One of the few other multi-page papers from recently, MinerU-Popo, treats fixing up multi-page outputs as a post-processing correction step (https://arxiv.org/abs/2605.24973). Interesting to see the drop-off in quality as you up page count, though.
I also think the attention approach (always attend to the image/prefix, with a sliding window for local context) is neat!
I do wish they updated their comparison table to include more recent work (that scores marginally better on OmniDocBench), like dots.mocr.
Very impressed with how much the Gemma ecosystem has advanced just this week.
Gemma 12B, multitoken prediction, and official quants released. Feels like Google is putting real effort into this string of releases, and I'm very excited to see that!
Very cool to see a company pushing what's possible with (relatively) tiny models! A 350M parameter trained on 28T tokens that, from the benchmarks, is competitive with Qwen3.5-0.8B.
Comparing the architecture to Qwen3.5, it seems:
- fewer, wider layers
- mixing full attention and conv's, instead of the full+linear attention of Qwen3.5
Shared this because I was having fun thinking through floating point numbers the other day.
I worked through what fp6 (e3m2) would look like, doing manual additions and multiplications, showing cases where the operations are non-associative, etc. and then I wanted something more rigorous to read.
For anyone interested in floating point numbers, I highly recommend working through fp6 as an activity! Felt like I truly came away with a much deeper understanding of floats. Anything less than fp6 felt too simple/constrained, and anything more than fp6 felt like too much to write out by hand. For fp6 you can enumerate all 64 possible values on a small sheet of paper.
For anyone not (yet) interested in floating point numbers, I’d still recommend giving it a shot.
I've been noticing a _lot_ more AI-generated/edited content of late, both comments and stories. It's gotten to the point that I spend a lot less time on HN than I used to, and if it continues to get worse I expect I'll quit altogether.
At the end of the day, I'm here because of all the thoughtful commenters and people sharing interesting stories.
I’ve been a monthly donor since ~the beginning when I was just an undergraduate, and I still read the stories and emails I receive. I’m glad that you opted for the steady growth path, and that you’ve made it a sustainable thing.
Aleksa’s blog is a bit in the weeds for my taste but it’s really worth working through.
A lot of the magic of vLLM happens in the PagedAttention kernels, which are really succinctly implanted in nanovllm. And the codebase is great and readable by itself!
That’s built on a dataset and paper I wrote called CommonForms, where I scraped CommonCrawl for hundreds of thousands of fillable form pages and used that as a training set:
Next step is training and releasing some DETRs, which I think will drive quality even higher. But the ultimate end goal is working on automatic form accessibility.
Existing “auto-fillable” tools are pretty lackluster in my experience. CommonForms is tooling that can automatically detect form fields in PDFs and turn those PDFs into fillable documents. The dataset is ~500k form pages pulled from Common Crawl, which I trained the object detectors on. For being vision only, the results are pretty remarkable!
Releasing the dataset, paper, models, and (imo most importantly) simple/convenient tooling to automatically prepare any PDF.
I also think the attention approach (always attend to the image/prefix, with a sliding window for local context) is neat!
I do wish they updated their comparison table to include more recent work (that scores marginally better on OmniDocBench), like dots.mocr.