IMHO, your article is missing an important point: 90% of implementations today flatten documents to plain text before chunking them. Why not consider the visual appearance that the human gave to the document?
Using layout information combined with semantics, you can increase rag performances by +160% (tested via benchmarks), so why do most of us only use text?
It seems it tries to always extract tables even if the content is just text. Is not working and at a similar price you can get high performing solutions like preprocess.co and similars.
- Titles matter, a lot: if you add the title of the section at the start of each chunk you will get 10x better embeddings and so more accurate results.
- The size doesn't matter: It depends on the combination of the layout and semantics of the content.
- Avoid garbage in / out: increased context windows don't mean you can put trash inside them. The more good you are at putting relevant information the more precise answers you get. Especially for enterprise-grade solutions, this is so important.
There are good emerging API solutions that implement semantic + layout-based chunking, which in my opinion is the best chunking strategy for PDF / Office files (the widest use case scenario for enterprises).
The real point that has been completely ignored is that data for real applications come from Office and PDF files, and making them plain text throws all the visual information.
You are both right about chunking, and i think is one of the main challenges.
About more intelligent chunking approaches, i think you have to give a try to to preprocess.co
It's able to preprocess and chunk PDFs, Office Files, and HTML content.
It follows the original document layout considering the content semantics so you get optimal chunks
Note: multimodal ≠ layout