I'm the maintainer of Kreuzberg, an open-source document intelligence library (https://github.com/kreuzberg-dev/kreuzberg). Some of you may have used it for RAG ingestion.
We're launching Kreuzberg Cloud, a SAAS API and a self-hosted system. It's in public beta, and I would like to invite you all to give it a try.
What out MVP offers: we offer very fast CPU optimized document and code intelligence. You can extract content from more than 90 document file formats and 300 code file formats into Markdown (or plaintext/djot), with additional features (same pricing tier) including chunking, embeddings, keyword extraction - and various types of intelligence.
The OSS library is used as the base engine of the cloud system. Our initial offering is $0.008/page, and you get the first 10K pages free, no card required.
We also offer our entire system for self-hosting - using helm charts. We are looking for design partners, so if thats relevant - shoot me a line.
In my tests I found tesseract quite good for regular text documents. For other kinds of texts it's not great.
As for using models - there are some good small language models as well, and of course LLMs.
I sorta feel though that if one needs complex OCR, or a vision model for layout, one should opt for either a commercial solution that abstracts the deployment and GPU management, or bake ones own system.
For most use cases involving text documents though, my subjective opinion is that tesseract is sufficient.
So, for PDF we need to distinguish between two types of text extraction-
1. Text extraction from a searchable PDF.
2. OCR.
For 1. Kreuzberg uses pypdfium2, which is a python binding for pdfium - the chromium PDF engine. In this regard Kreuzberg has top notch performance. Much faster than miner.six, PDFplumber etc.
Note PyMuPDF has top notch performance but also an AGPL license, and is almost unusable because of this without paying.
For 2. Kreuzberg uses Tesseract, which is very solid. Performance is good, and Kreuzberg utilizes async worker processes to optimize concurrency.
OCR though is a complex world. If what you need is to extract text from standard text documents (broadly speaking), Tesseract and hence Kreuzberg are a good choice.
If what you need is things like layout extraction, hand writing recognition, complete bonding box metadata etc. than you need to use an alternative - commercial one probably.
I'm the maintainer of Kreuzberg, an open-source document intelligence library (https://github.com/kreuzberg-dev/kreuzberg). Some of you may have used it for RAG ingestion.
We're launching Kreuzberg Cloud, a SAAS API and a self-hosted system. It's in public beta, and I would like to invite you all to give it a try.
What out MVP offers: we offer very fast CPU optimized document and code intelligence. You can extract content from more than 90 document file formats and 300 code file formats into Markdown (or plaintext/djot), with additional features (same pricing tier) including chunking, embeddings, keyword extraction - and various types of intelligence.
The OSS library is used as the base engine of the cloud system. Our initial offering is $0.008/page, and you get the first 10K pages free, no card required.
We also offer our entire system for self-hosting - using helm charts. We are looking for design partners, so if thats relevant - shoot me a line.