Lessons learned – 5x throughput on data pipelines with adaptive batching(cocoindex.io)
cocoindex.io
Lessons learned – 5x throughput on data pipelines with adaptive batching
https://cocoindex.io/blogs/batching
1 comments
Hi I’m George, I’d love to share lessons we made optimizing data pipelines with AI / embedding calls for our users, which increased the pipeline throughput 5x. We did adaptive batching - discussed in detail how we did it.
Developers still simply process data row-by-row, under the hood we queue requests and batch at the right moments (batching is effectively columnar), so no manual plumbing. Would love your thought.