Hi I’m George, I’d love to share lessons we made optimizing data pipelines with AI / embedding calls for our users, which increased the pipeline throughput 5x. We did adaptive batching - discussed in detail how we did it.
Developers still simply process data row-by-row, under the hood we queue requests and batch at the right moments (batching is effectively columnar), so no manual plumbing. Would love your thought.