Cutting LLM Batch Inference Time in Half: Dynamic Prefix Bucketing at Scaledaft.ai5 points·by ykev·قبل 8 أشهر·1 comments