Show HN: Pulsys – Pull-through cache for Hugging Face built with io_uring(github.com)
github.com
Show HN: Pulsys – Pull-through cache for Hugging Face built with io_uring
https://github.com/pulsys-io/pulsys
Pulsys is an open-source, self hosted, authenticated pull-through cache for Hugging Face. It acts as a drop-in replacement for existing HF clients and serves warm hits directly from disk, achieving up to 90 GB/s throughput (benchmarks are run on loopback on ec2) by bypassing standard syscalls in favor of io_uring (Linux) and sendfile (macOS).
1 comments
I’m the creator of Pulsys.
I originally built this to help with self-hosting Hugging Face models, but it evolved into an authenticated pull-through cache.
A big part of this project was an experiment to see how far I could get Cursor + (Opus/Fable) to optimize the hot path. I initially used fasthttp and Go's net/http, but after some deep optimizations, the only way to squeeze out more performance was to eliminate syscalls.
I ended up building two optimization paths that use a custom-built HTTP/1.1 parser: one for macOS using sendfile + sf_hdtr, and one for Linux using io_uring. For anyone interested in the threat model or how the custom parser is tested against Go's standard library, I wrote a detailed security breakdown here: https://pulsys.io/docs/security/
The result is that it can sustain 1.36M req/s at 4 KiB and 90 GB/s at 16 MiB on an EC2 instance (see benchmarks https://pulsys.io/docs/benchmarks/)
It drops right in front of existing clients (just set HF_ENDPOINT).
I’d love to hear your thoughts, feedback, or any questions on the AI-assisted optimization process!
I originally built this to help with self-hosting Hugging Face models, but it evolved into an authenticated pull-through cache.
A big part of this project was an experiment to see how far I could get Cursor + (Opus/Fable) to optimize the hot path. I initially used fasthttp and Go's net/http, but after some deep optimizations, the only way to squeeze out more performance was to eliminate syscalls.
I ended up building two optimization paths that use a custom-built HTTP/1.1 parser: one for macOS using sendfile + sf_hdtr, and one for Linux using io_uring. For anyone interested in the threat model or how the custom parser is tested against Go's standard library, I wrote a detailed security breakdown here: https://pulsys.io/docs/security/
The result is that it can sustain 1.36M req/s at 4 KiB and 90 GB/s at 16 MiB on an EC2 instance (see benchmarks https://pulsys.io/docs/benchmarks/)
It drops right in front of existing clients (just set HF_ENDPOINT).
I’d love to hear your thoughts, feedback, or any questions on the AI-assisted optimization process!