I originally built this to help with self-hosting Hugging Face models, but it evolved into an authenticated pull-through cache.
A big part of this project was an experiment to see how far I could get Cursor + (Opus/Fable) to optimize the hot path. I initially used fasthttp and Go's net/http, but after some deep optimizations, the only way to squeeze out more performance was to eliminate syscalls.
I ended up building two optimization paths that use a custom-built HTTP/1.1 parser: one for macOS using sendfile + sf_hdtr, and one for Linux using io_uring. For anyone interested in the threat model or how the custom parser is tested against Go's standard library, I wrote a detailed security breakdown here: https://pulsys.io/docs/security/
The result is that it can sustain 1.36M req/s at 4 KiB and 90 GB/s at 16 MiB on an EC2 instance (see benchmarks https://pulsys.io/docs/benchmarks/)
It drops right in front of existing clients (just set HF_ENDPOINT).
I’d love to hear your thoughts, feedback, or any questions on the AI-assisted optimization process!
I have 14 YOE and its the opposite. Really brilliant jrs don't have the experience to know the consequences of their designs, they build fast under pressure. They reinvent things that already exist (they don't know because they a Junior).
I worked in the music streaming industry, which involves ingesting 10's of millions of XML files. Absolutely do not use XSLT. Schema changes between different versions of DDEX were a nightmare for all the legacy XSLT we had.
I don't see any reason to use this for AWS over CDK. CDK already has some pretty higher level modules that make it really easy to setup ecs, k8s, vpc, ect.
Every framework have worked on for the last 12 years has bugs, breaks in certain situations. Every system in fact, i have worked on in 12 years has bugs and breaks. That is just the nature of software development.
You should be using frameworks instead of trying to reinvent things. It takes away decision fatigue and makes it easier for most teams to follow conventions that are already established.
auto scaling could also mean scaling to 0. If you are running GPU workloads in k8's, you would typically setup a node pool with gpus that can scale to 0 after the job runs.
I feel like the majority of the people who complain about orms, haven't taken the time to learn the orm. Often times, there is a DSL you need to learn, but for simple database relations and joins not using an ORM is a waste of time and resources.
orms are often opinionated and tables that are setup with an ORM may not be compatible with other ORMS. This is the only real downside. If you have a large legacy application that needs to be migrated to a new framework that uses different conventions for index names, table names, ect, then you are in for a bad time.
Django's orm is probably one of the best out there. Django also has orm manager classes you can extend with abstractions around orm functions or raw queries.
For complex queries, like with CTE's, complex joins, recrusive queries, index optimizations, there is no way around custom queries but any good ORM has an api around these raw queries that can hopefully also map to objects in your applications.
I've seen $85-$100. I paid $100/hr for the dev I hired. But they had multiple pro level certificates. I believe AWS takes as 15% fee though. The amount you will get paid is highly dependent on the certificates you have.
I am in the same boat. I have 12+ years of experience. I tired to launch a startup and it failed miserably.
I am considering Amazon IQ. If you are into AWS and can get the professional level certificates, you can use Amazon IQ platform to freelance. I have used Amazon IQ to hire before and I had good results. Your really need multiple pro level certificates to compete though.
I have tried Upwork, but it's been a not so good experience. I have my work history on the platform, but it doesn't seem to matter. I've worked at some top tier companies (one of the trillion dollar market cap SF big tech companies). I think Upwork requires building a portfolio tailored to the type of projects you would like to work on.
I originally built this to help with self-hosting Hugging Face models, but it evolved into an authenticated pull-through cache.
A big part of this project was an experiment to see how far I could get Cursor + (Opus/Fable) to optimize the hot path. I initially used fasthttp and Go's net/http, but after some deep optimizations, the only way to squeeze out more performance was to eliminate syscalls.
I ended up building two optimization paths that use a custom-built HTTP/1.1 parser: one for macOS using sendfile + sf_hdtr, and one for Linux using io_uring. For anyone interested in the threat model or how the custom parser is tested against Go's standard library, I wrote a detailed security breakdown here: https://pulsys.io/docs/security/
The result is that it can sustain 1.36M req/s at 4 KiB and 90 GB/s at 16 MiB on an EC2 instance (see benchmarks https://pulsys.io/docs/benchmarks/)
It drops right in front of existing clients (just set HF_ENDPOINT).
I’d love to hear your thoughts, feedback, or any questions on the AI-assisted optimization process!