HackerTrans
TopNewTrendsCommentsPastAskShowJobs

fullung

no profile record

Submissions

AWS Access with Rippling: Integrating IAM Identity Center and JIT Access

rippling.com
4 points·by fullung·il y a 3 ans·0 comments

How Rippling used Spot instances to save and scale CI/CD

rippling.com
3 points·by fullung·il y a 3 ans·0 comments

comments

fullung
·il y a 3 ans·discuss
Albert here :) Decision basically came down to the following: we needed to build a 1 million events/sec log processing pipeline, and had only 5 servers (more were on the way, but would take many months to arrive at the data center). So v1 of this pipeline was 5 servers, each running Kafka 0.8, a service to receive events from the edge and put it into Kafka, and a consumer to aggregate the data. To squeeze all of this onto this hardware footprint, I spent about a week looking for a format that optimized for deserialization speed, since we had a few thousand edge servers serializing data, but only 5 deserializing. Capnproto was a good fit :)