No I am not using Kafka. It is just basic python server. I want to calculate what is the nth percentile of size of my incoming request object over past one hour.
I don't want to store size of each request in memory. It will eat so much of my RAM.
Incoming traffic:
- 1st batch
--> 60 requests
---> size of 1st request is 10kb
---> size of 2nd request is 2kb
...
...
- 2nd batch
--> 10 requests
---> size of 1st request is 5kb
---> size of 2nd request is 8kb
...
- 100th batch
I am talking about percentile(10th, 50th, 95th) size of request.
Yeah, sometimes I am really pissed off because many people just try to find issues in my PR. They don't care for the rest of the work. I would suggest you talk to your manager about this topic. You need a better environment and which is fine.
I don't want to store size of each request in memory. It will eat so much of my RAM.
Incoming traffic:
- 1st batch
--> 60 requests
---> size of 1st request is 10kb
---> size of 2nd request is 2kb
...
...
- 2nd batch
--> 10 requests
---> size of 1st request is 5kb
---> size of 2nd request is 8kb
...
- 100th batch
I am talking about percentile(10th, 50th, 95th) size of request.