headlessvictim2

4 yıl öncediscuss

Thank you for this.

4 yıl öncediscuss

Thanks for the FastAPI explanation. This makes sense.

Right now, we use ELB (Elastic Load Balancer) to sit in front of multiple GPU instances.

Is this sufficient or do you suggest adding Celery to this architecture?

headlessvictim2

4 yıl öncediscuss

Thanks for the explanation.

Right now, we use ELB (Elastic Load Balancer) to sit in front of multiple GPU instances.

Is this sufficient or do you suggest adding Celery into this architecture?

headlessvictim2

4 yıl öncediscuss

A proxy like Cloudflare or a custom proxy that stores data?

Are there proxy examples you could point us to?

Thanks for your help.

headlessvictim2

4 yıl öncediscuss

Thanks for the suggestion.

How would this work with GPU-bound machine learning models?

The model processing takes > 30 seconds and would still represent the bottleneck?

headlessvictim2

4 yıl öncediscuss

To use premium settings without paying.

It appears less like malicious DDoS and more like pragmatic theft.

headlessvictim2

4 yıl öncediscuss

Thanks for the suggestion.

It is possible, but this degrades the experience for legitimate users.

We prefer solving this without impacting/taxing normal users if possible.

headlessvictim2

4 yıl öncediscuss

Thanks for the suggestion.

It is possible, but this degrades the experience for legitimate users.

We prefer solving this without impacting/taxing normal users if possible.

headlessvictim2

4 yıl öncediscuss

Consider us one. :)

We tried removing "async" -- thinking it would force sequential processing -- but it unexpectedly seemed to cause parallel processing of requests, which caused CUDA memory errors.

Before removing "async", this is the weird behavior we observed:

* Hacker blasts 50-100 requests.

* Our ML model processes each request in normal time and sequentially.

* But instead of returning individual responses immediately, the server holds onto all responses -- sending responses only when the last request finishes (or a bunch of requests finish).

* Normally, request 1 should return in N seconds, request 2 in 2N seconds, but with this, all requests returned in about N50 seconds (assuming batch size of 50).

1. Any suggestions on this?

2. Mind clarifying how sync vs aync works? The FastAPI docs are unclear.

Any help would be much appreciated.

This has been extremely frustrating.

headlessvictim2

4 yıl öncediscuss

This is interesting. Thanks for sharing.

Are you saying block form submission unless the cursor is over the event target?

If so:

* How to handle legitimate requests from mobile users?

* How to handle form submissions with the "return" key?

headlessvictim2

4 yıl öncediscuss

Thanks for the reply!

The freemium service provides access to machine learning models on GPU instances, served with FastAPI.

Each request invokes a compute-intensive ML model, but perhaps there is something wrong with the FastAPI configuration as well?

headlessvictim2

4 yıl öncediscuss

Off-topic, but our freemium website is under attack by headless browsers.

The freemium service provides access to compute-heavy machine learning models running on GPUs.

Hackers blast 50-100 requests in the same second, which clog the servers and block legitimate users.

We reported IPs to AWS and use Cloudflare "Super Bot Fight Mode" to thwart attacks, but the hackers still break through.

We don't require accounts, but could impose account requirements if this helps.

Any suggestions?

comments