Show HN: Deploy Scikit and Keras Models with a Simple Drag and Drop(inferrd.com)
inferrd.com
Show HN: Deploy Scikit and Keras Models with a Simple Drag and Drop
https://inferrd.com
11 comments
Is this able to support more than 50 requests per second? Are there any benchmarks on performance overhead of the underlying web server/routing that is handling the requests?
Theoretically we can support up to thousands of request per second under the entreprise plan.
We stopped at 50 request/s in the pricing table because that seemed like a reasonable number for most use cases.
We stopped at 50 request/s in the pricing table because that seemed like a reasonable number for most use cases.
Is there any reason that you select 50 requests per second? Anyway, I'm also interested in performance metrics.
It seems very interesting! What about support for pytorch models or .onnx? I usually use the pytorch->onnx->tensorrt to deploy models.
Hi! We do not currently have PyTorch model but it's one of the next items in our roadmap.
If you have a more custom pipeline, we have a custom environment where you can deploy any custom code with specific package versions!
If you have a more custom pipeline, we have a custom environment where you can deploy any custom code with specific package versions!
I see that you accept models up to 1 GB. It seems the inference time might be high for models of this size on CPUs. Do you use GPUs to speed up inference for deep learning models ?
We don’t offer GPU accelerated inference for now. However it’s on our roadmap, sign up for inferrd to get updates!
Looks interesting! How about models that require dictionaries - e.g. tf-idf to convert text into a feature vector? Does it allow for some preprocessing?
Hi! If don’t need any pre processing we have built in support for all the major librairies.
In addition we support custom pre and post processor via custom environments! Simply write your inference code into a predict.py and we take care of the rest.
(Btw I am really into CML.dev, great idea)
In addition we support custom pre and post processor via custom environments! Simply write your inference code into a predict.py and we take care of the rest.
(Btw I am really into CML.dev, great idea)