Show HN: Fullstack ML – From Notebooks to Deployment(github.com)
github.com
Show HN: Fullstack ML – From Notebooks to Deployment
https://github.com/xadrianzetx/fullstack.ai
24 comments
I have absolutely no idea what this is, supposed to do, and what the demo shows.
To get ML models into production is a lot of work involving tests, retraining, model iteration/management and model serving (often at the edge where latency is low).
ML-Ops is as deep as data engineering but not as often talked about. The linked article discusses one example.
ML-Ops is as deep as data engineering but not as often talked about. The linked article discusses one example.
From what I can tell it's an example application to show how a machine learning project could be turned into a real world web app more or less from start to finish.
That's my take as well. yhat, datascience.com, and Domino Data Lab were the first players in this space. Domino has a webinar and trial project to see how these platforms approach this type of workflow. https://www.dominodatalab.com/resources/webinars/web-apps-in...
Yes it covers a bit more than what people tend to see in Machine Learning which is the model training part. From the code base it looks to cover:
1) feature preparation and model training as part of notebooks
2) Creation of a Flask API to interact with the trained models. This includes feature enrichment based on APIs input, so as to match the expected inputs by the model
3) Creation of an UI to interact with the API/Model
4) Setup of NGinx and docker to surface that application
Each part is covered in a minimal manner compared to most enterprise data products, and only cover 1 of the approach for end to end ML, but I think it does the job well to demonstrate the scope of work that is needed to put ML data products into production, and could be used as good introduction.
1) feature preparation and model training as part of notebooks
2) Creation of a Flask API to interact with the trained models. This includes feature enrichment based on APIs input, so as to match the expected inputs by the model
3) Creation of an UI to interact with the API/Model
4) Setup of NGinx and docker to surface that application
Each part is covered in a minimal manner compared to most enterprise data products, and only cover 1 of the approach for end to end ML, but I think it does the job well to demonstrate the scope of work that is needed to put ML data products into production, and could be used as good introduction.
From the Github: End-to-end machine learning project showing key aspects of developing and deploying real life ml driven application ... a web based ml driven bike trip advisor with trip time prediction
A few days ago there was this related post on /r/machinelearning
> Book on practical considerations of ML
https://reddit.com/r/MachineLearning/comments/cxu9d7/
> Book on practical considerations of ML
https://reddit.com/r/MachineLearning/comments/cxu9d7/
I came here because I thought this was about the ML programming language, aka Lisp with Types™, but it's about machine learning.
I made the same mistake most of the time but here with notebook in the title is was obviously Machine Learning.
I made the same mistake for this post. I'm not very familiar with Machine Learning so notebook doesn't give it away for me. But then throw in "fullstack" and it makes it seem, to me, that it is a front end/backend web framework written in some variant of ML
Love the X-Clacks-Overhead header for Terry Pratchett. https://xclacksoverhead.org/home/about
This is very, very similar to what Yhat (YC W15) did.
You dropped some code in your Python or R project and it would slurp everything in to one package and, for Python, run it in Flask in a docker container using their home-grown container orchestrator.
https://www.welcome.ai/tech/data-science/yhat/
You dropped some code in your Python or R project and it would slurp everything in to one package and, for Python, run it in Flask in a docker container using their home-grown container orchestrator.
https://www.welcome.ai/tech/data-science/yhat/
While the example here is definitely trivial, it would be nice to see some more documentation. Particularly around project structure, internals, and thought process.
Seeing e2e projects is definitely a huge plus, but without relevant commentary I don't see how it is consumable enough to help a large enough mass.
Seeing e2e projects is definitely a huge plus, but without relevant commentary I don't see how it is consumable enough to help a large enough mass.
This feels like it doesn't live up to its own hype because it doesn't really exhibit a production-quality deployment of a Python application but something closer to a dev server. It uses a single-threaded flask server behind a nginx reverse proxy without a WSGI layer to handle multiple requests, etc. Here is the nginx code:
Mind you, I think this is a neat little demo, but as something to put into production - or to teach other people how to put REST APIs into production - it falls short in its current state.
[1]: https://flask.palletsprojects.com/en/1.1.x/tutorial/deploy/
[2]: http://docs.gunicorn.org/en/stable/deploy.html#deploying-gun...
[3]: http://supervisord.org/
[4]: https://www.nginx.com/blog/rate-limiting-nginx/
[5]: http://nginx.org/en/docs/http/load_balancing.html
server {
listen 80;
server_name: localhost;
location / {
proxy_pass http://fai:4242/;
proxy_set_header Host "localhost";
}
}
flask:
build:
context: ./
dockerfile: Dockerfile
container_name: flask
networks:
net:
aliases:
- faip
if __name__ == '__main__':
APP.run(host='0.0.0.0', port=4242)
Mind you, I think this is a neat little demo, but as something to put into production - or to teach other people how to put REST APIs into production - it falls short in its current state.
[1]: https://flask.palletsprojects.com/en/1.1.x/tutorial/deploy/
[2]: http://docs.gunicorn.org/en/stable/deploy.html#deploying-gun...
[3]: http://supervisord.org/
[4]: https://www.nginx.com/blog/rate-limiting-nginx/
[5]: http://nginx.org/en/docs/http/load_balancing.html
I don't think supervisord is necessary when you're in a container environment. Just write a healthcheck to bring up a new container when something goes wrong
[deleted]
Looks really nice! Love to see it next to some Rust-like or Golang=like backend. What do you think?
this is freaking awesome!
was looking for something like this for a long time. Much appreciated!!!
[deleted]