Show HN: A flask app to make dashboards, easily(github.com)
github.com
Show HN: A flask app to make dashboards, easily
https://github.com/christabor/flask_jsondash
41 comments
I started using Freeboard (http://freeboard.github.io/freeboard/) in the latest project. It's also really snick and requires no front-end coding. Worth checking out!
Yes, I used Freeboard (https://github.com/Freeboard/freeboard) couple of years ago for an IoT store. It's highly customizable and has fantastic UI. It can be used with minimal effort. Highly recommended.
The OP's project is also pretty cool.
The OP's project is also pretty cool.
I like how they introduced pricing options.
But how does it compare to OP's project? Does it have more graphics choices?
But how does it compare to OP's project? Does it have more graphics choices?
You can find all supported types here:
https://github.com/christabor/flask_jsondash/blob/master/sch...
The big ones are d3, c3, and plotly. Those cover 99% of what you would need.
https://github.com/christabor/flask_jsondash/blob/master/sch...
The big ones are d3, c3, and plotly. Those cover 99% of what you would need.
Really cool, thanks for sharing. Any idea how much data this can ingest before the rendering gets slow? I know horsepower on the client will vary, and the definition of slow is subjective, but just ballpark. Or is that an exercise for the reader?
Quite a bit - its powered by c3.js - which is what we are using for AppEnlight - its quite powerful(and fast).
All great questions. I haven't done any genuine analysis outside of "this seems fine on my MacBook" which I fully admit is not worth much.
Check out the example app and endpoints app, then load the poorly.json and kitchen sink.json files into a new dashboard. I wouldn't expect anyone to make dashboards that extreme and they all render fine (we're talking upwards of 30 charts, many using webgl for 3d).
All that being said, I will continue to make this more optimized as I can.
Check out the example app and endpoints app, then load the poorly.json and kitchen sink.json files into a new dashboard. I wouldn't expect anyone to make dashboards that extreme and they all render fine (we're talking upwards of 30 charts, many using webgl for 3d).
All that being said, I will continue to make this more optimized as I can.
Reminds me of http://atlasboard.bitbucket.org. Nice job!
Does anyone have experience running Flask at scale? I used it once, for a small internal reporting service, and was shocked to find that it was designed to process only a single request at a time, per process. Is this a problem in the real world, or do decent caching rules and query design render it a non issue?
What do flask apps do when they want a live connection to the client, or need to serve a heavy (slow) request? Communicate with a Node websocket server over a queue, and share a database?
I don't mean to disparage Flask. Their goal is to make it simple to stand up site with minimal boilerplate or bloat, and they succeeded at that.
What do flask apps do when they want a live connection to the client, or need to serve a heavy (slow) request? Communicate with a Node websocket server over a queue, and share a database?
I don't mean to disparage Flask. Their goal is to make it simple to stand up site with minimal boilerplate or bloat, and they succeeded at that.
You would typically use something like nginx as a proxy, in conjunction with uwsgi for managing a number of workers, and then offload slow operations to a task queue via redis or something similar. Caching obviously helps if it's applicable and it's also easy to expand to multiple servers with a load balancer. Websockets are a little bit more complicated but definitely possible.
As a side note, the synchronous request processing is more a consequence of Python rather than flask itself. I've personally found that I build more scalable things in Python, compared to something like node, because it lends itself to scalable architecture decisions. You can do a lot of things in node that are super convenient when you have a single server but that require major changes when you expand.
As a side note, the synchronous request processing is more a consequence of Python rather than flask itself. I've personally found that I build more scalable things in Python, compared to something like node, because it lends itself to scalable architecture decisions. You can do a lot of things in node that are super convenient when you have a single server but that require major changes when you expand.
> I've personally found that I build more scalable things in Python, compared to something like node, because it lends itself to scalable architecture decisions. You can do a lot of things in node that are super convenient when you have a single server but that require major changes when you expand.
That's one of the major plus points of PHP too.
That's one of the major plus points of PHP too.
[deleted]
Do you mean something like this?
https://www.youtube.com/watch?v=tdIIJuPh3SI
https://github.com/miguelgrinberg/flack
https://speakerdeck.com/miguelgrinberg/flask-at-scale
https://www.youtube.com/watch?v=tdIIJuPh3SI
https://github.com/miguelgrinberg/flack
https://speakerdeck.com/miguelgrinberg/flask-at-scale
So, move expensive operations outside of the web processes. Looks like the Flask community has worked out good ways of doing that. Seems legit, answers my question :)
If you want async, use gevent or other async implementation and you'll get ~same as node. If you want parallellism, you'll need multiple processes. Every framework in whatever language should move expensive operations outside of request-response.
I ran a small site that was written in Flask and received couple of millions requests a day without any problems.
I used Nginx as a HTTP proxy and gunicorn for WSGI. It worked decently well, although it was pretty resources intensive (CPU, RAM).
Today you can use something like Caddy[0] and Gunicorn[1]
[0] https://caddyserver.com
[1] http://gunicorn.org
I used Nginx as a HTTP proxy and gunicorn for WSGI. It worked decently well, although it was pretty resources intensive (CPU, RAM).
Today you can use something like Caddy[0] and Gunicorn[1]
[0] https://caddyserver.com
[1] http://gunicorn.org
We use flask backed by apache mod_wsgi running in containers or VMs at work for our micro Services and it's quite performant. I've run stress tests with this setup using freeBSDs siege command (awesome btw) and 1000+ concurrency and it ran without a hiccup.
I can't say at what level it would break, but I'm confident it would perform as well as django (and it's designed to be thread safe so assuming your following the 12factor app principles And use the process model/attached storage for persistence running elsewhere, you can run multiple threads, apache vhost multi threading notwithstanding.
I can't say at what level it would break, but I'm confident it would perform as well as django (and it's designed to be thread safe so assuming your following the 12factor app principles And use the process model/attached storage for persistence running elsewhere, you can run multiple threads, apache vhost multi threading notwithstanding.
Wait, Caddy supports WSGI now, or you're replacing nginx with Caddy and using gunicorn for WSGI?
Using gunicorn (even with Nginx).
flasks builtin web server is for debug/development use. Run your flask apps under gunicorn, twisted web or any of the other supported servers in production.
Yeah, I was running it under gunicorn. But that still leaves you with a small number of concurrent connections. If you want to have long connections, for something like long polling or websockets, then being limited to one request per CPU core seems a little sketchy.
Mind you, it could just be that I take high concurrency for granted. I build most web stuff on Node or Clojure, but now that I think about it, apps that require long quiet connections are actually not the norm.
Mind you, it could just be that I take high concurrency for granted. I build most web stuff on Node or Clojure, but now that I think about it, apps that require long quiet connections are actually not the norm.
> for something like long polling or web sockets
Are you imagine this scenario or you are actually using Flask for long polling? What's your Flask websocket setup looks like?
FYI Flask internal does not stop you from using thousands of threads of greenlets to process concurrency. And web request-response model is embarrassingly parallel on multi-core. Just spawn one worker per core.
For a simple API service if you can not handle 3K rps per Flask instance you are doing it wrong.
Are you imagine this scenario or you are actually using Flask for long polling? What's your Flask websocket setup looks like?
FYI Flask internal does not stop you from using thousands of threads of greenlets to process concurrency. And web request-response model is embarrassingly parallel on multi-core. Just spawn one worker per core.
For a simple API service if you can not handle 3K rps per Flask instance you are doing it wrong.
Python does high concurrency just fine, this example just isn't using it.
from gevent import monkey
monkey.patch_all()I have used Flask at significant scale for REST api requests.
I haven't done web sockets with it - what work I've done with sockets has been in Node.
For building REST apis, it doesn't get easier, IMO. It's very straightforward, it scales well, and it's simplicity makes troubleshooting a reasonable task.
It's appropriateness for slow requests may be questionable, but before spending too much time on a more robust solution it's worth looking into why requests are slow in the first place. Cacheing, message queues, etc. are easy solutions to implement. Data store optimization is generally a quick and easy win that should be done regardless. When it gets to the point where python is the limiting factor it's easy to replace because the client facing front end is generally a proxy.
I haven't done web sockets with it - what work I've done with sockets has been in Node.
For building REST apis, it doesn't get easier, IMO. It's very straightforward, it scales well, and it's simplicity makes troubleshooting a reasonable task.
It's appropriateness for slow requests may be questionable, but before spending too much time on a more robust solution it's worth looking into why requests are slow in the first place. Cacheing, message queues, etc. are easy solutions to implement. Data store optimization is generally a quick and easy win that should be done regardless. When it gets to the point where python is the limiting factor it's easy to replace because the client facing front end is generally a proxy.
I run some flask microservices and the key has been to use gunicorn and nginx. It scales up very well. Its not as quick as some other Go microservices I have but Python has an advantage in terms of libraries for my use case. Flask is simple and that helps to keel things under control.
IIRC, flask deploys straight to AWS's elastic beanstalk (gunicorn) with minimal configuration, I even think it's given as an example in EB's docs. I've deployed an EB hosted flask instance (2 ec2 instances behind a load balancer with zero issues) in production.
Nobody ever uses the built-in Flask server for production. The common deployment pattern is to load the Flask app into a WSGI or asyncio serve, which will then handle requests with scalable threading/process models.
Look into uwsgi or gunicorn, and you'll never look back :)
Look into uwsgi or gunicorn, and you'll never look back :)
We've made the mistake of using Flask in production...now we're trying to witch to aiohttp as fast as possible.
WSGI is partly to blame.
A web technology derived from the '93 has no place in 2016.
WSGI is partly to blame.
A web technology derived from the '93 has no place in 2016.
The problem is not Flask. Try to understand the technologies you are working with. If you cannot make Flask work for you chances are you won't be able to make it work with aiohttp/whatever.
Flask is used in production in sites serving thousands of requests/second.
Flask is used in production in sites serving thousands of requests/second.
Can you say more? I'm interested.
Async flask can do quite a bit of work with a single thread.
Use gunicorn for a simple solution.
It is supposed to process a single request at a time. You want more, instantiate several workers.
This is great! My weekend is officially opened up. ;)
Flask can be used with gunicorn's gevent (green threads) worker type to scale to many more concurrent connections.
What's the difference between this and d3js?
D3.js is one of the many libraries used to generate these kinds of graphics. This project wraps up a bunch of different charting thingies into an easy-to-use combination.
Exactly, thanks!