Voilà turns Jupyter notebooks to standalone web applications(blog.jupyter.org)
blog.jupyter.org
Voilà turns Jupyter notebooks to standalone web applications
https://blog.jupyter.org/and-voil%C3%A0-f6a2c08a4a93
51 comments
...a lot of us are practically addicted to Jupyter Notebook Extensions [1] features and these aren't compatibly with Jupyter Lab and Jupyter Lab didn't add equivalent builtin features.
Once you get used to little things like (a) auto-TOC/outline of current doc in a sidebar (with highlight for current running cells and other goodies), (b) auto scroll to end of output, (c) browser notifications for end of long running calculations (so you can be alerted of stuff being done while browsing in other tabs) etc. etc. you really can't live without them!
It's this little things that matter 100x more than freely rearrangeable tab-grids and detachable outputs and terminals... these are cool but it's the other small details that make a bigger difference in productivity!
(As someone who also has some frontend web experience I was planning to maybe try and code these and contribute, but I never find the time, and last time I looked I couldn't find a good entry point to start learning about jupyter lab's frontend code...)
[1] https://github.com/ipython-contrib/jupyter_contrib_nbextensi...
Once you get used to little things like (a) auto-TOC/outline of current doc in a sidebar (with highlight for current running cells and other goodies), (b) auto scroll to end of output, (c) browser notifications for end of long running calculations (so you can be alerted of stuff being done while browsing in other tabs) etc. etc. you really can't live without them!
It's this little things that matter 100x more than freely rearrangeable tab-grids and detachable outputs and terminals... these are cool but it's the other small details that make a bigger difference in productivity!
(As someone who also has some frontend web experience I was planning to maybe try and code these and contribute, but I never find the time, and last time I looked I couldn't find a good entry point to start learning about jupyter lab's frontend code...)
[1] https://github.com/ipython-contrib/jupyter_contrib_nbextensi...
I'm not sure what you mean. Jupyter-lab has extensions, such as the toc?
Without knowing anything about what I'm saying, I think parents complaint is that Jupiter notebook extension don't work in labs, and that the labs extension system does not allow for as much customisation, leaving parent to stick to the notebook server.
They haven't released 1.0 yet though which is probably why it hasn't been advertised more openly. Hope it happens soon so we don't have to install pre-releases to run new versions of Altair anymore (edit: Altair + Jupyter Lab is an excellent combination though so it's worth the extra steps).
Not that I'm in this stuff everyday, but it's the first I've heard of it. Their own website mentions notebooks and not this on their main page: https://jupyter.org/
Edit: Nevermind. I see it is a separate project.
Edit: Nevermind. I see it is a separate project.
There's a somehow similar project called Panel: http://panel.pyviz.org/
However it's limited to Bokeh widgets whereas Voila is framework and language agnostic. As I understand it Panel allows to start Bokeh servers from notebooks.
Panel can use a Bokeh server but does not require it; it is equally happy communicating over Bokeh Server's or Jupyter's communication channels. Panel doesn't currently support using ipywidgets, nor does Voila currently support Bokeh plots or widgets, but the maintainers of both Panel and Voila have recently worked out mechanisms for using Panel or Bokeh objects in ipywidgets or using ipywidgets in Panels, which should be ready soon. I'm not sure of the details of how one would use Voila with other languages, but Panel can already show anything that has an interface to Python, e.g. an R ggplot visualization.
I'd also like to know how Voilà compares to Panel.
The recent Panel announcement stated that it was widget/graphing library agnostic and mentioned plotly and Altair explicitly. I have not used it myself though.
The recent Panel announcement stated that it was widget/graphing library agnostic and mentioned plotly and Altair explicitly. I have not used it myself though.
Panel and Voila attacked the same problem (moving easily between Jupyter and standalone server contexts) from completely opposite directions. This difference has some implications on their design and function.
Voila is based on ipywidgets running in Jupyter notebooks, and to make a standalone dashboard they had to create a standalone server that can securely execute Jupyter cells and display the results without allowing arbitrary code execution. The server is thus a work in progress, while the Jupyter integration was already solid.
Panel is based on Bokeh models, and because Bokeh models already had a full-blown standalone server, the task for Panel was to make Bokeh models (a) work seamlessly in Jupyter (previously they were awkward and limited in that context), (b) support other plotting libraries (by wrapping everything as a Bokeh model), and (c) have an API that's easier to use than native Bokeh for easy prototyping and design. Solid, secure server support came for free.
Once both libraries support each others models (soon!) and get a bit more polished, then off the top of my head, the main differences will be:
- Panel can use a Jupyter notebook, but it works equally well with a plain Python file; the notebook is just a source of Python code for it. Panel can be used fully even without Jupyter installed. Voila is closely tied to the Jupyter cell-based execution model, which is good or bad depending on your point of view. - Panel allows you to construct a "server view" of your notebook that can be completely independent of what is shown in the notebook, even though it is specified inside the notebook. I use that capability to have the same notebook go step by step analyzing a given dataset in detail, and then separately designate what should be shown in the server context, which is very handy; the boss sees one view, I work on another, and it all stays in sync. Voila works with notebook cell outputs only, and so I don't think it's possible to have fully different views of your data in the two contexts. - Panel supports building complex GUIs, with hierarchies of nested objects that each define their own editable parameters, without having to tie any of that code to Jupyter, Bokeh, or any other GUI or plotting system. This approach is really important for building large, complex codebases (e.g. simulators or data-analysis systems) that sometimes are used in dashboards, sometimes in notebooks, sometimes in batch runs, and sometimes on e.g. large remote computing systems.
I'm sure there are lots more differences, but that's enough for now!
Voila is based on ipywidgets running in Jupyter notebooks, and to make a standalone dashboard they had to create a standalone server that can securely execute Jupyter cells and display the results without allowing arbitrary code execution. The server is thus a work in progress, while the Jupyter integration was already solid.
Panel is based on Bokeh models, and because Bokeh models already had a full-blown standalone server, the task for Panel was to make Bokeh models (a) work seamlessly in Jupyter (previously they were awkward and limited in that context), (b) support other plotting libraries (by wrapping everything as a Bokeh model), and (c) have an API that's easier to use than native Bokeh for easy prototyping and design. Solid, secure server support came for free.
Once both libraries support each others models (soon!) and get a bit more polished, then off the top of my head, the main differences will be:
- Panel can use a Jupyter notebook, but it works equally well with a plain Python file; the notebook is just a source of Python code for it. Panel can be used fully even without Jupyter installed. Voila is closely tied to the Jupyter cell-based execution model, which is good or bad depending on your point of view. - Panel allows you to construct a "server view" of your notebook that can be completely independent of what is shown in the notebook, even though it is specified inside the notebook. I use that capability to have the same notebook go step by step analyzing a given dataset in detail, and then separately designate what should be shown in the server context, which is very handy; the boss sees one view, I work on another, and it all stays in sync. Voila works with notebook cell outputs only, and so I don't think it's possible to have fully different views of your data in the two contexts. - Panel supports building complex GUIs, with hierarchies of nested objects that each define their own editable parameters, without having to tie any of that code to Jupyter, Bokeh, or any other GUI or plotting system. This approach is really important for building large, complex codebases (e.g. simulators or data-analysis systems) that sometimes are used in dashboards, sometimes in notebooks, sometimes in batch runs, and sometimes on e.g. large remote computing systems.
I'm sure there are lots more differences, but that's enough for now!
Last time I checked you could indeed plug different graphing libraries but Panel comes with its own widgets API which a set of wrappers around Bokeh widgets. So yes for Plotly and Altair but no ipywigets.
If this interests you, check out Papermill and NTeract:
https://github.com/nteract/papermill
https://github.com/nteract
https://github.com/nteract/papermill
https://github.com/nteract
Any real-life examples? The blog only seems to contain animated gifs.
http://voila-gallery.org is their gallery of examples. It's built using jupyter hub, which spawns a python kernel for every new user.
UX is not great, you have to go to http://voila-gallery.org/hub/logout after you've started an example dashboard to be able to see another.
UX is not great, you have to go to http://voila-gallery.org/hub/logout after you've started an example dashboard to be able to see another.
This is huge - This basically allows your analysts and data scientists to have a company wide impact by productionizing your code.
Plotly dash tried to do this, but was a bit weird to use.
Plotly dash tried to do this, but was a bit weird to use.
I've been using the Jupyter AppMode plugin for a few years, it has been very useful for internal tools.
Voilà looks similar, with more sheen, it's going to enable true rapid SPA development.
Voila author here: a major difference with AppMode is that AppMode is merely hiding the notebook interface, but still allows for arbitrary code execution.
Voila is meant for producing secure web apps.
Voila is meant for producing secure web apps.
hi - its very interesting that you talk about security. what's the deployment story here ?
in fact, now that i think of it - can you generate an output folder with a Dockerfile ? so a mere docker build will produce a running docker compatible "build" folder ?
Also, can you have a mode where you hide the source code ? i just get the output data ? for world viewable dashboards.
in fact, now that i think of it - can you generate an output folder with a Dockerfile ? so a mere docker build will produce a running docker compatible "build" folder ?
Also, can you have a mode where you hide the source code ? i just get the output data ? for world viewable dashboards.
There are some details about this in the blog post.
- no execution request is sent by the front-end.
- code is stripped out unless explicitely stated otherwise.
For deployment, we are working on several scenarios around JupyterHub.
your blog post still displays the code. Not sure if that can be turned off as well.
Please consider the request for a Docker based output build in addition to whatever you do for jupyterhub
Please consider the request for a Docker based output build in addition to whatever you do for jupyterhub
From the blog post:
> with --strip_sources=False, input cells will be included in the resulting web application (as read-only pygment snippets).
So the sources are presumably not there by default.
> with --strip_sources=False, input cells will be included in the resulting web application (as read-only pygment snippets).
So the sources are presumably not there by default.
I still feel like R/Shiny is miles ahead here, but Voilà looks like a step in the right direction
Shiny is okay as a way to share simple analytical apps but it's syntax and performance are atrocious.
I wonder if this works with Jupyterhub-spawned notebooks, definitely want to try.
This is awesome!
Is there a way to host these somewhere easily?
Like a git push of a notebook to a repo and you get voila powered renders (and I assume some kind of automagic lightweight kernel behind the scenes which executes the code if required).
It reminds me of DeepNote[0] though that is geared towards the producers of notebooks rather than consumers.
[0]: https://beta.deepnote.org/
Is there a way to host these somewhere easily?
Like a git push of a notebook to a repo and you get voila powered renders (and I assume some kind of automagic lightweight kernel behind the scenes which executes the code if required).
It reminds me of DeepNote[0] though that is geared towards the producers of notebooks rather than consumers.
[0]: https://beta.deepnote.org/
At Kyso [1] we let you do this (I'm one of the founders) - you can run Jupyterlab online and start servers inside Jupyterlab that you can share with the world (Jupyterlab will run on instance-id.cloud.kyso.io).
You can install viola and then just run it on port 8000 from within the jupyterlab terminal and you can share the url with anyone (it will be instance-id-8000.cloud.kyso.io)
We're actually thinking of supporting Viola natively in the next few weeks, what kind of app do you want to run?
[1] https://kyso.io
You can install viola and then just run it on port 8000 from within the jupyterlab terminal and you can share the url with anyone (it will be instance-id-8000.cloud.kyso.io)
We're actually thinking of supporting Viola natively in the next few weeks, what kind of app do you want to run?
[1] https://kyso.io
I'm going to try this out at work, using the following approach:
- gitlab repo that hosts the notebook
- gitlab cicd configured to build a docker container running Voila, on every push to dev or master branch
- automated deployment of freshly built containers to kubernetes
Don't know if there's easier solutions, this will require some setting up
Don't know if there's easier solutions, this will require some setting up
why just jupyter? why not pure python as well?
What do you mean? Jupyter notebooks are usually pure Python apart from (sometimes) interactive widgets, which have Javascript behind them since it's in a browser. I don't think this is much use without widgets, so maybe you mean something else?
I mean why can't I write python outside of jupyter and generate these dashboards? Or can I?
so, it has come to that
If it spawns a new kernel for each request it will be very slow no?
Or at least each page load will be slow and memory intensive for the server
It is a bit slow, but there's some exploration of keeping a hot pool of prexecuted kernels available. At the end of the day it's still 1 user 1 kernel though, and this is important for security/isolation reasons.
Is there any work on bringing Python into the browser like with JavaScript? At the beginning of the year, I was looking at the best way of making an interactive textbook either with Python (requiring a backend it seemed), or with JavaScript via D3. You can still use Jupyter Notebooks with JS; there's a library called BeakerX by Two Sigma that does this.
I went the D3 route but it was hard to stay motivated when most of the time I was just learning how d3 and JavaScript worked instead of the interactability part itself. Closet project I found is Pyodide which was posted here [1]. Wondering if I should wait for someone to come up with a magicul implementation before restarting since this area seems to be experimental (?).
[1] https://news.ycombinator.com/item?id=19677721
I went the D3 route but it was hard to stay motivated when most of the time I was just learning how d3 and JavaScript worked instead of the interactability part itself. Closet project I found is Pyodide which was posted here [1]. Wondering if I should wait for someone to come up with a magicul implementation before restarting since this area seems to be experimental (?).
[1] https://news.ycombinator.com/item?id=19677721
There is indeed!
I'll start with the one I founded: Anvil, which is an end-to-end Web dev system with Python (plus a drag-and-drop UI builder, and lots more good stuff): https://anvil.works
Here's our version of how to turn a Jupyter notebook into a web app: https://anvil.works/blog/jupyter-notebook-to-web-app
(tl;dr there's a library you can `pip install`. It makes a websocket connection to Anvil, and then your code - wherever it's running, including a Jupyter notebook - can do everything you can do in our serverless environment, such as handling requests from browser code.)
But of course, we are neither the first nor the last to think Python in the browser is a good idea. A colleague of mine did a survey and comparison of the major open source implementations (including PyIodide, which you mention, and Skulpt, which is what Anvil uses):
https://pythontips.com/2019/05/22/running-python-in-the-brow...
I'll start with the one I founded: Anvil, which is an end-to-end Web dev system with Python (plus a drag-and-drop UI builder, and lots more good stuff): https://anvil.works
Here's our version of how to turn a Jupyter notebook into a web app: https://anvil.works/blog/jupyter-notebook-to-web-app
(tl;dr there's a library you can `pip install`. It makes a websocket connection to Anvil, and then your code - wherever it's running, including a Jupyter notebook - can do everything you can do in our serverless environment, such as handling requests from browser code.)
But of course, we are neither the first nor the last to think Python in the browser is a good idea. A colleague of mine did a survey and comparison of the major open source implementations (including PyIodide, which you mention, and Skulpt, which is what Anvil uses):
https://pythontips.com/2019/05/22/running-python-in-the-brow...
The latest podcast from podcast.__init__ is an interview of Meredydd about anvil
https://www.pythonpodcast.com/anvil-web-application-developm...
https://www.pythonpodcast.com/anvil-web-application-developm...
[deleted]
You can check out: https://epiphany.pub/post?refId=d40c08258ef10f2fe9acff4ab738...
Sorry, i am not a native speaker.
What is a 'Closet project'? Do you mean it is not open-source?
Check out Metatron Discovery(https://metatron.app) They have jupyter notebook connection feature. And it is open source. Here is the github repo: https://github.com/metatron-app/metatron-discovery
For years I thought Jupyter Notebook was hardware, a specific laptop made for scientists. Am I alone thinking it's a confusing name?
Yes, it's very rare to have the term notebook mixed up in this case. Unless you've only heard of the term "Jupyter Notebook" and have zero context. By the same logic Python can get confused with snakes.
Yes
No you are not alone. I have no crossover with the Python world aside from hearing about it on HN and Reddit. I too assumed it was like a Bloomberg terminal for data scientists and was surprised to discover it’s just a virtual environment wrapper you install. Poor name for the project if you are marketing it outside of your domain which I doubt they care much about.
It's not a virtual environment wrapper... It's closer to an environment for coding in the browser.
https://blog.jupyter.org/jupyterlab-is-ready-for-users-5a6f0...