The Python scientific stack, compiled to WebAssembly(github.com)
github.com
The Python scientific stack, compiled to WebAssembly
https://github.com/iodide-project/pyodide
53 comments
> "Part of an effort to make a Jupytr notebook-like environment without relying on server side code."
So there won't be any server side code, you'd just be downloading the entire Python stack and dependencies every time you want to use this through your browser. This time it will be WASM though, instead of x86/x64/pyc.
That we can, doesn't mean that we should.
So there won't be any server side code, you'd just be downloading the entire Python stack and dependencies every time you want to use this through your browser. This time it will be WASM though, instead of x86/x64/pyc.
That we can, doesn't mean that we should.
I guess we need an in-browser package management like npm?
The plan is to build something based on conda.
> I guess we need an in-browser package management like npm?
It exists and it's called bower.
It exists and it's called bower.
That would be dumb, so why assume that would happen? You don’t download every picture on every refresh.
It would work on a Chromebook and a iPad also.
And yes I could pay for a Jupiter notebook hosted on some instance but I dont want to.
And yes I could pay for a Jupiter notebook hosted on some instance but I dont want to.
Maybe, but it might require more RAM and CPU horsepower than they provide in order to be usable.
Well, you'd download the entire stack indeed - but not every time. Your browser does caching too.
Plus, if it’s packaged as a PWA, it can run offline and seamlessly update the runtime in the background using a Web Worker.
It doesn't share the cache between the sites though. A local installation has only one runtime.
Absolutely; it's pretty much like installing an application that includes all its dependencies. Which, interestingly, is also the direction installed apps are moving towards somewhat, at least on Linux.
I can think of several scenarios where this could be useful.
There's an advantage even if you're running it locally. Now you can just load up some static HTML + JS files in your browser instead of spinning up a server and then pointing your browser at localhost.
As long as it's framed properly it can be acceptable. Ie. You get to a page with details and a "load environment" button with progress bars and all.
Of course when it's like 50-100MB you wouldn't use it in some cases. But once cached you're golden for your repeat visitor audience.
Of course when it's like 50-100MB you wouldn't use it in some cases. But once cached you're golden for your repeat visitor audience.
I benchmarked this recently. It comes in at 4-10x slower than cpython for random typical use cases on my machine, which is not too shabby at all. The author also states that numpy is slower than it should be due to lack of support for BLAS at this time.
The real impressive feat to me isn't hosting jupyter in the browser. It's access to a reasonably fast implementation of numpy in the browser which smokes native JS code for homogeneous array operations. I would love to see a minimalistic WASM implementation of numpy that can seamlessly interop with normal JS. Such a library would open up all sorts of possibilities that aren't currently feasible due to perf reasons.
The real impressive feat to me isn't hosting jupyter in the browser. It's access to a reasonably fast implementation of numpy in the browser which smokes native JS code for homogeneous array operations. I would love to see a minimalistic WASM implementation of numpy that can seamlessly interop with normal JS. Such a library would open up all sorts of possibilities that aren't currently feasible due to perf reasons.
I can't believe I'm saying this but it would be neat to have a browser native BLAS, or absent that perhaps cross-compiled to WASM from a RUST implementation or something! Makes me think of all the demos like the tensorflow playground (dunno if all that happens in the browser or not).
People are working on the pure JS version of BLAS: https://github.com/jacobbogers/blasjs . Hopefully that work leads to best practices and a standard
Numpy is fast thanks to having much of its code compiled as a native library.
They were purposefully working to reduce abstraction, since performance mattered from day one.
WASM is a huge step backwards in that regard. If you care about performance, just install Python on your box and "pip install numpy". Do you absolutely have to have it come in a browser now?
They were purposefully working to reduce abstraction, since performance mattered from day one.
WASM is a huge step backwards in that regard. If you care about performance, just install Python on your box and "pip install numpy". Do you absolutely have to have it come in a browser now?
Interesting, how do you derive at "4-10x slower"?
Running the exported notebook here takes
time py3 python.py real 0m0.145s
chrome: load html: 2sec run html: 13sec!
That's not 10 times slower, but a 100 times slower!
Still good start.
Running the exported notebook here takes
time py3 python.py real 0m0.145s
chrome: load html: 2sec run html: 13sec!
That's not 10 times slower, but a 100 times slower!
Still good start.
[deleted]
I have used Weblas "GPU Powered BLAS for Browsers" for matrix multiply and it smokes!
https://github.com/waylonflinn/weblas
Did you use js typed arrays (e.g. Uint32Array) to benchmark large array operations or just Array?
This is great. Many times I wanted to showcase to colleagues who don't use Python how things would work in Python, but they don't have either Python or Jupyter installed. So I had to run Jupyter server in my own environment and give them access to it. This is quite an inconvenience as well as a security risk. Having a completely server-less Python/Jupyter environment should make this kind of showcasing much easier.
Why not show them the basics in https://repl.it/ or something similar?
You could also use a Jupyter notebook uploaded to colab.research.google.com
I'll chime in to mention notebooks.azure.com as well.
(Typical disclaimer, I'm a dev on the azure notebooks team; we just try to solve exactly the scenario GP was asking so I would be remiss to not throw our hat in the ring)
(Typical disclaimer, I'm a dev on the azure notebooks team; we just try to solve exactly the scenario GP was asking so I would be remiss to not throw our hat in the ring)
Do they have Docker? Jupyter maintains a few docker images with various parts of the scientific stack.
How and why would they have docker and not python. Docker's like 10 rungs up the skill ladder.
If they're working with a different stack that includes docker?
Coming soon [...] Plotting using D3 from Python
Wouldn't Bokeh make more sense? I'd be very interested in a serverless Bokeh that didn't force me to ditch Python, for example.
Wouldn't Bokeh make more sense? I'd be very interested in a serverless Bokeh that didn't force me to ditch Python, for example.
Bokeh could probably be done, but most of Bokeh's client/server complexity becomes unnecessary in this execution model.
More recently, pyodide has grown full matplotlib support. See, for example, https://iodide.io/pyodide-demo/matplotlib-sideload.html?side...
More recently, pyodide has grown full matplotlib support. See, for example, https://iodide.io/pyodide-demo/matplotlib-sideload.html?side...
What happens to all of the C code in Numpy, and the calls to external libraries like BLAS?
Python C extensions like NumPy are compiled into shared libraries, and loaded using dlopen() etc. normally by the main compiled Python runtime. (The only special thing here is that all those components are compiled to WebAssembly so it can run on the Web.)
In particular, does this mean Javascript has access to Numpy? Sounds unbelievable, but would be extremely cool.
Yep. You can "import" python objects over to the Javascript side and start using them from there. It's not as pleasant as working with Numpy in Python, of course, due to lack of operator overloading, of course.
C can compile to WASM, but I have no clue about external tools like BLAS. I'd assume that things like Numpy are able to work without tools like BLAS, but you might also be able to compile the essential ones to WASM as well.
I do a lot of work with Skulpt, which included adding (somewhat superficial) support for MatPlotLib. You can make line plots, scatter plots, and histograms. It's a pretty cool system, though I will say developing in it isn't always very fun :)
The goal of http://www.graalvm.org/docs/why-graal/#for-ruby-r-or-python is to run scipy.
Is there a performance hit compared to "native" Python?
There's a chart and some discussion here: http://droettboom.com/blog/2018/04/04/python-in-the-browser/ (and some followup at http://droettboom.com/blog/2018/04/11/profiling-webassembly/ )
TL;DR: pyodide in Firefox was slower than cpython.
TL;DR: pyodide in Firefox was slower than cpython.
Thanks. Note that the author of that post said:
> UPDATE 2018-04-11: My hunch was wrong, and I was able to get to the bottom of the root cause and significantly speed up these benchmarks.
And there's an updated graph here:
http://droettboom.com/blog/2018/04/11/profiling-webassembly/
> UPDATE 2018-04-11: My hunch was wrong, and I was able to get to the bottom of the root cause and significantly speed up these benchmarks.
And there's an updated graph here:
http://droettboom.com/blog/2018/04/11/profiling-webassembly/
Without a doubt.
Huh I am getting
``` TypeError: window[s] is undefined ```
with firefox.
``` TypeError: window[s] is undefined ```
with firefox.
File a bug over at https://github.com/iodide-project/pyodide/
I'd love to get to the bottom of that.
I'd love to get to the bottom of that.
https://github.com/iodide-project/iodide