Run code inline in Atom using Jupyter kernels(github.com)
github.com
Run code inline in Atom using Jupyter kernels
https://github.com/nteract/hydrogen
55 comments
If you try hard enough, you could tie any new development to the past.
Lisp may have been the first language with a REPL (as far as I'm aware it was either Lisp or Forth), but REPLs have evolved over the years, and plenty of modern programmers were familiar with REPLs before Bret Victor's views on instant feedback became popular.
Lisp may have been the first language with a REPL (as far as I'm aware it was either Lisp or Forth), but REPLs have evolved over the years, and plenty of modern programmers were familiar with REPLs before Bret Victor's views on instant feedback became popular.
The way one interacts with the Xerox PARC and Lisp Machines environments is much richer experience than a plain REPL.
It was a graphical REPL that could embedd live objects, which were still active to further interactive changes.
Also the ability to have the whole OS exposed via the REPLs is hardly available in modern systems.
Not alk REPLs are made alike.
It was a graphical REPL that could embedd live objects, which were still active to further interactive changes.
Also the ability to have the whole OS exposed via the REPLs is hardly available in modern systems.
Not alk REPLs are made alike.
My point is that REPLs (and other dynamic programming tools) have been in constant use since they were invented, and although respect can be paid to the originators of the approach, we're not really experiencing a rediscovery of those first ideas, but rather we're experiencing an evolution of the ideas they started. In other words, there's been plenty of activity in this area between when they were invented and now, let's not discount that activity.
As for modern systems not allowing the whole OS to be exposed via a REPL, there is at least one actively developed OS that allows this, and that's Oberon. The most recent revision of the Oberon OS was from 2013.
https://en.wikipedia.org/wiki/Oberon_(operating_system)
As for modern systems not allowing the whole OS to be exposed via a REPL, there is at least one actively developed OS that allows this, and that's Oberon. The most recent revision of the Oberon OS was from 2013.
https://en.wikipedia.org/wiki/Oberon_(operating_system)
I was an Oberon user for a while, back when it was still being developed.
The version 3 with the Gadgets framework was quite cool.
The only modern OS environments that come close to these experiences are Windows with .NET/VS/Powershell and OS X with Playgrounds, the OS X Frameworks and Applescript.
Don't get me wrong, it is great what they are doing and I for one enjoy Notebooks, Playgrounds, modern IDEs, tooling that allows for both for JIT and AOT, while offering safety.
Just a bit nostalgic how things could have been different.
For example, when Google brought out ChromeOS and Dart, I thought there was a perfect opportunity to have a similar experience, but that is not how it turned out to be.
The version 3 with the Gadgets framework was quite cool.
The only modern OS environments that come close to these experiences are Windows with .NET/VS/Powershell and OS X with Playgrounds, the OS X Frameworks and Applescript.
Don't get me wrong, it is great what they are doing and I for one enjoy Notebooks, Playgrounds, modern IDEs, tooling that allows for both for JIT and AOT, while offering safety.
Just a bit nostalgic how things could have been different.
For example, when Google brought out ChromeOS and Dart, I thought there was a perfect opportunity to have a similar experience, but that is not how it turned out to be.
That REPL is called a command line. Sure, it's not the same, but then it does not run on a virtual machine either (OK, LISP isn't necessarily, either).
You can still hook up a remote debugger to your kernel, I guess, but that's not quite the same then is it.
You can still hook up a remote debugger to your kernel, I guess, but that's not quite the same then is it.
That is not even close.
EDIT: I guess I should expand a bit on that answer.
Take Oberon, which like Java has the misfortune of using the same name for the language and the respective OS.
It was based on Niklaus Wirth experience on using the Xerox Star system with Mesa/Cedar environment.
The original Native Oberon used as only executable format shared libraries. Depending on the OS variant, they were either fully AOT, or JITted on load.
There was a convention on procedures that could be made visible to the REPL or mouse commands and what type of input they would act upon.
So you could either do module.procedure parameters at the CLI, select some text at a given open window or embedded object (think OLE) and select a module.procedure at the toolbox of loaded modules.
Very simple set of primitives exposing the whole OS, both CLI window and other GUI elements as programmable elements.
You see a bit of this way of working on the ACME editor used in Plan 9, specially later on Inferno with Limbo.
EDIT: I guess I should expand a bit on that answer.
Take Oberon, which like Java has the misfortune of using the same name for the language and the respective OS.
It was based on Niklaus Wirth experience on using the Xerox Star system with Mesa/Cedar environment.
The original Native Oberon used as only executable format shared libraries. Depending on the OS variant, they were either fully AOT, or JITted on load.
There was a convention on procedures that could be made visible to the REPL or mouse commands and what type of input they would act upon.
So you could either do module.procedure parameters at the CLI, select some text at a given open window or embedded object (think OLE) and select a module.procedure at the toolbox of loaded modules.
Very simple set of primitives exposing the whole OS, both CLI window and other GUI elements as programmable elements.
You see a bit of this way of working on the ACME editor used in Plan 9, specially later on Inferno with Limbo.
> That is not even close.
Stream processing should be equally mighty as list processing. What about system level access?
Edit:
> both CLI window and other GUI elements as programmable
Sure, X sucks, but it supports that. X might be infamously complicated, but there's a security hole, either way.
Stream processing should be equally mighty as list processing. What about system level access?
Edit:
> both CLI window and other GUI elements as programmable
Sure, X sucks, but it supports that. X might be infamously complicated, but there's a security hole, either way.
Text stream processing is not as powerful as data structures and functional programming composition.
How do you in X protocol select an object that represents a table in a word document and add an extra column with the standard deviation of the other rows?
Just a possible example of the type of composition that was possible.
How do you in X protocol select an object that represents a table in a word document and add an extra column with the standard deviation of the other rows?
Just a possible example of the type of composition that was possible.
nowadays, theres possibly a related System-D feature ;)
Yet, if you do know Lisp, you realize that not too many languages feature a true REPL with the R not just reading in Text that gets evaluated, but actually being a Reader.
So, we actually did lose some things with the new and shiny REPLs - Macros.
No debating your statement that there are other good programming environments with other shiny features^^
So, we actually did lose some things with the new and shiny REPLs - Macros.
No debating your statement that there are other good programming environments with other shiny features^^
This is really cool and I think over time, Atom is going to become really compelling once their org mode implementation or its moral equivalent gets some legs.
I've used the Jupyter emacs integration (https://github.com/millejoh/emacs-ipython-notebook) but in the end, I stuck with org mode.
For an example of what you can do in org mode (in emacs), here is part of a "dashboard" I created for one of my products: https://i.imgur.com/QUFbIBk.png
And an example of how I design: https://i.imgur.com/MeRYn1L.png
You can essentially use org-babel to recreate an approximation to the notebook environment which si cool
I've used the Jupyter emacs integration (https://github.com/millejoh/emacs-ipython-notebook) but in the end, I stuck with org mode.
For an example of what you can do in org mode (in emacs), here is part of a "dashboard" I created for one of my products: https://i.imgur.com/QUFbIBk.png
And an example of how I design: https://i.imgur.com/MeRYn1L.png
You can essentially use org-babel to recreate an approximation to the notebook environment which si cool
Org mode rocks! And with ob-ipython you basically get same functionality as the notebook...
https://github.com/gregsexton/ob-ipython
https://github.com/gregsexton/ob-ipython
Ah yes, that's the one I tried.
I haven't tried either, but atom-notebook (https://github.com/jupyter/atom-notebook) looks more promising to me. I hope they can integrate it well.
With hydrogen, are there cells? What happens if I want to rerun everything? Will I get no output except the output of the last line?
With hydrogen, are there cells? What happens if I want to rerun everything? Will I get no output except the output of the last line?
I think they both have a place - atom-notebook (like the web version) for more structured use with cells, and hydrogen for ad-hoc ("Will this work?") anonymous execution while you're editing.
> Will I get no output except the output of the last line?
If the return value is one line, it prints neatly after the executed block. If it's more than one it inserts a little pane with the results, with options to copy the data or open it in a new buffer.
> Will I get no output except the output of the last line?
If the return value is one line, it prints neatly after the executed block. If it's more than one it inserts a little pane with the results, with options to copy the data or open it in a new buffer.
I agree, it looks like Hydrogen is a nice replacement for the REPL, whereas notebooks are a completely different workflow.
Many times I wanted to switch from Sublime Text to Atom, but I always went back for some reason (slowness, not remembering open files upon restart, etc). Maybe this will make me swich for good.
FYI: Sublime Text's plugin API has recently (v. 3118) been updated with some cool new features relating to inline minihtml, which makes something like this more feasible.
While it doesn't currently exist, someone might implement something like this for Sublime Text in the future.
While it doesn't currently exist, someone might implement something like this for Sublime Text in the future.
The fact that Atom was free and Open Source is mostly why I switched. You'll be happy to find that it does restore your last session as default now (add in the Project Manager plugin to make switching between projects easier too). Personally I haven't noticed any speed issues but YMMV.
I regularly about few hundred MB files in Sublime. The only alternative is VIM with plugins.
Jupyter released a cool computational environment(basically an IDE like Rstudio Web) - https://github.com/jupyter/jupyterlab
There is already discussion about Jupyter lab here on the front page - https://news.ycombinator.com/item?id=12098180
There is already discussion about Jupyter lab here on the front page - https://news.ycombinator.com/item?id=12098180
What Jupyter Notebook is missing is a good text/code editor and what good text/code editors are missing is Jupyter Notebook like write and execute.
This is the first example I've seen of the power that using Electron in Atom can give, and it makes me super exited!
This is the first example I've seen of the power that using Electron in Atom can give, and it makes me super exited!
What's missing from Edwin or Emacs with SLIME? Support for Python, obviously, but—that's solvable. Is there a more fluid execution environment in Jupyter?
Have you tried PyCharm with jupyter notebook?
No, I didn't know they worked together. I'll have a look!
If you're spending a lot of time doing data science and if you might be missing R studio for Python I work for Yhat and we make a product called Rodeo http://rodeo.yhat.com/
What are the major benefits of Rodeo compared to Spyder?
For any of those who may be doing Elixir development, https://github.com/pprzetacznik/IElixir works perfectly with this.
The MPW Shell (the Macintosh Programmer's Workshop) had a feature like this; every editor window was also a shell command window. You could select some text and run it as a shell command.
I regularly included build steps as comments in the first few lines of C and Pascal sources. No makefile necessary.
You could also arrange for code to run on actions like 'cd' (to set up environments, for instance; your editor window could become a context of readily selectable commands based on the directory you were in).
Oh, and 1987.
I regularly included build steps as comments in the first few lines of C and Pascal sources. No makefile necessary.
You could also arrange for code to run on actions like 'cd' (to set up environments, for instance; your editor window could become a context of readily selectable commands based on the directory you were in).
Oh, and 1987.
Advantages of this over running a repl in an Emacs buffer and evaluating code that way? I've never tried Atom.
Honestly, I would say the main advantage is you don't have to learn Emacs.
Emacs is a very polarising environment and there are many programmers for whom it is simply not the right code editor.
Emacs is a very polarising environment and there are many programmers for whom it is simply not the right code editor.
Same question, but s/running repl in an Emacs buffer/authoring in org-babel/ - I have a hard time seeing Jupyter and friends as anything other than a subset of what people have been using org-babel to do for years, and I'm not really sure whether that's an accurate perception or not.
Thank you for the babel reference, it looks interesting. From what I can tell, it's pretty similar. In a notebook I'm directly editing the code and seeing the results immediately, instead of switching back and forth between emacs and a browser window. That looks like a pretty big win for interactive development. Is there any babel feature that you think is missing in Jupyter?
Babel does the same thing; you just press Control-C twice while the cursor is in a code block, and it's evaluated and the results inserted immediately below the code block. (If there are already results there, they're replaced by default, but you can opt to have new ones appended instead.) It's no less interactive than Jupyter; it's just interactive in the paradigm of an Emacs buffer instead of a browser window.
A major feature Babel offers, which Jupyter doesn't seem to, is passing data among code blocks in different languages. Any language Babel supports [1] can receive results from any other in the receiving language's native idiom, so there's no need to marshal temporary files or what-have-you in order to exchange data among blocks - Babel takes care of that for you. In a case where multiple transformations are necessary to get from source data to results, this makes it very easy to express each in a language well suited to it, without having to spend a bunch of time and mental energy on menial file shuffling.
The same is true for most other representations of data that Org mode supports. In particular, Org tables, which themselves have many features of spreadsheets, can be sourced in code blocks in the same way as the results of other code blocks, and a code block can generate an Org table as a result. These tables can be imported and exported as CSV, too. My work is professional rather than academic, but I can imagine a lot of cases in which this capability would be useful in the latter context, and Jupyter doesn't seem to offer it. (It's hard to tell, because Jupyter's documentation seems pretty thin overall - not something which can be said of Org mode.)
On the whole, Jupyter looks like a pretty nifty tool, and one which would be accessible to people who don't already use Emacs. I'm not sure how I would feel about recommending Emacs to someone just for Org mode, although I've heard several people say they picked up Emacs for precisely that reason and have stuck with it. But, if you already use Emacs and don't have contextual reasons (such as everyone else around you using Jupyter) to choose one tool over another, I wouldn't hesitate to recommend investigating Org mode first, because it is very nearly a strict superset of Jupyter. (It'll be a few years before you can easily embed a video in an Emacs buffer; right now you'd need to use a pre-release version with some extra compile options.)
[1] http://orgmode.org/worg/org-contrib/babel/languages.html
A major feature Babel offers, which Jupyter doesn't seem to, is passing data among code blocks in different languages. Any language Babel supports [1] can receive results from any other in the receiving language's native idiom, so there's no need to marshal temporary files or what-have-you in order to exchange data among blocks - Babel takes care of that for you. In a case where multiple transformations are necessary to get from source data to results, this makes it very easy to express each in a language well suited to it, without having to spend a bunch of time and mental energy on menial file shuffling.
The same is true for most other representations of data that Org mode supports. In particular, Org tables, which themselves have many features of spreadsheets, can be sourced in code blocks in the same way as the results of other code blocks, and a code block can generate an Org table as a result. These tables can be imported and exported as CSV, too. My work is professional rather than academic, but I can imagine a lot of cases in which this capability would be useful in the latter context, and Jupyter doesn't seem to offer it. (It's hard to tell, because Jupyter's documentation seems pretty thin overall - not something which can be said of Org mode.)
On the whole, Jupyter looks like a pretty nifty tool, and one which would be accessible to people who don't already use Emacs. I'm not sure how I would feel about recommending Emacs to someone just for Org mode, although I've heard several people say they picked up Emacs for precisely that reason and have stuck with it. But, if you already use Emacs and don't have contextual reasons (such as everyone else around you using Jupyter) to choose one tool over another, I wouldn't hesitate to recommend investigating Org mode first, because it is very nearly a strict superset of Jupyter. (It'll be a few years before you can easily embed a video in an Emacs buffer; right now you'd need to use a pre-release version with some extra compile options.)
[1] http://orgmode.org/worg/org-contrib/babel/languages.html
That sounds really nice, thank you for taking the time to write all this. I do use Emacs for editing, and I miss all the features SLIME offered for Common Lisp. Unfortunately, I'm using Julia now, and while the language is really getting there, tooling support is minimal outside of Jupyter and Atom, so I wouldn't bet on Babel.
I should learn org-mode though. I've heard so much praise for it!
I should learn org-mode though. I've heard so much praise for it!
I have been using Julia in emacs, primarily using ESS [1] and ob-julia [2] (org babel julia package). It is relatively well integrated with some basic auto-complete features and nice org-babel integration (as good as python or r).
[1] https://github.com/emacs-ess/ESS/wiki/Julia
[2] https://github.com/gjkerns/ob-julia
[1] https://github.com/emacs-ess/ESS/wiki/Julia
[2] https://github.com/gjkerns/ob-julia
As an org-babeler myself, and someone who has also used RMarkdown and Jupyter notebooks, I feel like these newcomers are quite underpowered compared to Org. Not to mention that mixing languages in a single doc is not possible / cumbersome with the others.
I do recognize, however, that being tied to emacs (the only complete implementation of org-mode / org-Babel today) is a liability. I wonder if some smart person will figure out a portable implementation in something else. Of course, it won't match the flexibility or extensibility of emacs, but perhaps it can still surpass Jupyter / RMarkdown.
I mention RMarkdown together with Jupyter simply because org-mode / org-Babel nearly addresses both the dynamic notebook use case as well as the document publishing use case (including nice academic feature like cross-references, which aren't supported in Markdown).
I do recognize, however, that being tied to emacs (the only complete implementation of org-mode / org-Babel today) is a liability. I wonder if some smart person will figure out a portable implementation in something else. Of course, it won't match the flexibility or extensibility of emacs, but perhaps it can still surpass Jupyter / RMarkdown.
I mention RMarkdown together with Jupyter simply because org-mode / org-Babel nearly addresses both the dynamic notebook use case as well as the document publishing use case (including nice academic feature like cross-references, which aren't supported in Markdown).
I could never get this to work in Ubuntu 14.1 :/
There are issues with Jupyter in that distro too, But hydrogen had some nps js error regarding a missing lib or something. So many stacks in these things...
There are issues with Jupyter in that distro too, But hydrogen had some nps js error regarding a missing lib or something. So many stacks in these things...
For vim people living in terminal, I suggest cellmode plugin for vim (https://github.com/julienr/vim-cellmode). You can split tmux window into vim + ipython panes, and trigger the evaluation of selected vim lines by pressing Ctrl-C.
Suuure, now this gets posted after I JUST found out about it yesterday in a comment about the JupyterLab link.. Go ahead, rub it in more.. Seriously though, I never thought to look for something like this so I'm happy more people are finding out about it too. Even if there may be better alternatives, more people now know to look.
Is there any way you could have this running on the cloud (in Digital Ocean or AWS)?
Suppose you had Jupyter installed using the following process:
https://www.dataquest.io/blog/digitalocean-docker-data-scien...
Suppose you had Jupyter installed using the following process:
https://www.dataquest.io/blog/digitalocean-docker-data-scien...
I have been enjoying Python (or any other external command) execution on whole file or on selected text block in Vim and it has been great, even for some specialized mini applications and custom docbook/markup formatting. The ability to have graphics is intriguing, will give atom a try.
Works fine after adding a minimal config {"magicpython": "Python 3"} at the Kernel Mappings setting and setting PYTHONPATH at the command line when running atom. It's kinda cool to manage your kernels from the editor!
So I know Sage is quite a bit more hefty than the standard Jupyter kernel, but I wonder if this could work with the Sage kernel? I quickly attempted it with no luck but assume with the right setup it could. Maybe?
This looks cool but how does this translate to being beneficial in real world web applications?
Often times the result of running some bit of code is to see the end result in a browser because the code resulted in sending a bunch of HTML or JSON as a response.
Also in the above case your code isn't self contained to a block of code. Chances are you have a web server, along with a database and cache server. Most of my projects are also encapsulated in half a dozen Docker containers. How is this plugin going to know what to do with that?
Starting up an entire stack of programs to live preview a few lines of code would end up being slower than just saving the file and reloading the browser to see feedback.
Often times the result of running some bit of code is to see the end result in a browser because the code resulted in sending a bunch of HTML or JSON as a response.
Also in the above case your code isn't self contained to a block of code. Chances are you have a web server, along with a database and cache server. Most of my projects are also encapsulated in half a dozen Docker containers. How is this plugin going to know what to do with that?
Starting up an entire stack of programs to live preview a few lines of code would end up being slower than just saving the file and reloading the browser to see feedback.
Jupyter Notebooks are more for statistical, iterative coding, not running an application.
Thanks. I wasn't aware of what Jupyter was.
I only glanced the top of the README which compares the project to what Light Table was trying to solve and then showed an animated gif of inline code evaluation examples.
I only glanced the top of the README which compares the project to what Light Table was trying to solve and then showed an animated gif of inline code evaluation examples.
Super awesome! The gif can do with a better Atom theme, though!
The dark one isn't really ideal for a cell-by-cell Jupyter-like implementation
The dark one isn't really ideal for a cell-by-cell Jupyter-like implementation
Is there a specific theme you had in mind? Just curious.
[deleted]
Good work.
Although Bret Victor deserves lots of appreciation for his work, most of his ideas are actually how Xerox PARC workstations and Lisp Machines used to enable developers to program more productively.
So if anything the best part of his work is making young generations aware of what the industry lost when those systems failed to gain wide market acceptance.