Neuron, a new VS Code extension for data science(blogs.msdn.microsoft.com)

259 pointsby starbugs8 tahun yang lalu59 comments

blogs.msdn.microsoft.com

Neuron, a new VS Code extension for data science

https://blogs.msdn.microsoft.com/uk_faculty_connection/2018/10/29/data-science-in-visual-studio-code-using-neuron-a-new-vs-code-extension/

61 comments

greazer8 tahun yang lalu

Hey great to hear the excitement for this. In fact, we (the Python VSCode extension team) were so excited about it, that we've already begun integrating it into the python VSCode extension itself! It's a little rough yet for a full release but it's highly functional. If you want to play with it, just build the extension from https://github.com/Microsoft/vscode-python. Suggestions are welcome, but remember, it's still undergoing changes.

pavanagrawal1238 tahun yang lalu

Unfortunately this extension has already died in activity. I forked the extension here: https://github.com/pavanagrawal123/VSNotebooks . A lot of bug fixes and usability improvements have been put in!

jjuel8 tahun yang lalu

I am curious as to why Microsoft would release a post just two days ago about a dead plugin written by one of the developers of said plugin?

smortaz8 tahun yang lalu

See my note elsewhere in the thread. This was summer student project. Not sure why the blog was posted so late...

farresito8 tahun yang lalu

In what way has it died in activity when there are tens of commits around June and July? That's not even four months.

judge20208 tahun yang lalu

3 months can feel like a long time without any commits in the age of corporate-funded projects like Vue and React.

kbumsik8 tahun yang lalu

This is not a big project like Vue or React. It's just an add-on. It is very common that popular add-ons are not maintained for months.

chillee8 tahun yang lalu

More relevantly, if people actually read the blog post, they'd see that this has been merged into the core Python extension, and is being worked on there.

mplewis8 tahun yang lalu

Why not just contribute to the existing extension?

wakkaflokka8 tahun yang lalu

This is fantastic stuff, thank you.

pavanagrawal1238 tahun yang lalu

No problem, if anyone has any feedback, please leave it in the issues. I want to make this extension great, and I would love the community's input!

greazer8 tahun yang lalu

Just to make sure it's seen, the Python extension team at Microsoft has an upcoming release that will integrate this functionality. The lastest github repo for this extension has the code we're working on.

pavanagrawal1238 tahun yang lalu

oh interesting I did not know this... Is the code being integrated from Neuron?

greazer8 tahun yang lalu

No, Neuron was the inspiration, but the code was redesigned and reimplemented. Additional features have been added as well, such as defining a cell via #%% in code rather than having to select text.

pavanagrawal1238 tahun yang lalu

Ahh I see... I've been looking at the PR and the implementation is really nice, I'm excited to see what it turns into :)

smortaz8 tahun yang lalu

FYI - This was one of the two Cambridge University projects that was done with our team last summer. I'm not sure why the report came out this late. Both were well done, especially considering the learning curve and duration.

The 1st one was the data science plugin for vscode mentioned in the blog. The other one which I haven't seen published yet was a C# kernel for Jupyter which can be found here along w a video: https://github.com/MohamedEihab/icsharp/tree/Roslyn_2

[disclaimer - work at microsoft]

montenegrohugo8 tahun yang lalu

This looks awesome. I'm continually surprised by the improvements made to VS Code. I love both Jupyter and VS Code, and having both merged seems great for my use case (and anyone else doing exploratory code).

Props!

hessenwolf8 tahun yang lalu

(3)

ct08 tahun yang lalu

Anything that resembles RStudio for python makes me really excited to shift working from R to python for projects running in production.

prgmtcbhmn8 tahun yang lalu

Spyder was pretty nice coming from RStudio. I last used it a few years ago, when it came with the anaconda distribution.

https://www.spyder-ide.org

samfriedman8 tahun yang lalu

In that case you should check out Yhat's Rodeo IDE. Essentially built as "RStudio for Python".

https://rodeo.yhat.com/

leetrout8 tahun yang lalu

It was a fun project but it's no longer supported.

https://github.com/yhat/rodeo/issues/655

https://github.com/yhat/rodeo/issues/631

samfriedman8 tahun yang lalu

Ah, right you are. Even looks like the download links are broken.

ddavis8 tahun yang lalu

Any Emacs users looking for something similar, see EIN [0] and ob-ipython [1].

[0] https://github.com/millejoh/emacs-ipython-notebook

[1] https://github.com/gregsexton/ob-ipython

natrys8 tahun yang lalu

ESS [0] is fantastic, which I am using with R.

[0] https://ess.r-project.org/

deshpand8 tahun yang lalu

The next generation of Jupyter, JupyterLab is turning into a whole IDE/platform, not just a notebook. They are also trying to incorporate the monaco editor which drives VS Code. Wouldn't that be a better solution?

https://github.com/jupyterlab/jupyterlab-monaco

pavanagrawal1238 tahun yang lalu

that won't bring the rich support of extensions VSCode has.

VSCode's magic isn't just the monaco editor, IMO it's really the ecosystem and really nice Intellisense the other extensions provide.

though I do agree, this might be the better solution in the long run.

xrd8 tahun yang lalu

I spent time reading through Joel Grus' "I hate notebooks" slide deck and he makes a really compelling why they (Jupyter notebooks) are not the right tool for many reasons.

https://docs.google.com/presentation/d/1n2RlMdmv1p25Xy5thJUh...

It was eye opening.

His YouTube live coding on building a neural net from scratch really shows the power of using VSCode to do AI work and complements his assertions.

https://www.youtube.com/watch?v=o64FV-ez6Gw&feature=youtu.be

Both are well worth the time. All the reading I've been doing on ML/AI really came together after watching and taking notes in that YouTube video.

deshpand8 tahun yang lalu

You have to give credit to JupyterCon for inviting him! Most of his issues relate to Jupyter, he has/had not seen JupyterLab. You can trust that many of these issues will get worked out in the long run. I have nothing against VS Code but Jupyter(Lab) didn't win the ACM Software System award (Nobel price for software) for nothing.

sorenjan8 tahun yang lalu

This looks a lot like the Jupyter extension [0], which is made by a Microsoft employee and was last updated before the summer. I really like the cell format of Jupyter (inspired by Matlab I presume), I don't really want to select the code before I run it. Will Microsoft support both extensions? I would like support for actual notebooks as well, for when I want to mix text, code, and figures but without having to work in a browser.

[0] https://marketplace.visualstudio.com/items?itemName=donjayam...

williamstein8 tahun yang lalu

> (inspired by Matlab I presume)

I think Jupyter is inspired by Mathematica (and Macsyma) notebooks, not Matlab... Source: Fernando Perez started IPython, very much inspired by Mathematica (he was switching from Mathematica to Python). Then Alex Clemesha and I created the Sage notebook in 2007, partly inspired by discussions and demos from IPython developers, and also very much inspired by Mathematica (Alex Clemesha, a physics student, was also a heavy Mathematica user). Alex then made knoboo in 2008 https://github.com/knoboo/knoboo), which is another notebook that inspired the Jupyter notebook implementation.

sorenjan8 tahun yang lalu

I was referring to the cell mode, not notebooks. In Matlab you create a new cell by starting a line with two comment characters (%%), in the Jupyter extension you use #%%. You can then run the current cell with ctrl+enter, no need for selecting anything.

https://blogs.mathworks.com/community/2008/01/07/ive-got-som...

garyfirestorm8 tahun yang lalu

I think Spyder is inspired by Matlab https://www.spyder-ide.org/

pavanagrawal1238 tahun yang lalu

in my fork here: https://github.com/pavanagrawal123/VSNotebooks I'm almost done with cell support.

sandGorgon8 tahun yang lalu

This runs Jupyter in the background.

So it retains some of the issues with it - e.g. notebook diff,etc.

thanatropism8 tahun yang lalu

I think (and this isn't just personal experience) most of the appeal of Jupyter is the fractioned persistent state that allows you to write incrementally but not re-run increasingly long scripts. Inline plots are just gravy.

(Yes yes write documents export LaTeX. Not sure people care about this at all)

I've been developing a style where development "cells" are cached with joblib.Memory so re-running a script from command line doesn't take idiotic amounts of time. Further, I have litte context managers such that

   with Figure('filename.png'): plotting_logic()

exports a file that immediately shows in the folder navigation bar of Atom. It's overall a better experience.

What I would really like is being able to use joblib.Memory not as a decorator (which requires bunching up your "cells" in functions and calling them separately) but as a context manager.

---

To the tune of the "Portlandia" song: the dream of MUMPS state is alive in joblib...

mirashii8 tahun yang lalu

You can actually do this as a context manager, I have done something very similar in the past. I created a dummy context manager with a unique name and used an importhook to do an AST rewrite to the body of the context manager. It's a bit of a hack (okay, a lot of a hack), but allows for some really neat patterns that context managers didn't allow.

geekraver8 tahun yang lalu

You're confusing the Jupyter backend (which manages the Python kernels) with the front-end (which maintains the JSON document that reflects the notebook). You can have different front ends that use different formats than .ipynb.

mattnewton8 tahun yang lalu

It actually looks like it is creating the notebook format on the fly from user’s selections, so it probably doesn’t suffer the notebook json diff problem. Other issues you have though, probably are present.

pavanagrawal1238 tahun yang lalu

any exported notebook will suffer from the same issue, but the source file is just a normal python file, so diffs will be fine for those

minimaxir8 tahun yang lalu

So when I saw my "Advantages of Using R Notebooks Instead of Jupyter Notebooks" post was trending on HN this morning, I remembered Neuron is a strong idea to work outside both technologies, easily complemented by the multitude of VS Code extensions/theming.

There are still a few important missing features unfortunately (see the issues page: https://github.com/lorenzo2897/vscode-ipe/issues), but there's a lot of promise!

yodon8 tahun yang lalu

Looking through the issues list you linked to, lack of conda support [0] looks like a pretty big deal for most people doing heavy analysis in Python.

[0]https://github.com/lorenzo2897/vscode-ipe/issues/162

pavanagrawal1238 tahun yang lalu

This is a big issue, and fixed in my fork here: https://github.com/pavanagrawal123/VSNotebooks. Unfortunately, neuron already seems to be dead :( so I created a fork with a bunch of new features and additions!

curiousgal8 tahun yang lalu

What problem does conda try to solve?

dual_basis8 tahun yang lalu

In general, it is a package manager which supports different environments. It is similar to pipenv, but existed before. I believe there are still some differences, for instance conda is language agnostic, and it particularly was useful for data science where you would want the compiled version of certain dependencies (eg. MKL) for performance reasons. This SO answer seems like a good overview: https://stackoverflow.com/q/20994716/1060781

yodon8 tahun yang lalu

Can anyone comment on strengths or weaknesses of this compared to Hydrogen?

pavanagrawal1238 tahun yang lalu

(disclaimer, i'm working on a fork with better feature support)

Strengths: VSCode support

Weaknesses:

Bad code quality for the extension

No variable explorer

No extensibility

No custom env (fixed in my fork)

No cell support (almost fixed in my fork)

vbsteven8 tahun yang lalu

This looks really cool. Does anything similar exist for exploration on a SQL database?

I would love to see some notebook like interface for a SQL database with all the context awareness of Datagrips/IntelliJ’s editor but with similar visualisations like this tool.

StreakyCobra8 tahun yang lalu

Not exactly your answer, but microsoft is developing an open-source tool, that's visually similar to VSCode, to work on databases [1]. Sadly it only supports SQL Server, Azure SQL DB and SQL DW for now, but people are asking for additional database engine [2].

[1] https://github.com/Microsoft/azuredatastudio [2] https://github.com/Microsoft/azuredatastudio/issues/56

pavanagrawal1238 tahun yang lalu

this is a pretty interesting idea. I feel like this should just be implemented at a Jupyter (kernel) level and not in an editor though....

samfisher838 tahun yang lalu

Does this have a variable viewer like spyder (for data frames, lists, etc.)?

jakear8 tahun yang lalu

VSCode already provides that in the debug view

pavanagrawal1238 tahun yang lalu

Neuron doesn't support this, it won't work natively with the notebook.

meanmrmustard928 tahun yang lalu

looks similar to Hydrogen for Atom. I'd recommend that (since it is actively maintained) too