All excellent points. Right now, Neptyne works much like a Google doc in that it's private to you but is easily sharable to pretty much anyone. If our users are looking for it, a self-hosted option is certainly something we'll explore.
Then there's things like PII protections as you said, data provenance, that you could imagine baked into the spreadsheet directly. We haven't gone in that direction but I think it's interesting to consider.
What sort of features around data security would you hope to see in a spreadsheet?
Thanks! We've talked to a lot of people in hedge funds and investments, and you're right that a lot of them care about a self-hosted solution. This is definitely on our radar as a potential future path for the product.
It is true that the people who are most productive today in Neptyne are those who are pretty familiar with Python. Part of our long-term goal is to make this kind of programming more accessible to beginners, and we talk to a lot of people who wouldn't call themselves "programmers" but are familiar with spreadsheets. Our hypothesis is that if you give these people a frictionless way to incorporate programming into their workflows, they'll learn quickly and reap the benefits.
That does sound like a fitting use case. Technically speaking it should be OK with 70k rows, but we've got some optimizations that might need to go out before it performs well. Ideally, you should be limited only by the memory allocated to the Python kernel, but today there are some other limitations that get in the way.
All great tools to be sure! I am a huge fan of numpy and pandas especially. For plotting in Neptyne I usually opt for plotly over matplotlib/seaborn. Agreed too on the shell -- that's why we built one into our product.
I am curious though, do you typically share/collaborate on these spreadsheets with others? Do you have a neat way of packaging up the spreadsheet+Python code?
We do support live collaborative editing, though it isn't all built out yet. If two people are working on the same tyne, you'll see the presence of other collaborators and you'll see their changes reflected live, but the code panel updates with last-writer-wins at the moment. We're working on making that better.
The main technical difference between us and something like Equals is that we give you a full Python environment (in the form of a Jupyter kernel running in a Docker container) for maximum programmability and flexibility. This means that you, as a Neptyne user, can connect to virtually any data source by importing the appropriate package and setting credentials. And of course you can use Python in the actual functioning of your spreadsheet, not just as a one-way data import step.
That's probably something we should dig more into in the docs, yeah. Neptyne works like a ipython/Jupyter notebook in that everything is in scope all the time. So you can write your code in an imperative way (e.g. `B10 = some_value`) within a function, or you can do use a functional style.
We handle spreadsheet-style reactivity by building the dependency graph out of any cell addresses mentioned in the cell itself. So if you have `=my_func(A1)` in one cell, it will re-run when A1 changes. But if my_func reads from, say, B1, it will not automatically re-run when B1 changes.
We see Apps Script is a strong signal that this kind of thing is useful. Apps Script adds so much power to Sheets but it could be so much easier to use. With Neptyne, since your spreadsheet engine runs in a Jupyter kernel, you get an interactive REPL and a very straightforward bidirectional API for reading/writing data.
Thanks for calling that out! We made it possible to skip the sign-up to try our tutorial. https://neptyne.com/neptyne/tutorial should now let you do that
It should work. I haven't tried requests-cache exactly but in Neptyne you're running with a Jupyter kernel in a Docker container backend, so you do have a filesystem to work with. That file system is mostly ephemeral though -- when you leave and come back, you'll get a new container, so anything you need to persist should live in the spreadsheet itself.
We hear you and we do want to make it possible to try the tutorial without signing up for an account. We're working on making that possible shortly. I'll update here when that's possible
EDIT: this should work now. Just click the new button that says "Skip signup and let me try it out first"
We don't have a self-host option at the moment. We wanted to build Neptyne as a collaborative platform first, where users could build together and share their work.
I wouldn't rule it out for the future though -- a lot will depend on where we see Neptyne being used.
To answer your question, it's cloud-only -- we wanted to build Neptyne as a collaborative platform where teams of programmers and non-programmer types could get things done together. I think you're right in that this will mean it doesn't work for a certain class of user, but for now at least, we're focused on Neptyne as a web app.
Our biggest challenge as you say is definitely the fact that lots of users will be entrenched in Excel. Our goal right now is to appeal not to those who are happy in Excel today, but to those who have grown disillusioned: lots of users today build up amazingly complicated things in Excel and grow frustrated by the difficulty of maintaining that complexity. Python can be a much better fit in many cases for a lot of that complexity.
And as a platform for data science, we've found Neptyne really nice for sharing results. I was surprised at how often I heard from users: "well, we usually do everything in Jupyter, but then whenever management wants to see the output, they ask for it in a spreadsheet".
With Google Sheets you get a flavor of JS called Apps Script, a flavor of JS with some limitations. Some ways in which our use of Python differ are:
- run Python directly in the spreadsheet cells, not just as an "extension"
- a full runtime in a Jupyter kernel, so you can import and use effectively any Python package, as long as it runs on Linux
- an interactive REPL that gives you a nice test environment, but also a command line of sorts for working with your spreadsheet. (e.g. you can say `A1 = requests.get(URL).json()`) to do a one-off fetch of some data from an API.
Generally speaking we hope to give you a much more powerful/seamless integration between spreadsheet/Python than what you get with Sheets/JS