VisiData Lightning Demo at PyCascades 2018 [video](youtube.com)
youtube.com
VisiData Lightning Demo at PyCascades 2018 [video]
https://www.youtube.com/watch?v=N1CBDTgGtOU
42 comments
This is really great in helping me understand what I can do with it and how to do it.
The lightning talk is a really great sales pitch, but it's unclear how to actually replicate it. Another video of his, a recommended one after the lightning talk, again loses me a little and goes very fast. I could pause and try to understand... but it's not just about how to do it, but also about knowing that I should be looking to do certain things in the first place.
Your tutorial both gives a nice overview of what features exist, and explains very clearly how to do things. Thanks for taking the time!
The lightning talk is a really great sales pitch, but it's unclear how to actually replicate it. Another video of his, a recommended one after the lightning talk, again loses me a little and goes very fast. I could pause and try to understand... but it's not just about how to do it, but also about knowing that I should be looking to do certain things in the first place.
Your tutorial both gives a nice overview of what features exist, and explains very clearly how to do things. Thanks for taking the time!
One of the challenges with powerful tools is learning them. This is why many new people avoid things like Emacs, Vim, etc.
Some hand-holding and guiding is very welcome... so thanks!
Some hand-holding and guiding is very welcome... so thanks!
>(That convention, like several others in VisiData, is borrowed from the vim text editor.)
Thats all I needed to hear.
Thats all I needed to hear.
Great work
I almost didn't bother because it was a video of someone messing with a csv file, but I'm glad I took a few minutes out to watch that.
For everyone who hasn't watched it yet, from what I can tell, this is a clip of someone using a terminal tool they've made for hacking around with tablular data. It looks awesome for exploring data in a partially visual way, driven by commands. I'll often end up doing a bunch of stuff like he demonstrates directly in python — pull out the data, filter, find patterns, calculate new columns. You could probably do much of that with Pandas, but this definitely looks like it would be worth a play.
For everyone who hasn't watched it yet, from what I can tell, this is a clip of someone using a terminal tool they've made for hacking around with tablular data. It looks awesome for exploring data in a partially visual way, driven by commands. I'll often end up doing a bunch of stuff like he demonstrates directly in python — pull out the data, filter, find patterns, calculate new columns. You could probably do much of that with Pandas, but this definitely looks like it would be worth a play.
Thanks for your comment - I wouldn't have clicked on the video link otherwise, and it was well worth it.
For others who usually skip videos, here is a link which skips the intro, getting your time investment to under 4 minutes:
https://www.youtube.com/watch?v=N1CBDTgGtOU&t=0m52s
For others who usually skip videos, here is a link which skips the intro, getting your time investment to under 4 minutes:
https://www.youtube.com/watch?v=N1CBDTgGtOU&t=0m52s
Saul is the guy who uncovered the crossword plagiarism scandal
http://fivethirtyeight.com/features/a-plagiarism-scandal-is-...
VisiData is his vision of interactive data exploration at the text terminal.
VisiData is indispensable for data engineering tasks. I've been `head -n 1000 mybigfile.csv | grep somerandomthing | etc | etc`ing for so long, it's ability to give you fast summaries and exploration is such a breath of fresh air!
http://visidata.org/ is the page for the tool and https://github.com/saulpw/visidata is the repo
> removed support for .visidatarc in the current directory and via XDG; only $HOME/.visidatarc is used now;
That was an odd choice considering everything is moving the other way as a standard. I wonder what the authors reasoning behind this decision was.
That was an odd choice considering everything is moving the other way as a standard. I wonder what the authors reasoning behind this decision was.
https://github.com/saulpw/visidata/issues/93 looks like it wasn't implemented correctly ti begin with. Sad that it wasn't fix though.
A thread about the product release is at https://news.ycombinator.com/item?id=16515310. We merged some comments from that thread into this one because this one is currently ranked higher. Also the demo seems to be communicating more about what the project does than the project page.
Apparently the guy took an entire year's sabbatical just to work on this tool. Pretty impressive stuff.
Going to plug my own research project which does some things like this in a GUI: https://www.youtube.com/watch?v=y1Ca5czOY7Q
Why on earth are blaring horns the first thing on your video?
It's an immediate 'close tab' motion for a lot of people.
Cut that part out and you'll have a lot more eyeballs on your product.
It's an immediate 'close tab' motion for a lot of people.
Cut that part out and you'll have a lot more eyeballs on your product.
Very neat, especially useful for peaking into (SQLite) databases and getting a quick overview of the contents. Thank you for your efforts!
I always read about how awesome VisCalc/Lotus1-2-3 is but never been able to use one.
How does VisiData compare to them?
How does VisiData compare to them?
Congrats, this is awesome!
since v0.96 it can take input directly from stdin. I very much appreciated that feature as now you can pipe output from mysql (--batch -e)/sqlite and the likes. Would be interesting to use it as a pager inside the mysql prompt.
Obligatory motivational video of Martin Shkreli using Excel: https://news.ycombinator.com/item?id=12448545
How does this handle plotting huge dataset? E.g. 500000 points?
Is Windows version somewhat on the roadmap? E.g. For mingw-bash or cygwin?
Is Windows version somewhat on the roadmap? E.g. For mingw-bash or cygwin?
What does this give you that pandas lacks? Or is it more geared towards people that like to use the terminal?
I don't really see why those two should be compared? Pandas is a Python library for programmatic work with tabular data. Visidata gives you a live view of a file storing tabular data.
Thanks for clarifying, that makes sense
Wonderful tool. Thank you!
Brilliant work!
[deleted]
Pretty cool but ... why not use a GUI? It would make it 100 times easier to use. He even says at one point "this is all just in the terminal by the way" like that's a good thing!
Because I’m remoted into another machine in a terminal. Because I’m already coding in a terminal. Because if I want to use a GUI I can use Excel.
I am in favor of it being terminal-based, but imho you're giving all the wrong reasons.
If working on a remote machine were really an issue, you'd just mount the remote filesystem or something. That's a small convenience but not a real reason.
That you're already coding in a terminal, didn't disable alt-tab.
Spreadsheets (or to use the brand name, Excel) have a very different focus. You speak as if spreadsheets are just as good as, in this case, Visidata, but the whole point is that this is better. Else, why bother developing and using it? You can indeed just open up your favorite spreadsheet processor.
I think the real reason is that it's universal and convenient. You can use the multitasking features of your favorite shell, and it attracts a certain audience which indeed wants vim-style hotkeys and shortcuts. It also doesn't need a platform to create GUIs on top of, like Electron.
If working on a remote machine were really an issue, you'd just mount the remote filesystem or something. That's a small convenience but not a real reason.
That you're already coding in a terminal, didn't disable alt-tab.
Spreadsheets (or to use the brand name, Excel) have a very different focus. You speak as if spreadsheets are just as good as, in this case, Visidata, but the whole point is that this is better. Else, why bother developing and using it? You can indeed just open up your favorite spreadsheet processor.
I think the real reason is that it's universal and convenient. You can use the multitasking features of your favorite shell, and it attracts a certain audience which indeed wants vim-style hotkeys and shortcuts. It also doesn't need a platform to create GUIs on top of, like Electron.
There's room for both reasons. Sometimes when you're working with massive sets of tabular data you don't want to be doing it over a network mounted drive.
What they showed in the lightning demo is so much better done with fast keystrokes than moving a mouse around. I can't wait to try this tool, it will save me days of time per year.
In fact, I've been liking it so much that I wrote a tutorial for first-time users: https://jsvine.github.io/intro-to-visidata/
(Feedback very much welcome / appreciated.)