Know Your Tools – Terminal and Bash(spacecowboyrocketcompany.com)
spacecowboyrocketcompany.com
Know Your Tools – Terminal and Bash
http://www.spacecowboyrocketcompany.com/2015/02/know-your-tools/
47 comments
How do you typically go to e.g. beginning/end of line; via insert or normal mode and which key combinations do you prefer?
Always been using ctrl+a and ctrl+e in emacs mode myself . . .
Very true. I have difficulties in training new recruits with UNIX. For some reason, beginners have developed an aversion to UNIX/shell scripting. I myself have started more and more of Ruby/AWK/Python, but it is always SHELL that blows me off with the sort of impact it has on the daily work.
I ended up highly optimizing my work environment to replace long ssh connections with `conn prod` (will use my RSA key and connect). File tranfers with `go get prod filename` or `go send prod filename`. Another script that can remember directory paths (I have to go over lot of them every day, and typing long lengthy ones does not help).
To me, UNIX shell scripting is a lazy men's most effective tool, only if you know how to use that rightly.
Loved the post. And thanks much.
I ended up highly optimizing my work environment to replace long ssh connections with `conn prod` (will use my RSA key and connect). File tranfers with `go get prod filename` or `go send prod filename`. Another script that can remember directory paths (I have to go over lot of them every day, and typing long lengthy ones does not help).
To me, UNIX shell scripting is a lazy men's most effective tool, only if you know how to use that rightly.
Loved the post. And thanks much.
I'm a relative newbie in the field, but I really love using terminal (I do all my C work for grad school in ViM), it's really essential at work, there's no faster way to do batch file processing a lot of times (especially with ffmpeg, awesome).
I'm surprised people don't like terminal, I love using it. It makes me feel old school, sort of like one of those hackers in the movies. That's probably just a personal thing.
I was familiar with most of the concepts but the links section made things click in that regard. Enjoyed the post as well.
I'm surprised people don't like terminal, I love using it. It makes me feel old school, sort of like one of those hackers in the movies. That's probably just a personal thing.
I was familiar with most of the concepts but the links section made things click in that regard. Enjoyed the post as well.
It just takes a while to be faster in the terminal than with the gui, and most people won't switch from the thing they're used to until they see significant advantages.
I remember when I first started using the terminal, it was slow and confusing. You had to read the man pages for every little thing. Discovering the command you want is tough, I'd be screwed without the internet.
Now I hate when I need to take my hands of the keyboard to use the mouse. What a waste of time!
I remember when I first started using the terminal, it was slow and confusing. You had to read the man pages for every little thing. Discovering the command you want is tough, I'd be screwed without the internet.
Now I hate when I need to take my hands of the keyboard to use the mouse. What a waste of time!
If you didn't know you can put ssh configurations (hosts, ports, keys, etc) in ~/.ssh/config. You can use it in scp like 'scp prod:path/to/file path/to/destination'. Not sure if there's a pro/con to mapping an alias.
I decided not to use ~/.ssh/config because I wanted the program to be dynamic and record the sessions as I connect for the first time [https://raw.githubusercontent.com/guruparan18/conn/master/co...].
Same case with `scp`, I have bunch of production servers and I have to send/get files from production server all the time. This means, I have to come up with my own one-line command [https://raw.githubusercontent.com/guruparan18/go/master/go]
As I said earlier, to me, UNIX scripting is highly personal. The scripting reflects one's personality. That is why there is not always the right one. There are several way of doing.
Same case with `scp`, I have bunch of production servers and I have to send/get files from production server all the time. This means, I have to come up with my own one-line command [https://raw.githubusercontent.com/guruparan18/go/master/go]
As I said earlier, to me, UNIX scripting is highly personal. The scripting reflects one's personality. That is why there is not always the right one. There are several way of doing.
Bash scripts are great if they're yours. My current project has ~20 large bash scripts that do all sorts of different things. They were all written by people who haven't worked on the project for two years. Anyone who needs to make changes there spends far too long trying to figure out what's going on. The scripts could have been written in the same language as the project, and it wouldn't take a whole day to make changes to them.
I think they wrote in Bash to avoid the interpreter startup time. But the scripts are run very infrequently and most of them start/stop long-running services.
That being said, getting a bash script to do the right thing is very satisfying!
I think they wrote in Bash to avoid the interpreter startup time. But the scripts are run very infrequently and most of them start/stop long-running services.
That being said, getting a bash script to do the right thing is very satisfying!
Bash scripts for very simple series of commands; higher level scripts for when those sets of commands need logic wrapped 'round 'em. For example: provisioning for packer|vagrant, if you don't need to go overkill with puppet|chef.
It's not just UNIX, hey. I come across a lot of programmers that have no idea how to browse the basic command line in windows. I.e. They have no concept of relative paths, browsing through folders, command parameters. It's all incomprehensible "mumbo jumbo" for them.
Which is fine, not everyone has been exposed to it. Eventually, by the fifth time I have to tell them how to move one folder up (Cd ..), I realize that something is very, very wrong.
Which is fine, not everyone has been exposed to it. Eventually, by the fifth time I have to tell them how to move one folder up (Cd ..), I realize that something is very, very wrong.
When I was a student programmer, my boss came over to my desk to introduce my new project. He then fired up a terminal, ssh'd into the box where I would find the data. He then presented me the data by doing all kinds of bash magic. I was fairly impressed each time. I used some time just trying to make sense of what he did and how.
It's insanely useful.
It's insanely useful.
I forget where I got this from, but this is how I remember it:
tar -xzf # eXtract Ze Files!
I don't understand why you need options at all... If you point it at a file/directory it should automatically compress and if you point it at an archive it should automatically decompress. The options should be there for advanced tasks not 90% of the tasks you need the app for. It's really poor ui design.
After googling this three separate times I finally just aliased it to an 'untar' command. I'll remember this for when I'm ssh'ed somewhere!
Lately, all of my tar use has been `tar -xvf filename`. I don't remember it always being the case that those options would automatically gunzip/uncompress, but they do now!
The best mnemonic I've encountered for remembering those flags is visualizing Arnold Schwarzenegger yelling them: "eXtract Ze File!!!"
Hm, what OS? They didn't in the past. You needed to use 'tar -zxvf' for gunzip and 'tar -jxvf' for bzip2 IIRC. I still use it today to wrap files and move them on my server. On the server I use 'lzma' as default compression for archiving files (sql schemas, conf files, etc.)
Primarily Ubuntu, I'm not sure if this worked on my laptop when I was using Arch a couple years ago... I'd estimate the behavior has been around since 2008-2009.
I've found the option[1], but it's not in my alias list.
Edit: Changelog says Version 1.20 (2008-04-14) introduced the --auto-compress option
[1]: http://www.gnu.org/software/tar/manual/html_node/gzip.html#a...
I've found the option[1], but it's not in my alias list.
Edit: Changelog says Version 1.20 (2008-04-14) introduced the --auto-compress option
[1]: http://www.gnu.org/software/tar/manual/html_node/gzip.html#a...
haha exactly! :)
My most used bash command is probably "sudo !!", which runs the previous command with "sudo" prepended.
The book "Unix Power Tools" showed me that higher-level programming languages (Perl, Python, Ruby, etc) are not needed to accomplish most administrative tasks. Awk is incredibly useful (but also very confusing, imo).
The book "Unix Power Tools" showed me that higher-level programming languages (Perl, Python, Ruby, etc) are not needed to accomplish most administrative tasks. Awk is incredibly useful (but also very confusing, imo).
I really like '!$' and '!:1' in bash. '!$' holds the last argument of the last command and '!:n' holds the nth argument.
$ cd /big/long/path/to/this.txt Oops $ vim !$
$ cd /big/long/path/to/this.txt Oops $ vim !$
It should be noted for onlookers that this is the abbreviated form; !-3:4 goes back to the third-to-last command and refers to its fourth argument, and similarly !-3:$ is its last argument.
It's also apparently not universally known that bash has csh-style (similar to sed) editing, so "s^foo^bar" will repeat the previous command with "bar" substituted at for bar. So "ls foo" becomes "ls bar".
Similarly with !-3:s^foo^bar
I freely mix these things with readline-editing, depending on what needs to be done.
Since csh gets a lot of disrespect these days, to give credit where due: these are all Bill Joy features from the original csh -- it's a good thing that bash borrowed innovations from various shells beyond Bourne shell and ksh.
Historical footnote: We donated code to him to do readline-style editing circa 1978, but he said raw mode was bogging down the pdp 11 too much. Systems were a bit slower back then.
It's also apparently not universally known that bash has csh-style (similar to sed) editing, so "s^foo^bar" will repeat the previous command with "bar" substituted at for bar. So "ls foo" becomes "ls bar".
Similarly with !-3:s^foo^bar
I freely mix these things with readline-editing, depending on what needs to be done.
Since csh gets a lot of disrespect these days, to give credit where due: these are all Bill Joy features from the original csh -- it's a good thing that bash borrowed innovations from various shells beyond Bourne shell and ksh.
Historical footnote: We donated code to him to do readline-style editing circa 1978, but he said raw mode was bogging down the pdp 11 too much. Systems were a bit slower back then.
If using readline (emacs) keybindings, press Alt and period for the same effect as !$.
Two very basic tips for vi:
You can use slash(/) and question mark(?) to search for text (forward and backward) in vi. It's useful to know these, because they also work with less(1).
If you are Windows user and used to press Ctrl+S to save the file and you accidentally do that in Vi, your terminal would get stuck. But you can use Ctrl+Q to unfreeze it.
You can use slash(/) and question mark(?) to search for text (forward and backward) in vi. It's useful to know these, because they also work with less(1).
If you are Windows user and used to press Ctrl+S to save the file and you accidentally do that in Vi, your terminal would get stuck. But you can use Ctrl+Q to unfreeze it.
vi move commands works a lot of places in the UNIX world.
"Fact of life is, as you spend time in the terminal, you're probably going to launch Vim at some point, whether willingly or not...
For all other intents and purposes, just friggin use nano. Or sublime."
Words to live by.
Words to live by.
Tip: I recently looked up the man page for "which" and found out about the -a flag which will list all instances of the executable found (instead of just the first one). Incredibly useful when you have multiple installations of the same software and it's causing problems. e.g.:
$ which -a ruby
/Users/olalonde/.rbenv/shims/ruby
/usr/local/bin/ruby
/usr/bin/rubyTo exit vim use <ESC> :wq, not just :wq - most of the problem is it getting stuck in insert mode, and not knowing how to escape into normal mode.
My first experience of vim was trying to fix my xorg.conf from a tty. I figured out how to save the file, but couldn't exit. In the end I hard-rebooted :$
^C also pops up this helpful dialog (after you've already pressed it once to get to normal mode, but the use case is that you're probably mashing it trying to figure out why you can't exit this crazy editor):
Type :quit<Enter> to exit VimI could never remember that darned syntax. At some point, though, I came across an alternative to it:
shift+z shift+z
I.e. You're in command mode, and essentially type two capital Z letters. It saves your document and quits.
Anywho, it's stuck with me ever since.
shift+z shift+z
I.e. You're in command mode, and essentially type two capital Z letters. It saves your document and quits.
Anywho, it's stuck with me ever since.
Thanks guys, I'll collect these thoughts and add them to the blogpost. After all, I want this to be available to all and every one of us!
I love playing around in bash. Most of the scripts I have right now are just for setting up project templates, adding common packages etc.
Actually, I think most people who use computers at work, be they writers, designers, whatever, not just programmers, would benefit from learning some bash (or batch). You can automate a lot of useful processes with a few simple scripts.
Actually, I think most people who use computers at work, be they writers, designers, whatever, not just programmers, would benefit from learning some bash (or batch). You can automate a lot of useful processes with a few simple scripts.
don't type `cd ~` just type `cd`
Also: Accidentally ran `cd`? Run `cd -` to return
I think there are some pop/push mechanics for working directory too, but I can't find them right now.
I think there are some pop/push mechanics for working directory too, but I can't find them right now.
The commands are pushd and popd, and they're bash (and zsh, and most sane shell) builtins. Pushd to get to a new directory, popd to go back to the last one on the stack.
Tip : Setting cd as an alias for pushd make the whole thing much more intuitive.
Oh handy! Hadn't come across that before. Thanks.
Or type `cd ~username` to goto that user's home path
1. I like tab-completion, but I can't personally stand bash's completion feature. Be aware of differences.
2. If you want to save your command without running it... comment it out with a '#'. Bang, it's in your history.
2. If you want to save your command without running it... comment it out with a '#'. Bang, it's in your history.
This is probably misunderstanding the demographic, but why does he suggest using nano over vim?
Maybe they just prefer nano?
More likely, if this is meant to be a gentle introduction to the command line, vim is a pretty scary place. Nano works much more like a "normal" text editor.
More likely, if this is meant to be a gentle introduction to the command line, vim is a pretty scary place. Nano works much more like a "normal" text editor.
Until you want to undo something! Nano is definitely more user-friendly though. Some of the engineers I work with still don't know basic vim shortcuts, like 'w' and 'b'
knowing my demographic, that's what they prefer. If anyone wants to use vim, I point them to the vim person.
edit: Actually, something cool I learned recently is that if you put the line "set editing-mode vi" in your ~/.inputrc then vi-like editing keys are available in any program that uses readline. This includes bash and the python repl. Presumably, a bunch of other repls and interactive command lines too.