The bash book to rule them all(fabiensanglard.net)
fabiensanglard.net
The bash book to rule them all
https://fabiensanglard.net/bash/index.html
55 コメント
Thanks so much your book review, Fabien! I wrote "Efficient Linux at the Command Line" because I kept meeting Linux/Unix users (over the past 30 years) who had learned the command line mostly by trial and error. They could use Linux OK for their jobs, but they didn't know the underlying concepts and had all sorts of inefficiencies or superstitions. I'd spend a few minutes with these users, and their speed at the command line would literally double, just from showing them job control, CDPATH, the directory stack, and other features they'd missed. I'm glad you enjoyed the book.
I believe it's been posted here before, but I also use this for reference whenever I forget the syntax to something, or know there must be a better way than what I am doing.
https://github.com/dylanaraps/pure-bash-bible
https://github.com/dylanaraps/pure-bash-bible
Don't forget about Greg's Wiki
https://mywiki.wooledge.org/BashFAQ
(and of course we're all using ShellCheck to check our scripts for common mistakes)
https://mywiki.wooledge.org/BashFAQ
(and of course we're all using ShellCheck to check our scripts for common mistakes)
I usually recommend Data Science At The Command Line https://jeroenjanssens.com/dsatcl/
Not because it explains bash well, but because it fits with my data science/bioinformatics niche. It's all about these funky sort/uniq/rev/cut/uniq/awk/wc/sed patterns I use everyday fiddling with csv files.
Not because it explains bash well, but because it fits with my data science/bioinformatics niche. It's all about these funky sort/uniq/rev/cut/uniq/awk/wc/sed patterns I use everyday fiddling with csv files.
I've written books for GNU grep, sed, awk as well as one for coreutils. Free to read online. See https://github.com/learnbyexample/scripting_course#ebooks for links.
Have you looked at https://github.com/BurntSushi/xsv for csv processing?
Have you looked at https://github.com/BurntSushi/xsv for csv processing?
Hi, the part about bioinformatics caught my eye. Im a software engineer wanting to move into bioinformatics or somewhere between bio+CS. If you're up for it, can i email/DM you somewhere? Just want to learn more about what the industry is like, projects or ideas to break into the field etc. My email is also on my profile.
Thanks for the article!
Might want to check 'Small, Sharp Software Tools' from The Pragmatic Programmers also. It's a great book, and comparable to the O’Reilly book.
https://pragprog.com/titles/bhcldev/small-sharp-software-too...
Might want to check 'Small, Sharp Software Tools' from The Pragmatic Programmers also. It's a great book, and comparable to the O’Reilly book.
https://pragprog.com/titles/bhcldev/small-sharp-software-too...
> You want to understand why cd is not an executable but a builtin?
Because a sub-process cannot change the working directory of its parent. The shell process needs to change its own working directory. Therefore cd must be a builtin.
Although, I suppose that one might execve a cd binary, which would change directory and then in turn the cd binary would execve the shell that is specified in the $SHELL env var. That would be terribly silly though :p
Because a sub-process cannot change the working directory of its parent. The shell process needs to change its own working directory. Therefore cd must be a builtin.
Although, I suppose that one might execve a cd binary, which would change directory and then in turn the cd binary would execve the shell that is specified in the $SHELL env var. That would be terribly silly though :p
The title of the book (perhaps a more useful title) is
Efficient Linux at the Command Line: Boost Your Command-Line Skills by Daniel J. Barrett 2022
perhaps the book should be called Efficient Bash at the Command Line: Boost Your Linux Skills. It has a little table at the end that says "if you use a different shell". Not sure if this type thing suffuses the book a tall
Efficient Linux at the Command Line: Boost Your Command-Line Skills by Daniel J. Barrett 2022
perhaps the book should be called Efficient Bash at the Command Line: Boost Your Linux Skills. It has a little table at the end that says "if you use a different shell". Not sure if this type thing suffuses the book a tall
For such nuances I keep this page handy: https://hyperpolyglot.org/unix-shells
Controversial perhaps, but I now use Go where I used to be a prolific shell script script coder.
Main reasons:
Main reasons:
- Fewer opportunities for foot-guns
- Ease of portability, no dependencies, e.g. no more "is jq installed ?"
- Consistent cross-platforms, e.g. no more "is this system using GNU sed ?"
- Modern stuff like crypto, and "internet stuff" is just quicker and easier to do in Go.I can't go along with your reasons at all (installing go on every machine is not solving a portability problem, it is the portability problem) but I'm going to leave that aside.
can Go be used REPL as a shell CLI? and do everything I need to do from the CLI? because that's why I use bash for scripting. An equal amount of time I spend typing at the command line, and I one language is the language I want to use.
can Go be used REPL as a shell CLI? and do everything I need to do from the CLI? because that's why I use bash for scripting. An equal amount of time I spend typing at the command line, and I one language is the language I want to use.
It’s not go as a literal replacement for bash, as in you write golang when interacting with your own machine. It’s go binaries instead of bash scripts. You’re still distributing one file, but now with all the GP’s stated benefits
Bash is not a language, it's a shell. There is a big difference, and the differences will bite you in the ass until you learn to recognize them. For one, bash has no standard library. Different machines will have different things available. It's weird that you cite portability and then use the one solution that's pretty much the least portable. At least if you build on, say the Win32 API, you know exactly what machines are going to be able to run it. Bash? Well you pretty much have to know exactly what the other machine is running in order to have a clue about whether or not it will work.
I'd say the most portable thing you can build with that's script-y enough to be pleasant is likely Ruby. If you can avoid using things introduced with Ruby 3, keep in mind platform differences and the abstractions Ruby has built-in to deal with them, and stick with stdlib, it will likely run on just about every machine with just about any Ruby.
Otherwise maybe perl?
I'd say the most portable thing you can build with that's script-y enough to be pleasant is likely Ruby. If you can avoid using things introduced with Ruby 3, keep in mind platform differences and the abstractions Ruby has built-in to deal with them, and stick with stdlib, it will likely run on just about every machine with just about any Ruby.
Otherwise maybe perl?
I wrote another comment, I guess I didn't post it. Perl made a run at being the standard middle layer language like bash 30 years ago. Learning perl back then was different than for people to learn today. Back then, everybody would have known bash, and awk, and grep, and sed, and perl borrowed the their syntax/semeantics so as soon as you started using it, it was like you alrady knew it. But perl followed a trajectory of being more complicated and less intuitive, so nobody wanted it to replace the shell, but it could have had things gone differently.
bash used to be "a shell" and "the shell" before graphical shells became the norm. Now bash is a REPL and programming language, and you need to run an "ANSI terminal shell" to access it :)
1. the command line has a syntax of it's own ("readline", generally standard across different shells, in this instance enforced by bash) and then
2. bash has its own language you can use, turing complete, with if-branches and loops, functions, some return values, global values (env) but also file i/o can be used as up-down argument passing, i/o and you can call readline over and over
3. there are certain conventions for "filters" that fit into the above framework, and they impose a small variety of contentions onto the command lines which nest within readline syntax.
it has warts, and I really wish there was a general desire to fix them, but I guess I'm going to have to want it enough to do it myself because i've had the same ideas for 30 yrs now and nothings been happening :(
last point, people aren't picking up on the importance some of us place on a language which can script that is the same language that is the command line. We don't want two languages, we want one. i.e. a language with a REPL
bash used to be "a shell" and "the shell" before graphical shells became the norm. Now bash is a REPL and programming language, and you need to run an "ANSI terminal shell" to access it :)
1. the command line has a syntax of it's own ("readline", generally standard across different shells, in this instance enforced by bash) and then
2. bash has its own language you can use, turing complete, with if-branches and loops, functions, some return values, global values (env) but also file i/o can be used as up-down argument passing, i/o and you can call readline over and over
3. there are certain conventions for "filters" that fit into the above framework, and they impose a small variety of contentions onto the command lines which nest within readline syntax.
it has warts, and I really wish there was a general desire to fix them, but I guess I'm going to have to want it enough to do it myself because i've had the same ideas for 30 yrs now and nothings been happening :(
last point, people aren't picking up on the importance some of us place on a language which can script that is the same language that is the command line. We don't want two languages, we want one. i.e. a language with a REPL
I disagree that bash is fixable for the same reason I believe that javascript isn't fixable. It's too "low-level", by that I mean whoever implements it is also implementing the entire system it's going to be living in. Javascript runs in a browser, and standardizing it means standardizing the browser. And that's just not going to happen, there are literally thousands of versions of browsers.
And so bash exists in the context of an entire unix-y system. You can fix it for yourself, but that's not going to fix the problems that bash has. No standard library. You need to build on an abstraction that you control, that can accommodate for the differences between systems. And if you want a language with a REPL, it already exists. Ruby. You can do amazing things with Pry, Ruby's true REPL, and inline Bundler. Automatic dependency management in a single-file script? Yes please! Could also do Smalltalk or a Lisp if that way of working entices.
You literally never have to touch bash/zsh/fish/whatever if you don't want to. Set your shell to Pry. In a Ruby process you can abstract over literally everything. You can use Pry's shell-integration to do bash-y stuff or you can use editor integration to open up classes or methods in emacsclient or what have you. An actual, honest-to-god, real language, specifically engineered to be human-friendly, with everything you could ever need. It can even run it's own readline, which you can modify if you feel the need.
https://github.com/ConnorAtherton/rb-readline
And so bash exists in the context of an entire unix-y system. You can fix it for yourself, but that's not going to fix the problems that bash has. No standard library. You need to build on an abstraction that you control, that can accommodate for the differences between systems. And if you want a language with a REPL, it already exists. Ruby. You can do amazing things with Pry, Ruby's true REPL, and inline Bundler. Automatic dependency management in a single-file script? Yes please! Could also do Smalltalk or a Lisp if that way of working entices.
You literally never have to touch bash/zsh/fish/whatever if you don't want to. Set your shell to Pry. In a Ruby process you can abstract over literally everything. You can use Pry's shell-integration to do bash-y stuff or you can use editor integration to open up classes or methods in emacsclient or what have you. An actual, honest-to-god, real language, specifically engineered to be human-friendly, with everything you could ever need. It can even run it's own readline, which you can modify if you feel the need.
https://github.com/ConnorAtherton/rb-readline
There's a Go REPL called YAEGI, but it's not a similar experience as using a shell.
> (installing go on every machine is not solving a portability problem, it is the portability problem) but I'm going to leave that aside.
What a clueless reply. Complete FUD.
Yeah, you can "leave that aside" because you have no clue what you're talking about.
The ONLY place Go needs to be installed is on the developer's machine.
With Go you write, you compile/cross-compile and SHIP THE COMPILED BINARY job done.
Do some homework next time before engaging in completely unfounded FUD.
What a clueless reply. Complete FUD.
Yeah, you can "leave that aside" because you have no clue what you're talking about.
The ONLY place Go needs to be installed is on the developer's machine.
With Go you write, you compile/cross-compile and SHIP THE COMPILED BINARY job done.
Do some homework next time before engaging in completely unfounded FUD.
Can’t help but feel like the tone of your response is completely uncalled for.
> Can’t help but feel like the tone of your response is completely uncalled for.
Literally five seconds on Google could have shown the guy he was completely wrong.
But no, instead he posts unfounded FUD about how he will courteously "leave aside" the demonstrably INCORRECT claim that Go runtime needs to be installed on all machines that a Go program runs on.
So don't give me a hard time about it when it was that poster who made the original arrogant comment.
Literally five seconds on Google could have shown the guy he was completely wrong.
But no, instead he posts unfounded FUD about how he will courteously "leave aside" the demonstrably INCORRECT claim that Go runtime needs to be installed on all machines that a Go program runs on.
So don't give me a hard time about it when it was that poster who made the original arrogant comment.
I think the question assumed that you can run go in a script-language mode. Because you do want the source of shell-scripts on the local machine.
So it is easy to make changes by system admins/non-developers. Scripting is for glue functionality after all…
So it is easy to make changes by system admins/non-developers. Scripting is for glue functionality after all…
> So it is easy to make changes by system admins/non-developers. Scripting is for glue functionality after all…
You can also use env-vars and flags in Go.
Which many would argue leads to more robust "admin glue functionality", because the admin controls how much non-admins can mess around with instead of allowing them to mess around with the core shell-script functionality.
You can also use env-vars and flags in Go.
Which many would argue leads to more robust "admin glue functionality", because the admin controls how much non-admins can mess around with instead of allowing them to mess around with the core shell-script functionality.
(i don't mind if people use a rude tone, we're human. i just don't like when people are wrong, or interpret others wrong)
in corporate environments, it is not easy to (nor should it be) to require every desk to install Go. If you are hired to script something (and scripts are even more important in corporate environments) and require a new piece of software to be installed, and a new language for staff to maintain, it's imposing a great burden and could very well be rejected.
I don't not know what I'm talking about, and I'm not trying to FUD, I'm trying to suggest "gird your loins". But also, I said that was not the point of my comment, I just didn't want you to think that through my silence I had accepted your earlier suggestion.
in corporate environments, it is not easy to (nor should it be) to require every desk to install Go. If you are hired to script something (and scripts are even more important in corporate environments) and require a new piece of software to be installed, and a new language for staff to maintain, it's imposing a great burden and could very well be rejected.
I don't not know what I'm talking about, and I'm not trying to FUD, I'm trying to suggest "gird your loins". But also, I said that was not the point of my comment, I just didn't want you to think that through my silence I had accepted your earlier suggestion.
Two catches I could see:
* Great if you can do everything natively in Go, loses advantages and I suspect gets somewhat cumbersome if you have to shell out.
* Harder to inspect on target system. If I encounter a random shell script someone left on a server, I can read it; with a binary I'd have to hope I can find the source.
Otherwise... yeah, that seems reasonable, and if you're already good at Go, sensible.
* Great if you can do everything natively in Go, loses advantages and I suspect gets somewhat cumbersome if you have to shell out.
* Harder to inspect on target system. If I encounter a random shell script someone left on a server, I can read it; with a binary I'd have to hope I can find the source.
Otherwise... yeah, that seems reasonable, and if you're already good at Go, sensible.
Doesn't that mean that the binary is platform dependant? i.e. you need a different binary to run on Intel vs Arm?
Even more controversial, I use Swift.
Just as controversial, I use Haskell. Everything has a type, no more interpolation foot-guns, etc. And it pays off in slowly building a library on top. (Much if not all of it applies to Swift as well I believe.)
> slowly building a library on top
What does this part mean?
What does this part mean?
swift is so nice to code with
Controversial opinion: I use Python for scripting. Terse syntax, but more structured than bash.
If Go why not something like Python though...?
Because Python is godawful for dependencies.
With Go, you write, you compile, you ship the binary, job done.
With Go, you write, you compile, you ship the binary, job done.
Is Go genuinely as dependency independent as bash on a fresh Linux install? If so I'd find this very interesting and perhaps worthwhile to learn. I work in an environment where everything is done at the Linux command line. I'd be thrilled to re-do some of my my complex bash scripts in Go if there is a good advantage to doing so.
The idea is that if you need dependencies, you only need them present during compilation on the machine you develop on. You then compile a statically linked binary and copy this to the machine where it's needed.
There may be other advantages like static types, editor integration and so on, but that's beside the point.
There may be other advantages like static types, editor integration and so on, but that's beside the point.
Go's dependency management is in a really good state. For building your code, Go modules are generally easy to use and editor integrations exist to automatically update your dependencies as you type. Then you build a static binary and distribute it, no installers or packages required.
For the kind of things I do with bash scripts, the python standard library is usually more than enough.
That sounds like a lot of steps when you require interaction (e.g. for bash I list files, then try find, then xargs with echo, etc until I got what I wanted..)
Incidentally this is why I think an O'Reilly is one of the most cost-effective subscriptions that I pay. It is not cheap, for sure, but I can find most book recommendations like this on it.
Also worth checking your public library to see if they're already paying for you to have a subscription. The DC Public Library offers it here.
Ok this may be a little over the top but in my experience there’s a better way to learn bash: implement bash. I did this for a school project, along with job control, pipes, redirects, builtins. All of the trip ups mentionned in this article will become obvious.
This is good advice. Whenever I'm trying to recreate something that exists, I often run into what feels like forced decisions that make my reimplementation closer to the original. And then some seemingly random thing in the original becomes logical.
[deleted]
The only way to learn bash ;-)
https://leanpub.com/learnbashthehardway
https://leanpub.com/learnbashthehardway
$ uname
Darwin
$ which cd
/usr/bin/cdHuh ? Which OS X are you on ?
What you posted is not reproducable on Ventura (13.6)
What you posted is not reproducable on Ventura (13.6)
% uname
Darwin
% which cd
cd: shell built-in commandDepends on your shell, if you execute `which -a cd` it will show you /usr/bin/cd in addition to the built-in command.
(zsh has which as a built in command, apparently bash doesn't, which causes the different output). It's unclear to me what /usr/bin/cd actually accomplishes though, even in bash.
(final edit I hope, I found an explanation of sorts for /usr/bin/cd: https://unix.stackexchange.com/a/50060)
(zsh has which as a built in command, apparently bash doesn't, which causes the different output). It's unclear to me what /usr/bin/cd actually accomplishes though, even in bash.
(final edit I hope, I found an explanation of sorts for /usr/bin/cd: https://unix.stackexchange.com/a/50060)
> It's unclear to me what /usr/bin/cd actually accomplishes though, even in bash.
Seems to be something they adopted from FreeBSD if the comment in the script is anything to go by ? (`/usr/bin/cd` is a shell script, you can cat it).
It seems something to do with it being a POSIX system if this SO answer[1] is anything to go by.
Beyond my paygrade though, and given Apple's shell appears to default to the built-in it seems I don't need to care either. :-)
[1] https://unix.stackexchange.com/questions/50022/why-cant-i-re...
Seems to be something they adopted from FreeBSD if the comment in the script is anything to go by ? (`/usr/bin/cd` is a shell script, you can cat it).
It seems something to do with it being a POSIX system if this SO answer[1] is anything to go by.
Beyond my paygrade though, and given Apple's shell appears to default to the built-in it seems I don't need to care either. :-)
[1] https://unix.stackexchange.com/questions/50022/why-cant-i-re...
I’m on Ventura 13.6.1. Looks like your which is a shell built-in as well. Try
ls /usr/bin/cdIt was the second question that convinced me to buy it.
That’s a legitimate reason :D
I think the pretty animal illustrations is what initially caught my attention and helped me discover O’Reilly as a publisher to pay specific attention to in IT books. The animals trick you into opening their books, and the texts usually have you continue reading.
I think the pretty animal illustrations is what initially caught my attention and helped me discover O’Reilly as a publisher to pay specific attention to in IT books. The animals trick you into opening their books, and the texts usually have you continue reading.
goimonkeen(1)