Pure Bash Bible – A collection of pure bash alternatives to external processes(github.com)

291 pointsby rrampage4 tahun yang lalu96 comments

Pure Bash Bible – A collection of pure bash alternatives to external processes

https://github.com/dylanaraps/pure-bash-bible

99 comments

infogulch4 tahun yang lalu

It's Bash Month on Hacker News, apparently:

My thoughts on writing a Minecraft server from scratch in Bash | https://news.ycombinator.com/item?id=30347501

Bash Pitfalls | https://news.ycombinator.com/item?id=30345487

Show HN: Toolbox – A framework for modular Bash scripts | https://news.ycombinator.com/item?id=30067708

Use Bash Strict Mode | https://news.ycombinator.com/item?id=30217719

A Wordle clone in 50 lines of Bash | https://news.ycombinator.com/item?id=30174112

krnlpnc4 tahun yang lalu

I've been sensing a stigma against shell scripting increasingly over the past decade. It's nice to see bash being used in creative and elaborate ways like this.

AceJohnny24 tahun yang lalu

The stigma is well deserved and I will enthusiastically spread it.

Bash should be considered an esoteric programming language. Shell Programming Considered Harmful.

It's hard to program, and easy to make mistakes in. It should be left as a tool of last resort.

(that said, I understand the appeal of doing weird things with it, like the Minecraft server, for the same reason other esoteric programming languages have appeal)

asdff4 tahun yang lalu

What is so hard about it? It feels like python but with an even more english language like syntax, and the man files can be like tomes of knowledge. 'head file.txt' is a heck of a lot easier to wrap your own head around than reading something into an object and calling head on it. Python programmers today would probably import pandas for that.

scubbo4 tahun yang lalu

Not the person you replied to, but - yes, `head file.txt` is perfectly reasonable, as are a few other use-cases like `grep`/`sed`-ing. But when you go beyond "processing streams of lines of text" and start having to introduce conditionals, maps, or functions, the syntax is (for me, at least) opaque and easy to make mistakes in.

marginalia_nu4 tahun yang lalu

Isn't that also a case against learning new programming languages?

vore4 tahun yang lalu

I think bash has a lot of gotchas due to its original purpose, and it's definitely more error-prone to write bash scripts than to e.g. write Python scripts, so that should be a consideration.

Ideally we should all try to more programming languages that are harder for us to shoot ourselves in the foot with, rather than easier ;-)

chasil4 tahun yang lalu

I will just leave this here.

"I am a strong believer that Bourne-derived languages are extremely bad, on the same order of badness as Perl, for programming, and consider programming sh for any purpose other than as a super-portable, lowest-common-denominator platform for build or bootstrap scripts and the like, as an extremely misguided endeavor. As such you won’t see me spending many words on extensions particular to ksh, Bash, or whatever other shells may be popular."

http://www.etalabs.net/sh_tricks.html

marginalia_nu4 tahun yang lalu

I think it's easy to sort of gloss over how good shell scripts are at the sort of things they actually are good at.

Consider this:

  for f in *png; do convert $f $(echo $f | sed 's/png$/webp/'); done

This is like at least a screen of code in in most languages, you get that sinking feeling where you need to fork a process and deal with its lifecycle and deal with some standard library to list files and don't forget to close the file handle for the directory and god forgive if you run a program with actual output then you need to read from a pipe. Like, geesh, do I have to write a whole program to convert a few files?

In a shell, it's just something you type up because you needed to convert a few files. Don't get me wrong, shell scripts are extremely bad at most things involving non-trivial branches, long-lived state, really most things other languages are good at, but man are they ever good at some things most programming languages are bad at.

_kst_4 tahun yang lalu

Better:

    for f in *.png; do convert "$f" "${f/%.png/.webp}" ; done

Quoting the arguments lets it correctly handle file names with funny characters, including spaces. Using the `${parameter/pattern/string}` parameter substitution means the shell does all the work itself without invoking an external command (other than convert, of course). The '%' in the parameter substitution requires the pattern to match at the end, so "foo.png.png" becomes "foo.png.webp" and not "foo.webp.png" -- an unlikely corner case, but you might as well do it right. ("foo.png.blah" doesn't match "*.png" in the first place.)

For elaborate commands like this, I typically run it once with an "echo" inserted ("echo convert ..."), confirm that the resulting commands make sense, and then hit up-arrow to edit the command and delete the echo.

https://www.gnu.org/software/bash/manual/html_node/Shell-Par...

Flimm4 tahun yang lalu

Quoting is not enough, as that won't handle filenames beginning with dash correctly.

leephillips4 tahun yang lalu

    lee@sp3tmp1:ls *.png
    'a b.png'   c.png
    lee@sp3tmp1:for f in *png; do convert $f $(echo $f | sed 's/png$/web'); done
    convert-im6.q16: unable to open image `a': No such file or directory @ error/blob.c/OpenBlob/2924.
    convert-im6.q16: no decode delegate for this image format `' @ error/constitute.c/ReadImage/575.
    convert-im6.q16: unable to open image `b.png': No such file or directory @ error/blob.c/OpenBlob/2924.
    convert-im6.q16: unable to open image `b.png': No such file or directory @ error/blob.c/OpenBlob/2924.
    convert-im6.q16: no decode delegate for this image format `' @ error/constitute.c/ReadImage/575.
    convert-im6.q16: no images defined `b.webp' @ error/convert.c/ConvertImageCommand/3229.

[etc.]

I think you could be making a more complicated point.

chasil4 tahun yang lalu

Your script fails if the files referenced by $f contain white spaces or a newline.

You should script defensively, for when this is run on Windows or elsewhere [maybe Android] that will break this assumption.

Newlines can also be part of a filename. For extreme rigor, use find -print0 and xargs -0.

I have covered some of this here:

https://www.linuxjournal.com/content/parallel-shells-xargs-u...

(Somebody took me to the Hacker News front page for this, and nobody said that I made a mistake.)

marginalia_nu4 tahun yang lalu

> You should script defensively, for when this is run on Windows or elsewhere [maybe Android] that will break this assumption.

I think you are missing the point. I wouldn't run a shellscript on Windows or any other place than the terminal I entered it into. It's an expression of something I want to do at that particular time in that particular directory.

The entire point of my post is that if I wished to write something portable and robust, I wouldn't use bash in the first place. That's not what bash is good at. But quick one-offs, holy crap is it good at that.

chasil4 tahun yang lalu

I've run many Borne/POSIX shell scripts on Windows for things that I've needed to do.

The most straightforward POSIX shell on Windows is the Busybox port, which uses the Almquist shell (ash) with many bashisms reintroduced.

https://frippery.org/busybox/

kwhitefoot4 tahun yang lalu

> fails if the files referenced by $f contain white spaces or a newline.

This is said every time Bash scripting is mentioned. Why people can't get into the habit of quoting all variables in Bash is beyond me and the mention of white space makes sense; but how often, if ever, have you encountered a file name that contained a newline?

Most of the Unix tools are line oriented by default anyway and are frequently used to manipulate lists of file names and paths.

chasil4 tahun yang lalu

It's safe to omit quotes for POSIX shell variables that are initialized as numeric, and you enforce to remain numeric.

All else should be quoted, and newlines can happen, primarily from abuse. Do not trust user input!

Calling fopen() on a file with every ASCII character will work on POSIX, as long as it does not contain a null or forward slash (/). Coding defensively means that this does not crash your processing.

Flimm4 tahun yang lalu

I agree. It also fails if the filename starts with a dash.

pizza2344 tahun yang lalu

>> I think it's easy to sort of gloss over how good shell scripts are at the sort of things they actually are good at.

>> Consider this:

> for f in *png; do convert $f $(echo $f | sed 's/png$/webp/'); done

This is buggy, and it actually represents exactly the problem with Bash scripting.

marginalia_nu4 tahun yang lalu

I think considering the purpose of the language, it does rather well. It's really easy to spawn and pipe processes, far more so than in almost any alternatives. Like it's so easy that it's easy to forget that this is like a 5-10 line overhead dance to launch and read from a process in most languages.

The problem usually come when you attempt to use it for nontrivial programming. But that just not what it's for, any more than the difficulty in building an OS kernel is a legitimate complaint against PHP.

chasil4 tahun yang lalu

Stephen Borne famously disliked C, and much preferred Algol.

This is important to know in the context of what you are saying.

The Algol structures in the Borne shell are the most incongruous parts of UNIX.

https://research.swtch.com/shmacro

vore4 tahun yang lalu

I think this is what the post you were replying to is saying.

suifbwish4 tahun yang lalu

By that logic shouldn’t we also stop using programming languages that aren’t threadsafe? Segfaulting could be prevented if C++ wasn’t so esoteric.

_dain_4 tahun yang lalu

>What is so hard about it?

quick: without looking anything up, write something in bash that takes a string of comma-separated values and turns it into an array of those values.

then check to see if your answer is on this very long list of wrong implementations:

https://stackoverflow.com/a/45201229/6107981

asdff4 tahun yang lalu

Easy, I just avoid these sort of mind game situations when programming, and I come up with a simple 'wrong' approach that will probably work fine for my purposes anyhow. I don't care if my implementation breaks if I have a newline in my data or whatever, because I already know based on what I am working on that I won't have to worry about a newline in my data or any other weird quirks they bring up in stack overflow comment sections that I have yet to come across in my years of writing code that gets the job done. I don't care if my solution is fractions of a second slower, it's probably already going to be mindbendingly fast compared to the code running on hardware from 40 years ago when people had to be a lot more mindful about selecting the most performant solutions when writing in bash or anything else.

_dain_4 tahun yang lalu

Okay, but how do you know that those weird quirks won't be present in your data? Or what those weird quirks even are? The details of the stuff in that Stackoverflow comment have already fallen out of my head; I couldn't tell you about them now. With Bash there are so many traps and footguns and (subjective) unknown unknowns, it would keep me up at night worrying about it. There's always another maddening fact to discover.

I would just use Python, where strings are sane and lists are sane: my_string.split(", "), boom, done, no jumpscares.

pyuser5834 tahun yang lalu

I can tell you’ve dealt with lots of Python programmers.

rascul4 tahun yang lalu

> Bash should be considered an esoteric programming language. Shell Programming Considered Harmful.

I disagree. Bash is fine. Nothing esoteric about it.

> It's hard to program, and easy to make mistakes in. It should be left as a tool of last resort.

It's easy to program, but it has a learning curve. Use shellcheck. It's OK to go first to bash if you need a shell script. Other languages make pretty crappy shell scripts.

pizza2344 tahun yang lalu

I copy/paste from a previous post. "Exoteric" is a stretch (in particular, because there are no alternative glue languages, if we exclude all the shell variants), but cryptic, unintuitive and error-prone fits well.

Arrays:

  - `${#myvar[@]}`: unreadable

  - `mapfile myvar < <(grep pattern file)`: "mapfile" is not exactly a clear term (there's the synonym
    "readarray", but "mapfile" is the reference; worse, you may find both used inconsistently); the
    command as I wrote it also has two problems

Strings:

  - `${myvar##.*}`: unreadable

  - `echo $(IFS=,; echo "${myvar[*]}")`: unreadable; most also don't know the difference between `[@]`
    and `[*]`; this doesn't also work for two-char join operations

  - `[[ $myvar =~ $(echo '\bpattern\b') ]]`: unreadable, and most don't know why command substitution
    is needed

Redirections:

  - `3>&2 2>&1 1>&3`: most don't know what this means

marklgr4 tahun yang lalu

Yes, it can be cryptic and not beginner-friendly, but it's powerful and once you know it, you know it; it will be useful for years if not decades, as opposed to some tools or frameworks du jour that are sometimes as cryptic.

The main issue is that many people try (by necessity or not) to cobble scripts together without really learning the language and its idioms. It's not that hard to really learn it though, it's basically one man page then the mandatory [wooledge wiki][1].

[1]: http://mywiki.wooledge.org/BashGuide

rascul4 tahun yang lalu

I don't understand, none of that is unreadable or weird to me. Are you sure the issue is not a lack of familiarity with the language? The language can appear deceptively simple in the beginning.

kdtsh4 tahun yang lalu

And what a tool of last resort it is.

It may be much easier (and most of the time I would generally prefer) to use Python for scripting purposes, but Bash is a) almost always available and b) widely portable.

If you manage any servers where you never know if Python is available or if it’s version 2 or 3 (and you have no power to change this for one reason or other), you know the value of this. No, it’s not ideal, but we live in a non-ideal world. Long live Bash.

AtlasBarfed4 tahun yang lalu

Bash has the highest rate of Stackoverflow searches per line of code produced. I swear I do about 10 searches per line of code produced.

And when you consider there really isn't a standard library, mostly gee-I-hope-this-UNIX-util-is-installed...

francis-io4 tahun yang lalu

I think the big thing going for it is it's low footprint and mostly universal existence on all servers I work on.

For example, I'm working on a Hashicorp Vault server install and due to it's sensitive nature, I'm strongly considering using bash to set it up over something like python. I love the ability to test in a robust language, but the extra dependencies are hard to justify, at least on a security sensitive application.

einpoklum4 tahun yang lalu

> hard to program

Well, we "write" one-liner programs with it all day every day - since it's the shell.

I'm not saying bash doesn't have a bunch of warts, but - what other shell language would you suggest as an alternative to it?

usrbinbash4 tahun yang lalu

The stigma is not against shell scripting, the stigma is against using shell scripting for things and in settings it wasn't intended for.

Don't get me wrong, I have nothing against fun and creative projects. If someone wants to script something complicated in bash for the lulz, the challenge, to learn something, or just to show off bash-skills, I'm all for it.

What I am against, is when I get asked to debug a 2000 LOC system used in production that was thrown together in bash, waste 2 days on it, only to end up doing what should have been done in the first place, and rewrite the damn thing in 200 lines of Python or 400 lines of Go.

It can, and should, be used to automate some things, string some commands together, make certain tasks easier. I use and write bash scripts in my daily work, and make my life easier with it.

But there comes a point, and that is sooner rather than later, when something should no longer be written in bash, but in a language that wasn't designed primarily as a command line interpreter. Can I make large complicated systems in bash? Sure. Can I carve the handle of a hammer so that it can be used as a screwdriver? I'm sure someone more skilled in woodworking than me could do it. But sometimes the question isn't if it can be done, but if it should.

atarian4 tahun yang lalu

Yeah it gets bashed a lot.

einpoklum4 tahun yang lalu

Yet we continue to use it unabashed.

dspillett4 tahun yang lalu

There has always been stigma against shell programming, not shell scripting, and I think that more-or-less is still the case. It isn't so much anti-bash stigma as, using a less than ideal tool for the job being a smell that'll cost you to fix later.

Shell scripts to automate processes with other tools is exactly what they are designed for, and adding in other logic and loops is fine too, but if you are setting out to create what would be considered a full program rather than an automated task you should definitely consider alternatives (of which there are several suitable choices) that usually end up being easier to maintain.

It used to be that the main alternative was perl, but frankly that was/is no better (IMO much worse) than bash for encouraging the generation write-only (difficult to maintain) code. Python is often a good option, and I've seen node used, though these don't have the ubiquity that sh/bash does (or perl used to have) which is one advantage of bash: it is almost everywhere, even Windows these days (or stick with core posix sh and you'll find support a little wider again), though maybe without some of the supporting externals which is when tricks like these come in.

If you are writing some simple automation, or maybe slightly less simple, then bash is fine as a go-to, but if you are setting out to write something complex bash is a bad choice unless compatibility with many environments you can't dictate have another interpreter present is a non-negotiable requirement.

There is the grey area when a script, or collection of scripts, becomes a larger program/environment over time. Sometimes the effort to rewrite (and fully test) in another environment is not worth the small benefit of doing so, and is therefore a bad use of developer/admin time that could be used elsewhere, particularly when high performance is not an important feature. Though if you find yourself with a lot of bash script, do yourself (and your later replacement when you've moved on to newer projects) a massive favour and make sure it is well documented even if that is only in in-script comments.

jrm44 tahun yang lalu

As someone who is genuinely a fan of stupid pointless hacks, like Turing Completeness with CSS or Minecraft or whatnot, in my opinion this is definitely the stupidest. Kudos.

The entire point of Bash/Shell scripting is (or darn it, ought to be) to use it to string together OTHER programs to get stuff done. "Pure Bash" to me is like a bar of soap that never gets wet.

xelxebar4 tahun yang lalu

> The entire point...

https://en.wikipedia.org/wiki/The_purpose_of_a_system_is_wha...

Prescriptivism is perhaps helpful when starting out on a project and you need to prune down a vast tree of possibilities, but once you already have a thing, prescriptivist declarations are pretty impotent at helping you understand how that thing works.

If your bash script has some tight loop, calling out to external tools can incur a huge performance overhead. Rewriting in bash can aleviate that. How do you re-write it in bash? Well... this article provides good fodder for thought!

> like a bar of soap that never gets wet.

Here are some uses of soap that would be hampered by it getting wet:

- Make-shift lubricant on that sticky cabinet drawer

- Draw temporary markings, like a crayon

- Potpourri alternative

- Fragrant seal on your letters

- Deterrent to puppies chewing on your chair legs

chasil4 tahun yang lalu

Windows is your foil.

On the same hardware, Windows is 100x slower at forking an external process.

When you are using a (POSIX) shell on Windows, don't cause a fork, especially in a loop.

Do everything internally in a POSIX shell on Windows, unless an enormous time penalty is acceptable.

jrm44 tahun yang lalu

Sure. I mean, I'm not being pure prescriptivist here; always good to explore and mess about with things.

But if I'm getting a tad meta, I think there is generally enough of what you're doing (experts really hacking at things to push their limits) and not enough of what I think Bash/scripting could provide -- namely non-experts discovering "coding" NOT through the typical "start with a real language and do hello world" and instead through, "Here's bash; it's for stringing together other programs to do stuff."

jayceedenton4 tahun yang lalu

To be fair, when writing scripts it's easy to fall foul of differences between GNU and BSD tools, or using an external tool that isn't available by default on some systems.

Pure bash solutions avoid these problems, and can be just as simple as (or simpler than) an alternative that use external tools.

wink4 tahun yang lalu

Until you're happy you solved it in a nice way and then learn that your script will run on some ancient RHEL with bash 3 ;)

woudsma4 tahun yang lalu

Nice to see that the author uses Shellcheck. There is a very nice Shellcheck VS Code extension that links examples of good/bad practices when writing Bash. I can definitely recommend using it if you’re writing Bash scripts!

usrbinbash4 tahun yang lalu

    Use #!/usr/bin/env bash instead of #!/bin/bash.
    
    The former searches the user's PATH to find the bash binary.
    The latter assumes it is always installed to /bin/ which can cause issues.

And the former assumes that `env` is installed in `/usr/bin`, so I am trading one assumption for another.

fuzzybear39654 tahun yang lalu

Maybe I shouldn't pick a battle with someone that has that handle .

But, in all seriousness, you're right - you're making a trade. The question is which side of the trade is more portable. I use NixOS which doesn't have bash in /bin. So, I've had to ask people on my team to replace the hashbang. The /usr/bin/env hashbang has yet to fail on any machine so, anecdotally, it seems more portable.

usrbinbash4 tahun yang lalu

If something is portable to every *nix system which uses the Filesystem Hierarchie Standard (FHS), it's portable enough for every system I ever encountered or worked with.

This isn't me ranting against systems with new ideas about file system layout, this is just me being pragmatic: When it works on all my work-, private- and production-machines, I see no reason to run an extra binary in front of all my shellscripts.

fuzzybear39654 tahun yang lalu

If it doesn't support any of your users/co-workers then there's no need. But, there's nothing requiring the enforcement of the FHS in the linux kernel, so, generally, you can't take it's conformance for granted.

If something works for X% of people and a trivial change makes it work for Y% of people (Y > X) then I advocate for making that trivial change.

ChrisArchitect4 tahun yang lalu

Plenty of previous discussion:

1 year ago https://news.ycombinator.com/item?id=24827360

2 years ago https://news.ycombinator.com/item?id=21013150

kwertyoowiyop4 tahun yang lalu

I hope to never read or even use such code. I’ll install as many versions of Python as it takes to avoid this fate.

[deleted]4 tahun yang lalu

ilyash4 tahun yang lalu

Nice... but how you fix the way something fundamental as the way `if` works in bash? Each `if` is a bomb. `if grep -q ...;then ...;else ...;fi`. Two branches for three possible exit codes. Given a long script and big amount of output, you will miss the error message such as "file not found" and the execution will peacefully continue into the else branch.

Things like the above motivated me to create Next Generation Shell.

Edit: hint - yes branch, no branch, exception.

More about motivation: https://ilya-sher.org/2020/10/31/bash-or-python-the-square-p...

Project: https://github.com/ngs-lang/ngs

tomcam4 tahun yang lalu

Awesome that it was made free on github. Of course I donated.

xiaq4 tahun yang lalu

If you're interested in a shell that allows to trim a string without decoding the following:

    trim_string() {
        # Usage: trim_string "   example   string    "
        : "${1#"${1%%[![:space:]]*}"}"
        : "${_%"${_##*[![:space:]]}"}"
        printf '%s\n' "$_"
    }

You may be interested in Elvish (https://elv.sh), which simply has a builtin function for trimming strings: https://elv.sh/ref/str.html#str:trim-space

nomel4 tahun yang lalu

Related, here's a complexity vs effort graph that I made for our company wiki:

https://i.imgur.com/sGPPOhn.jpeg

chasil4 tahun yang lalu

In a POSIX shell, I would do it like this:

  trim_string () (
  x=${1%[:space:]}
  return ${x#[:space:]}
  )

Note the use of a subshell ( instead of command grouping { because POSIX shells do not have local function variables.

At least, I think that would work.

On Windows, forking is expensive, so I wouldn't do it this way.

dncornholio4 tahun yang lalu

After reading that function, I know why I stay the heck away from bash programming.

rossy4 tahun yang lalu

These might make your bash scripts a bit faster in Linux, but they will make them a lot faster in Cygwin and MSYS bash in Windows, where process spawning is very slow. Seriously, if there's even a chance of someone running your bash script in Windows (you might be surprised - every bash.exe is either Cygwin or MSYS, including the one that comes with Git for Windows,) your Windows users will thank you for using pure bash instead of external commands.

strzibny4 tahun yang lalu

Great to see more Bash here. I think for me it comes down to size. If I just need to spin up one server, it's surprisingly easy (as I show in Deployment from Scratch), but of course if I have to do something more complex, I would prefer Ruby/Python. Certainly bookmarking this resource, thanks!

xvilka4 tahun yang lalu

There's interesting cross-platform "shell" that compiles to either Bash or cmd (bat) - Batsh[1].

[1] https://github.com/batsh-dev-team/Batsh

frazbin4 tahun yang lalu

The manual is pretty good too https://www.gnu.org/software/bash/manual/bash.html

r3trohack3r4 tahun yang lalu

Dylan is an incredible mind - he’s behind KISS Linux and neofetch. KISS is worth looking into as an alternative to Linux From Scratch - I learned a lot with it.

cferr4 tahun yang lalu

Another way to touch a file:

cp /dev/null file

Test if a port is listening without telnet:

echo > /dev/tcp/address/port && echo Yes || echo No

That's not actually a file. It's just a path Bash recognizes and is the interface to socket functionality. udp works too.

IncRnd4 tahun yang lalu

> Another way to touch a file: cp /dev/null file

That destroys a file, making it 0 bytes. It doesn't touch a file, not at all.

_kst_4 tahun yang lalu

And if "file" is the name of an existing directory, it creates an empty file called "file/null".

"touch" exists for a reason.

cferr4 tahun yang lalu

If the file does not exist, it creates an empty file. That is what most people use touch for.

IncRnd4 tahun yang lalu

That's one use case.

The actual use case touch was designed for was to update the timestamp of a file. This is a huge case for developers, since timestamps are used in dependency resolution of partial compilation. [1]

[1] https://man7.org/linux/man-pages/man1/touch.1.html

  NAME
       touch - change file timestamps
  DESCRIPTION
       Update the access and modification times of each FILE to the
       current time.

[deleted]4 tahun yang lalu

jteppinette4 tahun yang lalu

I’m sorry, but I almost spit my drink out at how unreadable the very first example is.

  trim_string() {
      # Usage: trim_string "   example   string    "
      : "${1#"${1%%[![:space:]]*}"}"
      : "${_%"${_##*[![:space:]]}"}"
      printf '%s\n' "$_"
  }

pavon4 tahun yang lalu

To be fair that is one of the worst. I'm definitely bookmarking this site. Even if I won't use several of the functions, it is really valuable to have a single place to check to see "yeah, that really is the best you can do with bash, I'll exec out instead", rather than scouring through multiple StackOverflow answers that are subtly wrong.

And my bash code would probably be more readable if I started collecting common operations in a function library rather than repeating the same sed/awk incantations that aren't super readable. At that point might as well use pure bash/sh implementations to save spawning a new process when reasonable.

avgcorrection4 tahun yang lalu

String substitution (or whatever it is called?) looks so bad in Bash. Similar to the sigils for history expansion. I’ll just stick to Up, Down, and Ctrl+R.

lubesGordi4 tahun yang lalu

Bash syntax is so alien. The sentence before your example: "The : built-in is used in place of a temporary variable." So I was like wtf even before you spit your drink. Maybe someone can claim there's efficiency here?

chasil4 tahun yang lalu

It's actually a "no-op" -try it.

  if [ -z "$foo" ]
  then :
  else echo not blank "$foo"
  fi

mrspuratic4 tahun yang lalu

It is intrinsically a nop, but its context can have side effects because it's still a real command. One side effect is exploited here: setting $_ to the last argument of the previous command. The first description in the article of its use is unhelpful, it's not until the end in "Simpler case statement" that it's made clear.

I often use it instead of truncate or touch

: > somefile

: >> somefile

: is not lexically special (the way # is), you can use : in variable and function names, even create a function called just ":" if you're trying to confuse...

mttjj4 tahun yang lalu

I mean, does it really need to be readable? I kind of took this site to be “here’s some snippets of code that you can use in your scripts to use instead of calling an external process.” Which means to me, I’m going to copy/paste this function into my script, call the function (since it’s structured as one), and never think about it again.

“Learn how to write routines like this and make them readable” seems like it’s not the point of this “Bible”.

I’m not against readable code by any means (maybe there’s even a cleaner way to write this) but it seemed inconsequential in my opinion.

queuebert4 tahun yang lalu

> I mean, does it really need to be readable?

Yes, because it says it trims strings, but is otherwise unreadable to most people. If they have a bug in their script caused by this function not working correctly, good luck finding it.

chasil4 tahun yang lalu

Ideally, the function is POSIX-compatible, so it runs on the widest number of platforms.

Why do we want to adhere to POSIX?

  $ man bash | grep slow
         It's too big and too slow.

  $ rpm -qi dash | tail -3
  DASH is a POSIX-compliant implementation of /bin/sh that aims to be as small as
  possible. It does this without sacrificing speed where possible. In fact, it is
  significantly faster than bash (the GNU Bourne-Again SHell) for most tasks.

So let's find out how this function works in dash.

  $ ./testtrim
  ./testtrim

  $ sed -i 's/dash/bash/' testtrim          
  $ ./testtrim                              
  foo bar

I think I will pass.

bewuethr4 tahun yang lalu

There is https://github.com/dylanaraps/pure-sh-bible, including a trim function.

chasil4 tahun yang lalu

That is a better resource.

The two important places for portability are Ubuntu's /bin/sh (which is dash, not bash), and the Busybox shell.

Commercial unix would be important, but is likely a distant 3rd by the numbers.

kergonath4 tahun yang lalu

That’s why you put things like #!/usr/bin/env bash. You can argue about which version of bash supporting which feature, but being afraid to have your own scripts interpreted by POSIX sh is strange in the vast majority of situations. It’s like complaining that a C++ source would not be compiled by a strict C compiler. Or that Python 3 scripts should be valid Python 2 because you never know which version ‘python’ might be. Just use the right interpreter.

mkdirp4 tahun yang lalu

> Ideally, the function is POSIX-compatible, so it runs on the widest number of platforms.

But... Why? During my professional career of about 10 years, and even before that, I've not once had to require POSIX compatibility for any of the shell scripts I've written or come across. The few times I had a snippet that couldn't run I have been able to install bash 4+ with ease.

I'm sure there are times where POSIX is kind of needed, but it's becoming exceedingly rare.

_lqaf4 tahun yang lalu

The Linux monoculture in internet services does reduce the value of POSIX-compliance.

I got in the habit of at least gesturing towards POSIX back when it was more common to run a bunch of different unixes. And I still run both FreeBSD and Linux at home, so there's value there for me.

Mostly, I still do it because I'm always surprised by which artifacts I create end up being long-lived, and you never know what will be valuable in the future. One of the things I'm technically proudest of ended up only being useful for less than a year, whereas I know a dumb 4 line hack I wrote on a contract in 2002 is still running via cron every night, now on a VM in AWS.

chasil4 tahun yang lalu

I think it would be helpful if we can update the POSIX shell standard. This is the only way that the Debian/Ubuntu /bin/sh will ever change.

Microsoft was pivotal in the formation of the standard, as Korn was required to run in a 64k text segment for Xenix, and this was retained for POSIX.2.

The parser for the Borne family is very complex, and cannot be implemented with a yacc grammar. There is some value in starting again with something less convoluted.

This guy tried to implement a POSIX shell in OCaml, and he is rather perturbed with the entire Borne family (I wish I knew parsers this well):

https://m.youtube.com/watch?v=fiJR4_059HA

Bash has a number of real strikes against it... GPLv3 - which caused Apple to dump it, speed - which caused Debian/Ubuntu to demote it, and the lingering damage bashisms and their damage to portability.

chasil4 tahun yang lalu

This discussion of a high-performance POSIX shell for Windows is also enlightening.

https://video.fosdem.org/2019/AW1.125/ada_shell.webm

pavon4 tahun yang lalu

I can't think of any time where I have needed the same script to run on multiple systems, because that's not the sort of thing I use shell scripts for. But there are many times where I have needed to write a script for a specific system that was running busybox or dash or ksh, and it is worth knowing what features I can use in those cases.

kergonath4 tahun yang lalu

But then your source is probably not something called “Pure Bash Bible”.

chasil4 tahun yang lalu

I have a Korn shell script that I use to run SQL against a few dozen databases.

I recently rewrote this script, so it would run on Windows, via Busybox ash. It needed quite a few changes. I've become somewhat practiced on the removal of bashisms.

Oddly enough I had to put a few back. Busybox on Windows doesn't implement stty that I needed to read a password, but it did have read -s, so I used that instead. I threw several snippets of ash-emulated bash back in, that would never ever run on dash.

When you ask "but why?" this is precisely why - I need this functionality in a specific place, where you cannot go.

aloisdg4 tahun yang lalu

I am more and more on your side. If this function was a builtin, how many reader will check its implementation?

jteppinette4 tahun yang lalu

But it is not a builtin. Someone else is going to have to maintain this one day.

> insert quote about writing your code as if the person who will have to maintain it next is a violent psychopath who knows where you live

adhesive_wombat4 tahun yang lalu

This maybe true even if the next person to maintain that is yourself! I have certainly wished terrible vengeance to be visited on past me.

throwamon4 tahun yang lalu

Totally agreed. While we're at it, let's make Brainfuck the official POSIX shell language.

jamil74 tahun yang lalu

Yeah that’s pretty wild. This resource pops up here a bit and it’s cool but there’s always a point where I’m like why would I do this in Bash and not Python/Ruby or something else.

adhesive_wombat4 tahun yang lalu

I have found my life has improved where as soon as I can't find the answer to a "wtf how do I do this trivial in shell" question after a cursory search, it's a good time to stop, and seriously consider Python (or Perl, ok, ok).

Sometimes, it's genuinely better to plug on in shell (e.g. if you are just calling other scripts and chaining stuff), but, especially if you want some less-magical string or list manipulations, often Python/Perl will provide a more readable, more maintainable, more _testable_ script. YMMV with Perl.

INTPenis4 tahun yang lalu

Oh god, you know you've become anal about bash when you can actually vaguely understand what is going on.

I mean it's parameter expansion, just flip down to that section of the manpage, and then there is posix regex in there. Not too difficult to look up in the manpages.

[deleted]4 tahun yang lalu

freddref4 tahun yang lalu

It gets more readable the more often you read it

[deleted]4 tahun yang lalu