Ask HN: What to use instead of Bash / Sh for scripting?
105 コメント
A configuration management system may have you write e.g. YAML with Jinja2 so that you don't reinvent the idempotent wheel.
It's really easy to write dangerous shell scripts ("${@}" vs ${@} for example) and also easy to write dangerous Python scripts (cmd="{}; {}").
Sarge is one way to use subprocess in Python. https://sarge.readthedocs.io/en/latest/
If you're doing installation and configuration, the most team-maintainable thing is to avoid custom code and work with a configuration management system test runner.
When you "A shell script will be fine, all I have to do is [...]" and then you realize that you need a portable POSIX shell script and to be merged it must have actual automated tests of things that are supposed to run as root - now in a fresh vm/container for testing - and manual verification of `set +xev` output isn't an automated assertion.
It's really easy to write dangerous shell scripts ("${@}" vs ${@} for example) and also easy to write dangerous Python scripts (cmd="{}; {}").
Sarge is one way to use subprocess in Python. https://sarge.readthedocs.io/en/latest/
If you're doing installation and configuration, the most team-maintainable thing is to avoid custom code and work with a configuration management system test runner.
When you "A shell script will be fine, all I have to do is [...]" and then you realize that you need a portable POSIX shell script and to be merged it must have actual automated tests of things that are supposed to run as root - now in a fresh vm/container for testing - and manual verification of `set +xev` output isn't an automated assertion.
> avoid custom code and work with a configuration management system test runner
ansible-molecule is a test runner for Ansible playbooks that can create VMs or containers on local or remote resources.
You can definitely just call shell scripts from Ansible, but the (parallel) script output is only logged after the script returns a return code unless you pipe the script output somewhere and tail that .
> manual verification of `set +xev` output isn't an automated assertion.
From "Bash Error Handling" https://news.ycombinator.com/item?id=24745833 : you can display the line number in `set -x` output by setting $PS4:
ansible-molecule is a test runner for Ansible playbooks that can create VMs or containers on local or remote resources.
You can definitely just call shell scripts from Ansible, but the (parallel) script output is only logged after the script returns a return code unless you pipe the script output somewhere and tail that .
> manual verification of `set +xev` output isn't an automated assertion.
From "Bash Error Handling" https://news.ycombinator.com/item?id=24745833 : you can display the line number in `set -x` output by setting $PS4:
export PS4='+(${BASH_SOURCE}:${LINENO}) '
set -x
I’ve seen people use Go because it’s a simpler language. It’s what I would probably use if I didn’t want to use bash.
What do you think about editing the title to include “for scripting”? I thought you were asking for recommendations like zsh or fish when I first clicked
What do you think about editing the title to include “for scripting”? I thought you were asking for recommendations like zsh or fish when I first clicked
Good point, changed.
Also go has the advantage of producing a single static binary. Easy to build on your machine and run on another host.
Which is why I'm writing most of my tooling in Rust.
Deployment or installation is just one scp away.
not rewriting, just when It needs large refactoring, or is currently a mess, or brand new.
Deployment or installation is just one scp away.
not rewriting, just when It needs large refactoring, or is currently a mess, or brand new.
Assuming you're deploying to the same OS/architecture that you're working from, otherwise you'd have to rebuild (which Rust makes easy to do, but still)
True.
Though in all those cases, for me, all alternatives are just as bad or worse.
Getting ansible running on a windows server is... unfortunate. Ensuring that your bash script runs on OSX is easier, but I've had lots of unexpected issues with tools like grep or find working just a tiny bit different. And so on.
Though in all those cases, for me, all alternatives are just as bad or worse.
Getting ansible running on a windows server is... unfortunate. Ensuring that your bash script runs on OSX is easier, but I've had lots of unexpected issues with tools like grep or find working just a tiny bit different. And so on.
Node scripts, even with dependencies, are pretty much 100% equivalent across all platforms :)
How easy is it to cross-compile with Rust?
Trivial. All of the tooling, dependency management, linking, etc. is neatly bundled together under cargo. Just run "cargo build" for the current system, or specify one or more other architectures as arguments to get multiple binaries. Everything's statically-linked so you should be able to scp those wherever.
Still, when we're talking about casual "scripts" even this extra step seems a little bit burdensome
Still, when we're talking about casual "scripts" even this extra step seems a little bit burdensome
Have you tried other shells like csh or tcsh?
The oldie Tom Christiansen anti (t)csh post is worth a read for those considering alternatives.
https://shlomif.github.io/open-source/anti/csh/ has a link to it, and other things.
https://shlomif.github.io/open-source/anti/csh/ has a link to it, and other things.
or dtksh ?
You might want to experiment with Powershell. It runs on Linux.
You might not adopt it, but it can give perspective on the pros and cons that comes with Bash.
You might not adopt it, but it can give perspective on the pros and cons that comes with Bash.
Just use PowerShell. You can do so much in it without any dependency (jq is native in it and many other things).
It is cross-platform so you can use your scripts anywhere.
Don't use python or go or ruby or whatever as those are not scripting languages.
IMO Bash is horror and should die already.
It is cross-platform so you can use your scripts anywhere.
Don't use python or go or ruby or whatever as those are not scripting languages.
IMO Bash is horror and should die already.
I use Powershell in Windows and am a big fan. It is a biiiiiiiiiiiiiiiiiiiig download on Linux, making it pretty impractical managing many different Linux machines.
> It is a biiiiiiiiiiiiiiiiiiiig download
How big really? Can't be more then 200MB? Should be less then minute to download and install? If you have docker, can you go that route?
Also, if you account the fact that you have variants of many other popular tools included OTB is that really a downside ?
That is certainly different concern, you want fast and practical script programming and why would you care if it has first time latency of couple of minutes if you are set good forever after that ?
How big really? Can't be more then 200MB? Should be less then minute to download and install? If you have docker, can you go that route?
Also, if you account the fact that you have variants of many other popular tools included OTB is that really a downside ?
That is certainly different concern, you want fast and practical script programming and why would you care if it has first time latency of couple of minutes if you are set good forever after that ?
I just checked, if you already have .NET installed, the powershell package is about 20 MBs, which is… tolerable. However, I see that the latest version still starts very slowly compared to *nix shells (and also Python).
The shell was running from tmpfs (so basically a RAM disk). I have a decent processor and an NVMe SSD, and this is what I see (it was tested 10 times in a row to pull everything into the page cache properly, and the numbers are pretty stable):
It's even slower on an older Windows laptop which I have lying around (starts in about 3-4 seconds). Absolutely terrible for an interactive shell/script execution engine.
Just for comparison's sake, the same laptop can run an empty bash script in 2-3 milliseconds. This is very important when running build jobs, for example.
The shell was running from tmpfs (so basically a RAM disk). I have a decent processor and an NVMe SSD, and this is what I see (it was tested 10 times in a row to pull everything into the page cache properly, and the numbers are pretty stable):
$ time ./pwsh <<<"" >/dev/null
real 0m1.050s
user 0m1.464s
sys 0m0.067s
It's even slower on an older Windows laptop which I have lying around (starts in about 3-4 seconds). Absolutely terrible for an interactive shell/script execution engine.
Just for comparison's sake, the same laptop can run an empty bash script in 2-3 milliseconds. This is very important when running build jobs, for example.
How about pwsh -NoLogo -NoProfile ?
> Measure-Command { pwsh -NoProfile -NoLogo exit }
Days : 0
Hours : 0
Minutes : 0
Seconds : 0
Milliseconds : 254
Thanks for the suggestion. Adding these flags halves the startup time:
There's no defender (or anything similar) on that other machine. It also has an SSD. With an older HDD powershell started in about 10-15 seconds there (while cold startup time for bash was something like a second, and instantly afterwards).
$ time ./pwsh -NoProfile -NoLogo <<<'' >/dev/null
real 0m0.456s
user 0m0.388s
sys 0m0.040s
There's no defender (or anything similar) on that other machine. It also has an SSD. With an older HDD powershell started in about 10-15 seconds there (while cold startup time for bash was something like a second, and instantly afterwards).
I open few hundred terminals each day. There is something seriously wrong with your machines. Pwsh is not a king of speed but 10-15 is local problem. Anything more then 2 seconds is a local problem. You can maybe get to 5 with lots of modules in your profile.
Powershell is a horror you mean, it's got the verbosity of objective c with the consistency that comes with all Microsoft designed products.
Just use a proper scripting language.
Just use a proper scripting language.
No, I said what I mean. Bash and its disasters like [ as executable and mega error prone and awkward language designed 50 years ago IS horror.
Verbosity of powershell is optional, I don't use it.
> consistency that comes with all Microsoft designed products.
Oh, lets not troll here.
Verbosity of powershell is optional, I don't use it.
> consistency that comes with all Microsoft designed products.
Oh, lets not troll here.
Disasters like powershell returning a 0 exit code even if there is an error? Its a pile of trash that doesn't add anything new other than a big shiny MS logo.
That is simply not true. RTFM - $ErrorLevel is for native apps $? is for cmdlets
> its disasters like [ as executable
Why is that a disaster? What problem has it caused you that wouldn't have happened if it wasn't an executable?
[[ is a non-executable version of [. Does that solve your problem?
Why is that a disaster? What problem has it caused you that wouldn't have happened if it wasn't an executable?
[[ is a non-executable version of [. Does that solve your problem?
I don't have a problem - I don't use bash unless its few lines or absolute must.
Its disaster because launching process to achieve simple math expression is just ridiculous, besides being slow and besides having to terminate it with ;. Its wrong on so many levels I don't even know where to start, they must have been smoking some nasty things back in time.
Its disaster because launching process to achieve simple math expression is just ridiculous, besides being slow and besides having to terminate it with ;. Its wrong on so many levels I don't even know where to start, they must have been smoking some nasty things back in time.
> launching process to achieve simple math expression is just ridiculous
That's why you don't do that. $(( math goes here ))
That's why you don't do that. $(( math goes here ))
That is also ridiculous as it starts subshell/subprocess.
Neither [ nor $(()) start subshells/subprocesses. [ is built-in in pretty much every shell, and there's absolutely no reason for any shell to make a subshell/subprocess for $(()). You might be confusing it with $().
In any case, worrying about subprocesses and their slowness in a shell is completely moot. Shells aren't built to run fast, and are rather built to work with executables conveniently.
> besides having to terminate it with ;
Is that really a complaint? Does that bother you a lot? This seems so petty, I'm a bit lost for words.
[ is a regular command because the control structures of the shell are flexible enough to support any command for their conditions.
I don't know if you're asking for the shell to make a super-special exception in its syntax for that particular command just so that people won't have to type ; under certain circumstances, or if you're asking for control structures to be more restricted and allow only [ instead of any command for their conditions.
> they must have been smoking some nasty things back in time.
So far, you've given the impression of someone complaining that screwdrivers really suck for inserting nails. I wonder if the issue is that you've never seen hammers before or if you've never seen screws before. Or maybe the issue is you believe each tool should be equally fit for everything?
In any case, worrying about subprocesses and their slowness in a shell is completely moot. Shells aren't built to run fast, and are rather built to work with executables conveniently.
> besides having to terminate it with ;
Is that really a complaint? Does that bother you a lot? This seems so petty, I'm a bit lost for words.
[ is a regular command because the control structures of the shell are flexible enough to support any command for their conditions.
I don't know if you're asking for the shell to make a super-special exception in its syntax for that particular command just so that people won't have to type ; under certain circumstances, or if you're asking for control structures to be more restricted and allow only [ instead of any command for their conditions.
> they must have been smoking some nasty things back in time.
So far, you've given the impression of someone complaining that screwdrivers really suck for inserting nails. I wonder if the issue is that you've never seen hammers before or if you've never seen screws before. Or maybe the issue is you believe each tool should be equally fit for everything?
[deleted]
Demonstrably false.
And even if not, so what? Every argument you make is a shifting of a goalpost.
And even if not, so what? Every argument you make is a shifting of a goalpost.
Just looking at PS scripts makes my RSI flare up.
Perl is nice and ubiquitous.
Agreed. Also, Perl’s sweet spot is 1-500 lines of code. (Beyond that, not having types starts to hurt.)
Also, for scripting stuff, my (readable, extensible) perl is about 10% as long as other people’s python, so 500 lines of perl ends up being 5000 lines of python. That creates a huge difference in maintainability and correctness.
With code golf, my perl can be 10x shorter than that, but then it is neither readable nor extensible, and doesn’t scale to 500 (or even 50) lines before becoming more trouble than it’s worth.
Also, for scripting stuff, my (readable, extensible) perl is about 10% as long as other people’s python, so 500 lines of perl ends up being 5000 lines of python. That creates a huge difference in maintainability and correctness.
With code golf, my perl can be 10x shorter than that, but then it is neither readable nor extensible, and doesn’t scale to 500 (or even 50) lines before becoming more trouble than it’s worth.
I've been wondering the same myself... I want:
- reliable scripts::in my case this means strong typing. Because for example currently to see the exit status of a program one need to check the exit number 0 or non zero plus the string for the command.
- crosplatform:: this eliminates interpreted languages like ruby and python
- easy to interact with the environment :: is something that like bash can actually call programs installed on the computer.
As the song says: "But I still... haven't found... what I'm looking for."
As the song says: "But I still... haven't found... what I'm looking for."
Go might be a decent choice. It’s basically python, but with reasonable type and error handling.
It still has other python-style problems, like epic dependency graphs, with all the compatibility and bitrot nightmares those imply.
It still has other python-style problems, like epic dependency graphs, with all the compatibility and bitrot nightmares those imply.
I recommended Node in another comment, but if you want typing I'd recommend Deno. Similar reasons, but add the fact that you get type checks without a build step.
With typescript installed:
usr bin env ts-node
On phone but you get what I mean
usr bin env ts-node
On phone but you get what I mean
With nix you can make anything into a script, there's some people using haskell for example.
I tend to use common lisp but I'm learning guile scheme and will probably move to that instead.
I tend to use common lisp but I'm learning guile scheme and will probably move to that instead.
I don't understand (even though Nixos was my only OS for a couple of months).
How would Nix help me make Haskell into a script?
How would Nix help me make Haskell into a script?
I think OP means that it would allow you to pull it from anywhere you have nix installed...
https://github.com/BrianHicks/nix-script
This was on HN the other day. Nix allows you to crunchbang a `nix-shell -p` kind of expression.
This was on HN the other day. Nix allows you to crunchbang a `nix-shell -p` kind of expression.
I'd say Go might be your best bet - it's easy to generate executables, you get a simple type system, and it's pretty easy for anyone to pick up and modify a program in. However another alternative is to use the language of the project so it's uniform. Most languages are pretty usable for scripting.
I really like NodeJS for more complex system-scripting. In contrast with Python:
1) There aren't two disjoint versions to juggle
2) Dependency installation is clean and contained
3) Dependencies can easily be installed locally or globally, depending on your preference for isolation vs reuse
4) Lots of packages are available for doing cross-platform (read: Windows compatible) operations
5) JS makes it really easy to do efficient async tasks, for example high-IO scripting involving reading and writing lots of files and making network requests (without any dependencies, I might add)
At my last company I converted all our shell scripts to Node scripts when the pandemic hit, so they'd work predictably on Windows machines, and it was great. These days I'd do it that way to begin with.
1) There aren't two disjoint versions to juggle
2) Dependency installation is clean and contained
3) Dependencies can easily be installed locally or globally, depending on your preference for isolation vs reuse
4) Lots of packages are available for doing cross-platform (read: Windows compatible) operations
5) JS makes it really easy to do efficient async tasks, for example high-IO scripting involving reading and writing lots of files and making network requests (without any dependencies, I might add)
At my last company I converted all our shell scripts to Node scripts when the pandemic hit, so they'd work predictably on Windows machines, and it was great. These days I'd do it that way to begin with.
JavaScript as a shell replacement, what a time to be alive!
I am familiar with and use quite a bit of js on the front end, but how is node for shelling out? This is something I find Python to be quite terrible at.
I am familiar with and use quite a bit of js on the front end, but how is node for shelling out? This is something I find Python to be quite terrible at.
I wouldn't use it as an interactive prompt personally - it's a bit too verbose - though I did know someone in college who did that :)
Calling shell commands is reasonably straightforward:
I will admit though that in most of the cases where I've used it I didn't really do any shelling-out, so I can't speak much to how that experience scales in practice. Mostly I was either 1) doing self-contained logic with files, web requests, etc, or 2) delegating out to JS build tools (which even when they have a CLI, almost always expose a nice JS API too). Ymmv, etc.
Calling shell commands is reasonably straightforward:
const { exec} = require('child_process')
const child = exec('ls', ['-lh', '/usr'])
console.log('error', child.error)
console.log('stdout ', child.stdout)
console.log('stderr ', child.stderr)
I will admit though that in most of the cases where I've used it I didn't really do any shelling-out, so I can't speak much to how that experience scales in practice. Mostly I was either 1) doing self-contained logic with files, web requests, etc, or 2) delegating out to JS build tools (which even when they have a CLI, almost always expose a nice JS API too). Ymmv, etc.
It's far easier to do it on perl.
$output = `ls -lh /usr`;
I don't think I can remember all the boilerplate of using the wrong language for the job if I only happen to write that once a month or less.
Even PHP is easy.
$output = shell_exec('ls -lh /usr');
$output = `ls -lh /usr`;
I don't think I can remember all the boilerplate of using the wrong language for the job if I only happen to write that once a month or less.
Even PHP is easy.
$output = shell_exec('ls -lh /usr');
Let me put it this way; I recently wrote a script to parse the output of last and format it.
The Node version was some 90 LoC but if I included the dependencies it came to about 5 MB. Then I rewrote the same script in Python3 and it was 60 LoC and no dependencies that needed to be installed separately.
I will admit that the JS code looked _a lot_ cleaner than the the Python code though.
The Node version was some 90 LoC but if I included the dependencies it came to about 5 MB. Then I rewrote the same script in Python3 and it was 60 LoC and no dependencies that needed to be installed separately.
I will admit that the JS code looked _a lot_ cleaner than the the Python code though.
Node is great for shell scripts. Async programming is a breeze in JavaScript. I wrote a quick wrapper function around child_process.exec to promisify it, and this allows me to use features like async / await and Promise.all in my scripts.
I tend to write a lot of bash, partially for masochistic reasons, but I would probably choose Node over Python in 2021 for basic dev tooling scripts. Setting up a node environment with nvm can be done in a few lines of code, and it’s pretty reliable across platforms. Python is just too difficult to support consistently.
Really you should pick whatever you currently have in your codebase. We happen to have both Python and Node, with the languages generally being developed separately by different devs. So we’ve got tools in both, but a Node developer is unlikely to ever need/use the Python tools (although they can all run in Docker anyway). What annoys me a bit is that pre-commit requires a Python env to bootstrap itself, but it can install nvm without any fuss.
Really you should pick whatever you currently have in your codebase. We happen to have both Python and Node, with the languages generally being developed separately by different devs. So we’ve got tools in both, but a Node developer is unlikely to ever need/use the Python tools (although they can all run in Docker anyway). What annoys me a bit is that pre-commit requires a Python env to bootstrap itself, but it can install nvm without any fuss.
I use Fish as my shell instead of bash precisely because I really like the scripting language for my personal scripts.
It's not the best fit for everyone but it has served me well.
It's not the best fit for everyone but it has served me well.
#!/usr/bin/newlisp
Check out Babashka if you enjoy lisps/Clojure.
https://github.com/babashka/babashka
https://github.com/babashka/babashka
Just use shell or Python3 and stop nitpicking IMHO. There aren’t really any other choices.
Most OSes come with relatively modern versions of either. Pretty much everyone is or should be familiar with either. It’s not like it’s some Lispy thing.
Most OSes come with relatively modern versions of either. Pretty much everyone is or should be familiar with either. It’s not like it’s some Lispy thing.
Perl (5). It's an incredibly capable scripting language, many things essential for Shell programming are much better integrated than in Python (e.g. regular expressions). It gets a lot of hate these days but it's still a damn fine language and perfectly suited to shell programming (and larger projects).
One of the things about Perl is that I can't quite believe myself how far it has fallen.
I was never a Perl monk by any means but I knew enough to get by. And why did I not even think about it? I haven't coded Perl in years, but if my memory serves me correctly, bundling extra dependencies in Perl was a heck of a lot less painful than Python too.
I was never a Perl monk by any means but I knew enough to get by. And why did I not even think about it? I haven't coded Perl in years, but if my memory serves me correctly, bundling extra dependencies in Perl was a heck of a lot less painful than Python too.
Yes, Perl's package ecosystem CPAN (the "Comprehensive Perl Archive Network") was lightyears ahead of other systems at the time (late 90s / early 2000s), it provided modules for everything, which made you so incredibly productive. You need a DNS library? Just go to CPAN. CGI functionality? CPAN. Parsing some obscure file format? CPAN.
Really sad it lost so much traction in the last decade, I partially blame it on the failed Perl6 endeavour. I also mostly use Python these days but looking at my old Perl scripts Python really doesn't come close in terms of succintness.
Really sad it lost so much traction in the last decade, I partially blame it on the failed Perl6 endeavour. I also mostly use Python these days but looking at my old Perl scripts Python really doesn't come close in terms of succintness.
What dependencies? OP wants to replace a bash/sh script, not left pad a string.
Next Generation Shell was born exactly out of the pain of using bash and Python. I'm the author and I felt the same way.
> shelling out when needed is not pleasant,
Handled properly because the language was built ground up for Ops tasks. There is even a syntax for "run a command and parse the output". Rationale and examples - https://ngs-lang.org/doc/latest/man/ngswhy.1.html
Currently NGS supports Linux and MacOS.
> smallish binary
Unfortunately there is no static build yet so can't check this checkbox
Next Generation Shell - https://github.com/ngs-lang/ngs
Alternative Shells list by @chubot - https://github.com/oilshell/oil/wiki/Alternative-Shells
> shelling out when needed is not pleasant,
Handled properly because the language was built ground up for Ops tasks. There is even a syntax for "run a command and parse the output". Rationale and examples - https://ngs-lang.org/doc/latest/man/ngswhy.1.html
Currently NGS supports Linux and MacOS.
> smallish binary
Unfortunately there is no static build yet so can't check this checkbox
Next Generation Shell - https://github.com/ngs-lang/ngs
Alternative Shells list by @chubot - https://github.com/oilshell/oil/wiki/Alternative-Shells
If you are ready to invest some time then try `perl`. It is awesome.
I tried using Python with import sh for awhile, but...meh.
https://amoffat.github.io/sh/sections/faq.htmlThis won’t work for many people who don’t know the language or its ecosystem, but I’ve started doing my ad-hoc scripting in Rust using rust-script [0]. I’ve got a template with lots of boilerplate for everything I might need and I just copy it and delete the parts I don’t want. Having argument parsing, dotenv init, toml configuration and such ready to go with just a few tweaks, simple user prompting with dialoguer [1] and easy delegation to other tools with cmd_lib [2] means I can often write the tool I’m envisioning in just a couple of minutes.
[0] https://rust-script.org/
[1] https://crates.io/crates/dialoguer
[2] https://crates.io/crates/cmd_lib
[0] https://rust-script.org/
[1] https://crates.io/crates/dialoguer
[2] https://crates.io/crates/cmd_lib
Can you be more specific than "fatigue"? That sounds like ennui, in which case maybe consider a job switch instead of a language switch (not being sarcastic or intentionally mean). Bash plus all the Unix tools (awk, grep, sed, tsort, comm, the ones I don't know, etc.) have pretty long legs, in terms of functionality and the syntax is pretty understandable. They provide most of what Perl offers, apart from OO (-ish) and line noise.
If you want strong typing (are you scripting in the large?), then, almost by definition, no scripting language provides that. If you need performance, switch to a compiled language (or a jit-compiled interpreted language) that most of your co-workers understand.
If Python brings too many dependencies, I guess you're trying to do something that is way out of the realm of possibility for bash, so you're not looking for an "alternative" to bash, you're looking for the best language for whatever project you're undertaking. And you may not be able to get away from the dependencies if you're trying something specific/sophisticated.
If you want strong typing (are you scripting in the large?), then, almost by definition, no scripting language provides that. If you need performance, switch to a compiled language (or a jit-compiled interpreted language) that most of your co-workers understand.
If Python brings too many dependencies, I guess you're trying to do something that is way out of the realm of possibility for bash, so you're not looking for an "alternative" to bash, you're looking for the best language for whatever project you're undertaking. And you may not be able to get away from the dependencies if you're trying something specific/sophisticated.
> If you want strong typing (are you scripting in the large?), then, almost by definition, no scripting language provides that.
Powershell is basically strongly typed.
> Can you be more specific than "fatigue"? That sounds like ennui
I love my job, but bash makes me wanna hang myself. I can say the same about stored procedures. Its like going back to cobol days. Some people have strong tastes I guess.
Powershell is basically strongly typed.
> Can you be more specific than "fatigue"? That sounds like ennui
I love my job, but bash makes me wanna hang myself. I can say the same about stored procedures. Its like going back to cobol days. Some people have strong tastes I guess.
Yeah, i forgot to add my thoughts on PowerShell. It's great for Windows and nothing else. Its power isn't syntax (which is a bit weird) but the fact that it interfaces with all the Windows APIs. I can't really imagine it on any other platform. If you're on Windows and nowhere else, PS is superior to bash. Have you been using bash on Windows? If so, why? Cross-Platform?
PS is superior to bash, period, no matter platform. You are talking about something else entirely, like added benefit of lots of Windows tools supporting PowerShell automation. While that is nice, the major appeal of pwsh for me is not the ecosystem but the paradigm and language - in bash, you always do something irrelevant to your current task as anything requires excessive boilerplate and all supportive utilities are reinventing stuff like parameter names and parsing, error handling, etc. Also, those utilities are sometimes different on other *nix platforms, so you can't even count on linux portability.
So yeah, PowerShell is huge on Windows and just awesome on Linux. You must also take into account that its only few years old in that space.
So yeah, PowerShell is huge on Windows and just awesome on Linux. You must also take into account that its only few years old in that space.
> PS is superior to bash, period, no matter platform
Simply untrue. See [1]. PS _may_ be superior _where it exists_, but without question it is available in far fewer places than bash.
[1]: https://github.com/PowerShell/PowerShell/issues/12612
Simply untrue. See [1]. PS _may_ be superior _where it exists_, but without question it is available in far fewer places than bash.
[1]: https://github.com/PowerShell/PowerShell/issues/12612
I used “true” instead of “$True” in a power shell script. It didn’t error. It didn’t warn. It just evaluated to false.
So doesn’t behave like a strongly typed language to me.
So doesn’t behave like a strongly typed language to me.
RTFM - if you don't specify type, it is infered.
[bool] $x = "true"
MetadataError: Cannot convert value "System.String" to type "System.Boolean". Boolean parameters accept only Boolean values and numbers, such as $True, $False, 1 or 0.I don't think the breaking point is the number of lines in the script; your pain point is probably one or two specific concepts or features? (or lack of)
Shell is likely to remain the best way to connect together other pieces of software.
The tipping point, in my experience, is folks trying to use arrays or other buffering of data. At that point they're writing in shell, but wishing for a procedural language.
Whereas, shell works excellently (and incredibly efficiently) if you can compose your task strictly into streams of data (a more 'functional' feel)
So the real answer I think is specific to the types of script you are writing!
But also it could be worth revisiting the style in which you're using shell; especially if you can call upon your own helper programs (eg. in Python) where you require buffering or non-streamed access to data.
Whilst there's a lot of documentation of sh/bash functionality, I don't think there's anything like the same momentum behind 'best practices' as other languages like Python or even C.
Shell is likely to remain the best way to connect together other pieces of software.
The tipping point, in my experience, is folks trying to use arrays or other buffering of data. At that point they're writing in shell, but wishing for a procedural language.
Whereas, shell works excellently (and incredibly efficiently) if you can compose your task strictly into streams of data (a more 'functional' feel)
So the real answer I think is specific to the types of script you are writing!
But also it could be worth revisiting the style in which you're using shell; especially if you can call upon your own helper programs (eg. in Python) where you require buffering or non-streamed access to data.
Whilst there's a lot of documentation of sh/bash functionality, I don't think there's anything like the same momentum behind 'best practices' as other languages like Python or even C.
The question you should ask yourself is not what use instead of bash but what is the problem you are trying to solve and what is the best way to solve it.
It's hard for us to help you with you giving us some context.
150-500 loc of bash are not that much, specially when you combine bash with standard Unix utilities.
These scripts you wrote, for what do you use them? Are they executed by humans or machines? On which platform? Linux, Mac, Windows or all?
You mentioned jq. Are you parsing json from HTTP requests or files? And why are you parsing json with bash?
Without a better context, A lot of people will say to use Python and Python is a good choice with you are running you scripts on Linux because you probably already have Python 3 installed and whatever you are writing with 500 loc of bash can be accomplished with zero dependencies with Python standard library.
If relying on the Python runtime is really a problem for you, your next best option is Go. Go is easy to learn and easy to use for building small cli apps with only the standard library, but even if you need some dependency, you will ship a single binary, so no problems with runtime compatibility or dependencies.
It's hard for us to help you with you giving us some context.
150-500 loc of bash are not that much, specially when you combine bash with standard Unix utilities.
These scripts you wrote, for what do you use them? Are they executed by humans or machines? On which platform? Linux, Mac, Windows or all?
You mentioned jq. Are you parsing json from HTTP requests or files? And why are you parsing json with bash?
Without a better context, A lot of people will say to use Python and Python is a good choice with you are running you scripts on Linux because you probably already have Python 3 installed and whatever you are writing with 500 loc of bash can be accomplished with zero dependencies with Python standard library.
If relying on the Python runtime is really a problem for you, your next best option is Go. Go is easy to learn and easy to use for building small cli apps with only the standard library, but even if you need some dependency, you will ship a single binary, so no problems with runtime compatibility or dependencies.
Depends on the program. Sometimes I use shell scripts, and sometimes I use Node.js, and (more rarely) PostScript.
Like others said it depends on each problem you are solving. Consider Java, nodejs, go, rust, ruby, python, c# - the one you know best is the best. Build up some Util classes for stuff you do regularly. NPM has a package for everything eg there is a cron so I like it for that reason.
Consider docker for some situations. Consider kubernetes if the need is more devops/ running a production system.
Consider powershell if on windows.
Travis ci / Jenkins / appveyor etc. for CI
Be lazy - can I get out of writing this code should be in the forefront of your mind (for any code!). The answer might be no but consider it.
No code tools can be considered - Google sheets, Wordpress, plethora of saas to do this. Paas eg aws lambda etc.
So many options. Solve the problem, rather than transliterate bash.
Consider docker for some situations. Consider kubernetes if the need is more devops/ running a production system.
Consider powershell if on windows.
Travis ci / Jenkins / appveyor etc. for CI
Be lazy - can I get out of writing this code should be in the forefront of your mind (for any code!). The answer might be no but consider it.
No code tools can be considered - Google sheets, Wordpress, plethora of saas to do this. Paas eg aws lambda etc.
So many options. Solve the problem, rather than transliterate bash.
Honestly it depends on what you're solving, personally I use both.
For me: I use groovy. Reasoning behind it is that I am a software engineer specializing in JVM. On all the machines I work there is a JVM installed. Plus groovy is Java with extra stuff so it is a flavor of Java nothing new. And it does not require a compiler.
Same here! bonuses include:
- built in dependency manager so you can pull dependencies in just by putting @Grab at the top of your script
- interface with your actual code written in Java easily so you can easily have scripts that utilise your "real" code for business logic
- full java ecosystem of libraries trivially accessible
- genuine cross platform so none of the weird OS / architecture issues and conflicts you get with pre-installed python, etc etc
- built in dependency manager so you can pull dependencies in just by putting @Grab at the top of your script
- interface with your actual code written in Java easily so you can easily have scripts that utilise your "real" code for business logic
- full java ecosystem of libraries trivially accessible
- genuine cross platform so none of the weird OS / architecture issues and conflicts you get with pre-installed python, etc etc
I used to use Python for this also but now use Ruby and have no desire to ever go back.
Its Perl-ish roots just make it a fantastic tool for the job.
Its Perl-ish roots just make it a fantastic tool for the job.
I like to use Python with PyInstaller[1], but it results in fairly large binaries with a relatively slow startup time.
When binary size and startup speed matters, then I use Go instead.
Finally, sometimes I just write in Bash anyway, even though it is such a difficult language to write in.
[1]: https://pyinstaller.readthedocs.io/en/stable/
When binary size and startup speed matters, then I use Go instead.
Finally, sometimes I just write in Bash anyway, even though it is such a difficult language to write in.
[1]: https://pyinstaller.readthedocs.io/en/stable/
Without knowing what you're trying, I'm often using nim (https://nim-lang.org/) when I want more complexity than a basic bash script, cross-compiled binaries _and_ way smaller binaries than go. It's much like python, easy to pick up, and promising. But not as mature as python, obviously. Go is great, but the binaries are big... not a problem if you've got great bandwidth...
I use php(7.4) and python(3.x), works great. Alternatively, perl is also a great choice.
Is there a language with sane syntax (if, for, comparisons being a syntactic feature not commands requiring ; in weird places) that would still execute every line by default and support pipelines and && and || natively? Sort of bash that looks like a normal language.
Maybe even some library with stuff for parsing outputs of common commands?
Maybe even some library with stuff for parsing outputs of common commands?
Xonsh is a Python superset with shell-like operations. Also there's PowerShell which is arguably a better designed language than bash.
PHP is great, self-contained solution
As a devops engineer who also writes a lot of code, I started using Deno (https://deno.land) since I can write my code in JavaScript and TypeScript. I can also natively use wasm, and have an internal repo for hosting commonly used functions since Deno loads files using url imports.
Can recommend!
Can recommend!
For a quasi embedded project I picked lua. my requirements were small runtime size + fast startup time. I would say that the library availability is tremendously lower than perl,python,node but ample enough for most tasks (openwrt is almost completely built in it, including web views)
However I agree that nim / zig could also be good directions
However I agree that nim / zig could also be good directions
>I don't know what version of Python I'm going to have on the system, installing dependencies is not fun
If your colleagues can be expected to have Docker, you can wrap your Python script in an image that contains specific versions of Python and the dependencies.
If your colleagues can be expected to have Docker, you can wrap your Python script in an image that contains specific versions of Python and the dependencies.
> I don't know what version of Python I'm going to have on the system
You could have assumed Python 2.7 for 10 years or so.
Now you could safely assume Python 3 (find minimum version applicable for your case).
> installing dependencies is not fun,
Pure python dependencies are easy (e.g., pipenv). C extensions are not more difficult than corresponding dependencies for your bash script (likely via system packages).
> shelling out when needed is not pleasant, and the size of program always seemingly doubles.
Have you tried to use shell command for shell one-liners in Python? e.g., directly via `proc = subprocess.run(shell_pipeline, shell=True)` or dedicated libraries such as pyinvoke/fabric?
You could have assumed Python 2.7 for 10 years or so.
Now you could safely assume Python 3 (find minimum version applicable for your case).
> installing dependencies is not fun,
Pure python dependencies are easy (e.g., pipenv). C extensions are not more difficult than corresponding dependencies for your bash script (likely via system packages).
> shelling out when needed is not pleasant, and the size of program always seemingly doubles.
Have you tried to use shell command for shell one-liners in Python? e.g., directly via `proc = subprocess.run(shell_pipeline, shell=True)` or dedicated libraries such as pyinvoke/fabric?
Python.
With GraalVM, you can compile all JVM languages into binary files to run, including Python.
There are many other scripting languages to choose from, such as JavaScript，Ruby, Kotlin and Groovy.
Of course, you can also use Java，Scala, Clojure and so on.
There are many other scripting languages to choose from, such as JavaScript，Ruby, Kotlin and Groovy.
Of course, you can also use Java，Scala, Clojure and so on.
TXR Lisp
Nim
NodeJS
I recently changed my shell to elvish because of the data structures and extendability. It has a lot of sane defaults that make it very appealing to me. https://elv.sh/
Tcl. As with other shells, the scripting model is based on commands and substitutions, but in Tcl substitution is more sane because a substitution is never reparsed and reinterpreted. Tcl was designed to provide a uniform scripting environment that could be extended into any problem domain. In this sense, is specifically intended as a generalization of the concept of a shell beyond interactive use by a logged-in user and into programmatic control of any set of subsystems. Tcl's syntax is more simple and more regular than other shells, and its semantics are designed to make it easy to create a DSL to manage the task at hand. Values are immutable in Tcl so it has the mathematical purity that is attractive in a functional language, but is much easier to get started with. Because every value is a string, there is a fexibility in Tcl that can not be found in other languages. Stringiness is interesting, fun, and productive.
The common refrain of "use Python" hasn't really worked fantastically: I don't know what version of Python I'm going to have on the system, installing dependencies is not fun, shelling out when needed is not pleasant, and the size of program always seemingly doubles.
I'm willing to accept something that's not on the system as long as it's one smallish binary that's available in multiple architectures. Right now, I've settled on (ab)using jq, using it whenever tasks get too complex, but I'm wondering if anyone else has found a better way that should also hopefully not be completely a black box to my colleagues?