Writing simple tab-completions for Bash and Zsh(mill-build.org)
mill-build.org
Writing simple tab-completions for Bash and Zsh
https://mill-build.org/blog/14-bash-zsh-completion.html
98 comments
When I screen share, people don't realize I'm not using zsh and dozen plugins. It's just fish and it's beautiful out of the box.
I'll switch to fish after it stops expanding `car TAB` to `blkdiscard` when I don't have `cargo` in path. Non-prefix completion for commands is plain evil.
I’ll switch to fish when it comes preinstalled on all of the computers I use so I can write scripts in it.
I already avoid bash scripting so I lose very little. Shell scripting beyond throwaway one-liners is a problem not a solution.
(Well that and all my machines come from the same NixOS configs.)
(Well that and all my machines come from the same NixOS configs.)
Ha that’s funny, considering nixos is 99% stdenv which is one of the worst bash monstrosities in existence, and drives people ever further into the swamp of bash. (Ever tried to debug stdenv setup hooks? I still have water damage from the tears.)
I have personally embraced the insanity but let’s not kid ourselves about nixos basically just being three bashes in a trench coat.
Basically https://xkcd.com/224/ , but s/lisp/nix/ s/perl/bash/
I have personally embraced the insanity but let’s not kid ourselves about nixos basically just being three bashes in a trench coat.
Basically https://xkcd.com/224/ , but s/lisp/nix/ s/perl/bash/
It's my daily driver, but I am holding my nose.
Some year something better will show up from someone with more taste (and I know I am not alone in having thoughts on what it would look like), but for now the yak has already been shaven. At least the builds are sandboxed.
Some year something better will show up from someone with more taste (and I know I am not alone in having thoughts on what it would look like), but for now the yak has already been shaven. At least the builds are sandboxed.
Guix if you can stomach it.
Do you really limit your shell experience to plain POSIX? That sounds masochistic.
Ever since shellcheck came around I have been writing all my scripts in POSIX sh. Its not that bad actually and knowing stuff is just more compatible is worth that little friction.
Shellcheck has been a huge game changer for me. The lack of a linter is the main reason I don't consider switching to zsh anymore.
When I first found shellcheck, I'd sit on the train every morning and fix all the stupid little bugs in all our shell code. I learned a TON about shell scripting, because the shellcheck wiki is amazing. We had some redhat 6 init scripts checked into our repos, and shellcheck helpd me find syntax errors in those init scripts. If Redhat can't reliably write shell scripts without a linter, or catch bugs from upstream, what makes us think we're any better?
I wouldn't write serious programs in another language without a linter. Why should I write shell scripts without one?
When I first found shellcheck, I'd sit on the train every morning and fix all the stupid little bugs in all our shell code. I learned a TON about shell scripting, because the shellcheck wiki is amazing. We had some redhat 6 init scripts checked into our repos, and shellcheck helpd me find syntax errors in those init scripts. If Redhat can't reliably write shell scripts without a linter, or catch bugs from upstream, what makes us think we're any better?
I wouldn't write serious programs in another language without a linter. Why should I write shell scripts without one?
You shouldn't be writing serious programs in Bash/Zsh at all, is the issue. If it's complex enough to need linting, it's complex enough to require Python/Perl.
Writing all your scripts that way is one thing. I’m not that far off, although I write all my shell functions and things like that in fish. But avoiding using tools that may not be universally available means never using ripgrep or fd or fish or newer versions of bash or any of the other nice updates we have now, and that sounds awful.
I would rather limit myself to things I know are going to be preinstalled on the platforms I use than deal with the masochism that is scripted dependency management. No thanks.
All these comments here "Uhhh just use PythoN!" Yeah? I write scripts that need to run on macOS. No Python there. Yeah I can have my script sit there and pull the command line tools in an automated way and wait the 20 minutes it takes to install or I can pull on my big boy pants and just figure out how to do it in shell. It's turing complete. It works. Whatever.
All these comments here "Uhhh just use PythoN!" Yeah? I write scripts that need to run on macOS. No Python there. Yeah I can have my script sit there and pull the command line tools in an automated way and wait the 20 minutes it takes to install or I can pull on my big boy pants and just figure out how to do it in shell. It's turing complete. It works. Whatever.
That's why I use C to write all my shell scripts. Can't have something non-portable like bash, which is missing from 75% of computers by default.
(Seriously, what am I missing here? Is `apt-brew pacman install fish` really that hard to type? If cross-platform portability is actually a concern, shouldn't you be using Python anyway?)
(Seriously, what am I missing here? Is `apt-brew pacman install fish` really that hard to type? If cross-platform portability is actually a concern, shouldn't you be using Python anyway?)
I don't really care about portability as I do "how much shit do I need to install to actually get work done on my workstations and servers"
Maybe it's because I was a sysadmin in a former life, but overmanaging is almost always a mistake. If I had one single computer and only one, sure, who cares. But I have a homelab full of servers and a constant churn of VMs and I don't need to add in another step of "oh shit X isn't installed yet". Maybe it's also because of that that I don't really give a shit about how "bad" sh/bash/zsh are.
Yes, I can sync dotfiles and crap to git (which I also have to install to use), but it's just a small weight off my shoulders to not have to worry or think about that shit before I can use a server or workstation. I don't want to have to unlearn normal shell to learn fish and then have to switch back when I'm trying to get some thing online so that I can install fish. I'm just not fuckin interested.
(inb4 someone is like nix solves this, i don't give a shit about nix either)
Maybe it's because I was a sysadmin in a former life, but overmanaging is almost always a mistake. If I had one single computer and only one, sure, who cares. But I have a homelab full of servers and a constant churn of VMs and I don't need to add in another step of "oh shit X isn't installed yet". Maybe it's also because of that that I don't really give a shit about how "bad" sh/bash/zsh are.
Yes, I can sync dotfiles and crap to git (which I also have to install to use), but it's just a small weight off my shoulders to not have to worry or think about that shit before I can use a server or workstation. I don't want to have to unlearn normal shell to learn fish and then have to switch back when I'm trying to get some thing online so that I can install fish. I'm just not fuckin interested.
(inb4 someone is like nix solves this, i don't give a shit about nix either)
[deleted]
i'll just stick to bash
I bet this is configurable but I wanted to say that this is totally personal preference; I have the exact opposite opinion. Prefix only matching requires much more tab slapping in my experience.
I believe you would lose that bet. I look every few years and I don't see it
interestingly this also seems to have come up pretty recently in their discussions.
https://github.com/fish-shell/fish-shell/discussions/11670#d...
> I'm not sure that subsequence matching ever produces results that people expect (I feel like we've discussed this before but haven't had time to go digging)
No solution, but the discussion seems positive, from a maintainer too.
https://github.com/fish-shell/fish-shell/discussions/11670#d...
> I'm not sure that subsequence matching ever produces results that people expect (I feel like we've discussed this before but haven't had time to go digging)
No solution, but the discussion seems positive, from a maintainer too.
I gave up on fish after a few weeks because of a similar preference issue where the fish maintainers flat out refused to make a bad default configurable: https://github.com/fish-shell/fish-shell/issues/8618
> We don't really do config flags like this, as a philosophical point.
> I would be against accepting such a PR. I do not believe this should be changed.
I understand that they want to keep the list of configs short and manageable, but it means that's not a tool for me. I'm all for good, opinionated defaults but I want to be able to make some changes if I want to.
They apparently reconsidered and implemented the change since migrating fish to rust though.
> We don't really do config flags like this, as a philosophical point.
> I would be against accepting such a PR. I do not believe this should be changed.
I understand that they want to keep the list of configs short and manageable, but it means that's not a tool for me. I'm all for good, opinionated defaults but I want to be able to make some changes if I want to.
They apparently reconsidered and implemented the change since migrating fish to rust though.
Thank you for the comment.
https://github.com/umlx5h/zsh-manpage-completion-generator appears to adapt this to ZSH. Have yet to try through
For those programs that have betrayed shipping man pages, instead say relying only on a --help system, do you happen to know if the fish shell has an analogue to Zsh `_gnu_generic` and Bash `complete -F _longopt`? If not, do you have any insight into why not/what it would take to make that happen?
At least for a subset of Python CLI programs, I wrote this: https://github.com/AdrianVollmer/pycompgen
Still in an early stage, but it should work.
Still in an early stage, but it should work.
The OP mentions them in the last part of their comment, there is `complete …` commands for registering completions
It sounds like you don't know how the Bash/Zsh ideas I mentioned work. They run the command with --help, parse the output, and from that generate the completions, wiring them in to the completion system. That method is a zero config solution (well, you might need a list of such "command-names only" - no option names, which could change at any time -- so maybe "minimal config"). The OP & you mention a much heavier config solution which strikes me as against the vibe of Fish in general which is, supposedly, out-of-the-box niceness.
It's surprising that on OpenSUSE `zypper search fish-completion` returns more than 200 packages. Something is fishy here.
That is to get live data in your completions.
Say you have a widget that has 3 commands: list, start and stop.
With one of those completion files widget stop <tab> will show you a tab-able list of widgets which are running.
Say you have a widget that has 3 commands: list, start and stop.
With one of those completion files widget stop <tab> will show you a tab-able list of widgets which are running.
man pages are so underrated. I mean every project nowadays has README.md so I don’t see why we can’t just auto generate them with or without an LLM helping. Also I wish programs would use standardized generic arguments for help, config file, version, background the task, PID file, log file, and log level.
oh wow, it's parsing all 9461 man pages on my arch install, for a cute total of 13MB
thanks a lot
thanks a lot
Fish isn’t the only shell that does this. Murex has supported man page completions for years too. And it’s fully automatic in Murex.
I tried to use fish on some of my debian servers that i only rarely update packages/kernel, so I don't have to carry my bashrc there, but found their completion for apt is pretty naive. for example after updated kernel i would want to clean up the old ones, but `apt purge linux-image-<TAB>` would list all available kernel versions, not just the ones currently installed.
in the end i switched back to bash.
in the end i switched back to bash.
I feel that the ergonomics of bash completion took a hit as the configurations got “smarter” and “helpfully” started blocking file or directory name completion if it thinks it wouldn’t be appropriate to have a file name at the current cursor position. Instead of blocking, the default should always be to fall back to filename completion.
Sometimes I’m close to disabling/uninstalling all completion scripts out of irritation as decades of muscle memory are frustrated by this behavior.
It’s like that bad/annoying UX with text fields where the UI is constantly fighting against you in order prevent you from producing “illegal” intermediate input - e.g. let me paste the clipboard here goddammit - I know what I’m doing - I’ll correct it.
Sometimes I’m close to disabling/uninstalling all completion scripts out of irritation as decades of muscle memory are frustrated by this behavior.
It’s like that bad/annoying UX with text fields where the UI is constantly fighting against you in order prevent you from producing “illegal” intermediate input - e.g. let me paste the clipboard here goddammit - I know what I’m doing - I’ll correct it.
There is the complete-filename function that only completes filenames in bash, bound to M-/ by default. You can use that in any place you want a filename where "complete"(the function normally bound to tab) would do something you don't desire.
There are a collection of other non-context aware completion functions that are bound by default too, useful for example when you when you wish to complete hostnames in a for-loop.
zle has what is largely a significant superset of this, the documentation is spread about between the zshzle and zshcomp* manpages.
There are a collection of other non-context aware completion functions that are bound by default too, useful for example when you when you wish to complete hostnames in a for-loop.
zle has what is largely a significant superset of this, the documentation is spread about between the zshzle and zshcomp* manpages.
There is one particular command I occasionally use that has totally broken completion for files, so I've taken to just using 'ls X Y Z' to get the right completion behavior and then changing 'ls' to the right command as the last step.
Hopefully you saw one of the sibling comments: https://news.ycombinator.com/item?id=44855551
TLDR: M-/ (Alt /) will do file auto-completions regardless of the commands auto-comoletion. It is a different method, but maybe could help.
TLDR: M-/ (Alt /) will do file auto-completions regardless of the commands auto-comoletion. It is a different method, but maybe could help.
I agree that is annoying. It's waaay less confusing to complete a filename and then get an error from the actual program than it is for just ...nothing to happen so you get confused and have to `ls` to check if the file actually exists and it does and so you think tab completion is broken for some reason and you copy & paste the filename and then finally you get the error that explains what is going on.
It should at least print a message like "file foo.exe exists but it isn't executable".
It should at least print a message like "file foo.exe exists but it isn't executable".
If you get into this position what you can do is `ls <tab-completed-path>` or other command to put the filename in the previous command's argument, then you can access it via !$ or !^ (or use !!:1 or your shell's notation for indexing an argument that was already in the previous command).
It's not a fix but it'll save a little time sometimes.
It's not a fix but it'll save a little time sometimes.
An alternative way would be pressing M-. (assuming one is using Readline for typing text in the shell, which is the default for my bash shell).
I have come to absolutely despise web form inputs with front end email validators that are broken. Input field hints to type your email, so you start typing. As soon as you type the first letter it goes red and says “error!!! Invalid email!”
Unbelievably frustrating.
Unbelievably frustrating.
As a guy who is vicously guarding our company email valdation I can tell you that it is a rite of passage for new frontend hires to mess that up.
As it works as desired after running: complete -r; there is something broken about the bash-completion script.
I wrote this, hope everyone finds it as interesting reading this as I did figuring this out for the first time!
Thanks for sharing! I hope to incorporate your bash completion ideas into my CLIs (I've already got zsh completions).
Instead of sourcing the zsh completion script on every startup, you can install it into somewhere on $fpath and zsh will "compile" and cache the completions. This can really speed up shell startup time, but of course is harder to set up. Users have to understand $fpath to put the completions there.
I distribute my CLIs via Homebrew which can install completions automatically.
Instead of sourcing the zsh completion script on every startup, you can install it into somewhere on $fpath and zsh will "compile" and cache the completions. This can really speed up shell startup time, but of course is harder to set up. Users have to understand $fpath to put the completions there.
I distribute my CLIs via Homebrew which can install completions automatically.
It's a good first dive into zsh completion. The whole thing is quite the large system to wrap ones head around it and I'm still somewhat struggling.
But at work, I've been slowly adding auto completion to our ansible wrapper scripts, like explanations which playbooks to use when, smart `-l` completion based off a possibly selected playbook (so, if the playbook is postgres.yml, it doesn't suggest mariadb groups), tag autocompletion (with a few, admittedly, hardcoded explanations how these tags should be used) and such.
It's somewhat of a friday-afternoon struggle project, but it's making the big ansible project pretty approachable to use.
But at work, I've been slowly adding auto completion to our ansible wrapper scripts, like explanations which playbooks to use when, smart `-l` completion based off a possibly selected playbook (so, if the playbook is postgres.yml, it doesn't suggest mariadb groups), tag autocompletion (with a few, admittedly, hardcoded explanations how these tags should be used) and such.
It's somewhat of a friday-afternoon struggle project, but it's making the big ansible project pretty approachable to use.
Could you share how you do it? Ansible playbooks are ran via command `ansible-playbook` command and it surely has its own tab auto completion script.
It kinda does, but I've added a few bells and whistles to it. Mind you, this is zsh code I haven't really cleaned up, so it's kinda messy and most likely buggy in edge cases.
This needs to be in a directory in your FPATH.
At the core, it uses _arguments to dissect the command line. This both suggests that "-t" exists, is called "tags" and later on sets a state variable to "tags" or "limits" if we're completing these arguments.
All of that is shoved into `_values`, and then the usual zsh completion works, so with something like `./wrapper mariadb.yml -l prj4<TAB>`, zsh tries to filter the values based on the word, so this finds stuff like `prj4`, `prj49`, `dc2_prj45`, and so on, but not `prj5`.
- You can run `ansible-playbook "$WORD" --list-tags` to get all tags a playbook references in the current inventory.
- One can give `_values` descriptions. If `_values` sees 'foo[bar]' as an option, it will show the user something like: "foo: bar" and only auto-completes to foo.
- This means, we can give standard or well-established tags short descriptions in a directory or an array or whatever, and instead of offering just "postgres_client_tls_certs" as possible auto-completions for `-t`, we can give the user a prompt like "postgres_client_tls_certs: Ensures the postgres cluster has valid and up-to-date TLS certificates for mutual TLS with applications".
It took a bit of time to understand all of this. But now documenting a tag in a place that people actually look at is very easy and straightforward.
This needs to be in a directory in your FPATH.
At the core, it uses _arguments to dissect the command line. This both suggests that "-t" exists, is called "tags" and later on sets a state variable to "tags" or "limits" if we're completing these arguments.
#compdef corp-ansible-wrapper
_arguments '-t[tags]:tags:->tags' '-l[limit]:limit:->limits' '-D[diffmode]' '-C[checkmode]' '::optional arg:_files'
detect_playbook
case "$state" in
tags)
...
;;
limits)
...
;;
All of that is shoved into `_values`, and then the usual zsh completion works, so with something like `./wrapper mariadb.yml -l prj4<TAB>`, zsh tries to filter the values based on the word, so this finds stuff like `prj4`, `prj49`, `dc2_prj45`, and so on, but not `prj5`.
detect_playbook
# should be an array.
case $PLAYBOOK in
postgres.yml)
FILTER="pg_\|postgres_\|pgbackrest_"
;;
mariadb.yml)
FILTER="db_\|mariadb_"
;;
# quite a few more
*)
FILTER=""
;;
esac
# probably overly complex
ANSIBLE_GROUPS_RAW=$( cat $ANSIBLE_INVENTORY/groups | grep -E "$FILTER" | sort | uniq | tr '\n' ',')
IFS=',' read -r -A ANSIBLE_GROUPS <<< "$ANSIBLE_GROUPS_RAW"
_values -s: 'groups' "${ANSIBLE_GROUPS[@]}"
- You can run `ansible-playbook "$WORD" --list-tags` to get all tags a playbook references in the current inventory.
- One can give `_values` descriptions. If `_values` sees 'foo[bar]' as an option, it will show the user something like: "foo: bar" and only auto-completes to foo.
- This means, we can give standard or well-established tags short descriptions in a directory or an array or whatever, and instead of offering just "postgres_client_tls_certs" as possible auto-completions for `-t`, we can give the user a prompt like "postgres_client_tls_certs: Ensures the postgres cluster has valid and up-to-date TLS certificates for mutual TLS with applications".
It took a bit of time to understand all of this. But now documenting a tag in a place that people actually look at is very easy and straightforward.
Oh, I thought you didn't hardcore those as if conditions, but rather parsed inventory files to get suggestions for -t and parsed tasks in playbook's roles to get suggestions for -l.
Still cool though, thanks for sharing!
Still cool though, thanks for sharing!
Thanks, for the interesting read.
I've started using jdx's usage[1] for my clis. It integrates neatly into clap, and can be used stand alone in scripts. It can generate completions, argparse, manpages, and more
I'm still on the fence if replacing the argparse blocks in my fish scripts is worth the hassle, but against things like old school optparse, it's far better
[1]: https://usage.jdx.dev/
I'm still on the fence if replacing the argparse blocks in my fish scripts is worth the hassle, but against things like old school optparse, it's far better
[1]: https://usage.jdx.dev/
I’ve recently been building a similar tool [1] which defines a specification for CLIs, though the goals are slightly different to the tool you mention I think. I just added support for fish as it happens.
[1] https://github.com/fujiapple852/claptrap
[1] https://github.com/fujiapple852/claptrap
Nice. I'll have to add that to my tool belt, although I think usage will still sit front and center haha
JSON fields autocomplete right in bash/zsh: https://fx.wtf/install#autocomplete
Thanks for linking this! This is a lightweight solution, compared to ijq (interactive jq), but it still may come in handy.
https://github.com/gpanders/ijq
https://github.com/gpanders/ijq
Here's another tutorial for creating zsh completers using the built-in functions: https://github.com/vapniks/zsh-completions/blob/master/zsh-c...
Isn't there a standard flag which programs can implement to avoid writing this bash script?
Ideally this could all be part of a library such as argparse for typical cases, right?
Ideally this could all be part of a library such as argparse for typical cases, right?
Rust has the clap_complete package for its most popular arg parsing library: https://crates.io/crates/clap_complete
ripgrep exposes its (bespoke) shell completion and man page generation through a --generate option: rg --generate=man, rg --generate=complete-bash, etcetera. In xh (clap-based) we provide the same but AFAIK we're the only one to copy that interface.
Symfony (for PHP) provides some kind of runtime completion generation but I don't know the details.
ripgrep exposes its (bespoke) shell completion and man page generation through a --generate option: rg --generate=man, rg --generate=complete-bash, etcetera. In xh (clap-based) we provide the same but AFAIK we're the only one to copy that interface.
Symfony (for PHP) provides some kind of runtime completion generation but I don't know the details.
In zsh you can use the _gnu_generic function for simple completion of commands with a --help flag.
Just put a line like this somewhere in your startup file: compdef _gnu_generic <CMD>
_gnu_generic is fantastic. I use it all the time. If your CLI toolkit emits colorized help, but skips said colorization if NO_COLOR[1] is set then you can prefix _gnu_generic with NO_COLOR=1 as in
https://github.com/c-blake/cligen/wiki/Zsh-completion-for-cl... for the cligen Nim CLI toolkit.
A similar thing in Bash is `complete -F _longopt YourCmd`, but these will not work with "multi-commands" that have "sub-commands" as the article of this thread covers. Truth is, as non-standard as GNU long opts already are, how subcommands work is even more non-standard (are there even global options? Or between each subcommand? Is it how Python does it? Or Go? Or some one specific library or ..?)
[^1]: https://no-color.org/
A similar thing in Bash is `complete -F _longopt YourCmd`, but these will not work with "multi-commands" that have "sub-commands" as the article of this thread covers. Truth is, as non-standard as GNU long opts already are, how subcommands work is even more non-standard (are there even global options? Or between each subcommand? Is it how Python does it? Or Go? Or some one specific library or ..?)
[^1]: https://no-color.org/
I've wondered this as well - it would sure be nice if there was a standard --completion or something that common argument parsing libraries could automatically implement for us (much like they often implement automatic help text)
I did something similar to this for tab-completing server names for use with ssh. I went a step further and allowed pattern matching based on the server being qa/prod and on locale. so for instance you could type `ssh co prod <tab>` and it would tab-complete / suggest any servers that were production and located in the Denver datacenter (co is the state abbrev for Colorado, for non-US readers).
Unfortunately my work doesn't allow me to share code, but essentially I remapped ssh to a bash script that maintains an environment variable containing the args (you must do this because each <tab> press is an independent invocation of the script. Then you run into persistence problems, so I added a call to compute elapsed seconds so that it flushes the state variable after a 10s timeout).
The bash script then forwards the args to a python script that reads a JSON file and figures out which params (such as 'co' or 'qa') map to which hostnames. It also matches against partial hostnames, so when you see this after tab
qa-server-co1 qa-server-co2 pr-server-co3
you only need to add '3' to the list of args to narrow it down to 1 match, then hit <enter> to ssh to that host.
Unfortunately my work doesn't allow me to share code, but essentially I remapped ssh to a bash script that maintains an environment variable containing the args (you must do this because each <tab> press is an independent invocation of the script. Then you run into persistence problems, so I added a call to compute elapsed seconds so that it flushes the state variable after a 10s timeout).
The bash script then forwards the args to a python script that reads a JSON file and figures out which params (such as 'co' or 'qa') map to which hostnames. It also matches against partial hostnames, so when you see this after tab
qa-server-co1 qa-server-co2 pr-server-co3
you only need to add '3' to the list of args to narrow it down to 1 match, then hit <enter> to ssh to that host.
Basic completion in ksh is as easy as defining an array. From https://man.openbsd.org/ksh :
Custom completions may be configured by creating an array named ‘complete_command’, optionally suffixed with an argument number to complete only for a single argument. So defining an array named ‘complete_kill’ provides possible completions for any argument to the kill(1) command, but ‘complete_kill_1’ only completes the first argument. For example, the following command makes ksh offer a selection of signal names for the first argument to kill(1):
Custom completions may be configured by creating an array named ‘complete_command’, optionally suffixed with an argument number to complete only for a single argument. So defining an array named ‘complete_kill’ provides possible completions for any argument to the kill(1) command, but ‘complete_kill_1’ only completes the first argument. For example, the following command makes ksh offer a selection of signal names for the first argument to kill(1):
set -A complete_kill_1 -- -9 -HUP -INFO -KILL -TERMIs this in the Korn & Bolsy ksh88 book?
Or is this ksh93 syntax that oksh back ported?
Or is this ksh93 syntax that oksh back ported?
I don't know actually. Are there currently used version of ksh that are not derived from ksh93?
Very much so - pdksh was a bsd-licensed clone that is the ancestor of mksh (Android's system shell) and OpenBSD's oksh, which you previously mentioned.
David Korn thanked the pdksh authors and maintainers for making it available in the years when true ksh was closed (ksh88) or open but licensed with awkward terms (ksh93 for several years).
David Korn interview (I asked one of the questions):
https://m.slashdot.org/story/16351
"First of all pdksh is a ksh88 clone; and I might add a better clone than the MKS Korn Shell...
"I don't know the pdksh development team but I would like to thank them for the service they have done in making a version of ksh available while ksh was proprietary. I have noticed remarkable improvements in pdksh in its ability to mimic ksh88 functionality. I don't know what plans the pdksh development team has now that ksh93 is available in open source form, but I certainly would help them try to maintain compatibility if they do continue pdksh distribution. Otherwise, I would hope that they would pick up the ksh93 source and help support and enhance it."
David Korn thanked the pdksh authors and maintainers for making it available in the years when true ksh was closed (ksh88) or open but licensed with awkward terms (ksh93 for several years).
David Korn interview (I asked one of the questions):
https://m.slashdot.org/story/16351
"First of all pdksh is a ksh88 clone; and I might add a better clone than the MKS Korn Shell...
"I don't know the pdksh development team but I would like to thank them for the service they have done in making a version of ksh available while ksh was proprietary. I have noticed remarkable improvements in pdksh in its ability to mimic ksh88 functionality. I don't know what plans the pdksh development team has now that ksh93 is available in open source form, but I certainly would help them try to maintain compatibility if they do continue pdksh distribution. Otherwise, I would hope that they would pick up the ksh93 source and help support and enhance it."
Why doesn't someone (not me) just build a basic DSL and a transpiler that does this?
The answer to your question is that command-lines have a much larger diversity of syntax (even to get help!) than most people realize. Folks have their 30..60 commands they run frequently and don't run into many or conveniently forget/neglect older ones like `gcc` or `tar` or `dd`. Many people (not saying you specifically) do not even realize that double-dash long options are a GNU extension never standardized or that Python toolkits typically allow --my-opt for --my-option abbreviations, just to name a couple of the dozen variations (space or '=', or ':' or '/' or any of the above or etc., etc.). There are probably hundreds if not thousands of syntax possibilities, but people often act like there is only one.
As an example of diversity estimation that you can try at home, a couple of times I have run every single command in my command search PATH with --help </dev/null >/tmp/help.$c 2>&1 . Caution - be careful if you do this! Have backups/checksums of everything important and run as an unprivileged user. I always have to kill off several processes that just hang doing something or otherwise manually intervene. Anyway, this alone suggests data collection of help text is not a trivial problem.
Beyond data collection, many commands did not/do not use CLI toolkits at all. Their commands may have even less regular syntax. Freeform help makes it harder to produce a regular help syntax to convert into the interpreter needed by a completion system. That said, as elsethread commented for some toolkits the Zsh _gnu_generic works great! It essentially IS the "automagic" system you might want, just for a highly restricted circumstance.
Any CLI toolkit itself does have the data, by necessity. So, if the CLI framework supports the 2 or 3 common shells there is no need for a translator exactly. You just need a code generator. There is a stab at an auto-generation framework from said data for the Nim CLI toolkit, cligen, over at:
https://github.com/c-blake/cligen/blob/master/util/complgen....
but it only works for Zsh right now. Anyway, I don't think perfect should be the enemy of the good or anything like that, but you seemed to ask an earnest "why" question and these are some of the complexities.
As an example of diversity estimation that you can try at home, a couple of times I have run every single command in my command search PATH with --help </dev/null >/tmp/help.$c 2>&1 . Caution - be careful if you do this! Have backups/checksums of everything important and run as an unprivileged user. I always have to kill off several processes that just hang doing something or otherwise manually intervene. Anyway, this alone suggests data collection of help text is not a trivial problem.
Beyond data collection, many commands did not/do not use CLI toolkits at all. Their commands may have even less regular syntax. Freeform help makes it harder to produce a regular help syntax to convert into the interpreter needed by a completion system. That said, as elsethread commented for some toolkits the Zsh _gnu_generic works great! It essentially IS the "automagic" system you might want, just for a highly restricted circumstance.
Any CLI toolkit itself does have the data, by necessity. So, if the CLI framework supports the 2 or 3 common shells there is no need for a translator exactly. You just need a code generator. There is a stab at an auto-generation framework from said data for the Nim CLI toolkit, cligen, over at:
https://github.com/c-blake/cligen/blob/master/util/complgen....
but it only works for Zsh right now. Anyway, I don't think perfect should be the enemy of the good or anything like that, but you seemed to ask an earnest "why" question and these are some of the complexities.
Quickly skimming your code it seems like that's what I was asking for? basically you enter your auto completions without knowing the bash syntax?
As for the 30..60 commands, I use the following "tricks"
1) my ctrl-p is mapped in a way that it searches the prefix into history. (On phone right now)
2) long history which syncs with other machines in a machine-name-suffixed file via syncthing
3) justfile which are really a game changer. I used bare j to tell me what commands are available (as opposed to running the first command) and I basically know everything I can and do do in that folder
4)
As for the 30..60 commands, I use the following "tricks"
1) my ctrl-p is mapped in a way that it searches the prefix into history. (On phone right now)
2) long history which syncs with other machines in a machine-name-suffixed file via syncthing
3) justfile which are really a game changer. I used bare j to tell me what commands are available (as opposed to running the first command) and I basically know everything I can and do do in that folder
4)
People do!
See: https://pixi.carapace.sh/ or https://github.com/withfig/autocomplete
It's still a hard problem as lots of tools format --help differently. One of the things I'm jealous in Poweshell is their standardized completions
See: https://pixi.carapace.sh/ or https://github.com/withfig/autocomplete
It's still a hard problem as lots of tools format --help differently. One of the things I'm jealous in Poweshell is their standardized completions
Yes! I posed about pwsh here https://news.ycombinator.com/item?id=44860445
Here's zsh snippet I've came up with for my own simple functions. I'm using it as a base for other completions. In this example, function `set-java-home zulu-21` sets JAVA_HOME to `~/apps/java/zulu-21`. Here's `_set-java-home`:
#compdef set-java-home
local -a versions=(~/apps/java/*(:t))
_describe 'version' versions
Do many people do bash on osx or zsh on Linux, and would this make much of a difference?
I don't know about "use" — luckily, there's no opt-out telemetry — but enough of "enthusiast distribution" users who have also opted in (very biased sample) have explicitly installed zsh (not necessarily run it)
https://pkgstats.archlinux.de/compare/packages#packages=bash...
OTOH, it's only 4-7% on Debian (also opt-in):
https://qa.debian.org/popcon.php?package=zsh
https://pkgstats.archlinux.de/compare/packages#packages=bash...
OTOH, it's only 4-7% on Debian (also opt-in):
https://qa.debian.org/popcon.php?package=zsh
Why does this type of statistic require an opt-in solution? Can't the Arch mirrors just tally requests for each package without identifying information like IP adddress?
Maybe they can’t ask for info on every mirror.
I use zsh at work and bash at home. I am such an unsophisticated user that I haven't noticed a real difference! Other than I can install ohmyzsh on zsh.
I repeat the different variations of the same command so often that I get a large quality of life improvement by making that easier- that's why I take the trouble to install zsh.
Regrettably due to the licensing stupidity there's actually two bash on macOS: /bin/bash and $HOMEBREW_PREFIX/bin/bash they do differ (as is expected by a two major version spread)
I'm firmly in the never-zsh camp, but I'd guess so long as one doesn't do something silly like use #!/bin/zsh in shared scripts you can continue to use whatever interactive shell you like
I had a college who was hard core into fish but had to be completely self sufficient because he was the only user in the office. I presume it's the same as being on Linux in a Windows shop or being on Windows in a macOS shop: no one has the day to day experience to have the answers handy
I'm firmly in the never-zsh camp, but I'd guess so long as one doesn't do something silly like use #!/bin/zsh in shared scripts you can continue to use whatever interactive shell you like
I had a college who was hard core into fish but had to be completely self sufficient because he was the only user in the office. I presume it's the same as being on Linux in a Windows shop or being on Windows in a macOS shop: no one has the day to day experience to have the answers handy
Homebrew bash is my default shell on macOS.
Since auto text completion is the primary task of an LLM, I have seen GitHub copilot do that very well on both bash and zsh within vscode terminal!
I wish tcsh would get more love.
There is a famous paper on the perils of scripting in the csh. It is unfortunate that Bill Joy was not able to write a formal grammar or parser for his language. It was certainly a missed opportunity, and tcsh cannot fix the design.
That being said, csh advocates definitely influenced everything in the Bourne/POSIX family.
That being said, csh advocates definitely influenced everything in the Bourne/POSIX family.
It's a classic Usenet post that deserves linking at every opportunity!
https://harmful.cat-v.org/software/csh
https://harmful.cat-v.org/software/csh
Why, it's a dinosaur? Have you tried nushell, murex, oil shell or xonsh?
I don't know why you're being downvoted. You're correct, and as the other commenter mentioned, csh programming is evil in the eyes of the LORD. Starting last year, I've been maintaining a bash profile that implements (mostly) the same setup as my tcsh configuration. I suppose I ought to make the final leap into something more modern. It would be nice not to jump through hoops to redirect stderr somewhere different than stdout or [insert your favorite of the myriad examples of csh/tcsh weirdness here]! But I feel irrationally sad about it. I cut my teeth on Unix systems where csh was the default shell—namely, NeXTstep and sunOS 4—and it feels a little like saying the final goodbyes to a dead friend. Silly of me, I realize, but there it is.
zsh standard tab completion drives me batty. But I come from bash where 99% of the time I just want filename completion.
Shell syntax is the exact reason why we've needed LLMs in the first place.
You mean Unix shell syntax. Powershell has got this absolutely right ,and only this (which is probably still a 50% of what a shell is)
Can you expand on why it is good I have never read any good evangelisation for it. I can not stand PowerShell the syntax is backward, least important information first. It is also very slow for me but I have understood that it is a pebkac issue. So I am open to be corrected.
By "got this right", the parent poster probably meant tab completion. You get that for free when writing a script or function, as well as basic syntax documentation. Both can be improved gradually by adding hints / comments in your script.
If you are bothered by the verbose `Verb-Noun` structure of commands, those are mostly used in scripts (and you never type them fully, you use tab-completion of course). On the commandline you use aliases for many of them.
For example listing files and directories created in the last 10 days:
If you are bothered by the verbose `Verb-Noun` structure of commands, those are mostly used in scripts (and you never type them fully, you use tab-completion of course). On the commandline you use aliases for many of them.
For example listing files and directories created in the last 10 days:
ls | ? CreationTime -ge (Get-Date).AddDays(-10) | ft name, size
Get-ChildItem | Where-Object CreationTime -ge (Get-Date).AddDays(-10) | Format-Table Name, Size
I think they’re talking about powershells programming syntax which is much more sane than any Unix shell. There’s way less footguns and everything is typed. You’re not dealing with raw strings for everything.
If you don’t like the Verb-Noun nonsense I’d encourage you to look at the default aliases as they make everything a lot less verbose. For example Where-Object is just “where” or getchild-item is default aliased to ls and gci.
Id encourage you to look at NuShell as well since it is mostly the same philosophy as PowerShell.
If you don’t like the Verb-Noun nonsense I’d encourage you to look at the default aliases as they make everything a lot less verbose. For example Where-Object is just “where” or getchild-item is default aliased to ls and gci.
Id encourage you to look at NuShell as well since it is mostly the same philosophy as PowerShell.
On phone:
Bad for your reasons (slow etc), good for siblings (objects).
Also, understandable flags that auto-shorten. Eg:
Descriptive-Command-Name -DescriptiveOption 32
Can be typed as:
d-c-n <tab> -des 32
---
Interactively it's still far behind zsh sadly due to startup time but when I'm doing something thoughtful I prefer it. If you're not entering directories and doing small stuff, use pwsh. And for all shell scripting
Bad for your reasons (slow etc), good for siblings (objects).
Also, understandable flags that auto-shorten. Eg:
Descriptive-Command-Name -DescriptiveOption 32
Can be typed as:
d-c-n <tab> -des 32
---
Interactively it's still far behind zsh sadly due to startup time but when I'm doing something thoughtful I prefer it. If you're not entering directories and doing small stuff, use pwsh. And for all shell scripting
Perl solved that 20 years ago.
And `rc` under plan9/9front did a Unix shell better than the classic Unix itself.
And `rc` under plan9/9front did a Unix shell better than the classic Unix itself.
It parses all man pages on your system and generates completion files for you. By default, they go into ~/.cache/fish/generated_completions/*
If the man page was written poorly/is missing, you can always write your own completion (and hopefully send it upstream). fish uses such a simple format that I don't think there's any need for tutorials save the official doc:
https://fishshell.com/docs/current/completions.html
For example, here's an excerpt from curl