Bestline: Light self-contained readline alternative(github.com)
github.com
Bestline: Light self-contained readline alternative
https://github.com/jart/bestline
48 comments
Looks like this doesn't support Ctrl-O (run line and retrieve next line from history). I've found that incredibly useful for re-running a series of commands: Ctrl-R to reverse search, type a few letters, Ctrl-O Ctrl-O Ctrl-O.
The one shortcut I use the most, in addition to the listed ones, is "Ctrl-e Ctrl-e" which opens an editor then sends the resulting content to the shell.
Using an editor is great for loops, loops with nested conditions, etc.
Using an editor is great for loops, loops with nested conditions, etc.
Here's a (rather less sophisticated) personal favourite of mine: alt + # causes a # symbol to be prepended to the line, and then runs it. Useful for entering a nearly-finished line into your command history for later, commented out, if you realise you need to fix up something else beforehand.
Slightly faster than ctrl-a followed by # followed by return.
Slightly faster than ctrl-a followed by # followed by return.
Or, if you are using vim mode with set -o vi (works with zsh and bash) you can just type # in normal mode to comment that line, do other things then come back up in history with k and run that line with # too.
I usually use C-q for that: it stores the current line in a buffer which it restores after the next execution.
On my machine, ctrl-q appears to do nothing.
(Aside: I've never seen the sense in using C to denote ctrl. How do you denote C?)
(Aside: I've never seen the sense in using C to denote ctrl. How do you denote C?)
> On my machine, ctrl-q appears to do nothing.
Ah apparently it's a ZSH-specific shortcut for "push-line". readline follows Emacs' (not illogical but probably less useful here) "quoted-insert" (generally used to include non-literal code e.g. C-q a does nothing as "a" is already literal, but C-q C-q should insert 0x 11 / XON / DC1).
> (Aside: I've never seen the sense in using C to denote ctrl. How do you denote C?)
c. Or C for S-c (shift-c).
a-b indicates a single keystroke, meaning a modified key, so the first character can only signify a modifier key[0]. Which "c" is not. The sequence "upper case C then lowercase c" is "C c", not "C-c".
[0] well technically you could have keystrokes of non-modifiers as modern keyboards can handle concurrent keycodes, but historical systems don't so a keystroke is only a bunch of modifiers + a single key e.g. C-S-M-c is unambiguously Control-Shift-Meta-c
Ah apparently it's a ZSH-specific shortcut for "push-line". readline follows Emacs' (not illogical but probably less useful here) "quoted-insert" (generally used to include non-literal code e.g. C-q a does nothing as "a" is already literal, but C-q C-q should insert 0x 11 / XON / DC1).
> (Aside: I've never seen the sense in using C to denote ctrl. How do you denote C?)
c. Or C for S-c (shift-c).
a-b indicates a single keystroke, meaning a modified key, so the first character can only signify a modifier key[0]. Which "c" is not. The sequence "upper case C then lowercase c" is "C c", not "C-c".
[0] well technically you could have keystrokes of non-modifiers as modern keyboards can handle concurrent keycodes, but historical systems don't so a keystroke is only a bunch of modifiers + a single key e.g. C-S-M-c is unambiguously Control-Shift-Meta-c
Just "C".
Ctrl is "C-", prefixed to another key without a space, so it's unambiguous.
Ctrl is "C-", prefixed to another key without a space, so it's unambiguous.
> Aside: I've never seen the sense in using C to denote ctrl. How do you denote C?
To denote a capital c, use Shift-c or some variant thereof.
To denote a capital c, use Shift-c or some variant thereof.
This is cool, thanks for sharing! I have the following in my .inputrc which is in a similar vein (may be an anti pattern):
```
# F1, F2, F3 - calls alias1, alias2, alias3, helpful if you are executig the same ommands over and over
"^[OP": "alias1\r"
"^[OQ": "alias2\r"
"^[OR": "alias3\r"
```
Wow, I didn't know about this, thanks so much!
I'm a bit puzzled by "reducing binary footprint (surprisingly) by removing bloated dependencies" since linenoise has no dependencies (beyond libc) that I'm aware of.
$ size linenoise_example
text data bss dec hex filename
14718 948 184 15850 3dea linenoise_example
$ size bestline_example
text data bss dec hex filename
40830 1000 9280 51110 c7a6 bestline_example
Author here. It's a question of beneath the iceberg dependencies revealed by static linking.
It matters because a library that's this low-level and so fundamental to the needs of nearly every program, should impose as few choices upon the user as possible, in terms of what other things they're required to support. For example, many people don't like printf() style interfaces and therefore do not want them included in their address space. The same goes for huge libraries like Curses and ICU that require you to link in megs of code/data just to read a character correctly. You want the value those things offer, but you don't want the baggage if all you care about is just reading a line comfortably.
That's where Bestline comes in. It distills the value of all those heavyweight dependencies down into a single .c file focused on reading lines and only reading lines, which everyone can agree on, that's actually tinier than the original library at the end of the day. So you get a better value as a software developer in terms of the code complexity and dependency bloat you need to take on.
$ make clean && CC="x86_64-linux-musl-gcc -s -static -DNDEBUG -Os" make
$ ls -hal bestline_example
-rwxr-xr-x 1 jart jart 38K Sep 19 21:41 bestline_example
$ make clean && CC="x86_64-linux-musl-gcc -s -static -DNDEBUG -Os" make
$ ls -hal linenoise_example
-rwxr-xr-x 1 jart jart 50K Sep 19 21:41 linenoise_example
It matters because a library that's this low-level and so fundamental to the needs of nearly every program, should impose as few choices upon the user as possible, in terms of what other things they're required to support. For example, many people don't like printf() style interfaces and therefore do not want them included in their address space. The same goes for huge libraries like Curses and ICU that require you to link in megs of code/data just to read a character correctly. You want the value those things offer, but you don't want the baggage if all you care about is just reading a line comfortably.
That's where Bestline comes in. It distills the value of all those heavyweight dependencies down into a single .c file focused on reading lines and only reading lines, which everyone can agree on, that's actually tinier than the original library at the end of the day. So you get a better value as a software developer in terms of the code complexity and dependency bloat you need to take on.
The readme mentions
> Remove heavyweight dependencies like printf/sprintf
But that’s from libc, unless they measure by statically linking everything?
> Remove heavyweight dependencies like printf/sprintf
But that’s from libc, unless they measure by statically linking everything?
When you think about using the library from languages which don't link against libc by default (Go for example), then this is a significant difference.
How does Go have any relevance? This is a C program, it will link against libc.
You can easily call C libraries from languages like Go and Rust. It absolutely makes sense to write small libraries in C so that all kind of languages can use the library which are not C. Go programs by default do not link against libc, so it would be nice not introducing the dependency. So this would fall under "nice to have".
The example is still relevant, say if your program has no OS.
> The example is still relevant, say if your program has no OS.
Not only does it seem unlikely your program would have a terminal output and standard streams without having an OS, bestline still requires a libc. Just open `bestline.c` and you'll find:
Not only does it seem unlikely your program would have a terminal output and standard streams without having an OS, bestline still requires a libc. Just open `bestline.c` and you'll find:
#include <termios.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <errno.h>
#include <string.h>
#include <stdlib.h>
#include <ctype.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <sys/ioctl.h>
#include <sys/mman.h>
#include <unistd.h>
#include <setjmp.h>
#include <poll.h>
#include <assert.h>
#include <signal.h>
#include <fcntl.h>
#include <limits.h>You've never used a microcontroller. They are usually shipped with a partial libc that binds fd 0/1 to a USART. Sometimes POSIX threads can be mapped to FreeRTOS calls.
Most of the defines can have a null implementation and the core will still work. They just need to be defined.
Most of the defines can have a null implementation and the core will still work. They just need to be defined.
You’re not making any sense. If the system provides a libc stub then printf and whatever are not concerns, they’ll just go through the stub.
And the message I quoted says literally nothing about µc.
And the message I quoted says literally nothing about µc.
So I think (having read the readme but not the code and not being expert in the subject matter) this is a library that lets you do things like read line, but probably is not API or ABI compatible (because that seems like something that would be mentioned and I can't see anything about it)?
Also, I have to say I'm entertained at completely throwing out worrying about terminals and just declaring that everything supports vt100 so we're going to target that and be done with it. Kinda elegant, especially for the relatively limited functionality they need (not like it's curses where you're really going to exercise the terminal's capabilities).
Also, I have to say I'm entertained at completely throwing out worrying about terminals and just declaring that everything supports vt100 so we're going to target that and be done with it. Kinda elegant, especially for the relatively limited functionality they need (not like it's curses where you're really going to exercise the terminal's capabilities).
I suppose there are two kinds of terminals in real use today: terminal emulators (which all support vt100) and dumb / trivial consoles that use a framebuffer if not a segmented LCD display, that don't support anything standardized anyway, and you don't need much from them beside displaying characters.
So the choice to support the common case of the terminal emulator looks reasonable. Even when you attach a serial cable to a tiny controller to do something on its tiny command line, you likely do it from your laptop. Having a readline lookalike in this situation would be nice.
So the choice to support the common case of the terminal emulator looks reasonable. Even when you attach a serial cable to a tiny controller to do something on its tiny command line, you likely do it from your laptop. Having a readline lookalike in this situation would be nice.
The README mentions Antirez's coding philosophy as being an inspiration:
> This codebase aims to follow in Antirez's tradition of writing beautiful programs, that solve extremely difficult difficult technical problems in the simplest most elegant way possible.
Is something Antirez has talked about himself anywhere? I'm interested to know more, but am not fluent enough in C to understand what makes this code elegant.
> This codebase aims to follow in Antirez's tradition of writing beautiful programs, that solve extremely difficult difficult technical problems in the simplest most elegant way possible.
Is something Antirez has talked about himself anywhere? I'm interested to know more, but am not fluent enough in C to understand what makes this code elegant.
This library is a fork of Antirez's linenoise, so it's perhaps not only an inspiration but a quality that the code had to begin with, which the author of the fork declares that she intends to preserve.
The linenoise README mentions "sensibility for small easy to understand code". It's not really explained further in the README, but I suppose the point is that you can read the ~1200 lines of source code and get a feel for it yourself.
The linenoise README mentions "sensibility for small easy to understand code". It's not really explained further in the README, but I suppose the point is that you can read the ~1200 lines of source code and get a feel for it yourself.
Tbh, it doesn't seem to me like a great code.
https://github.com/jart/bestline/blob/master/bestline.c#L286...
This interface function is already terrible. Why doesn't it check the error code of bestlineHistoryLoad()? It smells like a sloppy C program with obscure failure modes.
https://github.com/jart/bestline/blob/master/bestline.c#L286...
This interface function is already terrible. Why doesn't it check the error code of bestlineHistoryLoad()? It smells like a sloppy C program with obscure failure modes.
I didn’t dive deep into this code but what I would assume makes this elegant is that it is mostly declarative/lookup style with only a little imperative code where necessary. This style doesn’t use fancy tricks/hacks and instead uses lookup tables where necessary to encapsulate logic.
Open this code side by side with the source code for Cyrus IMAP, or Perl core, or libssl, and then scroll through them. You will be enlightened.
In short, any code that is intended to work on a wide variety of systems is usually littered with conditional compilation directives and macros that make the code extremely difficult to follow without a extensive study of the author’s “setup”. This is because to compile/run on a wide variety of systems, you actually need wildly different C source code, and what is written in the C source file ends up being more like a template for a program than the actual program.
In short, any code that is intended to work on a wide variety of systems is usually littered with conditional compilation directives and macros that make the code extremely difficult to follow without a extensive study of the author’s “setup”. This is because to compile/run on a wide variety of systems, you actually need wildly different C source code, and what is written in the C source file ends up being more like a template for a program than the actual program.
That's only if you want your source code to compile on a lot of different systems. If your goal is to be able to have your program run on a lot of systems, then it's so much easier to just attack the binary interfaces straight on and ignore the platform tooling. That's how I got my executables to run on seven operating systems. See https://justine.lol/ape.html
At this point there are already hundreds of readline alternatives across various different languages as it’s actually surprisingly easy to write a readline library.
Given the capabilities of computers these days, I think if one is stuck choosing which library to use, I’d suggest get the one that’s most complete rather than most compact. I guarantee you that there will be one of your users who’ll use an Emacs or Vi shortcut and find the lack of support really prohibiting.
Given the capabilities of computers these days, I think if one is stuck choosing which library to use, I’d suggest get the one that’s most complete rather than most compact. I guarantee you that there will be one of your users who’ll use an Emacs or Vi shortcut and find the lack of support really prohibiting.
Curious about the UI paradigm here and still newbie after all these years.
Is there a well written explanation about the idea / logic behind the terminology of kill line / yank somewhere for people who are used to ... well, just what modern GUIs provide, i.e copy/paste etc? Pros/cons? What functionality do these operations actually provide? Thanks.
Is there a well written explanation about the idea / logic behind the terminology of kill line / yank somewhere for people who are used to ... well, just what modern GUIs provide, i.e copy/paste etc? Pros/cons? What functionality do these operations actually provide? Thanks.
The terminology comes from Emacs:
https://www.emacswiki.org/emacs/KillingAndYanking https://www.gnu.org/software/emacs/manual/html_node/emacs/Ki...
The basic idea is that your clipboard ("kill ring") can contain many entries, so you can kill several lines (or words, sentences, functions, what have you) and then yank them back at other locations. Yanking copies the most recently killed text, but you can use another command to replace it with previous entries until you find the right one. Thus you don't have to fear losing the clipboard while performing intermediate operations.
Readline-type libraries typically implement a small subset of the Emacs kill-ring features, so you really need to try Emacs to appreciate the full range of the concept.
https://www.emacswiki.org/emacs/KillingAndYanking https://www.gnu.org/software/emacs/manual/html_node/emacs/Ki...
The basic idea is that your clipboard ("kill ring") can contain many entries, so you can kill several lines (or words, sentences, functions, what have you) and then yank them back at other locations. Yanking copies the most recently killed text, but you can use another command to replace it with previous entries until you find the right one. Thus you don't have to fear losing the clipboard while performing intermediate operations.
Readline-type libraries typically implement a small subset of the Emacs kill-ring features, so you really need to try Emacs to appreciate the full range of the concept.
For instance, the kill ring allows you to easily move around two words in different parts of a long command.
- Move cursor to the first word, kill it with Alt+D (aka M-D). - Fearlessly move the cursor to another word, kill it, too. - Now move where you want to put the first word back, because it's nearby. Press Ctrl+Y, oops, it shows the second word. Never mind, press Alt+Y, and it pastes the first word on its place. - Now move elsewhere where the second word belongs. Ctrl+Y pastes (yanks) the first word again, the most recently used. Alt+Y changes it to the second word, the second most recently used.
This way you may have quite a few fragments in the ring and juggle them, repeat them, several different fragments, without the need to copy again and again.
- Move cursor to the first word, kill it with Alt+D (aka M-D). - Fearlessly move the cursor to another word, kill it, too. - Now move where you want to put the first word back, because it's nearby. Press Ctrl+Y, oops, it shows the second word. Never mind, press Alt+Y, and it pastes the first word on its place. - Now move elsewhere where the second word belongs. Ctrl+Y pastes (yanks) the first word again, the most recently used. Alt+Y changes it to the second word, the second most recently used.
This way you may have quite a few fragments in the ring and juggle them, repeat them, several different fragments, without the need to copy again and again.
Thanks. I suppose I would have to see this in practice to be able to understand and appreciate it.
My memory is limited so it doesn't seem worth the trouble to learn such a niche system. Still there's something intriguing about it as someone who tries to keep one's mind open about different design systems.
As a ui/ux designer what strikes me with these legacy systems is there seems to be no fluent support system in place for learning them. And even after you've learned, no visibility to support long term memorization.
Also, I suppose there is no clear visual panel to support memory as to what's actually in the ring but you kinda have to just browse it one by one with shortcuts, as I understand the above?
Do the terms kill/yank seem appropriate to others? It seems that there is a slang context I'm missing there that might make the terms more intuitive.
My memory is limited so it doesn't seem worth the trouble to learn such a niche system. Still there's something intriguing about it as someone who tries to keep one's mind open about different design systems.
As a ui/ux designer what strikes me with these legacy systems is there seems to be no fluent support system in place for learning them. And even after you've learned, no visibility to support long term memorization.
Also, I suppose there is no clear visual panel to support memory as to what's actually in the ring but you kinda have to just browse it one by one with shortcuts, as I understand the above?
Do the terms kill/yank seem appropriate to others? It seems that there is a slang context I'm missing there that might make the terms more intuitive.
People 40 years ago were no less smart than ourselves, but the machines they had in their disposal were much, much less mighty. Coding discoverability features into them was often very hard, especially with relatively slow terminals of the day. They relied on manuals instead.
Now we have inherited these technologies, but there is often no reasonable way to modernize their UX in the discoverability department, because of key assumptions made decades ago. All we can use now is still manuals; `man bash`, search for the READLINE section, and read the extensive explanation of the riches hiding there. You can search by regexp, so ^READLINE will bring you right to the section, but again there is no affordance to show you that a regexp is acceptable.
And yes, I think nobody thought through the terms kill / yank; their authors just picked some metaphors 40 or more years ago, because they were building a tool for themselves and their colleagues in a computer department of a university, all engineers used to weird abstractions.
The world is very different by now.
Now we have inherited these technologies, but there is often no reasonable way to modernize their UX in the discoverability department, because of key assumptions made decades ago. All we can use now is still manuals; `man bash`, search for the READLINE section, and read the extensive explanation of the riches hiding there. You can search by regexp, so ^READLINE will bring you right to the section, but again there is no affordance to show you that a regexp is acceptable.
And yes, I think nobody thought through the terms kill / yank; their authors just picked some metaphors 40 or more years ago, because they were building a tool for themselves and their colleagues in a computer department of a university, all engineers used to weird abstractions.
The world is very different by now.
In modern Emacs versions the top of the kill ring can be found in the Edit menu, and there are other packages to help navigate it: https://www.emacswiki.org/emacs/BrowseKillRing
I agree that the ancient terminology is a problem with Emacs. People who are used to Emacs don't want to change it, but it is alien to all new users. Funnily enough, in vi (a competing old text editor that predates current UI standards) "yank" means "copy".
Another example is "window" and "frame", which mean the opposite that you would think if you have used any modern windowing system. A similar problem (that is not strictly about terminology) is the way "undo" works. It is more powerful than the standard undo/redo system, but it can be very confusing when you encounter it the first time. Repeated "undo" commands work the way they do in other programs, but once you run any other command, the undo operations go in the undo history just like other commands and are themselves undoable, so the way to cancel an undo operation (which is called "redo" in many other programs) is to type some character and then run "undo" twice.
I agree that the ancient terminology is a problem with Emacs. People who are used to Emacs don't want to change it, but it is alien to all new users. Funnily enough, in vi (a competing old text editor that predates current UI standards) "yank" means "copy".
Another example is "window" and "frame", which mean the opposite that you would think if you have used any modern windowing system. A similar problem (that is not strictly about terminology) is the way "undo" works. It is more powerful than the standard undo/redo system, but it can be very confusing when you encounter it the first time. Repeated "undo" commands work the way they do in other programs, but once you run any other command, the undo operations go in the undo history just like other commands and are themselves undoable, so the way to cancel an undo operation (which is called "redo" in many other programs) is to type some character and then run "undo" twice.
Emacs gives you a pretty pampered experience. The subthread started with a discussion of the shell command line, and its Emacs-like editing mode, which remains quite Spartan in its affordances: if you don't know the keys, nothing will hint or guide you.
(With Emacs as my daily driver, I greatly miss its uniformity of window management even in environments that look like its spiritual successors, like VSCode.)
(With Emacs as my daily driver, I greatly miss its uniformity of window management even in environments that look like its spiritual successors, like VSCode.)
It would be cool to see a version of rlwrap [1] (blwrap in this case?) using this library.
[1]: https://man.archlinux.org/man/rlwrap.1
[1]: https://man.archlinux.org/man/rlwrap.1
What advantages would a blwrap have over rlwrap? As I understand it, bestline's advantages over readline (fewer dependencies, license) only apply when you are using it as a library. rlwrap is a separate program that weighs in at around 200k. That could only ever be a problem on an embedded system, where rlwrap/blwrap wouldn't make sense anyway.
Trying to compile it on osx. Several errors like this:
make
cc -I/usr/local/opt/ruby/include -c -o bestline.o bestline.c
In file included from bestline.c:122:
In file included from /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/termios.h:26:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/sys/cdefs.h:681:49: error: invalid token at start of a preprocessor expression
#if defined(_POSIX_C_SOURCE) && _POSIX_C_SOURCE == 1L
^The issue you encountered should be fixed! I've confirmed it's building on Mac OS X. Enjoy.
Apparently, it does not fully conforms to POSIX as README claims - AFAIK, that "_POSIX_X_SOURCE" feature test macro should be defined with positive integer value per POSIX standard, not empty as defined in bestline.c
Hm I don't see mention of vi key bindings? set -o vi is awesome in bash / Python / R, etc.
I have added bestline to "my butterflies", independent software reviews site, http://161.35.115.119/mbf/project/bestline/reviews , one can vote, write reviews here as well.