Scripting with Go(bitfieldconsulting.com)
bitfieldconsulting.com
Scripting with Go
https://bitfieldconsulting.com/golang/scripting
66 comments
I may be in the minority here, but I personally prefer the "wrong answer" highlighted at the beginning of this article. Scripts are source code. They go in the repo, and they have the same standards applied to them as any other source code. I would much prefer that the code be explicit and rely on few if any third party libraries. I have go scripts that have been functioning in production for half a decade now without modification. They are as "self documenting" as any other go code, and I do not require esoteric knowledge about a third party library to re-familiarize myself with them.
I think you are missing the point. The overarching motivation here is "Normally one would do this in bash, but I want to use go instead. Is there any way for me to do that while retaining the aspects of bash that make it good for these tasks?" And the author provides a solution. Dunno why you would do that, but not really important.
I agree. It would be much more readable if it simply had a few comments too (something that can't really exist in a 1-liner) and is infinitely flexible (ex: for each match, also make an API call).
I like me some bash scripting. And I do a lot of it. And I write a good amount of Go.
I read the intent of the original package as a bit different than how it’s presented here; a pain point in using go for systems programming on Posix systems with GNU tooling is interacting with all the other really excellent tools there. There’s an awful lot of cruft to go spawn a shell script in Go without this lib.
So, I cannot imagine using this to replace a short-ish bash script, it’s just too heavyweight to add go development into the workflow. But, I can easily imagine using this when I want to do some simple-to-medium-scale pipelined text processing from the middle of a go program.
I read the intent of the original package as a bit different than how it’s presented here; a pain point in using go for systems programming on Posix systems with GNU tooling is interacting with all the other really excellent tools there. There’s an awful lot of cruft to go spawn a shell script in Go without this lib.
So, I cannot imagine using this to replace a short-ish bash script, it’s just too heavyweight to add go development into the workflow. But, I can easily imagine using this when I want to do some simple-to-medium-scale pipelined text processing from the middle of a go program.
Thing is, bash is everywhere (or, at least a POSIX shell is) - for some use cases that matters, because you can't control what is available on the target machine. Even python isn't guaranteed - OK python is fairly ubiquitous, but what version?
Shell scripting is far from perfect, and sometimes it does take a while to figure out how to do something that would be easy in another language - but very often, she'll scripting is "good enough".
Shell scripting is far from perfect, and sometimes it does take a while to figure out how to do something that would be easy in another language - but very often, she'll scripting is "good enough".
This is my feeling too.
I’ve written plenty of bash scripts to automate common tasks. Colleagues won’t open them up and extend or maintain them (some developers won’t touch bash). But if they have an easy to use, well documented interface, they will happily use them to be productive.
The problem with Python is as you describe. It’s not always guaranteed and if you want everyone’s environment on the same page, you then have to start faffing with virtual envs and/or version managers.
This is totally fine on projects written in Python as a whole. But, otherwise, adding complexity to the tool chain is a tough sell to the team.
If things get hairy in bash, it’s probably better to extract those parts in the project language and call them from bash. Your build will already be half way there to accommodate this vs introducing a scripting language.
Not only that, maintaining those programs involves less context switching. Although mainstream languages blur with enough experience, there is still overhead between juggling idioms, syntax and what not in, say, Python and Go. I’m not perfect: if I’ve been working all day in one language, writing another is like a smack in the face. Sometimes I’ll forget semicolons, or include them. Sometimes I’ll use camel case when really the style in that language is snake case. Not a big deal but it is jarring.
There is still context switching with bash but if you use a terminal-based workflow, that does have less of an impact.
I’ve written plenty of bash scripts to automate common tasks. Colleagues won’t open them up and extend or maintain them (some developers won’t touch bash). But if they have an easy to use, well documented interface, they will happily use them to be productive.
The problem with Python is as you describe. It’s not always guaranteed and if you want everyone’s environment on the same page, you then have to start faffing with virtual envs and/or version managers.
This is totally fine on projects written in Python as a whole. But, otherwise, adding complexity to the tool chain is a tough sell to the team.
If things get hairy in bash, it’s probably better to extract those parts in the project language and call them from bash. Your build will already be half way there to accommodate this vs introducing a scripting language.
Not only that, maintaining those programs involves less context switching. Although mainstream languages blur with enough experience, there is still overhead between juggling idioms, syntax and what not in, say, Python and Go. I’m not perfect: if I’ve been working all day in one language, writing another is like a smack in the face. Sometimes I’ll forget semicolons, or include them. Sometimes I’ll use camel case when really the style in that language is snake case. Not a big deal but it is jarring.
There is still context switching with bash but if you use a terminal-based workflow, that does have less of an impact.
Bash yes, but commands like sed may or may not support certain flags depending in the platform.
https://stackoverflow.com/questions/5694228/sed-in-place-fla...
https://stackoverflow.com/questions/5694228/sed-in-place-fla...
> But, I can easily imagine using this when I want to do some simple-to-medium-scale pipelined text processing from the middle of a go program.
Yeah, this is where I see it.
I have a cross-platform utility in customer use that replaces a bash script or a .bat file that did a simple job with curl and a database client but was tricky for customers to configure and deploy; I replaced it with an interactively-configurable go command.
In the middle of it is the replacement pipe; this library would help here. Might use it in the rewrite.
Yeah, this is where I see it.
I have a cross-platform utility in customer use that replaces a bash script or a .bat file that did a simple job with curl and a database client but was tricky for customers to configure and deploy; I replaced it with an interactively-configurable go command.
In the middle of it is the replacement pipe; this library would help here. Might use it in the rewrite.
While I see the benefit of this approach, I’m often baffled why people want to go either 100 % POSIX builtins or 100 % scripting language.
The biggest benefit of the shell is its clearly defined input and output (and error) interfaces. Most programming languages can read from and write to stdin, stdout, and stderr.
Why not use it and stick to KISS, replacing one cumbersome POSIX utility at a time, suites for the task? Then you don’t need to chain methods using less idiomatic code. But then you wouldn’t need these kind of libraries either.
The biggest benefit of the shell is its clearly defined input and output (and error) interfaces. Most programming languages can read from and write to stdin, stdout, and stderr.
Why not use it and stick to KISS, replacing one cumbersome POSIX utility at a time, suites for the task? Then you don’t need to chain methods using less idiomatic code. But then you wouldn’t need these kind of libraries either.
I agree in principle, but mastering the syntactic quirk-fest of bash and other shells is really a bit weird, in that surprises arise at runtime.
maybe that's the compelling use of scripting with a statically typed, thus compile-time at least partially low-hanging-fruit-error-checked, language?
maybe that's the compelling use of scripting with a statically typed, thus compile-time at least partially low-hanging-fruit-error-checked, language?
We really need a 'Bash: The Good Parts' book like Doug Crockford did for Javascript. IMHO bash and the shell are in a state that Javascript was ~2005--an old language/tool full of complexities but that can be sharpened and honed down to something beautiful.
IMHO writing procedural style code with lots of if, loops, etc. in the shell can quickly turn into an anti-pattern. Try to stick to simple functions that are chained together in pipelines. The only loop is typically one that processes arguments and that's it.
IMHO writing procedural style code with lots of if, loops, etc. in the shell can quickly turn into an anti-pattern. Try to stick to simple functions that are chained together in pipelines. The only loop is typically one that processes arguments and that's it.
> We really need a 'Bash: The Good Parts' book like Doug Crockford did for Javascript.
Bash is incredibly less complex than Javascript and there is such a resource: the "Bash guide" [1] and "Bash pitfalls" [2] are both excellent resources that teach you how to use Bash properly.
[1] http://mywiki.wooledge.org/BashGuide
[2] http://mywiki.wooledge.org/BashPitfalls
Bash is incredibly less complex than Javascript and there is such a resource: the "Bash guide" [1] and "Bash pitfalls" [2] are both excellent resources that teach you how to use Bash properly.
[1] http://mywiki.wooledge.org/BashGuide
[2] http://mywiki.wooledge.org/BashPitfalls
I have a coworker who uses make for tasks that do not fit comfortably in a few lines of shell script, and it can make for very elegant and readable solutions. Without giving in to the procedural constructs that are so awkward in shell scripting languages.
i do that too...folks seem surprised, but it's handy
That opened my mind… 30 years after I started using make. D’oh!
Bash really just should die already. I really don’t see it being more available than python is, and the fact that basically every 3+ line bash script is logically faulty should be enough of a reason. Add that they are basically unreadable after writing.
> I really don’t see it being more available than python is
Oh, but which version, 2 or 3?
Bash availability may not be guaranteed, but a POSIX shell of some variety is.
Oh, but which version, 2 or 3?
Bash availability may not be guaranteed, but a POSIX shell of some variety is.
People talk about Bash quirks as if Python and Go don't have plenty of quirks that you also find out by trial and error. The difference is, most people hardly ever use Bash. If it were the reverse and people hardly ever used Python or Go or JavaScript, they'd be talking about how quirky those languages are. (They already do, in relation to other languages)
Shellcheck has dramatically increased the ease by which you can write decent scripts before ever running them. But I also have never seen any language that anyone could master without a year or more of constant use.
Shellcheck has dramatically increased the ease by which you can write decent scripts before ever running them. But I also have never seen any language that anyone could master without a year or more of constant use.
I think it boils down to the friction of starting and stopping external processes.
For example, you could in your scripting language use `find` to search for files in a folder and do something with them, but why do that when your language of choice almost certainly has globbing capabilities? You could grep a file for a line, but why do that when you can use your language's inbuilt regex system?
At least from the scripting side, the reason I tend to push more towards using the language and less towards using external processes is because most scripting languages can do what those external processes do in one go.
Perhaps it would make more sense if I were better at defining scope :)
For example, you could in your scripting language use `find` to search for files in a folder and do something with them, but why do that when your language of choice almost certainly has globbing capabilities? You could grep a file for a line, but why do that when you can use your language's inbuilt regex system?
At least from the scripting side, the reason I tend to push more towards using the language and less towards using external processes is because most scripting languages can do what those external processes do in one go.
Perhaps it would make more sense if I were better at defining scope :)
Cross-platform deployment is why I switched from script plus utilities to a go binary.
I did manage to make a Windows batch file that replicated the functionality of a linux/mac bash script, but configuring it was no fun for customers on any platform, and then there were the utilities themselves to deploy.
The replacement binary has a very small platform-dependent aspect, and I am not held back by the limits of batch files when trying to achieve feature parity.
It might be doable to deploy a powershell script, but then there's installation work to do on the unix side instead.
I did manage to make a Windows batch file that replicated the functionality of a linux/mac bash script, but configuring it was no fun for customers on any platform, and then there were the utilities themselves to deploy.
The replacement binary has a very small platform-dependent aspect, and I am not held back by the limits of batch files when trying to achieve feature parity.
It might be doable to deploy a powershell script, but then there's installation work to do on the unix side instead.
Im facing the inverse of this right now. We have some internal tools that are written in go and are triggered from our scm's equivalent of hooks. Unfortunately these are now failing on non Windows platforms (where we primarily develop) because I can't call a windows go binary on Linux or vice versa. So now I'm back to writing python scripts that wrap golang binaries...
I completely agree.
This is the approach that I took with murex. It comes with a stack of builtins for json, jsonlines, yaml, toml, csv, sqlite3, and others. So you have all the power of data types and intelligent management of structured data but also works with regular POSIX CLI commands without any additional boilerplate code when swapping between murex code and coreutils (et al). So one could say I’ve spent a fair amount of time studying this point you’ve raised :)
This is the approach that I took with murex. It comes with a stack of builtins for json, jsonlines, yaml, toml, csv, sqlite3, and others. So you have all the power of data types and intelligent management of structured data but also works with regular POSIX CLI commands without any additional boilerplate code when swapping between murex code and coreutils (et al). So one could say I’ve spent a fair amount of time studying this point you’ve raised :)
https://github.com/lmorg/murex
(Don't be afraid to self promote!)
One obvious benefit over what's suggested in the article is that you can use it interactively first, with autocomplete and such goodies, and transition smoothly to a script later.
(Don't be afraid to self promote!)
One obvious benefit over what's suggested in the article is that you can use it interactively first, with autocomplete and such goodies, and transition smoothly to a script later.
GP has promoted murex a few times on HN recently, which doesn't bother me, but if I were them I'd be sensitive to not wanting to overdo it as well.
Is murex open source? A quick search of murex shell presented me with beautiful seashells.
A quick Google for murex cli -sea yields: https://murex.rocks and https://github.com/lmorg/murex, looks like a promising tool
Not to mention the URL in @hnlmorg’s HN profile!
Is fish style autosuggestion possible or planned in murex? Thanks!
Let one-liners be one-liners, and bring in Go/etc when it ceases to be a one-liner. But not before.
I'll have to check this out properly later. 10 years ago go devs rejected the idea of introducing support hashbangs and now we're left with having to use gorun[0], meaning, people are unlikely to use go for script.
I hope the go devs will reconsider. I'd love to be able to use go for scripting. But as it stands, it's a sad state of affairs because you have to rely on hacks.
[0] https://github.com/erning/gorun/
I hope the go devs will reconsider. I'd love to be able to use go for scripting. But as it stands, it's a sad state of affairs because you have to rely on hacks.
[0] https://github.com/erning/gorun/
I'm not sure what definition you have for "scripts", but I'm fine with running `go run main.go` or whatever. I don't need my Go files to be executable (half the time I just run bash scripts via `bash script.sh` anyway because that always works). The biggest impediment is the ceremony for subprocessing out, but I'm sure that could be abstracted behind a library with a more pleasant veneer than os/exec, and even if not whatever you lose in subprocess ergonomics you make up for in static typing, not needing to Google/Stack Overflow everything, and general absence of weird quirks.
I mostly use executable scripts so I can put it in a dir in my PATH. That way I can just run globally. No need to know where it is, or how to run it.
> The biggest impediment is the ceremony for subprocessing out
Not sure I fully understand what you mean with this, but if I'm understanding it right, gorun attempts to solve this by doing `syscall.Exec()` on the compiled binary. Combined with slight variation on the "comment" hack (see examples in gorun's readme), signals get fully directed your binary.
> The biggest impediment is the ceremony for subprocessing out
Not sure I fully understand what you mean with this, but if I'm understanding it right, gorun attempts to solve this by doing `syscall.Exec()` on the compiled binary. Combined with slight variation on the "comment" hack (see examples in gorun's readme), signals get fully directed your binary.
// 2>/dev/null ; exec gorun "$0" "$@"
package main
// rest of the codeWhat about dependencies and libraries ? For tech that works very well for self-contained scripts, take a look at Deno. Supports she-bang. Also, you can have self-contained script with imports pointed to URL's or file system paths that are automatically resolved. This leads to easy management, maintenance and distribution.
you can now directly "go run" any import path
go run github.com/mikefarah/yq/[email protected]
#!/usr/bin/env -S go run github.com/mikefarah/yq/[email protected] r
---
some:
yaml: here
./test.yaml some.yaml
testMay be derailing discussion a little, but the nicest scripting language/environment I've ever used is Clojure's babashka[1].
Clojure's data first model + conveniences like the -> and ->> macros make simple data pipeline scripts a joy to write.
babashka of course suffers from the "it isn't everywhere" problem that is mentioned regarding python.
https://github.com/babashka/babashka
Clojure's data first model + conveniences like the -> and ->> macros make simple data pipeline scripts a joy to write.
babashka of course suffers from the "it isn't everywhere" problem that is mentioned regarding python.
https://github.com/babashka/babashka
i am not sure i would personally use this library/approach when scripting, a few reasons:
Using a third-party libraries as core part of your infrastructure (ci/cd scripts, provisioning, automation) implies a greater risk to potential security issues, "framework tax" ie. having to comply, learn, document, debug its custom APIs, having to deal with potential limitations, issues that either needs to be fixed upstream or result in the library being forked and therefore maintained in house. I would rather either put together a set of bash commands or - if the problem entails a more comprehensive endeavour with greater complexity - put together a in-house tool/library where i can make the right compromises from day one
Using a third-party libraries as core part of your infrastructure (ci/cd scripts, provisioning, automation) implies a greater risk to potential security issues, "framework tax" ie. having to comply, learn, document, debug its custom APIs, having to deal with potential limitations, issues that either needs to be fixed upstream or result in the library being forked and therefore maintained in house. I would rather either put together a set of bash commands or - if the problem entails a more comprehensive endeavour with greater complexity - put together a in-house tool/library where i can make the right compromises from day one
Honestly I'm not convinced. I understand the need of a so-called "real" programming language ready available, that's what all classic system have had, from SmallTalk workstation to LispM, but for them there is not just the language, there is the complete ecosystem build with the same language, so an user made script have no difference than a system part of the user desktop.
Unix decide to "simply" creating a system with a system language and an user system with an user language, the shell. I do not like much unix approach after having used Emacs a bit, but I do understand it. On contrary I always fails to "craft scripts" in "real" programming languages no matter what. I've tried in the past to "go Perl", "go Python", "go TCL", yes I can write scripts with them, I have written some etc but if I need something quick I go for the shell, zsh to be more precise (tried others, all failed at a certain point, from bash to elvish passing through oil shell and few others) or being in Emacs (EXWM is my WM/DE) Emacs itself depending on the case.
I read various similar article for a language or another but in the long run see no colleagues really choose a programming language behind shell itself...
Unix decide to "simply" creating a system with a system language and an user system with an user language, the shell. I do not like much unix approach after having used Emacs a bit, but I do understand it. On contrary I always fails to "craft scripts" in "real" programming languages no matter what. I've tried in the past to "go Perl", "go Python", "go TCL", yes I can write scripts with them, I have written some etc but if I need something quick I go for the shell, zsh to be more precise (tried others, all failed at a certain point, from bash to elvish passing through oil shell and few others) or being in Emacs (EXWM is my WM/DE) Emacs itself depending on the case.
I read various similar article for a language or another but in the long run see no colleagues really choose a programming language behind shell itself...
FWIW a lot of other host languages are more DSL-like, and have similar libraries:
https://github.com/oilshell/oil/wiki/Internal-DSLs-for-Shell
It looks like this particular one relies on method chaining
https://github.com/oilshell/oil/wiki/Internal-DSLs-for-Shell
It looks like this particular one relies on method chaining
script.Args().Concat().Match("Error").First(10).Stdout()If you’re willing to add a new binfmt to your system, here’s another method for using golang to write “scripts” (executed with gorun). It’s not the same as a shebang/hashbang, but it works.
https://blog.cloudflare.com/using-go-as-a-scripting-language...
https://blog.cloudflare.com/using-go-as-a-scripting-language...
I felt really fancy about a decade ago learning python and adopting it for even the most trivial tasks. Then I realized at least half the stuff I wrote doesn't even run now without modification. Meanwhile bash scripts I wrote that predated that all still work perfectly. Now I'm back on the bash big time.
This is done absolutely superb API design - taking inspiration from shell pipes and turning it into a Go library with syntax this clean is really impressive: https://github.com/bitfield/script
https://golangexample.com/making-it-easy-to-write-shell-like... with quick examples, everything is pipe based just like bash|fish|zsh|csh
Go script:
p := Exec("man bogus")
p.SetError(nil)
output, err := p.String()
fmt.Println(output)
man bogus
It would be nice if Go could do this sort of chaining like jQuery makes so easy with Javascript:
n, err := object
.Process1(arg1, arg2)
.Process2(true)
.Process3("foo")
You can do that now, just move the dots to end of line
https://github.com/89z/googleplay/blob/v1.8.1/header.go#L126
https://github.com/89z/googleplay/blob/v1.8.1/header.go#L126
I had a look at this library today.
Quote the author: "The script library is implemented entirely in Go, and does not require any external userland programs." Sorry but that's not scripting anymore. We all like using userland programs! it is The Way.
Most programs tell you why they failed on stderr. Seems like stderr is lost when a pipe component fails. Strangely stderr and stdout are conflated in the pipe structure - there is no way to get stderr!
You just get the numeric exit status.
IO redirection appears to be missing.
Difficult to use in production.
Quote the author: "The script library is implemented entirely in Go, and does not require any external userland programs." Sorry but that's not scripting anymore. We all like using userland programs! it is The Way.
Most programs tell you why they failed on stderr. Seems like stderr is lost when a pipe component fails. Strangely stderr and stdout are conflated in the pipe structure - there is no way to get stderr!
You just get the numeric exit status.
IO redirection appears to be missing.
Difficult to use in production.
I'm having a very hard time getting past the fact that the "wrong" code is wrapped very badly on both a mobile device and laptop. That inherently makes it harder to read ...
They had to really struggle to make that code not clearly readable, and the complete lack of any error handling and dependence on some random framework somehow more desirable to the reader in the second example.
It's interesting that go text templates have shell-like pipelines but that never bled over into the rest of the language: https://pkg.go.dev/text/template#hdr-Pipelines
It seems like a shame that this kind of power and expressiveness is reserved only for generating text.
It seems like a shame that this kind of power and expressiveness is reserved only for generating text.
Something which the shell script has which the go implementation lacks (unless I missed it) is concurrency. When you have a shell pipeline like 'cmd1 | cmd2 | cmd3' those cmds are actually started at the same time and run in parallel. This is really great for performance - you get multiple cores working on the problem with very little effort.
This Go library looks like a good one, with the caveats that it hasn't reached 1.0 yet, and does have a couple of dependencies that I didn't trace further.
At one time jQuery was popular and this seems like a similar thing, but for files?
It's small enough that if you're worried about "other people's code" you could fork it and maintain it yourself.
At one time jQuery was popular and this seems like a similar thing, but for files?
It's small enough that if you're worried about "other people's code" you could fork it and maintain it yourself.
While using the package seems nice I think what's missed here is that Go requires a module declaration and dependencies to be enumerated. That would mix your "scripts" with any production code written in Go. Bash and Python keep my scripts away from my app dependencies.
This article focused entirely on replacing the Unix shell. Does this library also work for the Windows shell or PowerShell (iirc PowerShell is POSIX)? I could see the value of having a single script in a Go repository which works for all build systems.
The programs should be OS agnostic as I assume that the majority of the functionality is based on the go standard library. Haven't tested this, but I've written heaps of go code for Windows and Linux and have rarely had to add OS specific info. There are a few small differences, but it's easy to abstract them away, which I assume has been done in this case.
It took me a while to find the link to the library "script" and it's repo - https://github.com/bitfield/script
this is personally challenging to me, and gets me out of my comfort zone -- in the right ways. Nobody needs to care about my own learning trajectory for server-side software, but, I will say that I have been hoarding 'good' bash scripts for a bunch of years, after having a bad start with Perl for server-side things. Bash needs no comments here, but I use it everyday. I avoided GO-lang partly due to an insider vibe from Xooglers, and partly internal prioritization. This article strikes that nerve today. thanks for posting .. no promises but bookmarked.
There was a recent discussion in /r/golang about chaining methods like this. The OP had a hard time writing tests for their implementation since each part of the chain returned interfaces that were relied on by following links within. The top answer in the thread was that Golang wasn't really meant to make "Promise-" type workflows possible due to it leaning heavily on code being intentional and explicit.
I agree with them.
One of the problems I have with things like LINQ or Promises is that they are immensely powerful but can be really difficult to debug when things go wrong because of implicit errors within the pipeline bubble up the stack without providing location context. While the standard boilerplate around errors in Go is annoying, I actually prefer it for two reasons:
1. It provides exactly where things went wrong while looking at an exception trace, and 2. From a readability perspective, it is much easier to understand what the author of the code meant to do and, more importantly, what the follow-on error s mean. Sometimes, errors aren't actually errors.
Either way, OCaml/Haskell/F#'s approach with match expressions and the `Result` type is the best way to deal with this, IMO. You get the best of both worlds: explicit declaration with a very expressive (triangular-like) "shape" to the code.
So something like this:
As far as the library itself, I personally wouldn't use something like this, though I see the appeal. I turn to statically-typed languages like Golang when a Bash script becomes too kludgey to stand on its own (usually when I need to begin relying on mild data structures) and when I care about type safety and portability more than what Ruby or Python can give me. When I'm writing a Go program, I want something that's testable that I know can work anywhere. With Ruby or Python, unless I'm distributing the script as a Docker image, I have to worry about versions, environments, etc., none of which are pleasant.
However, writing the error boilerplate is annoying, and I can see developers who want to write something quick but spend 99% of their time in Go using this to get something done with a tool they know well. I've seen similar things in the Java world; hell, that's the reason why Groovy and Kotlin exist!
TL;DR: The "wrong thing" in the Bitfield article isn't necessarily wrong; their library is useful, but niche; all hail pattern-matching and the Result type.
I agree with them.
One of the problems I have with things like LINQ or Promises is that they are immensely powerful but can be really difficult to debug when things go wrong because of implicit errors within the pipeline bubble up the stack without providing location context. While the standard boilerplate around errors in Go is annoying, I actually prefer it for two reasons:
1. It provides exactly where things went wrong while looking at an exception trace, and 2. From a readability perspective, it is much easier to understand what the author of the code meant to do and, more importantly, what the follow-on error s mean. Sometimes, errors aren't actually errors.
Either way, OCaml/Haskell/F#'s approach with match expressions and the `Result` type is the best way to deal with this, IMO. You get the best of both worlds: explicit declaration with a very expressive (triangular-like) "shape" to the code.
So something like this:
var data string = script.File("test.txt").Match("Error").CountLines()
var foundLines int32 = 0
var numLines int32 = match file.Open("test.txt") with
| Error e -> return e
| Ok f -> match f.Readlines() with
| Error e -> return e
| Ok lines -> match lines with
| /Error.*/ -> foundLines++
| _ -> // do nothign
return foundLines
foundLines := 0
lines, err := ioutil.ReadFile("test.txt")
if err != nil {
return 0, err
}
for _, l := range lines {
re, err := regexp.MustCompile(`Error.*`)
if err != nil {
return 0, err
}
matches := re.FindStringSubmatch(`Error.*`)
if len(matches) > 0 {
foundLines++
}
}
return foundLines, nil
As far as the library itself, I personally wouldn't use something like this, though I see the appeal. I turn to statically-typed languages like Golang when a Bash script becomes too kludgey to stand on its own (usually when I need to begin relying on mild data structures) and when I care about type safety and portability more than what Ruby or Python can give me. When I'm writing a Go program, I want something that's testable that I know can work anywhere. With Ruby or Python, unless I'm distributing the script as a Docker image, I have to worry about versions, environments, etc., none of which are pleasant.
However, writing the error boilerplate is annoying, and I can see developers who want to write something quick but spend 99% of their time in Go using this to get something done with a tool they know well. I've seen similar things in the Java world; hell, that's the reason why Groovy and Kotlin exist!
TL;DR: The "wrong thing" in the Bitfield article isn't necessarily wrong; their library is useful, but niche; all hail pattern-matching and the Result type.
Pattern matching and Result type are truly among the most revolutionary computer science constructions and I am glad to see their adoption starting to catch on. So much nicer to work with than exception handling or endless propagating of error constants a la Go. Python just got pattern matching (although it lacks pattern-match assignment a la Rust), now it just needs an actual Result type in the stdlib instead of Optional[T] which is really just T|None rather than an actual container. Golang can start delving into Return-type-looking generics but I suspect pattern matching is a long ways out.
Curious, what is your argument in favor of a container over a type union to represent Result? As long as the type checking system forces users to refine the union to use the value and is aware of blocks of code within type guards (I've heard this referred to as "flow-aware typing" but I'm not sure if that's a standard term) then the advantage of an explicit Result type isn't very obvious to me. What am I missing?
In terms of ergonomics, I'm more a fan of the fluent style [1] over flow-aware typing with guard blocks. I've never heard of till not but I presume you mean that's like where you have like
Also specifically in Python, the type system is kind of weak (especially those bundled with Pycharm), and a lot of functional operations which ought to be more strongly typed end up with their types erased for whatever reason. In particular functools.partial seems to discard type information more than I'd like. The Result python package [3] doesn't run into this problem near to the same degree.
1] https://martinfowler.com/bliki/FluentInterface.html
2] I find this term frustrating not only because it's infamously ineffable, but also because there appear to be disagreement whether Rust's containers are actual monads or not, leading me to dub what I call the "Engineer's Monad" which is a generic container you can map and flatmap over, even if isn't strictly an Applicative.
3] https://pypi.org/project/result/
foo: Optional[Foo]
if foo is None:
return some_error_handler()
bar = foo.qux() # the above guard "upgrades" O[Foo] to Foo
Also specifically in Python, the type system is kind of weak (especially those bundled with Pycharm), and a lot of functional operations which ought to be more strongly typed end up with their types erased for whatever reason. In particular functools.partial seems to discard type information more than I'd like. The Result python package [3] doesn't run into this problem near to the same degree.
1] https://martinfowler.com/bliki/FluentInterface.html
2] I find this term frustrating not only because it's infamously ineffable, but also because there appear to be disagreement whether Rust's containers are actual monads or not, leading me to dub what I call the "Engineer's Monad" which is a generic container you can map and flatmap over, even if isn't strictly an Applicative.
3] https://pypi.org/project/result/
None != Error would be my beef with it. Sometimes I'm expecting a NoneType in the happy path.
Tagged Unions (“containers”) vs. simple Unions is basically about composability.
You can't emulate Result[Result[T]] with a simple Union.
You can't emulate Result[Result[T]] with a simple Union.
(Mostly trolling) You must be missing extension method that other modern languages have, let alone the pipe operator of Elixir, R, etc. ;-)
The places I've "shell scripted with Go" involved also using some other code I already had in Go. The referenced page doesn't show it, but being able to do a bit of quick shell scripting-type manipulation on a file, then feed it to the Go JSON decoder to get Go objects, call a few methods or something, then maybe manipulate it a bit more on the way out, is sort of thing that is the real killer feature, IMHO.
The space of "shell-like libraries", not least of which is shell itself, is so rich it's hard to beat out everything else on its own merits. But having something as easy as shell that integrates back into your Go ecosystem is very convenient.
And I'm sure similar things are true for the other libraries in other languages.
So I would personally not present this as "here's something awesome Go can uniquely do", but, "if you already have Go, be aware of this tool/technique".
The space of "shell-like libraries", not least of which is shell itself, is so rich it's hard to beat out everything else on its own merits. But having something as easy as shell that integrates back into your Go ecosystem is very convenient.
And I'm sure similar things are true for the other libraries in other languages.
So I would personally not present this as "here's something awesome Go can uniquely do", but, "if you already have Go, be aware of this tool/technique".
Yeah, I agree with your point and can see the usefulness. Sorry for trolling, just couldn't resist :-/
Seems like this could pair well with something like yaegi.