Higher-order functions in Go(eli.thegreenplace.net)
eli.thegreenplace.net
Higher-order functions in Go
https://eli.thegreenplace.net/2023/higher-order-functions-in-go/
43 comments
I can’t think of any modern languages without closures.
Zig...
You can emulate closures with anonymous structs.
struct {
fn foo() { /**/ }
}.foo
Ouch
When Go first started to get traction at Google, I saw it mostly replacing Python (and some Java), but not a whole lot of C/C++. As the adage goes, you can write Java/Python in any language, and a lot of people did, and from that time we have a lot of Go code with two specific smells:
1) A lot of channels, because Python coders wanted iterators. (This got so bad I've seen memos circulated about channels not being allowed to cross public API boundaries.) 2) A lot of interfaces, because of Java
I am really happy about the recent trend towards a more functional style in the language, it's so much closer to what I think the authors intended the language to look like.
1) A lot of channels, because Python coders wanted iterators. (This got so bad I've seen memos circulated about channels not being allowed to cross public API boundaries.) 2) A lot of interfaces, because of Java
I am really happy about the recent trend towards a more functional style in the language, it's so much closer to what I think the authors intended the language to look like.
> I am really happy about the recent trend towards a more functional style
There is no such trend, at least not from the Go maintainers' side. Trying to write Haskell in Go is just as misguided and futile as trying to write Java in Go.
> in the language, it's so much closer to what I think the authors intended the language to look like.
I don't think so. The wanted it to be imperative like C or Pascal/Modula/Oberon, otherwise they would have started the design with ML or Lisp instead of C.
There is no such trend, at least not from the Go maintainers' side. Trying to write Haskell in Go is just as misguided and futile as trying to write Java in Go.
> in the language, it's so much closer to what I think the authors intended the language to look like.
I don't think so. The wanted it to be imperative like C or Pascal/Modula/Oberon, otherwise they would have started the design with ML or Lisp instead of C.
I think you're responding to something I didn't (intend to) say.
It is a lot closer in the sense that it is less indirected and simpler. The Go team have circulated style guidance to groups I was part of about how to properly structure public APIs, and for example functional callbacks are nowadays preferred over channels.
I completely agree that Go is not designed to be Haskell, and I personally cringe every time I see a library of map/reduce/filter functions. I still prefer this over the older sins of stuff like ITreeIterator.
When I say "functional style" I don't mean lambda calculus. I mean more recent libraries don't tend to return interface-typed values as much or use channels as iterators.
It is a lot closer in the sense that it is less indirected and simpler. The Go team have circulated style guidance to groups I was part of about how to properly structure public APIs, and for example functional callbacks are nowadays preferred over channels.
I completely agree that Go is not designed to be Haskell, and I personally cringe every time I see a library of map/reduce/filter functions. I still prefer this over the older sins of stuff like ITreeIterator.
When I say "functional style" I don't mean lambda calculus. I mean more recent libraries don't tend to return interface-typed values as much or use channels as iterators.
There's a widening generational gap in what "functional programming", in a practical sense, means. "Back in the day" a language got called 'functional' if it had anonymous closures, upwards funargs, tail calls, you could implement a decent `reduce`, etc. Today it means purity and immutability, monadic comprehensions, and type systems to support all that.
Go's the first kind of functional language. But so is almost everything else, even Java, now.
Go's the first kind of functional language. But so is almost everything else, even Java, now.
"Back in the day" we didn't call a language functional. The 1967 Lisp 1.5 Primer on my desk for example doesn't use the term at all. The widespread use of language categorization came with object oriented languages which used that as a marketing term.
The functional Lisp (Lisp without assignment) was called "Pure Lisp" back then. Sometimes also there was the term "applicative language".
IIRC lisp was already called "functional" in the 1980s. It was in contrast to purely procedural languages. I cannot comment on the 1970s.
"Lisp, an acronym for list processing"
OK boomer.
> “Back in the day” a language got called ‘functional’ if it had anonymous closures, upwards funargs, tail calls, you could implement a decent `reduce`, etc.
Back in the day, we talked about programming paradigms and whether or not a language supported them; we didn’t usually label languages by paradigm unless there was a fairly tight association of the paradigm with the language.
Back in the day, we talked about programming paradigms and whether or not a language supported them; we didn’t usually label languages by paradigm unless there was a fairly tight association of the paradigm with the language.
Higher-order functions improve readability, maintainability and correctness. The idiomatic supposed alternative of using e.g. for-loops instead of map/fold/... is a disease that needs to be shunned, like goto was for cases where better constructs exist.
If performance suffers, it is up to the language to implement proper inlining and fusion.
If performance suffers, it is up to the language to implement proper inlining and fusion.
I mean, if you strongly hold this opinion then Go is really not the language for you. And if you don't use Go then it shouldn't really bother you either. Let Go devs be Go devs.
I assume you're either trolling or have neither commercial nor compiler-writing (edit; nor significant functional programming) experience.
I do have commercial experience and I did a class on compiler-writing at the university. Therefore I am not trolling.
Well you answered moderately and you didn't hammer the downvote button so perhaps we can make something constructive of this.
> Higher-order functions improve readability, maintainability and correctness
Perhaps correctness, but readability and its close relation, maintainability, well that's another question. I think it depends partly on the person (I'm not very good at reading code) and very definitely on the quality of the written code. I have seen some very, very horrible functional code. Simply declaring that it is more readable doesn't fly.
Back to correctness: well, functional code tends to be shorter than non-functional code, and it's a known fact that the number of bugs seems to be a constant per the lines of code (or at least that's the accepted wisdom, let's go with it) so is the lower number of bugs in functional code directly proportional to its reduced size, or is there something extra work? If it's directly proportional to the reduced size, then you often can do a heck of a lot with procedural code to reduce its size as well, usually by simply defining extra procedures/functions instead of copy/pasting code (which is a disease that functional code can very definitely suffer from also!)
> If performance suffers, it is up to the language to implement proper inlining and fusion
And in the meantime you lose sales, your competitors do better and start eating your market share – if you were running a company how would that make you feel? You have to be realistic.
Anyway, I do appreciate that you weren't trolling and weren't dismissive of my post, so I've upvoted your response.
> Higher-order functions improve readability, maintainability and correctness
Perhaps correctness, but readability and its close relation, maintainability, well that's another question. I think it depends partly on the person (I'm not very good at reading code) and very definitely on the quality of the written code. I have seen some very, very horrible functional code. Simply declaring that it is more readable doesn't fly.
Back to correctness: well, functional code tends to be shorter than non-functional code, and it's a known fact that the number of bugs seems to be a constant per the lines of code (or at least that's the accepted wisdom, let's go with it) so is the lower number of bugs in functional code directly proportional to its reduced size, or is there something extra work? If it's directly proportional to the reduced size, then you often can do a heck of a lot with procedural code to reduce its size as well, usually by simply defining extra procedures/functions instead of copy/pasting code (which is a disease that functional code can very definitely suffer from also!)
> If performance suffers, it is up to the language to implement proper inlining and fusion
And in the meantime you lose sales, your competitors do better and start eating your market share – if you were running a company how would that make you feel? You have to be realistic.
Anyway, I do appreciate that you weren't trolling and weren't dismissive of my post, so I've upvoted your response.
Map explicitly says what it does. A loop forces you to carefully (re)read it to see whether it ever skips an element, appends more than one result, or doesn’t propagate an error, because findFirst, findLast, findAll, filter, map, flatMap, and reduce all look very similar as loops.
If you need a loop to work around a poor optimizer, at least use a preprocessor to generate it. Devs are expensive.
If you need a loop to work around a poor optimizer, at least use a preprocessor to generate it. Devs are expensive.
[deleted]
Higher order functions are super useful for middleware in Go. Functions are also nillable which makes them useful as options. Combined with variatic functions you can build quite a nice API.
Does that mean you use variadic functions to adapt to different structs and their properties/fields and they all return the same "base struct"?
Do you have example code? I am now really curious, but don't know what to google.
Do you have example code? I am now really curious, but don't know what to google.
I think you are looking for “functional options”[0]
[0] https://dave.cheney.net/2014/10/17/functional-options-for-fr...
[0] https://dave.cheney.net/2014/10/17/functional-options-for-fr...
Oh dear I love that. Thanks for sharing.
I use samber/lo in every project https://github.com/samber/lo
(Go is still plenty fast with it)
(Go is still plenty fast with it)
That's really misguided. Go is not a functional programming language, it's an imperative programming language. You're going against its design. You're like these people who write Java and "class hierarchies" in Go, just from a different angle. If you like FP you should use Haskell instead of Go. I say this as a fan of Go.
I don't think things are that simple. I like functional programming, but I also like good tooling, having a rich ecosystem, job opportunities, backward compatibility, simplicity, people to talk to. There are lots of things that can weigh on your choice of language. Semi-functional Go might be the sweet spot for some people, and absolute hell for others.
A map() is much easier to read and understand than create slice, for loop, return slice. Coding is about reading code.
An unfused map-filter chain is a performance disaster waiting to happen.
In Go or in general? I've written code for companies with 50k+ concurrent writing users, I've bootstrapped and scaled and sold a startup on Scala (which has bad performance b/c of immutable lists but with usual list <100 elements no problem) and I haven't seen this in other languages, but I'm not experienced enough in Go and haven't deployed something major in Go yet.
Other popular languages will fuse either via JIT or via some kind of linear types.
You should preen less in your comments.
You should preen less in your comments.
I think it's always easier to understand a point of view if you know where someone is coming from and the context.
"An unfused map-filter chain is a performance disaster waiting to happen."
Could you share your experience with that perfromance disaster here?
"An unfused map-filter chain is a performance disaster waiting to happen."
Could you share your experience with that perfromance disaster here?
> Go is still plenty fast with it
It is not. Your program might still hit acceptable speed targets, but you're taking enormous absolute perf hits for extremely questionable gains.
https://news.ycombinator.com/item?id=30696210
It is not. Your program might still hit acceptable speed targets, but you're taking enormous absolute perf hits for extremely questionable gains.
https://news.ycombinator.com/item?id=30696210
Thanks for the tip, I've written code for 40+y and have been paid in 10+ languages, I'm writing the code that I think is easiest to maintain for the years coming in the language I consider the best for the job.
As long as it's only you who has to maintain the code and not another Go programmer ...
The Go developers I know understand map() and filter() perfectly well.
What a condescending comment in response of someone trying to help
s/trying to help/trying to be right/g
The linked comment was about an implementation of partition with the specific problem of allocating too many slices in the opinion of the author. It was not about lo in general, did not help or shared any performance numbers E.g. for a size of a usual lists of <50 elements with filter() or the use case of three chunks, that people use when writing web applications (which I do, I don't write a database in Go - something the author could have found out my asking instead of assuming).
The linked comment was about an implementation of partition with the specific problem of allocating too many slices in the opinion of the author. It was not about lo in general, did not help or shared any performance numbers E.g. for a size of a usual lists of <50 elements with filter() or the use case of three chunks, that people use when writing web applications (which I do, I don't write a database in Go - something the author could have found out my asking instead of assuming).
There's no ergonomic loss to using the faster approach. It's just cargo culting patterns from other languages even though they're less accessible and less performant in Go.
The problem with your comments isn't the context for what you're doing, it's the "40y, scaled and sold" bullshit. That's bragging, not an argument nor context. Or as a senior colleague used to remind me, "shipping is easy if you're willing to ship shit." I'm not here to discuss shipping.
The problem with your comments isn't the context for what you're doing, it's the "40y, scaled and sold" bullshit. That's bragging, not an argument nor context. Or as a senior colleague used to remind me, "shipping is easy if you're willing to ship shit." I'm not here to discuss shipping.
"shipping is easy if you're willing to ship shit."
What a nasty comment.
What a nasty comment.
For more on the modern functional style in Go, including Map, Reduce, Filter, Sort, and so on: https://bitfieldconsulting.com/golang/functional
The Rust support for closures is more pleasant than that of C but the innate `Fn`/`FnMut`/`FnOnce` dichotomy makes things more complicated.
[1] https://en.wikipedia.org/wiki/Funarg_problem#Upwards_funarg_...