Go beyond Goroutines: introducing the Reactive paradigm(samuelberthe.substack.com)
samuelberthe.substack.com
Go beyond Goroutines: introducing the Reactive paradigm
https://samuelberthe.substack.com/p/go-beyond-goroutines-introducing
42 comments
I like how he kept "tabs" (and display it as 9 spaces) to make it as ugly as possible for the bad example, then proceed to use 4 spaces for the other examples.
I suspect this is substacks default styling. It actually doesn’t set the tab-size which defaults to 8. However there seems to be some funky font-rendering going on so it adds an extra half (or maybe three-quarters) of a space per tab-character.
You can see this your self if you edit the markup in your browser’s inspector and add `contenteditable` attribute to the surrounding <pre>, then navigate down a line... it will jump forward just by slightly less then a column per indent level.
You can see this your self if you edit the markup in your browser’s inspector and add `contenteditable` attribute to the surrounding <pre>, then navigate down a line... it will jump forward just by slightly less then a column per indent level.
Tabs are the recommended indentation method for Go code. It is interesting that they switched after the first example, though.
I'm well aware of it. But I don't think I have ever seen an editor/website which would display it as 9 spaces. Github for example default to 4 spaces width.
Yes, this is very weird. But Go Playground[1] has always been using 8 spaces per tab for some reason. I always found that very jarring, particularly where almost every other editor or documentation has settled on 4 spaces per tab.
[1] https://go.dev/play/
[1] https://go.dev/play/
Seems like it got fixed just now. Thanks for doing it. It looks much better now :)
I’ve come to believe that a direct implementation of something in the language is, where possible, more readable and maintainable in the long term.
Libraries that enable terse seemingly magical things tend to just hide a lot and make code harder to read. You end up having to become an expert in what amounts to a DSL on top of the language.
Libraries that enable terse seemingly magical things tend to just hide a lot and make code harder to read. You end up having to become an expert in what amounts to a DSL on top of the language.
i like reactive programming in other languages but it is at odds with Go's philosophy of simplicity and avoiding big abstractions
I am sorry I might be a little naive here. The hardest thing about RxJava or RxJS was actually understanding the reactor pattern and then the nuances of threading (in Java) and how it made simple linear looking code callback hell. Go from ground up was built on promise of having the simple no-callback hell code that will be easy to read on eyes. Why do we keep going back to these pattern that over long term have proven to be hard to debug, thus hard to maintain well. Go is not even good at syntactic sugar like couple of other languages that might make it easy, why are people so excited about this? Should the reactivity and yielding of a go routine be taken care of under the hood?
Edit: I've maintained a full codebase with R2DBC and I can assure you many developers came scratching their heads to me sometimes on tell me why are we doing this again when we can only have finite connections to DB and those connections can be 1-1 mapped to worker threads.
Edit: I've maintained a full codebase with R2DBC and I can assure you many developers came scratching their heads to me sometimes on tell me why are we doing this again when we can only have finite connections to DB and those connections can be 1-1 mapped to worker threads.
It's really tough to read through a text where you can't go more than a few sentences without having something be described as *an adjective, a second adjective, and a third adjective*. Just let the code example speak for itself, man.
Speaking of the code examples, I am not convinced at all. The supposedly bad example is idiomatic and understandable to me, and I have a much better idea of what's going on than in the supposedly good examples. It contains the kind of constructions I expect to see when working on a Go codebase, and it will be easier to modify correctly for another Go developer coming in later. Please, work with the language instead of forcing it to wear a programming paradigm it doesn't like.
Speaking of the code examples, I am not convinced at all. The supposedly bad example is idiomatic and understandable to me, and I have a much better idea of what's going on than in the supposedly good examples. It contains the kind of constructions I expect to see when working on a Go codebase, and it will be easier to modify correctly for another Go developer coming in later. Please, work with the language instead of forcing it to wear a programming paradigm it doesn't like.
It has to be AI generated or at least edited right? The reliance on bulleted lists. The endless adjectives and declarations. ...but also the subtle... well not exactly errors, but facts I think are open to dispute?
Such as:
> Together, these tools make Go perfect for microservices, real-time systems, and high-throughput backends.
Real-time systems?! I have never heard of anyone using Go for realtime systems because of its GC and preemptive scheduler. Seems like the sort of thing an LLM would slip in because it sounds good and nails that 3 item cadence.
> Built on top of Go channels → broken backpressure.
But then the example is about ordering. Maybe I'm being pedantic or missing the specific nomenclature the ReactiveX community uses, but backpressure and ordering are different concerns to me.
Then the Key Takeaways at the end just seems like an LLMism to me. It's a short article! Do we really need another 3 item list to summarize it?
I'm not anti-LLM, but the sameness of the content it generates grates on me.
Such as:
> Together, these tools make Go perfect for microservices, real-time systems, and high-throughput backends.
Real-time systems?! I have never heard of anyone using Go for realtime systems because of its GC and preemptive scheduler. Seems like the sort of thing an LLM would slip in because it sounds good and nails that 3 item cadence.
> Built on top of Go channels → broken backpressure.
But then the example is about ordering. Maybe I'm being pedantic or missing the specific nomenclature the ReactiveX community uses, but backpressure and ordering are different concerns to me.
Then the Key Takeaways at the end just seems like an LLMism to me. It's a short article! Do we really need another 3 item list to summarize it?
I'm not anti-LLM, but the sameness of the content it generates grates on me.
I have never heard of anyone using Go for realtime systems because of its GC and preemptive scheduler.
I don't know. I've seen "realtime" used quite often in a sort of colloquial sense, where it means something fairly different from "hard realtime system" as an embedded systems person might use the term. I think there's a pretty large base of people who use "realtime" to mean something that others might call "near real-time" or just "continually updating" or something along those lines, where's there's no implication of needing nanosecond level predictable scheduling and what-not.
That's not to say that the article isn't AI generated, or whatever. Just that I wouldn't necessarily see the use of the "realtime" nomenclature as strong support for that possibility.
I don't know. I've seen "realtime" used quite often in a sort of colloquial sense, where it means something fairly different from "hard realtime system" as an embedded systems person might use the term. I think there's a pretty large base of people who use "realtime" to mean something that others might call "near real-time" or just "continually updating" or something along those lines, where's there's no implication of needing nanosecond level predictable scheduling and what-not.
That's not to say that the article isn't AI generated, or whatever. Just that I wouldn't necessarily see the use of the "realtime" nomenclature as strong support for that possibility.
Fair enough. I suppose there aren’t many hard real time posts these days in general.
My understanding was that Go intentionally avoided patterns like this to improve readability.
Sometimes it feels like that Go mistakes readability for comprehendability. Sure, I can read every single line of Go I've ever read. But can I comprehend the bigger picture? Can I understand why? Isn't the actual meat buried under piles of non-abstraction?
This is precisely the premise for their library: I don't have the mental context to fit all the boilerplate in, nor do I have the brainpower to sift through it.
Sure, assembly is readable: every line is one machine instruction. But that's way too many details. On the other hand, C++ templates are not enough details.
This is precisely the premise for their library: I don't have the mental context to fit all the boilerplate in, nor do I have the brainpower to sift through it.
Sure, assembly is readable: every line is one machine instruction. But that's way too many details. On the other hand, C++ templates are not enough details.
I feel like Go's concept of readability is very Blub-oriented[1]. Can a Blub programmer read this line? Then it's readable. Sometimes Go fans would say that this code:
I more readable than this code:
Of course, once you do understand these very complicated concepts, the code above is far more readable: it clearly states WHAT the program does (filtering all values below the threshold and converting them to string) rather than HOW it does that (which nobody cares about). "readability" here is only counted from the narrow perspective of an imperative programmer who is not familiar with functional declarative data processing.
I feel the same about the "ro" examples in the OP. I don't particularly like that ro takes (mostly because Go forces its hand, I assume), like having to put everything in an explicit pipe, but I find the example far more readable than the pure Go example which combines loops, channels and WaitGroups. That's far worse than the loop example I gave in this reply, to be honest, and I really don't know why people say this example is readable. I guess you can optimize it a little, but I always found both channel and WaitGroups waitable, unreadable and error prone. They are only "readable" in narrow perverted sense that has somehow become prevalent in the Go community, where "readability" is redefined to mean: no closures, no immutable values, no generics, no type safety and certainly nothing that smells like FP.
[1] https://wiki.c2.com/?BlubParadox
var result := []string{}
for i = 0; i < items.Length(); i++ {
item := items.Get(i)
if item < threshold {
result = append(result, converToString(item))
}
}I more readable than this code:
return items
.filter { it < threshold }
.map { convertToString(it) }
Of course, once you do understand these very complicated concepts, the code above is far more readable: it clearly states WHAT the program does (filtering all values below the threshold and converting them to string) rather than HOW it does that (which nobody cares about). "readability" here is only counted from the narrow perspective of an imperative programmer who is not familiar with functional declarative data processing.
I feel the same about the "ro" examples in the OP. I don't particularly like that ro takes (mostly because Go forces its hand, I assume), like having to put everything in an explicit pipe, but I find the example far more readable than the pure Go example which combines loops, channels and WaitGroups. That's far worse than the loop example I gave in this reply, to be honest, and I really don't know why people say this example is readable. I guess you can optimize it a little, but I always found both channel and WaitGroups waitable, unreadable and error prone. They are only "readable" in narrow perverted sense that has somehow become prevalent in the Go community, where "readability" is redefined to mean: no closures, no immutable values, no generics, no type safety and certainly nothing that smells like FP.
[1] https://wiki.c2.com/?BlubParadox
> But can I comprehend the bigger picture? Can I understand why? Isn't the actual meat buried under piles of non-abstraction?
The way you approach this in Go (and I would argue in any other language) is by building small abstractions when and if it makes sense to do so. Not by introducing abstractions early, or in order to make a piece of code slightly easier to follow. A simple comment might help to explain tricky logic, but otherwise you shouldn't need explanations about core language features.
Abstractions are not free. They can be poorly written, leaky, abstract too much or too little, inflexible, increase overall complexity and thus cognitive load, impact performance, introduce bugs, etc.
So relying on them only when absolutely necessary is often the sensible choice.
Also, if possible, building your own bespoke abstraction is often preferable to using an external package. You can tailor it to your exact use case, without adding another dependency, which carries its own risks and problems.
This specific package seems designed to be syntax sugar over language features, which is not a good idea for future maintainers of the code that uses it. They would need to understand how this 3rd-party library works, even if the author claims it to be more readable, ergonomic, or what have you.
The way you approach this in Go (and I would argue in any other language) is by building small abstractions when and if it makes sense to do so. Not by introducing abstractions early, or in order to make a piece of code slightly easier to follow. A simple comment might help to explain tricky logic, but otherwise you shouldn't need explanations about core language features.
Abstractions are not free. They can be poorly written, leaky, abstract too much or too little, inflexible, increase overall complexity and thus cognitive load, impact performance, introduce bugs, etc.
So relying on them only when absolutely necessary is often the sensible choice.
Also, if possible, building your own bespoke abstraction is often preferable to using an external package. You can tailor it to your exact use case, without adding another dependency, which carries its own risks and problems.
This specific package seems designed to be syntax sugar over language features, which is not a good idea for future maintainers of the code that uses it. They would need to understand how this 3rd-party library works, even if the author claims it to be more readable, ergonomic, or what have you.
IMO, this is much more readable.
So many Go developers ignore some tools because they consider them "not idiomatic".
But why not use abstractions when available ??? Did we forget to be productive ?
So many Go developers ignore some tools because they consider them "not idiomatic".
But why not use abstractions when available ??? Did we forget to be productive ?
What makes sense depends on the agreement of the programming team.
And the smallest possible team is the programmer and their future self.
Even then the hard thing is to predict what will be better for our future selves. Maybe we will be rusty in our Go skills or maybe we will have embraced idiomatic Go, and the approach that makes sense now will require our future self to puzzle through the code.
Of course maybe we will have kept programming the same way because it still feels like the better way and our future self will be efficient. But again that's only for the smallest possible team. If the team expands, then all bets are off.
And the smallest possible team is the programmer and their future self.
Even then the hard thing is to predict what will be better for our future selves. Maybe we will be rusty in our Go skills or maybe we will have embraced idiomatic Go, and the approach that makes sense now will require our future self to puzzle through the code.
Of course maybe we will have kept programming the same way because it still feels like the better way and our future self will be efficient. But again that's only for the smallest possible team. If the team expands, then all bets are off.
It wasn’t to improve readability—it was for performance. There is no question what is happening in a triple-nested for loop. Once you start hiding it behind deeply-nested function calls, you get hidden performance penalties. If you have a functional language that is actually able to do some of this lazily, great, but that’s not Go.
[deleted]
I'm curious if someone could chime in on the state of adoption of these these Rx libraries in other language's ecosystems.
My poor memory seems to recall them gaining traction ~10 years ago, but they've fallen hard off my radar.
My fear with adopting a library like this for Go is actually that it might end up being very unfriendly to the profiler once bottlenecks start occurring.
My poor memory seems to recall them gaining traction ~10 years ago, but they've fallen hard off my radar.
My fear with adopting a library like this for Go is actually that it might end up being very unfriendly to the profiler once bottlenecks start occurring.
I use rxjs day in day out for my oss work (eg: https://github.com/mickael-kerjean/filestash/blob/master/pub...)
It's quite common to see job description where I live (Sydney) with rxjava but reactive libs are a bit of a niche thing mostly because it takes a bit of time to be proficient at it + not many people talk about it but it's not sexy
it's used extensively in java and it would be my first choice when starting a java project. I don't think I'd use it in Go though.
reactive is on the decline in the java world post-loom (virtual threads) and should be nobody's first choice. writing plain old imperative code is vastly simpler to write / debug / reason about.
Brian Goetz even went as far as saying loom is going to kill reactive entirely: https://www.youtube.com/watch?v=9si7gK94gLo&t=1165s
Brian Goetz even went as far as saying loom is going to kill reactive entirely: https://www.youtube.com/watch?v=9si7gK94gLo&t=1165s
I think there is an issue where reactive frameworks are massively overused in languages that have (or had) weak concurrency patterns. This was true for a while in JavaScript (before async/await became universally supported), but it's especially endemic in the Java world, especially the corners of it which refused to use Kotlin.
So yes, in this particular case, most of the usages of RxJava, Reactive Streams and particularly Spring Reactor is just there because programmers wanted pretty simple composition of asynchronous operations (particularly API calls) and all the other alternatives were worse: threads were too expensive (as Brian Goetz mentions), but Java did have asynchronous I/O since Java 1.4 and proactor-style scheduling (CompletableFuture) since Java 8. You could use CompletableFutures to do everything with asynchronous I/O, but this was just too messy. So a lot of programmers started using Reactive framework, since this was the most ergonomic solution they had (unless they wanted to introduce Kotlin).
That's why you'd mostly see Mono<T> types if you look at a typical reactive Spring WebFlux project. These Mono/Single monads are essentially CompletableFutures with better ergonomics. I don't mean to say that hot and cold multi-value observables were used at all, but in many cases the operator chaining was pretty simple: gathering multiple inputs, mapping, reducing, potentially splitting. Most of this logic is easier to do with normal control flow when you've got proper coroutines.
But that's not all what reactive frameworks can give you. The cases when I'd choose a reactive solution over plain coroutines are few and pretty niche: to be honest, I've only reached to a reactive 3 or 4 times in my career. But they do exist:
1. Some reactive operators are trivially mapped to loops or classic collection operators (map/reduce/filter/flatMap/groupBy/distinct...). But everything that's time-bound, is more complicated to implement in a simple loop, and the result is far less readable. Think about sample or debounce for instance.
2. Complex operation chains do exist and implementing them as a reactive pattern makes the logic far easier to test and reason about. I've had a case where I need to implement a multi-step resource fetching logic, where an index is fetched first, and then resources are fetched based on the index and periodically refreshed with some added jitter to avoid a thundering herd effect, as well as retries with exponential backoff and predictable update interval ranges which is NOT affected by the retries (in other words: no, you can't just put a delay in a loop). My first implementation tried to model that with pure coroutines and it was a disaster. My second implementation was RxJava, which was quite decent, and then Kotlin Flow came out, which was a breeze.
3. I'm not sure if we should call this "Reactive" (since it's not the classic observable), but hot and cached single values (like StateFlow in Kotlin) are extremely useful for many UI paradigms. I found myself reaching to StateFlow extensively when I was doing Android programming (which I didn't do a lot of!).
In short, I strongly disagree with Brian Goetz that Functional Reactive Programming is transitional. I think he's seeing this issue from a very Java-centric perspective, where probably over 90% of the usage we've seen for it was transitional, but that's not all that FRP was all about. FRP will probably lose its status a serious contender for a general tool for expressing asynchronous I/O logic, and that's fine. It was never designed to be that. But let's keep in mind that there are other languages than Java in the world. Most programming languages support some concept of coroutines that are not bound to OS kernel threads nowadays, and FRP is still very much alive.
So yes, in this particular case, most of the usages of RxJava, Reactive Streams and particularly Spring Reactor is just there because programmers wanted pretty simple composition of asynchronous operations (particularly API calls) and all the other alternatives were worse: threads were too expensive (as Brian Goetz mentions), but Java did have asynchronous I/O since Java 1.4 and proactor-style scheduling (CompletableFuture) since Java 8. You could use CompletableFutures to do everything with asynchronous I/O, but this was just too messy. So a lot of programmers started using Reactive framework, since this was the most ergonomic solution they had (unless they wanted to introduce Kotlin).
That's why you'd mostly see Mono<T> types if you look at a typical reactive Spring WebFlux project. These Mono/Single monads are essentially CompletableFutures with better ergonomics. I don't mean to say that hot and cold multi-value observables were used at all, but in many cases the operator chaining was pretty simple: gathering multiple inputs, mapping, reducing, potentially splitting. Most of this logic is easier to do with normal control flow when you've got proper coroutines.
But that's not all what reactive frameworks can give you. The cases when I'd choose a reactive solution over plain coroutines are few and pretty niche: to be honest, I've only reached to a reactive 3 or 4 times in my career. But they do exist:
1. Some reactive operators are trivially mapped to loops or classic collection operators (map/reduce/filter/flatMap/groupBy/distinct...). But everything that's time-bound, is more complicated to implement in a simple loop, and the result is far less readable. Think about sample or debounce for instance.
2. Complex operation chains do exist and implementing them as a reactive pattern makes the logic far easier to test and reason about. I've had a case where I need to implement a multi-step resource fetching logic, where an index is fetched first, and then resources are fetched based on the index and periodically refreshed with some added jitter to avoid a thundering herd effect, as well as retries with exponential backoff and predictable update interval ranges which is NOT affected by the retries (in other words: no, you can't just put a delay in a loop). My first implementation tried to model that with pure coroutines and it was a disaster. My second implementation was RxJava, which was quite decent, and then Kotlin Flow came out, which was a breeze.
3. I'm not sure if we should call this "Reactive" (since it's not the classic observable), but hot and cached single values (like StateFlow in Kotlin) are extremely useful for many UI paradigms. I found myself reaching to StateFlow extensively when I was doing Android programming (which I didn't do a lot of!).
In short, I strongly disagree with Brian Goetz that Functional Reactive Programming is transitional. I think he's seeing this issue from a very Java-centric perspective, where probably over 90% of the usage we've seen for it was transitional, but that's not all that FRP was all about. FRP will probably lose its status a serious contender for a general tool for expressing asynchronous I/O logic, and that's fine. It was never designed to be that. But let's keep in mind that there are other languages than Java in the world. Most programming languages support some concept of coroutines that are not bound to OS kernel threads nowadays, and FRP is still very much alive.
> In short, I strongly disagree with Brian Goetz that Functional Reactive Programming is transitional.
Yes, a transitional technology for the _Java Language/Platform_ post Loom. He never said anything about Functional Reactive Programming in general.
Yes, a transitional technology for the _Java Language/Platform_ post Loom. He never said anything about Functional Reactive Programming in general.
But even this statement is incorrect. FRP frameworks with Observables will remain useful in Java (as they have in other languages that already had coroutines). It's only the use of Observables as _an alternative for coroutines_ that is a transitional technology.
Maybe this is what Brian Goetz meant to say, but this is not what he said.
Maybe this is what Brian Goetz meant to say, but this is not what he said.
I think this article does an alright job selling ro over RxGo, but doesn’t really explain why using a reactive library is better than plain go. The channel/goroutine example is fine, as they say, but they hand wave how this will fall apart in a more complex project. Conversely, their reactive example is mapping and filtering an 4 item array and handwave how the simplicity will remain no matter the size of the codebase.
I’ve worked in a few complex projects that adopted reactive styles and I don’t think they made things simpler. It was just as easy to couple components with reactive programming as it was without.
I’ve worked in a few complex projects that adopted reactive styles and I don’t think they made things simpler. It was just as easy to couple components with reactive programming as it was without.
I'm interested, but when I see a large coordination system sitting on top of any language's primitives, I'm immediately curious what kind of overhead it has. Please add some benchmarks and allocation reports.
Interesting read. Reactive programming fits well for complex, event-driven systems where Goroutines can get messy. It’s great to see more Go devs exploring patterns that improve concurrency control and data flow clarity.
The explanation of RxGo being "wrong" or "out of order" seems very confusing? The case labeled "wrong" might aesthetically not look good, but from a data dependency perspective it seems totally valid to me. Why is it not valid to have the producer (map-A) resume and immediately do work "before" the consumer starts doing its work? Like, isn't that the point of breaking the "worker" bits up into separate functions? If I wanted the output of the `ro.Map()` example, wouldn't I just... not have map-A and map-B be separate functions?
I am curious on this. My reading was largely that this seemed to be a side effect of the format message, and not something that is in your control?
This looks like it uses the unsafe library, proceed with caution
I love these "we invented erlang/elixir!" hn posts (1/3 of them?).
(snark aside, that syntax actually looks pretty nice).
(snark aside, that syntax actually looks pretty nice).
Decent idea, but much to big a library. Would have been better to just have the core primitives/plumbing w/o the library of all the different operators/filters/etc. Allow the user to implement the logic, use the library for the piping to hook them together.
Thanks for this! I never used go but planning for a while and if I do so, I’d like to do it in a reactive way. As I’ve seen there are a few other reactive go libs as well, may I ask how this new lib relates to them?
I also explored Rx in Go several years ago.
https://dev.to/rix/rx-with-go-generics-2fl6
Think this looks beautiful, any idea as to the performance penalty vs say the new wg.Go(func({doSomething()}) sugared syntax in Go 1.25?
This is not the Go way. This abstraction appears to come for free, but it does not.
It was wrong of the author to not link to RxGo:
https://github.com/ReactiveX/RxGo
https://github.com/droxer/RxGo
https://github.com/ReactiveX/RxGo
https://github.com/droxer/RxGo
Meanwhile the samber/ro example is incomplete (Subscribe(...)?), and the source includes some weird stuff: https://github.com/samber/ro/blob/22b84c8296c01c4085e8913944...
Not to mention heaps of reflection and panics: https://github.com/samber/ro/blob/22b84c8296c01c4085e8913944...
The functionality and expressiveness might be fantastic, but I would evaluate it very carefully before use.