Understanding Singleflight in Go(codingexplorations.com)
codingexplorations.com
Understanding Singleflight in Go
https://www.codingexplorations.com/blog/understanding-singleflight-in-golang-a-solution-for-eliminating-redundant-work
11 comments
> So many variants of that code
Indeed… That evolution makes perfect sense. A lot of Go developers independently arrived at similar request coalescing patterns around that time, especially in caching, RPC, and high concurrency systems. I have an older implementation from personal 2013-era Go projects that follows almost the same approach.
What is nice about open source is not necessarily the novelty of every individual idea, but having a well-tested, shared implementation the community can converge on. Your work on singleflight clearly became that reference point for the Go ecosystem, and it is cool to see the lineage from dl.google.com to groupcache, x/net, the standard library, and now all the downstream variants.
Indeed… That evolution makes perfect sense. A lot of Go developers independently arrived at similar request coalescing patterns around that time, especially in caching, RPC, and high concurrency systems. I have an older implementation from personal 2013-era Go projects that follows almost the same approach.
What is nice about open source is not necessarily the novelty of every individual idea, but having a well-tested, shared implementation the community can converge on. Your work on singleflight clearly became that reference point for the Go ecosystem, and it is cool to see the lineage from dl.google.com to groupcache, x/net, the standard library, and now all the downstream variants.
"Understanding _using_ singleflight in Go" would be a better title. This generated article doesn't give the reader a real understanding of the implementation and its various tradeoffs that you might care about depending on your workload (e.g. should the first execution to reach a key spawn a goroutine or is that allocation too much)
That being said, singleflight is a fantastic library and pattern that helps so much with p95 latency. It's a little noisy code-wise, but if you use it in the right places the gains are huge.
Also, totally agree with the below comment that recommends janos/singleflight -- start there, but most of the critical projects at my company, AuthZed, have reimplementations with tailored semantics.
That being said, singleflight is a fantastic library and pattern that helps so much with p95 latency. It's a little noisy code-wise, but if you use it in the right places the gains are huge.
Also, totally agree with the below comment that recommends janos/singleflight -- start there, but most of the critical projects at my company, AuthZed, have reimplementations with tailored semantics.
This implementation is better than stdlib's implementation in my opinion, since it respects context:
https://github.com/janos/singleflight
https://github.com/janos/singleflight
The problem with singleflight is that when a goroutine that grabs it first happens to be a slow request, it slows down entire group. What's worse is if it returns error, rest of group share the fate. That sometimes shows in bursts of cancellations on one of goroutines timing out.
Better would be if it was semaphoreflight e.g. allow up to N concurrent requests to proceed to amortize this.
Another issue is there's per group lock that can generate meaningful contention just to check the result.
Better would be if it was semaphoreflight e.g. allow up to N concurrent requests to proceed to amortize this.
Another issue is there's per group lock that can generate meaningful contention just to check the result.
First I've heard of this. Lazyweb, why would I use this over sync.Once?
singleflight and sync.Once handle slightly different use cases.
sync.Once - Run exactly once, only the first call is invoked.
singleflight - Merge concurrent requests into one request and return the response to all the callers.
Where I tend to use both, is sync.Once tends to get used for lazy init code, the first caller does any client initializations, and subsequent callers wait, and then the lazy init is done and never done again in the lifetime of the application.
For singleflight, it tends to be on merging relatively expensive requests together. Like retrieving and parsing a JSON object from a server in concurrent requests. Merging those requests together, doing the expensive work once and distributing the results to each concurrent caller type of idea. With the results becoming invalid over time so a later set of requests need to do it all over again.
sync.Once - Run exactly once, only the first call is invoked.
singleflight - Merge concurrent requests into one request and return the response to all the callers.
Where I tend to use both, is sync.Once tends to get used for lazy init code, the first caller does any client initializations, and subsequent callers wait, and then the lazy init is done and never done again in the lifetime of the application.
For singleflight, it tends to be on merging relatively expensive requests together. Like retrieving and parsing a JSON object from a server in concurrent requests. Merging those requests together, doing the expensive work once and distributing the results to each concurrent caller type of idea. With the results becoming invalid over time so a later set of requests need to do it all over again.
sync.Once is for performing lazy initialization once in a process' lifetime. This is for duplicative requests for the same resource. E.g. concurrent requests for the same cache key can be coalesced into a single call. You can use this to prevent the connection setup thundering herd that often occurs when a bunch of requests come in for the same destination with an empty connection pool, etc.
It has error handling.
> This pattern is particularly useful in scenarios where caching isn't suitable or when the results are expected to change frequently
Any suitable examples? In the linked posts it gives querying weather service as an example but still uses a cache. Even with a normal concurrent request to a single function, a caching layer can be added before it makes any external request. Or am I misunderstanding this use case?
Any suitable examples? In the linked posts it gives querying weather service as an example but still uses a cache. Even with a normal concurrent request to a single function, a caching layer can be added before it makes any external request. Or am I misunderstanding this use case?
I think the idea is to avoid problems with functions or APIs that have side effects. You want to make sure you perform the task only once. Or you want to make sure you’re not burning money on some commercial API through redundant calls. It‘s not necessarily about performance, hence why the weather service example also uses an additional cache specifically for performance.
I then open sourced it in Jan 2013 in what was then named Camlistore (now Perkeep) in https://github.com/perkeep/perkeep/commit/6f9f0bdda9c9c1f147... d
And later I put it in https://pkg.go.dev/github.com/golang/groupcache/singleflight (groupcache was written for dl.google.com)
And a private copy in Go's net package in Jun 2013: https://github.com/golang/go/commit/61d3b2db6292581fc07a3767...
It later moved to golang.org/x/net, and later to the Go standard library (well, internal: https://pkg.go.dev/internal/singleflight)
We now even have a copy with generics in Tailscale's tree at https://pkg.go.dev/tailscale.com/util/singleflight
So many variants of that code :)