More Gotchas of Defer in Go, Part II(blog.learngoprogramming.com)
blog.learngoprogramming.com
More Gotchas of Defer in Go, Part II
https://blog.learngoprogramming.com/5-gotchas-of-defer-in-go-golang-part-ii-cc550f6ad9aa
54 comments
Defers just feel like a watered down version of what you get with good scoping and RAII. They're a half measure for something programming languages solved decades ago.
Go has a garbage collector. RAII is only viable in languages where destruction is predictable, like C++ or PHP.
"defer" is effectively used for predictable destruction in Go, and a more automated RAII scheme could be used in its place.
C++/CLI is a language that targets a garbage-collected runtime, yet has full-fledged RAII semantics (they're mapped to CLR Dispose pattern).
C++/CLI is a language that targets a garbage-collected runtime, yet has full-fledged RAII semantics (they're mapped to CLR Dispose pattern).
Oh, I totally forgot about C++/CLI! You're absolutely right; that's my mistake. I do agree that RAII is preferable.
PHP also has a garbage collector.
Swift and D also have garbage collectors and support RAII.
And since I already see it coming, reference counting is a garbage collection implementation algorithm in computer science literature.
Swift and D also have garbage collectors and support RAII.
And since I already see it coming, reference counting is a garbage collection implementation algorithm in computer science literature.
The tracing GC certainly complicates things, but you can make it work with a "scope" keyword like in D.
I code in go every day professionally. These are not gotchas, they are well defined behaviors.
Check out the first part too. Btw, gotcha means what you say: "a gotcha ... works as documented but is counter-intuitive and almost invites mistakes ..." https://en.wikipedia.org/wiki/Gotcha_(programming)
Every point mentioned was not a gotcha to anyone who read the official introductory tour of the language. The behaviour may be unexpected to newbies, but if you spent an hour or so learning the language, you shouldn't be falling for these.
Completely agree. This is probably the most comprehensible list of programming language « gotchas » i’ve ever read. It feels really a confirmation that the language authors have succeeded in their goal for simplicity.
Check out the first part too: https://blog.learngoprogramming.com/gotchas-of-defer-in-go-1...
I fell to these when I was learning Go for the first time and then I've seen many people also did so. There is no rule like these are gotchas for everyone.
I fell to these when I was learning Go for the first time and then I've seen many people also did so. There is no rule like these are gotchas for everyone.
Also read the official blog post on defer: https://blog.golang.org/defer-panic-and-recover
It covers about half of these gotchas by laying out how the defer statement works in plain English. The other half (like how closures work inside loops in Go) are covered elsewhere in the language tour.
I do really like the visualizations! Makes it very clear how these mechanics work.
It covers about half of these gotchas by laying out how the defer statement works in plain English. The other half (like how closures work inside loops in Go) are covered elsewhere in the language tour.
I do really like the visualizations! Makes it very clear how these mechanics work.
The content is good, and it is accurate, but even though defer can be a bit tricky this blog post series seems to literally be describing the entire semantics of "defer" as "gotchas". It's not that tricky!
Well, it depends. Upgoing posts will go deeper on defer.
Thanks! That post definitely a must read.
Btw, the first part was more advanced (but again for beginners and intermediate gophers) -> https://blog.learngoprogramming.com/gotchas-of-defer-in-go-1...
Btw, the first part was more advanced (but again for beginners and intermediate gophers) -> https://blog.learngoprogramming.com/gotchas-of-defer-in-go-1...
I think most of the people who understand the basic principles of how computers (and many languages) work wouldn’t be surprised by any of the „gotcha” described in this article. It seems to be written for people who just started working with golang without any or little knowledge about information technology.
I am not sure about this. Swift for example also has `defer` statements but its behavior differs from Go's. Swift executes defer statements at the end of the block, not the end of the function. And AFAICR Swift doesn't evaluate the parameters right away like Go does. If you have already read how defer works in detail in Go, you probably already know this. Devs new to the language or those that haven't used defer in those cases might still be surprised when this shows up.
[deleted]
The semantics of many languages' for loops are explicitly designed to avoid this problem: for example, for-of in ES6. Generally, modern language design considers the interaction between loop iteration variables and closure captures. It's reasonable to criticize Go as a language for not addressing this problem rather than users.
In only surprised because so many of them are such well-known things to avoid when designing a language and runtime.
#4 can also be fixed with:
for i := 0; i < 10; i++ {
i := i // Creates a *new* `i`
defer func() { … something with i … }
go func() { … something with i … }
}It would be nice if Go made a new i for each iteration of the loop.
Lua 5.0 made just one loop variable per loop, like Go apparently does. When I first ran into this behavior, I thought that, although it wasn't what I expected, it made at least as much sense as what I was expecting. Since then, every single time I've been in a situation where the two ways of doing it gave different results, I've wanted the "separate variable for each iteration" behavior, so I was glad when Lua's creators changed loop variables to work like this in Lua 5.1.
(Before this change I would just do the Lua equivalent of your "Creates a new `i`" line.)
Lua 5.0 made just one loop variable per loop, like Go apparently does. When I first ran into this behavior, I thought that, although it wasn't what I expected, it made at least as much sense as what I was expecting. Since then, every single time I've been in a situation where the two ways of doing it gave different results, I've wanted the "separate variable for each iteration" behavior, so I was glad when Lua's creators changed loop variables to work like this in Lua 5.1.
(Before this change I would just do the Lua equivalent of your "Creates a new `i`" line.)
C# also switched to a new i for each iteration.
Crikey! You wouldn't wanna allocate closures in a loop anyway, would you? Even if the compiler is smart about it. Doesn't read so well either.. what's wrong with adding a simple argument to your anonymous-func allocated outside of a loop? It's readable, and there are no 'gotchas'. The args are evaluated at the point of defer/go, not func execution. Simples.
Shadowing bites you sooner or later if it becomes a habit and gains you almost nothing in real terms. I keep running into subtle nasties in coworkers' commits from quick'n'dirty-that-stayed "convenience" shadowings. Need I say `err`.. of course linters help, but aren't a given in a "bring-your-own chosen dev-env" team. =)
Shadowing bites you sooner or later if it becomes a habit and gains you almost nothing in real terms. I keep running into subtle nasties in coworkers' commits from quick'n'dirty-that-stayed "convenience" shadowings. Need I say `err`.. of course linters help, but aren't a given in a "bring-your-own chosen dev-env" team. =)
> Crikey! You wouldn't wanna allocate closures in a loop anyway, would you?
Very often yes I do.
Go encourages synchronous APIs, making it up to the caller to add concurrency. This is great, in my opinion. E.g. this is a common pattern:
Very often yes I do.
Go encourages synchronous APIs, making it up to the caller to add concurrency. This is great, in my opinion. E.g. this is a common pattern:
var wg sync.WaitGroup
ch := make(chan int, len(items))
for _, item := range items {
item := item
wg.Add(1)
go func() {
defer wg.Done()
ch <- doSomething(item)
}()
}
wg.Wait()
close(ch)
for res := range ch {
…
}
foo, err := openFileOrSomething()
defer foo.Close()
[… do something …]
err := foo.Close()
None of your explanations/examples counter the fact that you could also declare your anonymous-func just above your loop in a local, ie. `doit:= func(item...` and then simply `go doit(item)` in the loop. You get: a leaner terser loop to later have to read through, no iteration-scope "gotcha", no need to elaborately spell out explicit shadowings for what are naturally semantically really func "args" already/anyway, at worst identical cost (or better)..
But well, guess it comes down to subjective stylistic preferences here =)
> So that even if "do something" throws exception… err… panics… the file gets closed.
Your defer as placed in your above example is already scheduled to run always, even on a later panic. (After all, how else could one `recover` from a `panic` if it wasn't for `defer`?) I don't see the point of the double-closing at all here..
But well, guess it comes down to subjective stylistic preferences here =)
> So that even if "do something" throws exception… err… panics… the file gets closed.
Your defer as placed in your above example is already scheduled to run always, even on a later panic. (After all, how else could one `recover` from a `panic` if it wasn't for `defer`?) I don't see the point of the double-closing at all here..
Yeah, I could declare it above. But it's subjectively harder to read, having to jump around. Like you say: subjective.
> Your defer as placed in your above example is already scheduled to run always
Yes, brainfart. Sorry. This is from a completely separate recommendation to defer a close (dropping error return), but also manually close so that closing errors can be surfaced. Matters for e.g. writing files (not so much reading), especially when you don't flush manually.
> Your defer as placed in your above example is already scheduled to run always
Yes, brainfart. Sorry. This is from a completely separate recommendation to defer a close (dropping error return), but also manually close so that closing errors can be surfaced. Matters for e.g. writing files (not so much reading), especially when you don't flush manually.
This is too sneaky and seems like poor form.
I'm curious what good use-case there is for this, that the language bothered to support and allow this to even compile.
Duplicate variable name declaration + referencing in the same scope is unintuitive at best, and just seems wrong.
I'm curious what good use-case there is for this, that the language bothered to support and allow this to even compile.
Duplicate variable name declaration + referencing in the same scope is unintuitive at best, and just seems wrong.
Shadowing can be useful if you want to make sure the "old" variable is never referenced again in the same scope.
The reason why this idiom feels "weird" is that the loop construct of the language really ought to automatically make a fresh instance of the loop counter for each trip through the loop.
The reason why this idiom feels "weird" is that the loop construct of the language really ought to automatically make a fresh instance of the loop counter for each trip through the loop.
I don't feel that it's in more poor form than the solution suggested in the article:
for i := 0; i < 3; i++ {
defer func(i int) {
fmt.Println(i)
}(i)
}
I personally prefer the version presented in the article, but your version has at least one nice feature in that you can cleanly specify at the top of the loop body which variables are being shadowed.
It's indefensible that defer works on the function and not on the scope.
Both seem like fair options to me. With function scope, you can
On the flip-side is of course that this doesn't work like most would probably want in function-scoped:
func f() {
x := ...
if x.something() {
x.doSomethingEarlier()
defer x.cleanup()
}
// use x however you like
}
func f() {
x := ...
if x.something() {
x.doSomethingEarlier()
defer x.cleanup()
// use x however you like
} else {
// use x however you like
}
}
On the flip-side is of course that this doesn't work like most would probably want in function-scoped:
for i := 0; i < 4; i++ {
x := get(i)
defer x.cleanup()
x.whatever()
}
for i := 0; i < 4; i++ {
x := get(i)
func() {
defer x.cleanup()
x.whatever()
}()
}
I've never seen the first pattern in any other language. Aside from downcasts as in your other example, why would you only want to clean up an object if some condition is true? The 99% use case of defer is for resource destruction, which you nearly always want to do in the same scope the object was initialized in (and that observation in fact is what underlies RAII).
I think function scope is a good default, and wrapping in an anonymous function and calling it (like your last example) is a simple workaround to get the scope_defer behavior. If it was scope based there's nothing you could do to get func_defer behavior.
Yeah, I generally feel the same way. For fairly simple use, scope is more consistent (all scopes / closures are identical), but func is a bit more flexible if you're willing to pay with simple boilerplate.
I mean, you can convert them into each other. Scoped can do something like this (go+python blended code 'cuz lazy):
I mean, you can convert them into each other. Scoped can do something like this (go+python blended code 'cuz lazy):
func f(){
deferred := []
defer func() { for d in deferred.reverse(): d() }() // plus error handling
if x.something() {
deferred.push(func(){ cleanup() });
}
// same as func scope
}
It's more explicit, which is a good thing, as it makes the intent clear. This matters if, for example, the function is later refactored to inline into a caller.
It also allows more flexibility (do you execute them in the order they were enqueued, or in reverse?), more room for errors, confusion between different patterns / lack of consistency across different codebases, etc.
Explicit-all-the-things isn't an unambiguous Good Thing™. If it were, we wouldn't even be discussing this - it's an abstraction, which is less explicit than e.g. building defer out of a list and using GOTO.
Explicit-all-the-things isn't an unambiguous Good Thing™. If it were, we wouldn't even be discussing this - it's an abstraction, which is less explicit than e.g. building defer out of a list and using GOTO.
> If it was scope based there's nothing you could do to get func_defer behavior.
That's clearly false. You could set up a list to hold objects to be disposed (or, more generally, closures to execute) and defer a simple procedure that disposes of all objects in the list. This is in fact what the implementation of defer must do internally.
That's clearly false. You could set up a list to hold objects to be disposed (or, more generally, closures to execute) and defer a simple procedure that disposes of all objects in the list. This is in fact what the implementation of defer must do internally.
I think by that he means it wouldn't be possible to get func_defer behavior in the example with a single keyword, or without some form of qualifier. The compiler wouldn't be able to differentiate the behavior.
Well I think the function scope the most useful, but wish there was block scope available too.
So what I really wish was that languages used dot notation to go up scope eg ..name is that name two blocks out. This was one of the good parts of VB syntax which I miss in other languages. Think how much nicer it is than python's nonlocal and global, for example!
So what I really wish was that languages used dot notation to go up scope eg ..name is that name two blocks out. This was one of the good parts of VB syntax which I miss in other languages. Think how much nicer it is than python's nonlocal and global, for example!
What's the use case for function scoped defer? I have never once needed function scoped RAII in C++ or any other language.
It's not terribly uncommon for larger functions to do something like
func someSQLStuff() {
tx, err := createTx()
defer func() {
if err != nil {
tx.Rollback()
log(err)
} else {
tx.Commit()
}
}()
rows, err = tx.QueryContext( ... )
// more SQL
}
I have a comment at a higher level with a broader example, but for Go at least this is somewhat common:
func f(i interface{}) {
if closable, ok := i.(closable); ok {
defer closable.close()
}
// do stuff with i, maybe other casts, etc
}
deferrable := func(){}
if closable {
deferrable = closable.close
}
defer deferrable()
Couldn't you do:
func closeIfNecessary(object interface{}) {
closable, needsClosing := object.(closable)
if needsClosing {
closable.close()
}
}
func f(i interface{}) {
defer closeIfNecessary(i)
...
}
As long as you can always call .close() regardless of the code below, yep - that'd work, and is definitely more readable.
If you can't call it unless [some other conditions], it goes back to the same kind of problem though. "closable" may not be a good choice on my part, as they're often called unconditionally.
If you can't call it unless [some other conditions], it goes back to the same kind of problem though. "closable" may not be a good choice on my part, as they're often called unconditionally.
Collecting tasks from all iterations to await before returning.
It's true that defer is more powerful at function scope. You can always recover block scope with an unnamed func. But it doesn't fit with normal lexically scoped constructs. You get gotchas.
I would love to be able to explicitly affect other scopes as you mention, for example to define two classes at the same time.
I would love to be able to explicitly affect other scopes as you mention, for example to define two classes at the same time.