Go says Wat(about.sourcegraph.com)
about.sourcegraph.com
Go says Wat
https://about.sourcegraph.com/go/gophercon-2018-go-says-wat/
104 comments
> Now, sure, slices and the way append work are a bit complicated to grasp at first
slices are not complicated, the issue is append, mutable slices and backing arrays being shared, that's what's really fucky about it.
And that interaction is absolutely a WAT. Especially the ability to separately append to two slices with the same backing array.
> A lot of WATs are more bad practice than problems in the language, IMO
Much the same could be said about the original presentation.
There absolutely are non-WAT sections in that article though. (4) is one.
slices are not complicated, the issue is append, mutable slices and backing arrays being shared, that's what's really fucky about it.
And that interaction is absolutely a WAT. Especially the ability to separately append to two slices with the same backing array.
> A lot of WATs are more bad practice than problems in the language, IMO
Much the same could be said about the original presentation.
There absolutely are non-WAT sections in that article though. (4) is one.
[deleted]
> And that interaction is absolutely a WAT
I'm not sure views into a fixed-sized array is really confusing/strange at all. Whether you create multiple views in the array, mutate the views, etc. All interactions that occur on slices and arrays make plenty of sense. How you'd write something like append yourself if "Slice" was your type w/ a backing array, a start index, and a length is quite straightforward.
I'm not sure views into a fixed-sized array is really confusing/strange at all. Whether you create multiple views in the array, mutate the views, etc. All interactions that occur on slices and arrays make plenty of sense. How you'd write something like append yourself if "Slice" was your type w/ a backing array, a start index, and a length is quite straightforward.
This is absolutely one of the worst aspects of Go. When you call append() on a slice, and then modify the contents of the result, you may or may not modify the original slice.
In a language that encourages concurrency, this indeterminism should be jaw-dropping. How many Go programs are safe only accidentally via an array's allocation policy (especially given how much code it requires to duplicate a slice)? Can you think of any other language that has such a design?
In a language that encourages concurrency, this indeterminism should be jaw-dropping. How many Go programs are safe only accidentally via an array's allocation policy (especially given how much code it requires to duplicate a slice)? Can you think of any other language that has such a design?
It's not indeterminate, just conditional at runtime, no different than any other may-mutate-parameter-reference function. It fits Go's philosophy of foot guns for the sake of performance/simplicity (be it compiler or runtime performance/simplicity). Use of append by devs should invalidate any future uses of the parameter, but this is better than forcing a copy or abstracting implementation details of linked arrays.
> Can you think of any other language that has such a design?
Sure, happens all the time because it fits the machine naturally though sometimes it may be hidden with unnecessary runtime costs by many languages that do force copies or reallocations due to immutability. C++ is a language that doesn't hide it. Your argument is like saying you're surprised that std::vector::emplace_back may or may not alter the array I had kept from std::vector::data previously.
> Can you think of any other language that has such a design?
Sure, happens all the time because it fits the machine naturally though sometimes it may be hidden with unnecessary runtime costs by many languages that do force copies or reallocations due to immutability. C++ is a language that doesn't hide it. Your argument is like saying you're surprised that std::vector::emplace_back may or may not alter the array I had kept from std::vector::data previously.
> I'm not sure views into a fixed-sized array is really confusing/strange at all.
Again, as I already stated in the previous comment the issue is not the slices themselves.
It's that they're doubling up as vectors and they're shitty at it: you can share a backing buffer between mutable slices and append to both and then all hell breaks loose.
That is the issue. And that is why Rust doesn't have that issue despite having the same concept of slices: you can't grow a slice, and you can't have multiple mutable slices to the same backing buffer.
Again, as I already stated in the previous comment the issue is not the slices themselves.
It's that they're doubling up as vectors and they're shitty at it: you can share a backing buffer between mutable slices and append to both and then all hell breaks loose.
That is the issue. And that is why Rust doesn't have that issue despite having the same concept of slices: you can't grow a slice, and you can't have multiple mutable slices to the same backing buffer.
I mean, by that measure C/C++ and many others have the same problems. Your issue about lack of compiler-enforced reuse checks after invalidation is more with the language than this feature. While I agree generally, I don't see how slices are any more of a problem or surprising here than any other mutable reference that may or may not be mutated conditionally by a function.
> I mean, by that measure C/C++ and many others have the same problems.
No. C only has raw pointers so it's not trying to pretend slices are vectors, and C++ has actual functioning vectors with "slicing" being either a copy of the subvector or an iterator. Either way there is no confusion as to the capabilities of what you're given and its relation to the original data.
No. C only has raw pointers so it's not trying to pretend slices are vectors, and C++ has actual functioning vectors with "slicing" being either a copy of the subvector or an iterator. Either way there is no confusion as to the capabilities of what you're given and its relation to the original data.
> Whether you create multiple views in the array, mutate the views
Whether there are multiple views or not is actually an implementation detail, and whether "append" chooses to allocate memory or not will change whether there are multiple copies or multiple views of the array.
All of that behavior is something you absolutely cannot rely on.
Whether there are multiple views or not is actually an implementation detail, and whether "append" chooses to allocate memory or not will change whether there are multiple copies or multiple views of the array.
All of that behavior is something you absolutely cannot rely on.
Can I not rely on append only allocating when the values to append won't fit in the remaining capacity of the underlying array starting from the end of the slice? Some articles indicate this and definitely don't seem surprising [0]. Granted I don't know the implementation details, but how do copies of the array or numbers of slices of the array affect that? Regardless, with Go as opposed to lower level languages, building reliance on whether functions do memory allocation (even built in ones) is folly.
0 - https://blog.golang.org/go-slices-usage-and-internals
0 - https://blog.golang.org/go-slices-usage-and-internals
The behaviour of append and the conditions where it will allocate a new backing array are specified and part of the language. Implementation details don't even enter the picture here. You can absolutely rely on that not changing.
>A lot of WATs are more bad practice than problems in the language, IMO (like WAT 8, modifying the return value twice in deferred functions, with the order being significant: who the hell writes such a code?)
Isn't that the tired old argument from the C camp? "If you are careful you can write perfectly fine C without bugs".
DUH!
The whole idea is for the compiler and types to catch as much of this crap as possible, not to add accidental mental burden to the programmer.
Isn't that the tired old argument from the C camp? "If you are careful you can write perfectly fine C without bugs".
DUH!
The whole idea is for the compiler and types to catch as much of this crap as possible, not to add accidental mental burden to the programmer.
While I agree that compiler should eventually be updated to prevent this sort of behavior or something similar, I agree with the grandparent here. I had same sentiment as the grandparent, in that for several of these errors, if you end up causing them it just belies a lack of understanding of software development in general. It’s great if we can improve compilers to teach people about these sort of behaviors, but they don’t really fall under the definition of WAT that I think is being referenced in the article. That is, a WAT behavior like this is something you would normally expect to behave in a certain way, but because of some language idiosyncrasy or design decision in the language it actually does something different.
> It's pass-by-reference 101.
Is it though? Slices seem to be passed by value, where the value is a pointer and some bookkeeping information (like a C struct of (data, len, capacity)).
Some operations manipulate just data (which is visible in the caller, since that 1/3 of the "struct" was a pointer passed by value) and some operations manipulate len and/or capacity (which is not visible in the caller, since that 2/3 of the "struct" was also passed by value).
Seems more complex than just hand-waving the details away with "pass-by-reference 101".
Is it though? Slices seem to be passed by value, where the value is a pointer and some bookkeeping information (like a C struct of (data, len, capacity)).
Some operations manipulate just data (which is visible in the caller, since that 1/3 of the "struct" was a pointer passed by value) and some operations manipulate len and/or capacity (which is not visible in the caller, since that 2/3 of the "struct" was also passed by value).
Seems more complex than just hand-waving the details away with "pass-by-reference 101".
An illustation of append being confusing: https://play.golang.org/p/WGwp2cM6E1B
[deleted]
It's pass by reference 101 where by reference is actually passing a struct by value, and the struct has a reference to more data, so sometimes the backing data is updated and the struct isn't...
So, probably more like pass-by-reference-and-by-value 201.
So, probably more like pass-by-reference-and-by-value 201.
Go is only pass by value. Much in the same way JavaScript is (the best description for JS, IMO, is reference by value).
Even if you pass a pointer, the pointer is explicitly copied to a new location in memory, which is why assignment to the pointer can overwrite the pointer, but not affect the original pointer (the memory location value inside the pointer-typed variable) outside the function scope.
Pass by reference is when assignment to a parameter name is transitive to the caller's scope, and can be seen in C#. [1]
[1] https://docs.microsoft.com/en-us/dotnet/csharp/language-refe...
Even if you pass a pointer, the pointer is explicitly copied to a new location in memory, which is why assignment to the pointer can overwrite the pointer, but not affect the original pointer (the memory location value inside the pointer-typed variable) outside the function scope.
Pass by reference is when assignment to a parameter name is transitive to the caller's scope, and can be seen in C#. [1]
[1] https://docs.microsoft.com/en-us/dotnet/csharp/language-refe...
Now that C# has ref locals, it's not really pass-by-reference so much so as C++-style (or Algol-68 style, if you go far enough) references that are implicitly dereferenced on access and cannot be rebound.
Go is not really 'pass by value' when passing slices. It just passes the pointer value instead of doing a deep copy(copying the backing array). When trying to grow a slice in a function, the result is unpredicted because of 'apend' implicitly choosing to allocate memory or not. In addition, it is not concurrent-safe to pass mutable data. To avoid the safety problem, you have to copy stuff and consequently suffer a performance penalty. A better way to solve the problem is to introduce immutibilty. If one day immutibilty is introduced in Go 2(or 3), i wish slices are really passed by value. Then everything is passed by value as default with the immutable data passed by reference. You can still use pointers to pass mutable data just like using '_' to explicitly ignore error handling.
Now in WAT 1,
Now in WAT 1,
func grow(s []int) { // s is deep copied.
s = append(s, 4, 5, 6) // changing 's' does not effect original slice.
}
func grow(s *[]int) { // s is referenced.
*s = append(*s, 4, 5, 6) // changing 's' will always effect origin slice.
}
In the given examples, it is. But I agree slices are complex in Go, probably the most complex thing for newcommers, and the main reason why I wouldn't recommend go as a first language.
My first thought was that a lot of these WAT should elicit an eye roll from anyone who took Comp Sci. My eventual conclusion was that this was just a form of stealth basic Comp Sci education for language hipsters.
The slices are actually confusing and I've written quite a few test programs to teach myself the WATs because the behavior is not obvious from the syntax. Remember, slices are very mutable, regardless of the lack of a * in the function parameter. This works as you'd expect:
As you write more go, you will begin to fully see the existence of a backing array. Try this program:
Now, add some knowledge from the documentation about how append() works. You use the idiom slice = append(slice, element) because the documentation says, sometimes the slice is updated in place when there is enough capacity for that to occur, and sometimes the slice is copied to a new slice and returned.
This then leads to your "WAT" in WAT 2. Because you _know_ that the data you appended is actually in the backing array, but for some reason Go isn't showing it to you.
You can see this in action with a very poor example I just wrote:
For that reason, I think WAT 2 is worthwhile. Most people know _just enough_ about slices to be dangerous, and so are surprised when there is an additional complication that they haven't thought about.
type thing struct {
foo int
}
func f(x thing) {
x.foo = 42
}
func main() {
var x thing
x.foo = 1234
fmt.Printf("x before: %v\n", x)
f(x)
fmt.Printf("x after: %v\n", x)
}
func f(x []int) {
x[1] = 42
}
func main() {
x := []int{1, 2, 3}
fmt.Printf("x before: %v\n", x)
f(x)
fmt.Printf("x after: %v\n", x)
}
As you write more go, you will begin to fully see the existence of a backing array. Try this program:
func main() {
x := []int{1,2,3,4}
y := x[1:2]
y[0] = 42
fmt.Printf("x: %v, y: %v\n", x, y)
}
Now, add some knowledge from the documentation about how append() works. You use the idiom slice = append(slice, element) because the documentation says, sometimes the slice is updated in place when there is enough capacity for that to occur, and sometimes the slice is copied to a new slice and returned.
This then leads to your "WAT" in WAT 2. Because you _know_ that the data you appended is actually in the backing array, but for some reason Go isn't showing it to you.
You can see this in action with a very poor example I just wrote:
func main() {
x := []int{1, 2, 3, 4, 5, 6}
y := x[0:0]
fmt.Printf("x: %v, y: %v\n", x, y)
fmt.Printf("intermediate result that's not saved: %v\n", append(y, 9, 8, 7, 6, 5))
fmt.Printf("x: %v, y: %v\n", x, y)
}
For that reason, I think WAT 2 is worthwhile. Most people know _just enough_ about slices to be dangerous, and so are surprised when there is an additional complication that they haven't thought about.
> This prints 1234 and 1234 as you'd expect.
Until it doesn't.
Until it doesn't.
type thing struct {
foo *int
}
func f(x thing) {
*x.foo = 42
}
func main() {
v := 1234
var x thing
x.foo = &v
fmt.Printf("x.foo before: %v\n", *x.foo)
f(x)
fmt.Printf("x.foo after: %v\n", *x.foo)
}
[deleted]
"The order of iteration for a Go map (a hashmap) is unspecified, by design."
I that surprising behavior? It is not only well-documented, but is in line with most other languages that offer a hash table / map / dictionary / whatever in the base language or standard library.
I that surprising behavior? It is not only well-documented, but is in line with most other languages that offer a hash table / map / dictionary / whatever in the base language or standard library.
The Go runtime explicitly randomizes the iteration of map, because the language designers noticed 'Programmers had begun to rely on the stable iteration order'[0]. Most languages, as far as I can tell, tell you that the iteration order is undefined but frequently have some stable iteration order that's consistent for the compiler or the system architecture, and don't take the extra step of randomization.
[0]https://blog.golang.org/go-maps-in-action - Header: Iteration Order
[0]https://blog.golang.org/go-maps-in-action - Header: Iteration Order
Those other languages are very arguably making a mistake.
I dislike Go for a billion reasons but I appreciate when they "break" APIs that explicitly never made those guarantees.
I dislike Go for a billion reasons but I appreciate when they "break" APIs that explicitly never made those guarantees.
Having done the work of observing langugage users apparently want an ordered map, were there any steps taken to add such a facility, or did efforts stop at letting developers know they were they were wrong by taking away a de facto feature?
Ordered maps require other tradeoffs. For example, c++ has both: std::map and std::unordered_map. std::map is a treemap, and so has logn access/insertion times. It also does a ton of small allocations, and scatters your data across memory, leading to poor cache performance at scale.
But it has ordered output, and doesn't invalidate iterators on insertion (hashmaps might, because they sometimes need to rehash).
Generally, the suggestion that I've heard is to avoid using std::map unless you really really what it specifically provides, because it's expensive and it's hard to know if you can safely relax those constraints.
But it has ordered output, and doesn't invalidate iterators on insertion (hashmaps might, because they sometimes need to rehash).
Generally, the suggestion that I've heard is to avoid using std::map unless you really really what it specifically provides, because it's expensive and it's hard to know if you can safely relax those constraints.
> Ordered maps require other tradeoffs.
Ordered maps don't require trees, and don't have that high a tradeoff: https://morepypy.blogspot.com/2015/01/faster-more-memory-eff... https://github.com/rust-lang/rust/pull/45282#issuecomment-33...
Ordered maps don't require trees, and don't have that high a tradeoff: https://morepypy.blogspot.com/2015/01/faster-more-memory-eff... https://github.com/rust-lang/rust/pull/45282#issuecomment-33...
C++ ordered maps are in sorted order whereas php-style hashes are in insertion order (recent versions of Ruby and Python have adopted the php semantics; in JS maps are php style but JS objects are unordered)
Small note, the JS objects are in fact ordered. The lack of determinism caused issues between browsers so the spec now defines the following order:
1. Integer indexes (string keys) in ascending numeric order
2. Other string keys (non-integer indexes) in insertion order
3. Symbols in insertion order
https://www.ecma-international.org/ecma-262/#sec-ordinary-ob...
1. Integer indexes (string keys) in ascending numeric order
2. Other string keys (non-integer indexes) in insertion order
3. Symbols in insertion order
https://www.ecma-international.org/ecma-262/#sec-ordinary-ob...
The biggest advantage of trees is deterministic complexity. There's a bunch of DoS attacks that exploit hashtables, because they can be O(n) in the worst case - and if you control the input, you can deliberately make it worst case. Most "secure C++" guidelines recommend defaulting to std::set and std::map for this reason alone.
If you really need an ordered map, implement one in Go isn't too difficult (but the usual no-generic caveat applies). You can just define a struct that wraps a map and a slice (or some other data structure) to record the order of the keys.
If you want it to be performant, a linked list is better than a slice. Then in your map you store structs that have both the stores value and a pointer to the corresponding node in the linked list. That way random deletions are constant time since they don’t require copying the slice.
Some language implementors deliberately introduce gotchas to bite you in the ass should you rely on implementation-specific, rather than standardized, behavior.
Take this example from Scheme: R5RS specified two procedures, integer->char and char->integer, to bijectively map characters to integer character codes, but it said nothing about a specific encoding. Most Scheme implementations happily mapped characters to their ASCII codes or Unicode codepoints.
Not so Scheme48, which mapped each character to its ASCII code number plus a magic constant (believe it was 1000). Scheme48 provided library functions ascii->char and char->ascii, that did what most people actually wanted and its maintainers insisted that people use those.
Had significant code appeared in the wild that, say, simply subtracted the magic constant from the value returned by char->integer and re-added it again before calling integer->char, the maintainers probably would have either changed the magic constant, or changed the mapping entirely, in a future version.
Take this example from Scheme: R5RS specified two procedures, integer->char and char->integer, to bijectively map characters to integer character codes, but it said nothing about a specific encoding. Most Scheme implementations happily mapped characters to their ASCII codes or Unicode codepoints.
Not so Scheme48, which mapped each character to its ASCII code number plus a magic constant (believe it was 1000). Scheme48 provided library functions ascii->char and char->ascii, that did what most people actually wanted and its maintainers insisted that people use those.
Had significant code appeared in the wild that, say, simply subtracted the magic constant from the value returned by char->integer and re-added it again before calling integer->char, the maintainers probably would have either changed the magic constant, or changed the mapping entirely, in a future version.
Many (probably most) popular languages nowadays change the hash function and, with it, iteration order between runs to make it much harder to do a denial of service attack using hash collisions (https://www.securityweek.com/hash-table-collision-attacks-co...)
Reading your [0], I couldn’t say whether go changes iteration order between iterations in the same run. I would guess it doesn’t.
Reading your [0], I couldn’t say whether go changes iteration order between iterations in the same run. I would guess it doesn’t.
It's not that uncommon, actually:
For the same reason (too many people depending on non-stable orders) - LLVM now deliberately shuffles things when you use non-stable sorts to provoke different results.
One motivation for this is the herculean effort it took to move from an older to newer version of java.
The stable order that exists turns into another footgun when people start to hardcode that into tests, and it breaks out from under them when the compiler changes slightly.
The stable order that exists turns into another footgun when people start to hardcode that into tests, and it breaks out from under them when the compiler changes slightly.
Thanks for the clarification!
When I first learned about hash tables (in Perl), the book I used said that the iteration order over a hash should never ever be relied upon, and that it could change arbitrarily from one release to the next. If the order mattered, one should use a different approach (get the keys and sort them by whatever criteria), plain and simple. I guess that has sunk in pretty deeply with me. ;-)
The .Net framework provides an OrderedDictionary and a SortedDictionary, both of which make some kind of promise regarding iteration order, but I have never used them myself.
When I first learned about hash tables (in Perl), the book I used said that the iteration order over a hash should never ever be relied upon, and that it could change arbitrarily from one release to the next. If the order mattered, one should use a different approach (get the keys and sort them by whatever criteria), plain and simple. I guess that has sunk in pretty deeply with me. ;-)
The .Net framework provides an OrderedDictionary and a SortedDictionary, both of which make some kind of promise regarding iteration order, but I have never used them myself.
I believe that Google randomized the iteration order of the versions of Java and Python that they use too.
It's easy to print a map in arbitrary order. All you have to do is create an array, iterate the map append to the array, create a new type, define the Len() and Swap() and Less() functions for that type, cast your array to that type, and finally call Sort on it. It's literally less than 20 lines of code.
Wat 4 is just setting up for wat 5 though. On its own it isn't suprising, many languages use insertion order for iteration, but many others use the hash order. But wat 5 is truly surprising, no other language that I can think of iterates through the same map in different orders (different maps, even clones - sure different order makes sense).
I'm actually impressed by the go devs here, it's a great way to avoid people making assumptions if your order is legitimately randomized.
I'm actually impressed by the go devs here, it's a great way to avoid people making assumptions if your order is legitimately randomized.
It's not a Go-specific thing, although I'm not aware of anybody else doing it like this, producing different results at runtime for the same map. The more usual approach is to make the hashes themselves random, e.g. by utilizing a seed that's different for every new process. This is the case in .NET Core by default, for example, and in the legacy .NET Framework it's been an off-by-default switch for several years now.
Well map in C++ is ordered, while unordered_map is not.
If the name was hashmap then it wouldn't be surprising, but for "map" and especially "dictionary" I can see how people might expect them to be ordered without further information.
If the name was hashmap then it wouldn't be surprising, but for "map" and especially "dictionary" I can see how people might expect them to be ordered without further information.
There has been a recent trend to preserve insertion order, at least among more dynamic languages (PHP, Python, Ruby).
I was really expecting the slice WATs to carry on to talk about what I consider a real WAT: you have to perform robust checks on your input slice's capacity and length if you use append in a function, because append makes its own choice whether the array of a slice is reused or copied.
Here's a basic demonstration:
https://play.golang.org/p/8zMmPwtjcxG
In particular, note how this behavior can easily be forgotten when the semantics are hidden through variadic arguments that are passed an existing slice (instead of the automatically created one when you pass in actual variadic arguments).
Here's a basic demonstration:
https://play.golang.org/p/8zMmPwtjcxG
In particular, note how this behavior can easily be forgotten when the semantics are hidden through variadic arguments that are passed an existing slice (instead of the automatically created one when you pass in actual variadic arguments).
> I was really expecting the slice WATs to carry on to talk about what I consider a real WAT: you have to perform robust checks on your input slice's capacity and length if you use append in a function, because append makes its own choice whether the array of a slice is reused or copied.
At a fundamental level, the issue is that Go's design decisions make any modification of a shared slice dangerous, and the language provides no way to mitigate it.
A really fun one is divergent appends to slices with a shared backing array and enough capacity for an in-place append: https://play.golang.org/p/ZHWo3bFOR0X
At a fundamental level, the issue is that Go's design decisions make any modification of a shared slice dangerous, and the language provides no way to mitigate it.
A really fun one is divergent appends to slices with a shared backing array and enough capacity for an in-place append: https://play.golang.org/p/ZHWo3bFOR0X
That's the one I show in my example above :) The final example shows how paired with variadic arguments very "weird" things can happen since your slice may come from an entirely different codebase and you may get very difficult to debug results.
I would add WAT 2.5: https://play.golang.org/p/gwcHh4-QhHK
However appending to a slice-of-the-slice leaves capacity in mutate's copy-of-the-slice to append without allocating a new backing array. Therefore mutate happy bumps the len of its slice copy and mutates the callers slice!
This bit me once in real production code when reusing a []byte array to avoid allocations. The bug was obviously my fault, but this behavior can be easy to inadvertently trigger if you're trying to avoid allocations!
Edit: fixed code formatting. It's 2018 YC, please please please implement at least a subset of markdown.
func main() {
x := []int{0, 1, 2, 3, 4, 5, 6, 7, 8, 9}
fmt.Println("orig: ", x)
mutate(x)
fmt.Println("append: ", x)
mutate(x[:4])
fmt.Println("sliced: ", x)
}
However appending to a slice-of-the-slice leaves capacity in mutate's copy-of-the-slice to append without allocating a new backing array. Therefore mutate happy bumps the len of its slice copy and mutates the callers slice!
This bit me once in real production code when reusing a []byte array to avoid allocations. The bug was obviously my fault, but this behavior can be easy to inadvertently trigger if you're trying to avoid allocations!
Edit: fixed code formatting. It's 2018 YC, please please please implement at least a subset of markdown.
Indeed, that's why one of the go versions (I don't remember which) added the capacity as the third argument when "slicing."
https://play.golang.org/p/FCm8nOElz2n
https://play.golang.org/p/FCm8nOElz2n
Nils are among the biggest wats in Go in my experience. "useful nils" just compounds "the billion dollar mistake" into something even more nefarious and even harder to identify during code review.
Along similar lines: WAT 15. Under what circumstances do you expect a nil var return to become non-nil? Did you even know that was possible?
Along similar lines: WAT 15. Under what circumstances do you expect a nil var return to become non-nil? Did you even know that was possible?
> Along similar lines: WAT 15. Under what circumstances do you expect a nil var return to become non-nil? Did you even know that was possible?
Yeah. Typed nils are a horrible, horrible feature: when you cast a value to an interface, it creates a fat pointer of (Type, Value). From a nil, that's (nil, nil) but from a nil Foo that's (Foo, nil). And since `==` just does a straight value comparison, (nil, nil) != (Foo, nil).
This actually has an official FAQ entry telling you to go fuck yourself: https://golang.org/doc/faq#nil_error
Yeah. Typed nils are a horrible, horrible feature: when you cast a value to an interface, it creates a fat pointer of (Type, Value). From a nil, that's (nil, nil) but from a nil Foo that's (Foo, nil). And since `==` just does a straight value comparison, (nil, nil) != (Foo, nil).
This actually has an official FAQ entry telling you to go fuck yourself: https://golang.org/doc/faq#nil_error
I really enjoyed Go until I started trying to work with interfaces. The fact that you have to understand the implementation details at this level in order to use them reliably, IMO, shows that when the Go team talks about language simplicity, what they really mean is _compiler_ simplicity. At some point, the complexity starts getting offloaded onto the end developer.
That said, the fact that the crowd only offered an incorrect guess four times out of 16 is telling. This really didn't have the same feel to me as the original WAT talk, which is really full of truly strange things.
That said, the fact that the crowd only offered an incorrect guess four times out of 16 is telling. This really didn't have the same feel to me as the original WAT talk, which is really full of truly strange things.
IMO, Go is an attempt to design a modern language using C philosophy, and it shows. Like, this whole thing about slices is because they insist on trying to make arrays + slices into a data structure that does everything "good enough", while mostly retaining their performance profile. And so you end up with all of those abstractions that look kinda sorta like what you'd expect in a more typical modern language - but the moment you start poking hard at them, they leak like a sieve.
It doesn't have anything to do with implementation details, even though the weird set of rules from the article implies that.
Dump those rules for a much simpler insight:
When comparing interfaces, type and value must match to compare equal.
So, (nil, nil) compares NE to (*myFancyErrorType, nil) for the same reason that (float64, 0) and (uint16, 0) do.
It isn't all that complicated. I don't see a real and non-insane alternative to it. Do you?
Dump those rules for a much simpler insight:
When comparing interfaces, type and value must match to compare equal.
So, (nil, nil) compares NE to (*myFancyErrorType, nil) for the same reason that (float64, 0) and (uint16, 0) do.
It isn't all that complicated. I don't see a real and non-insane alternative to it. Do you?
Sure there is. Do what the vast majority of other languages do - don't bundle behavior and nil like this. Or even better, make nullability a proper first-class citizen so this nonsense can be prevented at compile time.
> Go avoids magic
What? Go is all about magic. All the std lib stuff that is able to take any type works by magic.
Those "obscure rules" (which I prefer to call being able to reason about code) look like they are going to get added in Go 2.
It's odd how programmers think polymorphism is this obscure thing when it's available in almost every typed language.
What? Go is all about magic. All the std lib stuff that is able to take any type works by magic.
Those "obscure rules" (which I prefer to call being able to reason about code) look like they are going to get added in Go 2.
It's odd how programmers think polymorphism is this obscure thing when it's available in almost every typed language.
I would add garbage collection to that. Being able to ignore memory management and trust it to be handled at a non-deterministic time by a separate entity outside of your control is certainly useful, but it's also about as "magic" as programming gets at the language level.
Hi, I'm the presenter AMA.
One general comment; not every question was intended as a WAT. Some were setups to introduce a WAT.
One general comment; not every question was intended as a WAT. Some were setups to introduce a WAT.
"The default type (used for inference) for an int constant is int, which is a 32-bit type" -- this is not true, the size of int is platform-defined. See https://golang.org/ref/spec#Numeric_types. The error you're seeing is because on 32-bit platforms where int is 32 bits, 2^64-1 does not fit an int, and on 64-bit platforms where int is 64 bits, 2^64-1 does not fit an int either (int's are signed). This will work on 64-bit platforms though:
a := math.MaxInt64
fmt.Println(a)Unfortunately, the transcription doesn't match what I said. The video is now available at https://youtu.be/zPd0Cxzsslk
At first I was wondering, what is this? Anyone with a Comp Sci degree who read the docs should already know most of this! Then I thought, "This is one of the best stealth Go basic Comp Sci education presentations I've seen in awhile!"
I feel like WAT3 is the only real WAT.
Looking up a key in a map you didn't "make" gives you the zero value. I think that's consistent with much of Go. But if appending to a nil slice works, assigning to a key should never panic, even if you didn't "make" it first.
Looking up a key in a map you didn't "make" gives you the zero value. I think that's consistent with much of Go. But if appending to a nil slice works, assigning to a key should never panic, even if you didn't "make" it first.
I feel like every language should have at least one "wat" style talk or article. Even for languages you adore, it's important to understand the quirks, shortcomings, and issues in a context that might seem absurd to an outside developer.
WAT 16 is the most shocking really. Reminds me of Python 2 where True and False can be overridden as well, it was fixed in Python 3.
This was a design chose to minimize the number of keywords in the language.
> The rationale is pretty simple: Only identifiers that must be keywords for syntactic reasons are keywords.
> In fact, in the very beginning there was some discussion as to whether things like nil, iota, etc. should be keywords. Eventually we agreed on the rule above which settled it.
https://github.com/golang/go/issues/18193#issuecomment-26492...
> The rationale is pretty simple: Only identifiers that must be keywords for syntactic reasons are keywords.
> In fact, in the very beginning there was some discussion as to whether things like nil, iota, etc. should be keywords. Eventually we agreed on the rule above which settled it.
https://github.com/golang/go/issues/18193#issuecomment-26492...
On the one hand, I find it revolting that the language will let me do that.
On the other hand, why would I ever want to do that? It's not something one might do by accident. It's along the line of C letting you say
On the other hand, why would I ever want to do that? It's not something one might do by accident. It's along the line of C letting you say
idx[arr]
arr[idx]
Who does `true := false` in actual code, though? However, in real life, I've already wondered if using `new` as a variable is bad practice or not:
old := x
new := foo(&x)
if old != new {
return &Bar{} // oops, return new(Bar) won't compile
}
WATs are usually never real code, some of the ones in there are but I would encourage you to look at the linked video in the article for a fun look at other languages WATs: https://www.destroyallsoftware.com/talks/wat
Some of those in the original presentation can actually happen accidentally because of dynamic typing, though. For example, you could get a value from a JSON object, expect it to be a number, but have objects instead, and perform a {} + {} without actually knowing it, have the interpreter execute it with no error, and wonder where in your code you caught that NaN value (because it would propagate all along).
OTOH, it's very unlikely you type accidentally `true := x` in your code, and even if you did, that would very probably be caught by the compiler anyway.
OTOH, it's very unlikely you type accidentally `true := x` in your code, and even if you did, that would very probably be caught by the compiler anyway.
>Who does `true := false` in actual code, though?
Anybody who is still human and can do a typo.
Anybody who is still human and can do a typo.
What typo can that be though? If you wanted to do `if true == x` and typed `if true := x`, the compiler will tell you. If you mistyped your variable name, and wanted to do `tue := false` for instance, but inadvertantly typed `true := false`, the compiler will tell you "true declared and not used". I can't really find a compelling example.
>If you mistyped your variable name, and wanted to do `tue := false` for instance, but inadvertantly typed `true := false`
You don't need to have a variable like "tue" to make such a typo. Since your writing an if statement, you already have "true" and "false" in mind, so you just need to be a little absentminded and voila, instead of foo := false you've written true := false. Plus, lots of naive editors will offer to autocomplete something starting with t to true (if they find the token true used elsewhere within the file).
>the compiler will tell you "true declared and not used".
Only if you don't use it. But a few lines below you could very well be using true legitimately too, in which case it wont tell you:
You don't need to have a variable like "tue" to make such a typo. Since your writing an if statement, you already have "true" and "false" in mind, so you just need to be a little absentminded and voila, instead of foo := false you've written true := false. Plus, lots of naive editors will offer to autocomplete something starting with t to true (if they find the token true used elsewhere within the file).
>the compiler will tell you "true declared and not used".
Only if you don't use it. But a few lines below you could very well be using true legitimately too, in which case it wont tell you:
true := false
...
if condition == true {
// ...
}> Only if you don't use it. But a few lines below you could very well be using true legitimately too, in which case it wont tell you:
Good point!
Edit: but then, your expected original variable name, `tue` in my example, would be used while not yet defined:
Good point!
Edit: but then, your expected original variable name, `tue` in my example, would be used while not yet defined:
true := false
if foo() || bar {
tue = true // Compilation error
}
...
if condition() == true {
...
}
tue := true
...
if cond() {
true := false
if foo() || bar() {
tue = true // Will compile, but not the tue you expected
}
...
if condition == true {
...
}
}The "it's 3AM and I want to write 'myvar := true' and am thinking about 'true' so I write 'true := true' and move on and don't notice" typo.
JS used to do something similar with being able to redefine undefined.
tbh I think it's a lot less of an issue. It's creating a new var named `true` (or `false`) - that only exists in the scope it's declared in, not globally.
I think it's crazy that this isn't a compile-time error, but it's not even close to "you can change the global value of builtin values".
I think it's crazy that this isn't a compile-time error, but it's not even close to "you can change the global value of builtin values".
nil is weird in Go, and in most languages with an equivalent concept. It's a "hole in the type system"
In Smalltalk, nil just contains the sole instance of the UndefinedObject class. It's not a hole in the type system. Instead, it becomes a paradox in the inheritance system, because it's used as a superclass.
Here's a WAT. Smalltalk is actually strongly typed. It's just that everything has the same type of Object. (The type system has no holes. However, it's the size of a thimble!)
In Smalltalk, nil just contains the sole instance of the UndefinedObject class. It's not a hole in the type system. Instead, it becomes a paradox in the inheritance system, because it's used as a superclass.
Here's a WAT. Smalltalk is actually strongly typed. It's just that everything has the same type of Object. (The type system has no holes. However, it's the size of a thimble!)
I didn't finish reading all the WATs, but if these are the best WATs about Go, it must be a very consistent language.
The first two, for example, aren't WATs at all. The Go book is very clear that you need to use the result of `append` to get the modified slice. WAT 4, map traversal is unordered, is not at all a WAT. WAT 7, maps are reference types, is also not surprising at all. WAT 8 is also not a WAT, defers are defined to be processed in LIFO order, and the rest falls out of how named returns work. This was a waste of time :(
The first two, for example, aren't WATs at all. The Go book is very clear that you need to use the result of `append` to get the modified slice. WAT 4, map traversal is unordered, is not at all a WAT. WAT 7, maps are reference types, is also not surprising at all. WAT 8 is also not a WAT, defers are defined to be processed in LIFO order, and the rest falls out of how named returns work. This was a waste of time :(
I think that what you are saying is true, in a manner.
To my eyes, it is however in the manner that for the small syntax and the small number of idioms, it consistently surpises me how weirdly they interact.
> it consistently surpises me how weirdly they interact.
What exactly surprises you?
I just read the Go book and none of what I saw was surprising in the least. I consider a WAT something like Python mutable default arguments that are certain to surprise every Python programmer at some point.
What exactly surprises you?
I just read the Go book and none of what I saw was surprising in the least. I consider a WAT something like Python mutable default arguments that are certain to surprise every Python programmer at some point.
You call that a WAT? I'll show you a WAT.
https://play.golang.org/p/ePRpDbaFaRl
https://play.golang.org/p/ePRpDbaFaRl
It is confusing, but at least the io.Writer documentation calls it out as bad code
https://golang.org/pkg/io/#Writer
>Implementations must not retain p.
https://golang.org/pkg/io/#Writer
>Implementations must not retain p.
This old blog post has a pretty good explanation about how Go slice works: https://blog.golang.org/go-slices-usage-and-internals
Once you realize slices are just (pointer, length, capacity) structures, and the structure itself is copied by value, the first 2 WAT is pretty trivial.
Once you realize slices are just (pointer, length, capacity) structures, and the structure itself is copied by value, the first 2 WAT is pretty trivial.
It's kind of broken that slices are immutable but maps aren't, and that reading a slice can panic but reading a map can't.
Reading from a nil slice and map feel consistent to me. If you read from a nil map you get the same result as reading a key that is not in a non-nil map (because that key is clearly not in the map because it's nil). Similarly, if you read from a nil slice you get the same result as reading an index that's not in the slice, an out of bounds access.
Trivial in isolation, but it is much harder to see the problem in a heap of source code. In general such "surprise moments" should be regarded seriously however it seems trivial in isolation.
WAT 3 and WAT 10 were surprising for me.
WAT 3 becomes less surprising when you consider that a method on a pointer can be called and can return a valid result even if the pointer is nil.
WAT 10 just seems inconsistent. I said this before, and I'll say it again: shadowing does more harm than good.
The rest are pretty trivial for anyone who worked with the language for over a year or read the documentation carefully.
WAT 3 becomes less surprising when you consider that a method on a pointer can be called and can return a valid result even if the pointer is nil.
WAT 10 just seems inconsistent. I said this before, and I'll say it again: shadowing does more harm than good.
The rest are pretty trivial for anyone who worked with the language for over a year or read the documentation carefully.
Now combine that with the extremely confusing net.IP and net.IP.To16() and you can get a sneaky bug that I've seen in practice.
https://play.golang.org/p/SGgR4RSCPvM
https://play.golang.org/p/SGgR4RSCPvM
The video for the talk is now available: https://youtu.be/zPd0Cxzsslk
[deleted]
So, for those of us who are not GO-fers, what is a WAT?
It's not a Go-specific thing. I think this presentation started it:
https://www.destroyallsoftware.com/talks/wat
https://www.destroyallsoftware.com/talks/wat
Except for the first two I don't these are any where near as bad as the original Wat ones.
Still, nice list of small gotchas.
Still, nice list of small gotchas.
Feels like a lot of work went into coming up with these WATS
Number 4 isn't really a wat, it's just a design choice. other languages offer maps that do retain order.
A lot of WATs are more bad practice than problems in the language, IMO (like WAT 8, modifying the return value twice in deferred functions, with the order being significant: who the hell writes such a code?)
A few ones, though, really are problematic. For instance I've been bitten more than once by variable shadowing, especially with error values. IMO, this is the weakest point of Go.