Go gets preliminary WebAssembly support(go-review.googlesource.com)
go-review.googlesource.com
Go gets preliminary WebAssembly support
https://go-review.googlesource.com/c/go/+/102835
98 comments
Half the size of Go binaries you encounter in the wild are debugging symbols. Compile without them and use `upx --ultra-brute` and the binary will be about ~15% of its original size. You still get goroutine stack traces from panics without the debugging symbols.
Even building without debugging symbols, a lot of the remaining size comes from runtime type information and method/function/struct names, used in reflection, which there's no way to remove. That also has some reverse-engineering implications.
I didn't know go compiler allowed to build without those, thanks.
I've just tried on a go binary that was 2.4MO. After building it with `go build -ldflags=-s .`, it's 1.6MO. After stripping with upx, it's 480ko (and still works, and has panic stack traces).
I've just tried on a go binary that was 2.4MO. After building it with `go build -ldflags=-s .`, it's 1.6MO. After stripping with upx, it's 480ko (and still works, and has panic stack traces).
Don't forget the "-w" linker flag, too.
Please note that upx also compresses your binary contents and decompresses it at runtime.
When downloading a WASM file over HTTP, the server can gzip it transparently. It'd be interesting to compare the effects on the size with what upx does (which is more specialized than gzip).
When downloading a WASM file over HTTP, the server can gzip it transparently. It'd be interesting to compare the effects on the size with what upx does (which is more specialized than gzip).
I see, thanks. I used vanilla `strip` on my binary and it did not produce any size gain (so the strip flag of go compiler is plenty enough).
Indeed, given how browsers already handle gunziping resources, it's probably best to rely on it. My binary, stripped by the go compiler and gziped, is 525ko (against 480ko using upx).
Indeed, given how browsers already handle gunziping resources, it's probably best to rely on it. My binary, stripped by the go compiler and gziped, is 525ko (against 480ko using upx).
Could it also be that Go encodes symbols in a way that strip can't read, or is is just standard ELF stuff?
I remember UPX packed binaries would trigger some antivirus software in the past because packing was so frequently used by virus authors. Does anyone know if this is still the case?
Also, can you use UPX on WebAssembly binaries? If so, will that trigger any widely used antivirus software?
Also, can you use UPX on WebAssembly binaries? If so, will that trigger any widely used antivirus software?
UPX just compresses the machine code on disk and decompresses it in memory before execution. WebAssembly executables are delivered via http and can just be compressed with gzip/brotli etc as part of the existing protocols, completely transparently to WA itself.
Granted, UPX is specialized so it might have a slight compression edge over gzip. But layering UPX over WA will require three passes over all the code before it's ready to start (1: browser downloads and executes UPX stub and compressed data. 2: UPX unpacks and reloads uncompressed code. 3: browser re-compiles and executes unpacked code again.) This would completely decimate browsers attempts to make WA startup fast like firefox's streaming compiler [1] that starts compiling it before it's even done downloading.
[1]: https://hacks.mozilla.org/2018/01/making-webassembly-even-fa...
Granted, UPX is specialized so it might have a slight compression edge over gzip. But layering UPX over WA will require three passes over all the code before it's ready to start (1: browser downloads and executes UPX stub and compressed data. 2: UPX unpacks and reloads uncompressed code. 3: browser re-compiles and executes unpacked code again.) This would completely decimate browsers attempts to make WA startup fast like firefox's streaming compiler [1] that starts compiling it before it's even done downloading.
[1]: https://hacks.mozilla.org/2018/01/making-webassembly-even-fa...
Upx also breaks memory mapping.
I don't know much about wasm, but I imagine that most wasm binaries will be statically linked. The runtime does add overhead but I'd like to know how much compared to what you see with reasonably complex rust wasm code.
Yeah indeed, and if web development prefigure anything for wasm, we can expect apps to quickly have a lot of dependencies built in. I guess we'll get used again to "please wait, loading" screens :)
I still get "please wait, loading" screens in JS heavy sites, specially those that go to the extreme of using JS to display static text.
If you read the document link seen in the article you will see that one of their priorities is focusing on efficiency and size output of wasm.
Hmm, maybe language runetimes can be dynamically linked and hosted by the language maintainers. Everyone imports the official library so the odds of the user already having most of everything cached is high.
This would be cool: shared objects over internet. Interesting direction to explore.
AppFS (http://AppFS.rkeene.org/) supports this
Aren't cache gained savings a myth? Browsers have cache in magnitude of tens of megabytes and most sites have different versions of everything.
Well, subresource integrity was not a thing before.
There are still some commits under review. I would wait until the tracking issue is closed.
Somehow I have the feeling this whole Webassembly thing will be a giant flop.
Tech people seem to think that making code run twice as fast is a huge step forward. But in the grand scheme of things its nothing.
Tech people seem to think that making code run twice as fast is a huge step forward. But in the grand scheme of things its nothing.
This is great; looks like it's still fairly rough but interesting that GC languages are making due with what's there in the MVP of WebAssembly right now.
Wonder how it works? does it use the regular call stacks or construct it's own and so on?
Wonder how it works? does it use the regular call stacks or construct it's own and so on?
From this: https://docs.google.com/document/d/131vjr4DH6JFnb-blm_uRdaC0...
> Go’s garbage collection is fully supported. WebAssembly is planning to add its own garbage collection, but it is hard to imagine that it would yield a better performance than Go’s own GC which is specifically tailored to Go’s needs.
Presumably a runtime will be compiled into the wasm output, including Go's garbage collector.
> Go’s garbage collection is fully supported. WebAssembly is planning to add its own garbage collection, but it is hard to imagine that it would yield a better performance than Go’s own GC which is specifically tailored to Go’s needs.
Presumably a runtime will be compiled into the wasm output, including Go's garbage collector.
But Go's garbage collector can't traverse the DOM.
The design doc talks some about how this works: https://docs.google.com/document/d/131vjr4DH6JFnb-blm_uRdaC0...
A GCed language Go is with split stack semantics Webassembly will only be a second class citizen.
This looks like two or three sentences mashed together. I can't figure out how to parse it.
Split stacks are no longer used, as of Go 1.3 (2014).
Off topic: is it possible to build a JIT compiler for Go or does the runtime prevent that?
I think it's certainly possible. The Go compiler is AOT and there are Go interpreters (in various stages of completeness), it would be a matter of interpreting the application - finding hot functions in execution traces - and swapping them out with background-compiled versions.
Some work would be needed to keep the GC happy between interpreted and compiled functions, but no more complex than JS JITs have to do. Go's GC at least already has the capability to move objects, that's a help.
In reality, JITs are used to infer typing information that dynamic languages only discover at runtime. Go doesn't have this problem, and an AOT can probably discover stronger optimizations.
Some work would be needed to keep the GC happy between interpreted and compiled functions, but no more complex than JS JITs have to do. Go's GC at least already has the capability to move objects, that's a help.
In reality, JITs are used to infer typing information that dynamic languages only discover at runtime. Go doesn't have this problem, and an AOT can probably discover stronger optimizations.
there's actually (the beginning of) an interpreter in Go, for Go, that consummes wasm :)
https://github.com/go-interpreter/wagon
https://github.com/go-interpreter/wagon
Any language can have a JIT compiler, that is just an implementation detail.
What are the challenges to provide a standard GC interface that all managed languages can leverage.
I believe that the LLVM project is trying to accomplish this [1].
However, the concurrent, non-pausing GC of GoLang is not easy to match in efficiency. I'd be interested to hear about actual high-quality implementations of GCs in LLVM.
And then there's the problem of making it all work efficiently in WASM (which doesn't yet support advanced concepts needed by GCs such as memory barriers).
[1] https://llvm.org/docs/GarbageCollection.html#goals-and-non-g...
However, the concurrent, non-pausing GC of GoLang is not easy to match in efficiency. I'd be interested to hear about actual high-quality implementations of GCs in LLVM.
And then there's the problem of making it all work efficiently in WASM (which doesn't yet support advanced concepts needed by GCs such as memory barriers).
[1] https://llvm.org/docs/GarbageCollection.html#goals-and-non-g...
I believe the only user of the LLVM GC infrastructure is Azul's JVM. Azul is of course famous for Azul C4, which is a superior GC to that of Go.
Tracing objects in the heap (and/or stack). The GC has to be able to find all the outgoing pointers from any object, and usually the types of the objects pointed to by those pointers as well. Different language runtimes store objects in memory in different ways.
I think the interface can provide the semantics - pass by reference or value etc. Which can be standardized. Then it is upto the languages to implement the interface.
IMO the languages designers and developers are better off by looking at the GC solution from top-down rather than bottom-up.
IMO the languages designers and developers are better off by looking at the GC solution from top-down rather than bottom-up.
Maybe you should think about what this would mean from the perspective of language designers. That would unearth the main difficulty of such an approach:
Every language designer would have to completely redesign it's runtime system just to be able to run on this one backend (in addition to having a new compiler target and linker and debugger and ...). Hoping, of course, that this is possible at all, as many languages have various ways to store pointers to things in memory, which including not storing one at all (ie. pointer arithmetic).
Why would they do this ? Why should they do this ? You could simply make the existing runtime libraries run.
Every language designer would have to completely redesign it's runtime system just to be able to run on this one backend (in addition to having a new compiler target and linker and debugger and ...). Hoping, of course, that this is possible at all, as many languages have various ways to store pointers to things in memory, which including not storing one at all (ie. pointer arithmetic).
Why would they do this ? Why should they do this ? You could simply make the existing runtime libraries run.
> Why would they do this ? Why should they do this ?
One answer could be that it would enable multiple languages to interface in a better way, and allow garbage to be collected across language boundaries.
One answer could be that it would enable multiple languages to interface in a better way, and allow garbage to be collected across language boundaries.
allocation characteristics of different languages. Go copies when passing args. java is pass by value, but those values are usually pointers to objects. very different needs.
Hey so question - WebAssembly can't access the DOM right? What's the whole point then, avoid Javascript? If we can't just swap out all the gnarly JS garbage whole-hog, I don't see where the business value is, given that there's plenty of language-to-js libraries.
DOM access is planned. https://github.com/WebAssembly/design/issues/1079. (first bullet point)
WebAssembly actually CAN access the DOM, but it requires some glue code.
Essentially, from the developer perspective WebAssembly is like embedding native code as a scripting language in the browser.
Only primitive data types can be passed back and forth between the foreign function interface between the two.
Essentially, from the developer perspective WebAssembly is like embedding native code as a scripting language in the browser.
Only primitive data types can be passed back and forth between the foreign function interface between the two.
So there's a lot of other things that can be done on the client other than interacting with the DOM. In particular, applications that render intense graphics, or manage a local database could GREATLY benefit from more efficient compiled code, and this would open up possibilities that the current JS engine wouldn't be able to deliver.
That's indeed true, but I wouldn't count Go as an example of “efficient compiled code”. It's way behind of GCC or LLVM in terms of optimizations, and being ahead-of-time-compiled is a drawback, not an asset.
Both GCC and LLVM can compile Go code.
That's a bit of an oversimplification.
About llvm, there's a WIP prototype that doesn't quite work yet[0].
And about GCC, GccGo is behind in Go's version. GCC 7 only supports Go 1.8 (with caveats). Also, it had issues at some point[2] (but this is from 2014, it may have been fixed since then).
And, on a more pragmatic point of view, when talking about Go, the huge majority talks about Go compiled with the standard Go compiler. That's why you here a lot of positive feedback about compile-times for instance.
[0]: http://llvm.org/svn/llvm-project/llgo/trunk/README.TXT [1]: https://golang.org/doc/install/gccgo#Releases [2]: https://talks.golang.org/2014/gocon-tokyo.slide#54
About llvm, there's a WIP prototype that doesn't quite work yet[0].
And about GCC, GccGo is behind in Go's version. GCC 7 only supports Go 1.8 (with caveats). Also, it had issues at some point[2] (but this is from 2014, it may have been fixed since then).
And, on a more pragmatic point of view, when talking about Go, the huge majority talks about Go compiled with the standard Go compiler. That's why you here a lot of positive feedback about compile-times for instance.
[0]: http://llvm.org/svn/llvm-project/llgo/trunk/README.TXT [1]: https://golang.org/doc/install/gccgo#Releases [2]: https://talks.golang.org/2014/gocon-tokyo.slide#54
Twitch rewrote their entire video player in C++ and compiles it down to WebAssembly with Emscripten. That way the same player can be compiled to the web, as to iOS, Android, game consoles etc. That’s a huge business case when you want to create a module to support a wide variety of platforms and targets.
As weird as this might sound, I see WebAsssembly more like a portable shared library format than a JavaScript replacement.
That's pretty much what it's supposed to be. Much of the browsery stuff is still supposed to be handled by JS.
On a related note: does anyone know if there's wasm support with ocaml/reasonml?
It seems running reasonml/ocaml under nodejs with wasm (and in the browser) should be a great fit for ocaml modules?
It seems running reasonml/ocaml under nodejs with wasm (and in the browser) should be a great fit for ocaml modules?
Webassembly uses a javascript "glue" for accessing the DOM. The point of Webassembly is not to just be an alternative to javascript; it's to have a binary web program format that runs at close to native speeds. This makes it extremely useful for porting C/C++ applications, especially games, to the browser.
It can be a lot faster and less memory-hungry than javascript. That opens up a lot of things you can't feasibly do in JS. Game engines, image decoding, even video editing. http://webassembly.org/docs/use-cases/
WebAssembly allows you to build "native" libraries that can be accessed from JavaScript. There is huge value in that.
[deleted]
>language-to-js libraries
With huge performance cost and complicated toolchains.
With huge performance cost and complicated toolchains.
Could this be used to get native gRPC support for the browser?
WebAssembly doesn't provide any functional capabilities that aren't already accessible to JavaScript. This change does not really enable a browser to do anything more "natively" than it already could, other than run code that's written in Go.
And that's not really new either, due to GopherJS [1]
https://github.com/gopherjs/gopherjs
https://github.com/gopherjs/gopherjs
Do 64-bit integers count? Because those aren't available in JavaScript (yet.)
In case you didn't see recently, https://github.com/grpc/grpc-web came out of hidden-repo status recently. Of course there are caveats to its use, but I believe a server side gateway may always be needed due to how http/2 is used but I'm unsure.
Why would you use this and not GC-free C bindings? (Clang can compile to WASM).
I really wish WebAssembly did not use the word "Assembly". It doesn't really resemble any actual assembly language. And if you look at the comments in this thread, for example, a lot of people think it's letting you have something like arbitrary machine code.
Especially with upcoming features like GC, threads, exceptions, polymorphic inline cache, and more[1]. Looks somewhere in between ASM and something like bytecode for a VM.
[1]http://webassembly.org/docs/future-features/
[1]http://webassembly.org/docs/future-features/
Well, ASM is a bytecode for a VM too … CPU are really complex theses days and you definetly don't run your asm on a dumb piece of silicon, as the phrase “bare metal” seams to imply.
And, as a reminder, there's often bugs in that VM. Meltdown was one of them, but there's tons of them.
And, as a reminder, there's often bugs in that VM. Meltdown was one of them, but there's tons of them.
I see your point, but threading, PIC, exceptions, DOM access, etc, are a level above ASM.
To add to that, that it was always how most CPUs were developed.
Pure RISC CPUs were the outliers, but coupling RISC ideas with micro-code turned out to win the match.
Even modern ARM versions use micro-code on their CPUs.
Pure RISC CPUs were the outliers, but coupling RISC ideas with micro-code turned out to win the match.
Even modern ARM versions use micro-code on their CPUs.
From one of the guys who work on WebAssembly: "WebAssembly: neither Web nor Assembly"
https://twitter.com/jfbastien/status/880831562207473664
https://twitter.com/jfbastien/status/880831562207473664
I really wish WebAssembly didn’t use the word “Web”. Its best usecase is for the server. It just happens to also be great for challenging JavaScript in the browser.
Why do you say its "best" usecase is for the server? I'm actually asking, not disputing, but I'm asking because it seems the big selling point of WASM, running new types of code that you couldn't run before, was never a problem on the server, where you can run whatever you like.
So, I'm assuming you have something else in mind, and I'm wondering what it is.
So, I'm assuming you have something else in mind, and I'm wondering what it is.
It doesn't challenge JS in the browser, it augments it.
Though I guess you could say it challenges stopgap measures like asm.js.
Though I guess you could say it challenges stopgap measures like asm.js.
Can this be used with CGO_ENABLED=1 if CC is emscripten?
Currently it's not supported, see https://github.com/golang/go/blob/a9ba3e30acf004a57fbc1288d5...
In theory (if someone does the non-trivial work), probably.
Article mentions that the compiled code will also run on Node.js, does it mean it will be possible to use all zillions of NPM modules from Go at some point in time?
Thus solving Go's dependency management problem once and for all.
Solving it by giving it a worse one.
That is the joke.
Worse than pulling master on github, really ?
Npm has issues, but it's still way better than the state of the art of dependency management in Go.
Left-pad was bad but the problem was fixed on npm's side after a couple hours. Go's equivalent isn't fixable: https://mobile.twitter.com/davecheney/status/855049071307792...
Npm has issues, but it's still way better than the state of the art of dependency management in Go.
Left-pad was bad but the problem was fixed on npm's side after a couple hours. Go's equivalent isn't fixable: https://mobile.twitter.com/davecheney/status/855049071307792...
Pulling master on github isn't the "state of the art" for Go (it's dep and vgo).
`dep` is pulling directly from github … And `vgo`, while promising, is still a protype not ready for production[0].
[0]: https://github.com/golang/go/wiki/vgo#current-state
[0]: https://github.com/golang/go/wiki/vgo#current-state
commit your vendor folder
In Chrome and Node, Wasm code is executed by V8.
V8 can be embedded in C/C++/Go/etc applications. That allows you to use JS for scripting. This is quite different from compiling to Wasm, though.
V8 can be embedded in C/C++/Go/etc applications. That allows you to use JS for scripting. This is quite different from compiling to Wasm, though.
How are features like go routines and channels etc going to be supported? or they are not?
Dig a little deeper: https://docs.google.com/document/d/131vjr4DH6JFnb-blm_uRdaC0...
Mentions:
> Currently WebAssembly has no threads, but they are on the roadmap. Most Go code can run fine on a single thread. The only drawback is that “sysmon” is not available, thus there is no preemption of goroutines.
Mentions:
> Currently WebAssembly has no threads, but they are on the roadmap. Most Go code can run fine on a single thread. The only drawback is that “sysmon” is not available, thus there is no preemption of goroutines.
"thread" here refers to logical thread. A single logical thread can be used to run any number of goroutines, just one at a time.
Go does both preemptive and cooperative multitasking, so there is no relation to the number of goroutines "running" and the number of threads (or for that matter, cpus or cores).
https://en.wikipedia.org/wiki/Cooperative_multitasking
How this works in Go ? Thoroughly simplified it happens because at every flow control statement in Go, including function calls, what really happens is that, first, Go asks it's runtime to switch goroutines, and only then goes into the if, or subroutine, or ...
https://en.wikipedia.org/wiki/Cooperative_multitasking
How this works in Go ? Thoroughly simplified it happens because at every flow control statement in Go, including function calls, what really happens is that, first, Go asks it's runtime to switch goroutines, and only then goes into the if, or subroutine, or ...
Go binaries, being static and embedding the whole go runtime, tend to be quite big. Now, I don't care about it because 5 or 10MO on my server filesystem is nothing. If I ask every visitor to download 10MO before running my webpage, that will be a problem, though.
Is this something we will have to deal with? Or will this be a lesser featured go implementation?