Blink virtual machine now supports running GUI programs(twitter.com)
twitter.com
Blink virtual machine now supports running GUI programs
https://twitter.com/JustineTunney/status/1621415193296388096
132 comments
If someone is looking for some low hanging fruit in the way of contributing to cosmopolitan, they could teach its scanf()[1] family about floats[2]. (May not be as low-hanging as it first appears though, as converting floats correctly is trickier than it might first appear to be. The comments in the code say it can do floats, but the comments are mistaken.)
[1] https://github.com/jart/cosmopolitan/blob/master/libc/fmt/vc... [2] https://github.com/jart/cosmopolitan/issues/456
[1] https://github.com/jart/cosmopolitan/blob/master/libc/fmt/vc... [2] https://github.com/jart/cosmopolitan/issues/456
It'd be really nice to have a good scanf() function. Worth noting that we do have Google's outstanding double-conversion library in the cosmo codebase https://github.com/jart/cosmopolitan/tree/master/third_party... as well as gdtoa (which is what we use for printf's floating point support) https://github.com/jart/cosmopolitan/tree/master/third_party... but we simply haven't found the time yet to incorporate this stuff into scanf() so contributions are very much welcome.
Recent and related:
A 116kb WASM of Blink that lets you run x86_64 Linux binaries in the browser - https://news.ycombinator.com/item?id=34367767 - Jan 2023 (69 comments)
Emulating an emulator inside itself. Meet Blink - https://news.ycombinator.com/item?id=34250352 - Jan 2023 (107 comments)
A 116kb WASM of Blink that lets you run x86_64 Linux binaries in the browser - https://news.ycombinator.com/item?id=34367767 - Jan 2023 (69 comments)
Emulating an emulator inside itself. Meet Blink - https://news.ycombinator.com/item?id=34250352 - Jan 2023 (107 comments)
Note that this is not Blink the rendering engine that powers Chromium and related browsers, but https://github.com/jart/blink, “a virtual machine that runs x86-64-linux programs on different operating systems and hardware architectures”.
Where this talks of running GUI programs, I presume it means that the VM can act as an X client. You would thus still need an X server. I don’t know what may exist along those lines for the web. As for native/desktop hosts, well, I hope you weren’t ever trying to use emulation as any form of security, because the likes of Xorg really aren’t designed for that use case.
Where this talks of running GUI programs, I presume it means that the VM can act as an X client. You would thus still need an X server. I don’t know what may exist along those lines for the web. As for native/desktop hosts, well, I hope you weren’t ever trying to use emulation as any form of security, because the likes of Xorg really aren’t designed for that use case.
> the likes of Xorg really aren’t designed for that use case.
Bullshit. Would be nice if people were to stop spreading these outright lies.
Bullshit. Would be nice if people were to stop spreading these outright lies.
I’m curious how you think it an outright lie. I’m speaking of the consequences of Xorg’s basic lack of isolation (because it wasn’t designed for use cases where that matters): that granting access to the X socket allows you to inspect, intercept and modify inputs and outputs for things like key strokes, window contents (yours or others’) and clipboards.
> I’m speaking of the consequences of Xorg’s basic lack of isolation (because it wasn’t designed for use cases where that matters): that granting access to the X socket allows you to inspect, intercept and modify inputs and outputs for things like key strokes, window contents (yours or others’) and clipboards.
This is not true! Both the protocol and the implementation allow for significant separation of server permissions (for example, notice the difference between ssh -X and ssh -Y), and the peer-to-peer nature of things like clipboard means it is quite easy to deny requests on an application level if they're written for it.
This is not true! Both the protocol and the implementation allow for significant separation of server permissions (for example, notice the difference between ssh -X and ssh -Y), and the peer-to-peer nature of things like clipboard means it is quite easy to deny requests on an application level if they're written for it.
XTerm has basic keyboard locking support.
[deleted]
[deleted]
Straight from the horse's mouth:
https://www.x.org/wiki/Development/X12/
> In short, X11 was designed for a different era of computing.
> This is not to say that there's an X12 project. There isn't. But if one day there is...
> Systems need to be secure. X12 needs to be designed with security in mind.
Do you think the Xorg devs are spreading lies about Xorg?
https://www.x.org/wiki/Development/X12/
> In short, X11 was designed for a different era of computing.
> This is not to say that there's an X12 project. There isn't. But if one day there is...
> Systems need to be secure. X12 needs to be designed with security in mind.
Do you think the Xorg devs are spreading lies about Xorg?
> Do you think the Xorg devs are spreading lies about Xorg?
Some are, yes, this is incontrovertibly true. But this one? You should hit the "history" button that's in the corner of every wiki. That page hasn't been edited at all since 2013, imported from some other document written I don't know when, and literally the only thing it actually says about security is that "needs to be designed with security in mind".
What does that mean? What specific shortcomings have they identified?
Some are, yes, this is incontrovertibly true. But this one? You should hit the "history" button that's in the corner of every wiki. That page hasn't been edited at all since 2013, imported from some other document written I don't know when, and literally the only thing it actually says about security is that "needs to be designed with security in mind".
What does that mean? What specific shortcomings have they identified?
The fact that any Xorg client can become a key logger without any user input or authentication is a pretty big security hole imo.
By design Xorg has no isolation between clients so they can all read each others input, control others windows, and inject keystrokes into other applications. That’s unacceptable in the modern age and makes any attempt at sandboxing or separation of privileges for GUI applications completely pointless.
By design Xorg has no isolation between clients so they can all read each others input, control others windows, and inject keystrokes into other applications. That’s unacceptable in the modern age and makes any attempt at sandboxing or separation of privileges for GUI applications completely pointless.
> The fact that any Xorg client can become a key logger without any user input or authentication is a pretty big security hole imo.
This "hole" doesn't exist. For an X client to capture input, it must be authenticated by either the unix user permission or by an access control list (where the default is to deny). Individual clients can also be marked untrusted which sandboxes them to some extent (though not as much as using a separate X server of course).
I'll grant that in practice, most the time these restrictions are very lax... in part because they can break some applications. But at the same time, in practice, it doesn't seem to matter that much since either you're running things you trust anyway or if a malicious application has access to your X connection they also have access to all your other files so you're in trouble anyway.
This "hole" doesn't exist. For an X client to capture input, it must be authenticated by either the unix user permission or by an access control list (where the default is to deny). Individual clients can also be marked untrusted which sandboxes them to some extent (though not as much as using a separate X server of course).
I'll grant that in practice, most the time these restrictions are very lax... in part because they can break some applications. But at the same time, in practice, it doesn't seem to matter that much since either you're running things you trust anyway or if a malicious application has access to your X connection they also have access to all your other files so you're in trouble anyway.
Simple solution: isolate, by running 1 X server per client (or set of clients if you want gimp and krita in the same sandbox)
Is there any tutorials/examples of how to do this?
Apologies for not answering your question directly, but I'm pretty sure this is what XWayland does to allow for compatibility of X apps ontop of wayland.
I imagine it's mostly the issue related to applications having too much access to thinks like the keyboard and other windows without a permission system? That's a large part of what makes Wayland different from X11.
It runs the X server in the VM.
Not in this demo. It probably could though.
There's a X with sixel support: https://github.com/saitoha/xserver-sixel
I played with this before, and I could use X11 within a mlterm terminal.
I should try to recompile it with cosmopolitan to have a single X server binary both for Windows and Linux
I played with this before, and I could use X11 within a mlterm terminal.
I should try to recompile it with cosmopolitan to have a single X server binary both for Windows and Linux
That's amazing!
And this is why I'm so interested in Sixel support for the Windows Terminal (https://github.com/microsoft/Terminal/issues/448)
With that and a X server using sixels, there would no longer be a need for something as complex as wslg for most apps: you could just export DISPLAY in your .bashrc, then use one Windows Terminal tab to start X apps, and the other one to display/interact with them.
With that and a X server using sixels, there would no longer be a need for something as complex as wslg for most apps: you could just export DISPLAY in your .bashrc, then use one Windows Terminal tab to start X apps, and the other one to display/interact with them.
If the X server were running in the VM, how would it display things, or get keyboard and mouse input? You’d need to provide emulated hardware devices, which could be something like /dev/fb0 for output and /dev/input/ devices for keyboard and mouse. All that is a very much larger undertaking, and probably a good deal less useful in general. The likes of X are a much more suitable abstraction.
The change here is implementing sendmsg and recvmsg: socket communications, implying communicating with something outside the VM, which in the context of GUI stuff almost certainly means an X server from outside.
The change here is implementing sendmsg and recvmsg: socket communications, implying communicating with something outside the VM, which in the context of GUI stuff almost certainly means an X server from outside.
[deleted]
[deleted]
From https://twitter.com/StriderMyshkin/status/162141843157589606...
>A pathway to a blazingly fast WSL?
>A pathway to a blazingly fast WSL?
WSL2 is already “blazingly fast”, and WSL was no slouch either. They are both impacted by storage overhead, which makes that tweet look kind of misguided.
Also WASM is fast compared to javascript, slow compared to most other things.
Depending on your benchmarks, WASM can be 5 times slower than C, or equally as fast (sometimes faster). https://programming-language-benchmarks.vercel.app/cpp-vs-wa...
WSL2 is not blazingly fast.
I use Haskell for my ever continuing self-education and compiling and linking times with ghc under WSL2 is at least twice as slow as the same operations under native Linux on a much less capable machine.
It is not just me, it is acknowledged WSL2 is very slow in I/O: https://www.phoronix.com/review/windows10-okt-wsl/2
I use WSL2 for reasons but I think I will be much better off with Linux.
I use Haskell for my ever continuing self-education and compiling and linking times with ghc under WSL2 is at least twice as slow as the same operations under native Linux on a much less capable machine.
It is not just me, it is acknowledged WSL2 is very slow in I/O: https://www.phoronix.com/review/windows10-okt-wsl/2
I use WSL2 for reasons but I think I will be much better off with Linux.
If you cross the IO boundary to the Windows FS, yes, it is slow and your compilation will be slow. You need to have the project in the WSL FS, it's always been this way and I get pretty much native performance this way. Your editor can navigate to the WSL filesystem instead.
> If you cross the IO boundary to the Windows FS, yes, it is slow and your compilation will be slow
If you need good IO performance, WSL1 is still a valid option.
I need good IO performance, and it's not clear how much of the overhead is due to NTFS, and how much is due to 9p (to pass the VHDX file on the NTFS partition to WSL2)
So I'm planning some experiments with OpenZFS for windows and NTFS3 (https://old.reddit.com/r/zfs/comments/10lcnpk/zfs_raw_passth...), first using a WSL2 kernel with the option enabled (https://github.com/microsoft/WSL/issues/8564 and https://github.com/bioluks/WSL2-Linux-Kernel-NTFS) then WSL1.
I have a dual NVMe setup and W11 for Workstations, so it should be possible to run a passthrough of both partitions to the kernel.
However, it's not clear yet how to do that with a VHD on VHDX: most of the documentation I've read about VHD (or VHDX) refers to a passthrough as an "old" option: https://www.altaro.com/hyper-v/hyper-v-pass-through-disks/
If you need good IO performance, WSL1 is still a valid option.
I need good IO performance, and it's not clear how much of the overhead is due to NTFS, and how much is due to 9p (to pass the VHDX file on the NTFS partition to WSL2)
So I'm planning some experiments with OpenZFS for windows and NTFS3 (https://old.reddit.com/r/zfs/comments/10lcnpk/zfs_raw_passth...), first using a WSL2 kernel with the option enabled (https://github.com/microsoft/WSL/issues/8564 and https://github.com/bioluks/WSL2-Linux-Kernel-NTFS) then WSL1.
I have a dual NVMe setup and W11 for Workstations, so it should be possible to run a passthrough of both partitions to the kernel.
However, it's not clear yet how to do that with a VHD on VHDX: most of the documentation I've read about VHD (or VHDX) refers to a passthrough as an "old" option: https://www.altaro.com/hyper-v/hyper-v-pass-through-disks/
Have you considered swapping Windows and Linux? Depending on your use case, you can still get pretty good performance out of virtualized Windows if you use PCI-E passthrough
I have: I generally prefer Windows, but depending on the test results, this option is definitely on the table!
As the computer has 2x NVMe and 2x GPU, it should be easy to do a GPU passthrough (with one GPU for Linux native, the other in passthough to Windows) but this brings a little extra complexity I'd rather avoid if I can.
As the computer has 2x NVMe and 2x GPU, it should be easy to do a GPU passthrough (with one GPU for Linux native, the other in passthough to Windows) but this brings a little extra complexity I'd rather avoid if I can.
WSL2 is actually just a Linux VM running directly on Hyper-V (not under Windows). It provides 99.5% native performance.
If you're accessing files through SMB or something then sure, it'll be slow.
If you're accessing files through SMB or something then sure, it'll be slow.
I agree. If I would stay solely inside the WSL2 VM, it might be fast but my problem is that I need a tight integration between the Windows and the WSL2 file systems. In effect, this is a real pain, especially when it comes to I/O. But WSL2 also constantly loads one or more CPU cores to 100%+. My whole system feels handicapped. I probably have additional problems due to MS Defender, all known measures (that I can implement at all in a corporate-managed environment) have hardly remedied it.
It's probably due to hooks on NTFS file open. It shouldn't happen for files inside the ext4.vhdx (the / partition)
I wonder how VirtualBox would fare in your use case then. Especially since one can tweak I/O settings easily inside Linux too.
VMWare is a must for Windows. Code compiles 10x faster in a virtual machine on Windows than it does if you compile on Windows natively. Or 100x faster if you forgot to disable Windows Defender. I've always found that amusing.
Full transparency, I work at Microsoft on the hypervisor.
What do you find that VirtualBox does better than Hyper-V?
What do you find that VirtualBox does better than Hyper-V?
I use VMWare Workstation Player to run a bunch of different operating systems on Windows, e.g. FreeBSD, OpenBSD, Windows 7, etc. It's served my needs pretty well for some time. Hyper-V as I understand it is pretty new. I think I'm using it to run WSL2 and I'm pretty happy with its performance. However I don't know how to use Hyper-V as a VMWare replacement. I'd imagine if it's able to do that, the learning curve would be a bit higher due to the CLI.
Any idea of why this is?
Most compilation processes create a lot of very small files. Or a metric ton of them if you're unlucky enough to do js.
NTFS is awful at doing lots of small accesses on very small files, at least compared to Ext4. Add to that the fact that if you're on WSL, it needs to write POSIX attributes (which NTFS has no support for and therefore WSL must emulate), and that's a recipe for builds being slowed massively by IO.
As for Defender, well, Defender sees new files and halts everything while it scans them. You can see how that can be a problem for a process that pops out tens of thousands of temporary files, which it will all go and scan, locking them while it does so.
NTFS is awful at doing lots of small accesses on very small files, at least compared to Ext4. Add to that the fact that if you're on WSL, it needs to write POSIX attributes (which NTFS has no support for and therefore WSL must emulate), and that's a recipe for builds being slowed massively by IO.
As for Defender, well, Defender sees new files and halts everything while it scans them. You can see how that can be a problem for a process that pops out tens of thousands of temporary files, which it will all go and scan, locking them while it does so.
Another thing that makes Windows builds go slow is the lack of vfork(). On Linux, vfork() lets you create and teardown a new process in ~30 microseconds. On Windows it takes about 6000 microseconds to do the same thing, which is in large part due to the lack of copy-on-write memory (which Windows won't support, due to Microsoft corporate culture being heavily biased against overcommit models).
> which NTFS has no support for and therefore WSL must emulate
With ntfs3 (kernel 6) this is no longer true, as can be seen for example in Arch install living next to windows on C:\ https://gist.github.com/motorailgun/cc2c573f253d0893f429a165...
Also NTFS has had POSIX attributes support - they just weren't used before.
With ntfs3 (kernel 6) this is no longer true, as can be seen for example in Arch install living next to windows on C:\ https://gist.github.com/motorailgun/cc2c573f253d0893f429a165...
Also NTFS has had POSIX attributes support - they just weren't used before.
Note that I said WSL, not WSL2.
And I insist: NTFS has support for POSIX attributes. NTFS3 proves that: you can install arch on C:\, and use it normally.
WSL1 doesn't support POSIX attributes, just like ntfs-3g didn't support them, but it's not a limitation of NTFS.
WSL1 doesn't support POSIX attributes, just like ntfs-3g didn't support them, but it's not a limitation of NTFS.
That's not my experience with C++.
relevant bits from the comment you're replying to
> They are both impacted by storage overhead
> They are both impacted by storage overhead
WSL2 with code inside the Linux file system? Because if you're doing Linux work on the Windows file system, it will be bad. It is at native speeds inside the Linux file system though.
Yes, the file system appears to be Windows one.
I remember no questions being asked when I set up WSL2.
I really doubt that WSL2 will be at native speed with the Linux file system, because reads and writes still have to go through Windows kernel to be mapped into actual hardware reads/writes.
I remember no questions being asked when I set up WSL2.
I really doubt that WSL2 will be at native speed with the Linux file system, because reads and writes still have to go through Windows kernel to be mapped into actual hardware reads/writes.
> I really doubt that WSL2 will be at native speed with the Linux file system
I had the same doubts, but when I finally took the time to install it, I found out that I was wrong. One Java program that I use, which is very heavy on the filesystem and slow like hell under Windows it a LOT faster under WSL2.
As Java profiler support is poor on Windows, I now use WSL2 to run async-profiler sessions and get results as accurate as on native Linux.
EDIT to add: you really have to run things fully in the WSL2 filesystem, be it for compiling or using a program. The performance is awful if you run WSL2 processes under the Windows filesystem.
I had the same doubts, but when I finally took the time to install it, I found out that I was wrong. One Java program that I use, which is very heavy on the filesystem and slow like hell under Windows it a LOT faster under WSL2.
As Java profiler support is poor on Windows, I now use WSL2 to run async-profiler sessions and get results as accurate as on native Linux.
EDIT to add: you really have to run things fully in the WSL2 filesystem, be it for compiling or using a program. The performance is awful if you run WSL2 processes under the Windows filesystem.
You are incorrect in the last paragraph but that was already explained here. Try putting the files in the Linux file system and only using that.
They will go through Hyper-V, not Windows
> I really doubt that WSL2 will be at native speed with the Linux file system, because reads and writes still have to go through Windows kernel to be mapped into actual hardware reads/writes.
No, Linux is running alongside Windows, not under it. Windows itself is virtualized when WSL2 is turned on, and both Linux and Windows run under Hyper-V. It's quite fancy actually.
No, Linux is running alongside Windows, not under it. Windows itself is virtualized when WSL2 is turned on, and both Linux and Windows run under Hyper-V. It's quite fancy actually.
> Windows itself is virtualized when WSL2 is turned on
Is there documentation/explainer for this? I've been trying to understand more about this (was this a change in Windows 11? Is it only when WSL2 is installed?) for a while now, but not found anything apart from stray HN comments every now and then.
Is there documentation/explainer for this? I've been trying to understand more about this (was this a change in Windows 11? Is it only when WSL2 is installed?) for a while now, but not found anything apart from stray HN comments every now and then.
It's the fundamental difference between WSL1 and WSL2. As designed in the 90s, the Windows NT kernel can have different subsystems that offer different userspace APIs to different software. Originally there were the Win32 subsystem, the Security subsystem, the Posix subsystem and the OS/2 subsystem. The latter two fizzled out, but WLS1 is basically just another subsystem that provides a linux-compatible API to the Windows kernel. As it turns out providing full compatability is really hard though, so WSL2 is a complete rewrite, ignoring the subsystem concept. It basically just runs linux in Hyper-V, and adds some UI/UX to make that convinient.
That Windows is then running virtualized is just how Hyper-V works, as soon as you turn on Hyper-V the host Windows runs as a guest in Hyper-V, though with special privileges [1]
https://learn.microsoft.com/en-us/virtualization/hyper-v-on-...
That Windows is then running virtualized is just how Hyper-V works, as soon as you turn on Hyper-V the host Windows runs as a guest in Hyper-V, though with special privileges [1]
https://learn.microsoft.com/en-us/virtualization/hyper-v-on-...
https://github.com/microsoft/WSL/issues/2395
This is still a problem with WSL2, at least on my WSL2 with Windows file system mappings.
I tried to use instructions that use fossil from the hctree page: https://sqlite.org/hctree/doc/hctree/doc/hctree/index.html
Fossil employs sqlite to store SCM information and it fails with "SQLITE_IOERR(1290): os_unix.c:39533: (22) fsync(...)" error, which is the same with VirtualBox: https://www.fossil-scm.org/forum/forumpost/7fb6c96d80?t=c
So WSL2, having to put up with Windows quirks, is not much a Linux anymore. It is slow and some programs can't even run.
This is still a problem with WSL2, at least on my WSL2 with Windows file system mappings.
I tried to use instructions that use fossil from the hctree page: https://sqlite.org/hctree/doc/hctree/doc/hctree/index.html
Fossil employs sqlite to store SCM information and it fails with "SQLITE_IOERR(1290): os_unix.c:39533: (22) fsync(...)" error, which is the same with VirtualBox: https://www.fossil-scm.org/forum/forumpost/7fb6c96d80?t=c
So WSL2, having to put up with Windows quirks, is not much a Linux anymore. It is slow and some programs can't even run.
A valid critic given that there's no native filesystem support yet.
But if WSL2 is given its own drive to mount and do what it wants with it, the issue should disappear.
And when WSL2 supports passing partitions, the issue will vanish.
But if WSL2 is given its own drive to mount and do what it wants with it, the issue should disappear.
And when WSL2 supports passing partitions, the issue will vanish.
> WLS1 is basically just another subsystem that provides a linux-compatible API to the Windows kernel
WSL1 is not an NT subsystem, either. WSU (the POSIX subsystem) was an actual NT subsystem, but with WSL, they abandoned this concept and went with a different design for performance reasons.
https://news.ycombinator.com/item?id=25249262
WSL1 is not an NT subsystem, either. WSU (the POSIX subsystem) was an actual NT subsystem, but with WSL, they abandoned this concept and went with a different design for performance reasons.
https://news.ycombinator.com/item?id=25249262
WSL2 is powered by Hyper-V, which is a "type 1 hypervisor":
https://en.m.wikipedia.org/wiki/Hypervisor#Classification
https://learn.microsoft.com/en-us/windows-server/administrat...
https://en.m.wikipedia.org/wiki/Hypervisor#Classification
https://learn.microsoft.com/en-us/windows-server/administrat...
I wrote something on this topic a while ago: https://jmmv.dev/2020/11/wsl-lost-potential.html
Note that WSL 1 doesn't use the NT subsystem functionality. See this sibling comment for details https://news.ycombinator.com/item?id=34640434
> I remember no questions being asked when I set up WSL2.
If the (WSL) path to your project is inside ~/ it's on the linux file system. If it's in /mnt/c/ it's in the Windows filesystem. In WSL1 those were about equally fast, both going throught the windows kernel. With WSL2, accessing /mnt/c/ from Linux has to go through Windows, and accessing \\wsl$ from Windows has to go through Linux, so where you put the files decides where they are fast.
If the (WSL) path to your project is inside ~/ it's on the linux file system. If it's in /mnt/c/ it's in the Windows filesystem. In WSL1 those were about equally fast, both going throught the windows kernel. With WSL2, accessing /mnt/c/ from Linux has to go through Windows, and accessing \\wsl$ from Windows has to go through Linux, so where you put the files decides where they are fast.
> In WSL1 those were about equally fast, both going throught the windows kernel
Yes, WSL1 was great. I hope it will never be sunset, as I need good IO performance.
> where you put the files decides where they are fast.
I also also hope we'll eventually get an option to pass partitions to HyperV, which is currently possible only with full disks.
It would be a return to the previous WSL1 level of performance for files within a NTFS partition: even if you'd need a separate NTFS partition, that'd be a minor nuisance to guarantee performance.
Yes, WSL1 was great. I hope it will never be sunset, as I need good IO performance.
> where you put the files decides where they are fast.
I also also hope we'll eventually get an option to pass partitions to HyperV, which is currently possible only with full disks.
It would be a return to the previous WSL1 level of performance for files within a NTFS partition: even if you'd need a separate NTFS partition, that'd be a minor nuisance to guarantee performance.
I hate how low are our expectations of "blazingly fast" these days.
Nope, this won't really be an improvement.
I wonder if blink could be used as a lightweight sandbox. Looking at PR46[0], it seems sandboxing is not one of the current features, but it would be cool to have a way to run arbitrary code (e.g: Python) in a sandboxed environment. Even cooler if you could limit the amount of memory/CPU used.
[0]: https://github.com/jart/blink/pull/46#pullrequestreview-1264...
[0]: https://github.com/jart/blink/pull/46#pullrequestreview-1264...
Author of Blink here. I wrote another command called pledge.com which does sandboxing: https://justine.lol/pledge/ Let's say for example you want to be able to run the `ls` command under Blink in a way that (1) restricts which parts of the file system it's allowed to use and (2) prevents it from talking to the Internet. You could say:
~/blink$ pledge.com -v/lib -v/bin/ls -v. -p 'stdio rpath tty prot_exec' o//blink/blink /bin/ls
HTAGS LICENSE Makefile README.md TAGS blink build o test third_party tool
Awesome! I was just reading about pledge. I think I need to find some time to play around with it. Right now, there's a lot of interest in using WASM as a sandbox for untrusted code, but pledge looks easier to use and more versatile (you don't need any kind of WASM compatibility, just a binary to run).
Offline WASM runtimes like Wasmtime are pretty cool, however I'd call SECCOMP BPF with Landlock LSM a shining beacon of light. The problem is that (1) coding BPF assembly has always been terrifying to the uninitiated and (2) Landlock LSM only came out in the past year. I believe pledge.com is the first tool that makes using both these Linux security tools together universally accessible for everyone. Be sure to run a bleeding edge Linux kernel if you use the `-v PATH` flag. If you're running an older kernel, then the pledge.com command will treat unveiling as a no-op in the interest of compatibility. I use Landlock LSM on my desktop, which runs Alpine Linux 5.15.74-0-lts. I also use Landlock in production on GCE, but I needed to `apt install linux-image-5.18.0-0.deb11.4-cloud-amd64` in order for it to work.
[deleted]
But this is not cross platform, right? It is specific to x11
I don't think there are any remotely mainstream platforms there isn't an x11 server for.
Worst case, there's always
http://www.jcraft.com/weirdx/
http://www.jcraft.com/weirdx/
There's also xpra, which gets you a browser interface to X apps.
Good point. Blink could even ship with a X11 server. (Or Wayland compositor? To be more modern)
[deleted]
XQuartz does not properly support HiDPI, so it's never going to be great on Macs anymore:
https://github.com/XQuartz/XQuartz/issues/91
Besides that, even Linux is slowly moving to Wayland.
https://github.com/XQuartz/XQuartz/issues/91
Besides that, even Linux is slowly moving to Wayland.
Well, there is a Wayland Compositor for macOS:
https://github.com/owl-compositor/owl
It still lacks a lot of features though (I think, I never tried it out)
https://github.com/owl-compositor/owl
It still lacks a lot of features though (I think, I never tried it out)
Moving to Wayland does not prevent X11 apps. And I don't think anyone would expect this to be a good solution for crossplatform guis. More of a fun starting point.
I'm interested in using this for running x86 games on aarch64 Linux using Wine, do you think this would be useful as part of something like box64?
Are there any limitations in the linux binaries that can run in the VM ? (statically linked only ?)
Blink can run both static and dynamic binaries. There are of course caveats. See this section in the README: https://github.com/jart/blink#compiling-and-running-programs...
For dynamic libraries, does it mean a program running inside of blink can make calls to dynamic libraries installed on the host system? If so, cool!
It depends what you mean. For example, I can run `blink curl ...` on Alpine Linux and blink will run the the dynamically linked curl binary that Alpine provides in /usr/bin/curl. So blink programs can leverage system libraries on x86_64 systems. What Blink does not do is translate dynamic ABIs. Here's what I mean by that. Let's say I write an x86_64-linux binary that calls dlopen(). It'll work on Linux. But if I scp that Linux program over to an Apple system, then it's not going to be able to dynamically open MachO .dylib files, because Blink only translates man(2) SYSCALL APIs. It would be able to dynamically load any Linux .so files I copied over though. Some of our users have experimented with this in cool ways. One contributor created a chroot'd folder in a Cygwin environment that contains an rsync'd Alpine Linux userspace (binaries, libraries, configs, and all) and then used blink to launch its /bin/bash shell. This astoundingly produced a mostly working Linux system inside Cygwin!
Is it possible to run Python on it? I guess network I/O is a limitation due to the browser environment?
Could this be ported to run on PowerPC Macs?
PowerPC is one of the 11 hardware architectures on which we regularly test Blink using Qemu https://github.com/jart/blink/blob/c8dbbc8cf553680ab48d8a304... so my guess would be yes. However it depends on how old the Mac software is. Blink currently assumes modern POSIX system calls are available, e.g. fstatat(), O_CLOEXEC, etc. i.e. you currently need a host operating system published after ~2010ish. That could change in the future if we end up writing a ./configure script.
Art!
From the same author that did Cosmopolitan C and... Occupy Wall Street?
Damn, I don't do enough things
Damn, I don't do enough things
"In March 2014, Tunney petitioned the US government on We the People to hold a referendum asking for support to retire all government employees with full pensions, transfer administrative authority to the technology industry, and appoint the executive chairman of Google Eric Schmidt as CEO of America"
So... at least you haven't done that :)
So... at least you haven't done that :)
Justine also wrote Rampage toolz 2.0, one of the early "AOL Progz" [1]. Can't quite remember which ones I used as a kid back then.
She also claimed to have written hacks for Kevin Mitnick.
Justine was somewhat controversial inside the Occupy movement, she was outside most circles and was criticized for extremely strongly-voiced technocratic opinions (see "Nerds Should be Segregated" [2], since deleted), but she's incredibly, incredibly talented. A good profile of Justine x Occupy post-mortem is here [3]
[1] https://imgur.com/a/uSMr7
[2] https://web.archive.org/web/20141102140507/https://justinetu...
[3] https://www.thenation.com/article/archive/breaking-occupy/
She also claimed to have written hacks for Kevin Mitnick.
Justine was somewhat controversial inside the Occupy movement, she was outside most circles and was criticized for extremely strongly-voiced technocratic opinions (see "Nerds Should be Segregated" [2], since deleted), but she's incredibly, incredibly talented. A good profile of Justine x Occupy post-mortem is here [3]
[1] https://imgur.com/a/uSMr7
[2] https://web.archive.org/web/20141102140507/https://justinetu...
[3] https://www.thenation.com/article/archive/breaking-occupy/
[deleted]
So we invented Java again?
Author of Blink here. Yes. Absolutely. I've always wanted to have Java's tools (e.g. "build once run anywhere" simplistic development) except I want to program in C / C++ instead of Java. The main platform I target is x86_64 desktops and servers, on which I want to run natively (rather than Java, which would force my code to always be virtualized) and Blink lets me pull out the virtual machine trick when I need it (e.g. someone wants to run the apps I've released on an esoteric platform like MIPS OpenBSD). All the x86_64 Linux executables I spent the past several years writing and releasing are suddenly able to be enjoyed by users on platforms I'd never dreamed of, like Haiku, and I didn't have to do anything to "port" and re-release those binaries on my website in order to do that. We just had to build one tiny 180kb virtual machine (unlike the JVM which is more like 100mb and might bundle a browser toolbar in the installer) and I made it open source (ISC, unlike Qemu which is GPL) so that you can enjoy these benefits too.
Reading this, I began to think about cosmopolitan libc, and the "actually portable executable".
I haven't played around with it, but I remember being in awe of its ambition.
I haven't been following cosmopolitan, nor blink closely, but I just looked up cosmopolitan and realized you are the author of both.
Just want to say amazing work! Hope to get a chance to try out blink soon.
I haven't played around with it, but I remember being in awe of its ambition.
I haven't been following cosmopolitan, nor blink closely, but I just looked up cosmopolitan and realized you are the author of both.
Just want to say amazing work! Hope to get a chance to try out blink soon.
Somebody should write fanfic where jart goes back in time and forestalls the Java revolution in ~roughly 1995 by releasing Blink. Instead of the entire several decades of Java (and Javascript), Blink becomes the technology underlying web (the browser is really just a portable presentation layer) server and client programming. Lightweight phones with long battery life and portable architectures revolutionize mobile computing 5 years earlier. World GDP is 35% higher, and life is generally happier, and much easier.
(seriously though, thanks for doing all this. I like your philosophy compared to what the world ended up with as the dominant paradigm)
(seriously though, thanks for doing all this. I like your philosophy compared to what the world ended up with as the dominant paradigm)
I still find it surreal that these operating systems share so many mechanisms that these portable EXEs are even possible.
Someone should write some kind of historical explanation of how this could have happened. And how critical is it for the "APE" scheme to work? Could APE binaries work on AROS x86_64 for instance? (Without emulation.)
I also find it surreal that it took this long for someone to do something like Actually Portable Executable. I mean, it's slick and easy to use, but I haven't even seem assembler demos of this.
Is the same polyglot EXE possible also on 32-bit x86? If so, I find it even weirder this have not been done long ago, considering the strange hacks people have pulled over the years.
I'm fascinated by the whole thing, as you can tell. Back in the day, Windows and Unix seemed so far from each other, like oil and water.
Someone should write some kind of historical explanation of how this could have happened. And how critical is it for the "APE" scheme to work? Could APE binaries work on AROS x86_64 for instance? (Without emulation.)
I also find it surreal that it took this long for someone to do something like Actually Portable Executable. I mean, it's slick and easy to use, but I haven't even seem assembler demos of this.
Is the same polyglot EXE possible also on 32-bit x86? If so, I find it even weirder this have not been done long ago, considering the strange hacks people have pulled over the years.
I'm fascinated by the whole thing, as you can tell. Back in the day, Windows and Unix seemed so far from each other, like oil and water.
In this blog post https://justine.lol/sizetricks/#elf I talk about how simple it is to create a single file binary from scratch that runs on Linux, FreeBSD, OpenBSD, and NetBSD. All Actually Portable Executable does is it takes that trick three steps further, to support Windows, MacOS, and BIOS from a single file binary too. That requires a bit more effort.
These are simply the operating systems I knew best, because I grew up using them, and as a result was able to discover the right combination of tricks to get it to work. I'm sure someone else who loves AROS and knows AROS would also be able to discover a way to trick it out so that APE could support it too. Same goes for other platforms. For example, one of the things on my TODO list this year is to find out if the APE x86_64 hacks are replicable for the ARM64 architecture.
The closest thing to a scientific theory behind what APE is would be what the information security community calls a "polyglot". Researching that is another great way to learn more similarly inspired tricks.
> Is the same polyglot EXE possible also on 32-bit x86? If so, I find it even weirder this have not been done long ago, considering the strange hacks people have pulled over the years.
It's actually even easier to do for i386 than it is for x86_64. The blog post above explains why that's the case, due to the syscall magic numbers being more similar. Also I'm equally astonished no one did this before I came along. I think part of why that's the case, is that I couldn't just write APE. Doing this project meant that I had to implement a C library from scratch too, which is a years long coding marathon.
These are simply the operating systems I knew best, because I grew up using them, and as a result was able to discover the right combination of tricks to get it to work. I'm sure someone else who loves AROS and knows AROS would also be able to discover a way to trick it out so that APE could support it too. Same goes for other platforms. For example, one of the things on my TODO list this year is to find out if the APE x86_64 hacks are replicable for the ARM64 architecture.
The closest thing to a scientific theory behind what APE is would be what the information security community calls a "polyglot". Researching that is another great way to learn more similarly inspired tricks.
> Is the same polyglot EXE possible also on 32-bit x86? If so, I find it even weirder this have not been done long ago, considering the strange hacks people have pulled over the years.
It's actually even easier to do for i386 than it is for x86_64. The blog post above explains why that's the case, due to the syscall magic numbers being more similar. Also I'm equally astonished no one did this before I came along. I think part of why that's the case, is that I couldn't just write APE. Doing this project meant that I had to implement a C library from scratch too, which is a years long coding marathon.
I think for something like AROS the combination of support for ELF and such a small community would mean that if someone wanted to do this for AROS the easier path might well be to add support directly in the loader.
Thanks, great explanations. I think the most astonishing thing to me is that I never saw such a polyglot EXE even print "hello" on stdout, written in assembler.
Nobody dared to eat the fruit of the unknown tree. What if it was poisonous? Until a brave soul tried it, and saw that it was good.
> Blink lets me pull out the virtual machine trick when I need it (e.g. someone wants to run the apps I've released on an esoteric platform like MIPS OpenBSD).
Me and the other user of Mips on OpenBSD are both glad you do!
Me and the other user of Mips on OpenBSD are both glad you do!
Which ironically were invented before Java was even an idea.
https://en.wikipedia.org/wiki/EM_intermediate_language
https://en.wikipedia.org/wiki/TenDRA_Compiler
https://en.wikipedia.org/wiki/EM_intermediate_language
https://en.wikipedia.org/wiki/TenDRA_Compiler
curious about "except I want to program in C / C++ instead of Java", does that mean that you prefer writing C code instead of e.g. Rust or Zig?
In other words, what is your sincere opinion about Rust or Zig?
Thank you!
Theoretically, if Sun and Oracle would have cared for the experimental runtimes which compiled to native instead of JVM. I think GNU had one going, but RMS hated that too, so there was kind of a gap in the marketplace for such a thing. You can sort of do it with .NET now.
Edit: I completely misunderstood what Blink is in relation to WASM. I'm leaving my original comment below, but it is quite wrong.
Still, Blink is a VM in the same sense as the JVM, and it does JIT compilation (or it can interpret program ASM) just as much as that. As far as I understand from the git page, it is using x86_64 assembly as the "bytecode" though, so the JIT step may be quite trivial if running on the same arch and/or OS.
Old post:
This isn't compiling to native either, not ahead of time anyway. It is compiling C or C++ to WASM, and then running that in a browser. The browser will JIT compile the WASM, just like the JVM would.
The biggest difference is that everyone has a browser, while not everyone has a JVM pre-installed.
Still, Blink is a VM in the same sense as the JVM, and it does JIT compilation (or it can interpret program ASM) just as much as that. As far as I understand from the git page, it is using x86_64 assembly as the "bytecode" though, so the JIT step may be quite trivial if running on the same arch and/or OS.
Old post:
This isn't compiling to native either, not ahead of time anyway. It is compiling C or C++ to WASM, and then running that in a browser. The browser will JIT compile the WASM, just like the JVM would.
The biggest difference is that everyone has a browser, while not everyone has a JVM pre-installed.
This doesn't use WASM or a browser at all. This is an new, original x86-64 machine code interpreter with a JIT compiler that supports x86-64 and aarch64. This is an extension of Justine's APE project to use x86-64 ELF as the core of write-once-run-anywhere software.
Wait, where is WASM used here? I thought that was a possible future development?
They cared about the commercial third party ones, just not about the free beer ones.
no, we reinvented Smalltalk. It only took 45 years
EDIT: to realize it
EDIT: to realize it
We did, but not in this project. :-) This is indeed more like an embedded JVM, not totally unlike early J2ME, but with x86_64 instead of JVM instructions.
Ruby is more like Smalltalk.
Ruby is more like Smalltalk.
not at all a critique on your project, if that was the impression I gave :)
I'm old enough to remember when Java applets were invented, grew up using Visualage Smalltalk on OS/2 at school, and am still bitter that nobody ever said: but this is Smalltalk, why don't we use Smalltalk then? :)
I'm old enough to remember when Java applets were invented, grew up using Visualage Smalltalk on OS/2 at school, and am still bitter that nobody ever said: but this is Smalltalk, why don't we use Smalltalk then? :)
Because Smalltalk big supporters like IBM, decided to repurpose their Smalltalk offerings into Java.
Visual Age Smalltalk had a role similar to .NET on the OS/2 ecosystem, SOM (OS/2 version of COM) supported Smalltalk, implementation inheritance and meta-classes, so yeah it didn't help to go full on Java.
Also another Visual Age thing, Visual Age for C++ v4 also had a Smalltalk like experience for C++, unfortunely it was too demanding in resources.
Visual Age Smalltalk had a role similar to .NET on the OS/2 ecosystem, SOM (OS/2 version of COM) supported Smalltalk, implementation inheritance and meta-classes, so yeah it didn't help to go full on Java.
Also another Visual Age thing, Visual Age for C++ v4 also had a Smalltalk like experience for C++, unfortunely it was too demanding in resources.
Smalltalk was ahead of its time. So much so that its VM was unbearably slow. Thirsting for CPU power that just wasn't there. I recall using a Pentium-based machine and waiting up to five seconds to open a context menu, and the IDE (VisualWorks?) could not keep pace with my typing. I could type in a message name and watch the letters appear one-at-a-time. Compared with Borland's TurboPascal and similar IDEs of the day, Smalltalk was a slog, albeit a powerful one.
There are other issues with Smalltalk that we know now that we didn't then. Such as broad and deep class hierarchies are hard to use. Smalltalk didn't enforce type-safety when I used it (i.e., you can have runtime faults because you can send any message to any class and if you didn't implement "does not understand", you won't know until exercising that branch).
There are other issues with Smalltalk that we know now that we didn't then. Such as broad and deep class hierarchies are hard to use. Smalltalk didn't enforce type-safety when I used it (i.e., you can have runtime faults because you can send any message to any class and if you didn't implement "does not understand", you won't know until exercising that branch).
> So much so that its VM was unbearably slow
AFAIR Java applets, and the JVM in general, were not better.
My experience with Smalltalk is from high school in the middle 90s, Java came out in 1996 and it was much slower than Smalltalk in my opinion.
Also JavaScript in the browser was basically a joke and a nightmare to write and debug.
Type safety is over-hyped IMO, JavaScript isn't type safe, look were it arrived.
Even C++ is partially type safe and Java type safety is unsound
https://dl.acm.org/doi/10.1145/3022671.2984004
AFAIR Java applets, and the JVM in general, were not better.
My experience with Smalltalk is from high school in the middle 90s, Java came out in 1996 and it was much slower than Smalltalk in my opinion.
Also JavaScript in the browser was basically a joke and a nightmare to write and debug.
Type safety is over-hyped IMO, JavaScript isn't type safe, look were it arrived.
Even C++ is partially type safe and Java type safety is unsound
https://dl.acm.org/doi/10.1145/3022671.2984004
[deleted]
I always think these announcements are about https://blink.sh/
I don’t know why it’s okay to steal product names like that.
I don’t know why it’s okay to steal product names like that.
Trademark protection is around market concerns, and in general, open source projects side-step questions of "market" by giving everything away for free.
... apart from that, it's a legit hard problem to name things. As in "One of the two recognized hardest problems in software engineering" (alongside cache management and off-by-one errors).
... apart from that, it's a legit hard problem to name things. As in "One of the two recognized hardest problems in software engineering" (alongside cache management and off-by-one errors).
Yeah, it sucks that Blink (the shell) stole it from Blink (the browser engine) that stole it from Blink (the SIP client) that stole it from...
Maybe you build proton and have any windows game (or software?) in the browser?