Emulating win32(neugierig.org)
neugierig.org
Emulating win32
https://neugierig.org/software/blog/2023/01/emulating-win32.html
71 comments
Ah, but can you emulate calls to a shared library, translating ABI's and binary structure definitions seamlessly across architectures? I suppose that this may well be impossible in the general case.
If shared libraries are the level at which your OS exposes its interfaces, then yes.
winevdm came to mind as a rather different Windows API + CPU emulator project - it glues WinAPI thunking to MAME's x86 emulator to run Win16 apps against the Win32 API. On 32-bit x86 this is redundant, because the built-in WoW32 will do the same job. winevdm enables running Win16 apps on 64-bit Windows on x64 and ARM.
The closed Github issues are a trip. https://github.com/otya128/winevdm/issues?q=is%3Aissue+is%3A...
In the words of its own README:
The closed Github issues are a trip. https://github.com/otya128/winevdm/issues?q=is%3Aissue+is%3A...
In the words of its own README:
CPU Emulator
wine based Win16->Win32 conversion codes:
Relay routines from 16-bit to 32-bit are autogenerated by convspec
DOS emulation for Win16
16-bit <=> native HANDLE conversion
Fix compatibility problems, fix compatibility problemsIt's a wonderful project. Allowing me to run a 30 years old Win16 companion program (for a HP 53310A frequency domain analyzer) on modern 64bits Windows.
I have been saying that we need a “dosbox for the next generation”. Wine is great but due to Apple dropping 64 bit support it’s been decidedly less useful for me.
DosBox works well enough for most pre-Windows stuff, DosBox-X can handle the Windows 95 era well enough, but Windows 2000-Windows 7 stuff is actually kind of hard to get fully working on modern computers, and I feel emulation has not been able to keep up.
I wish a lot of success on this project. It would be great to get Jazz Jackrabbit 2 working on my Mac without mucking with VMWare.
DosBox works well enough for most pre-Windows stuff, DosBox-X can handle the Windows 95 era well enough, but Windows 2000-Windows 7 stuff is actually kind of hard to get fully working on modern computers, and I feel emulation has not been able to keep up.
I wish a lot of success on this project. It would be great to get Jazz Jackrabbit 2 working on my Mac without mucking with VMWare.
Wine can run 32 bit Windows applications on the latest versions of macOS just fine these days - easiest way to do this is via the (paid) Crossover, who actually wrote the insane code to make it work and contributed it back to Wine as usual, but forks of Wineskin work too. Both on Intel and M1.
Unfortunately 16 bit Windows applications no longer work. I don't believe that is being actively worked on.
Unfortunately 16 bit Windows applications no longer work. I don't believe that is being actively worked on.
I use CrossOver to run a 32 bit Windows app designed in the early 2000s on my M1 Mac, works great. Used to use Parallels but found it to just not be as good, plus they got really spammy with the marketing emails for a while so I just cut it out.
There's a pretty easy, cheap, and straight forward fix for this.
Stop using apple products. They are luke-warm in design and terrible in execution.
Get a PC: Supports Linux, windows, all your games (ALL) and has a mouse with more than one button.
Stop using apple products. They are luke-warm in design and terrible in execution.
Get a PC: Supports Linux, windows, all your games (ALL) and has a mouse with more than one button.
That’s really great advice and I think you’re very smart and intelligent for having such a brave opinion, but that does not help me as I already have a MacBook, and moreover it really doesn’t actually affect my comment.
> all your games (ALL)
This is measurably untrue; modern computers have trouble with the Windows XP era of games a lot of the time. Many games require tinkering and some will just crash after a minute.
> all your games (ALL)
This is measurably untrue; modern computers have trouble with the Windows XP era of games a lot of the time. Many games require tinkering and some will just crash after a minute.
> This is measurably untrue; modern computers have trouble with the Windows XP era of games a lot of the time. Many games require tinkering and some will just crash after a minute.
This is mostly the fault of games and not Windows. Software usually breaks because of unfounded assumptions, for example, that user documents are always at a specific location or that CPU frequency is always constant. Instead of using Windows API functions to retrieve the path or keep track of frequency, lazy developers often halfass the implementation, hardcode values into their code and that breaks sometime in the future when those API functions become meaningful.
This is mostly the fault of games and not Windows. Software usually breaks because of unfounded assumptions, for example, that user documents are always at a specific location or that CPU frequency is always constant. Instead of using Windows API functions to retrieve the path or keep track of frequency, lazy developers often halfass the implementation, hardcode values into their code and that breaks sometime in the future when those API functions become meaningful.
That CPU frequency could change after boot, was once an insane proposition.
My first computer had a button to do it whenever you felt like.
I'm pretty sure MacBooks have extremely high resale values; it should be possible to sell yours and get a PC without spending an extra dime if you want to follow the OP's advice.
[deleted]
You only get the one button mouse if you specifically go out to buy it instead of a normal mouse. Same as on Windows and Linux really.
Do you have any good articles on how to cheaply and easily stop using apple product, if one is photographer, filmmaker, musician (edit: or iOS developer), and has thousands of hours and dollars invested in MacOS software?
Easy to do if one is starting now with computers but not an option for those who already invested time and money on Apple hardware and use it for their daily work.
In these context I would rather suggest to buy (or build, which is also fun and instructive) a 2nd PC machine running whatever the user wants. This would also helps to keep the two worlds separated (work|tinkering). That involves some money for sure, but if the user has a job involving Apple products, chances are that a much more affordable platform like a PC won't be a problem.
It must be repeated that Microsoft's absolute devotion(servitude?) to backwards compatibility is nigh unparalleled.
Yes, there will be some amount of inevitable jank, but by and large the latest version of Windows will happily run software originally written almost 30 years ago for Windows 95.
Yes, there will be some amount of inevitable jank, but by and large the latest version of Windows will happily run software originally written almost 30 years ago for Windows 95.
On this very thread we have the example of WineVDM, a product whose mere existence shows that these days it is Wine who has better backwards compatibility. Better than Windows itself.
You’ve ignored the simple fact that Mac ARM laptops are not merely better than every other laptop around, they are almost in another market space.
Macs have had a second mouse button for 20ish years.
Macs have had a second mouse button for 20ish years.
They've actually never had a second mouse button. They started letting you fake having a right mouse button with touch but they never put the second switch in.
The first switch is also faked. :-) Just try tapping it with the Mac turned off.
Well on trackpads. Their mice actually do still have a switch. Also Apple's definition of "off" these days keeps the trackpad and keyboard alive so you can click or press a key to turn it on. Why? I have no idea and it makes doing things like cleaning the keyboard really annoying.
So overall it's more of a movable trackpad that pretends to be a mouse :D
Apple does support Wine on modern macOS, with specific segmentation support to run 32-bit apps in that environment.
How do I go about doing that? Whenever I try and run Wine it gives me weird errors.
I have no idea what's happening under the hood, but PlayOnMac[0] seems to poke WINE the right way to make it work as expected on newer macOS.
[0]: https://www.playonmac.com/en/
[0]: https://www.playonmac.com/en/
There is 86Box and PCBox which are emulators with retro PC gaming in mind. PCBox has Pentium III support which would be good for Windows 2000 or XP. They don't support ARM or macOS though.
They do, at least 86box
Try `gcenx` distribution of wine with crossover patches (https://github.com/Gcenx/homebrew-wine).
Work fine on M1, enough to play Heroes of Might and Magic III with various mods. I haven't tried playing the Jazz2 though.
This article is a rabbit hole of links that I happily wasted two hours of work reading. Well done.
Nitpick: Wine is as much a windows emulator as xterm is a terminal emulator.
xterm is a terminal emulator. It has to pass a compliance test suite that ensures it correctly replicates the behavior of DEC terminals up through the VT220 (plus some VT320 features). Programs that do this at the protocol level have been called "terminal emulators" since forever ago; even 70s and 80s vintage physical terminals have been said to "emulate" other makes of terminal by interpreting their wire protocols.
It's all the other "terminals" for Linux, especially the libvt ones, Kitty, and Alacritty, that are more like shitty xterm emulators.
It's all the other "terminals" for Linux, especially the libvt ones, Kitty, and Alacritty, that are more like shitty xterm emulators.
Yes, correct, but to disprove my equation, you should show what Wine does differently to not meet the criteria to be called emulation.
Honest question: does xterm emulate teletypes by hooking in shared libraries and re-implementing functions? This is what I understand wine is doing
xterm hooks into the tty subsystem by creating a pseudoterminal, not into shared libraries. The handling of control sequences and the buffer is re-implemented in C.
Different kinds of emulators have different kinds of interfaces they emulate.
Different kinds of emulators have different kinds of interfaces they emulate.
From `man xterm`:
xterm - terminal emulator for X
Wine (formerly a recursive backronym for Wine Is Not an Emulator (...))[deleted]
Neither wine nor xterm have virtualized cpu or memory, they both imitate (= emulate) some system via their interfaces, yet one is primarily an emulator and the other is not one. That does not make sense.
The wine project changed name to avoid the bad reputation emulators had back then due to their poor performance. "Wine is not an emulator" is not factually accurate, its a social workaround.
The wine project changed name to avoid the bad reputation emulators had back then due to their poor performance. "Wine is not an emulator" is not factually accurate, its a social workaround.
Words can have different meanings in different contexts. "Wine is not an emulator" was a very useful tagline to indicate that it was fast, as opposed to virtual machines like Virtual PC, PCTask etc.
"My bicycle is not a vehicle because it doesn't have a engine"
> To execute a Windows .exe then you just load it into memory (which requires unpacking the .exe file format) and tell the processor to jump to the first instruction.
Obviously a simplification, worth mentioning maybe at a slightly deeper level https://learn.microsoft.com/en-us/cpp/c-language/thread-loca...
I think a PE32 (.exe) can have multiple sections like this that are implicitly started before main()/entrypoint basically
Obviously a simplification, worth mentioning maybe at a slightly deeper level https://learn.microsoft.com/en-us/cpp/c-language/thread-loca...
I think a PE32 (.exe) can have multiple sections like this that are implicitly started before main()/entrypoint basically
TLS callbacks are a relatively common trick used by packers/obfuscators. AFAIK, the only other code that can get executed before the main entrypoint of the PE would be the entrypoints ("DllMain") of its imported DLLs.
Lots of 32-bit Windows apps shipped with 16-bit installers (old versions of InstallShield and the like). When MS dropped Win16 support from x64 Windows, they built native implementations which "ran" the 16-bit installer based on the resources packed into it. In console emulation terms, they supported 16-bit InstallShield and friends with a HLE (High Level Emulation) approach rather than continuing to execute the original binaries on top of a low-level emulation.
For a hobby/minimalist Win32 implementation I wonder if HLE support for popular packers/obfuscators would be easier than getting the low level details right?
For a hobby/minimalist Win32 implementation I wonder if HLE support for popular packers/obfuscators would be easier than getting the low level details right?
> In the specific case of Windows the kernel interface is pretty fiddly (the syscall ids vary between Windows releases)
Interesting - I wouldn't have expected that. Anyone know why that's the case?
Interesting - I wouldn't have expected that. Anyone know why that's the case?
Windows never used syscalls officially - the Windows API evolved from essentially having the operating system be inside few shared libraries that your program linked with. Even once the idea of syscall started showing up with protected mode and NT, the applications wouldn't directly invoke them, but through interfaces in the libraries.
Thus there was never really a proper need to have stable syscall interface, because at least officially no application was supposed to use them - they were private implementation details of various DLLs.
Thus there was never really a proper need to have stable syscall interface, because at least officially no application was supposed to use them - they were private implementation details of various DLLs.
These days, you have DRM solutions doing syscalls directly. I think Jurassic World evolution is an example of software using DRM that does that. The wine developers had to implement the ability to emulate windows syscalls because of it.
I wonder how Microsoft will react to third parties beginning to do syscalls.
I wonder how Microsoft will react to third parties beginning to do syscalls.
"beginning to"? This has been around since the early NT days, basically as long as DRM (and thus anti-DRM) has existed. Microsoft's reaction? Nothing but occasionally changing the syscall numbers.
I'm shocked that that works - I was given to understand that MS outright shuffles syscalls per build to make them completely unreliable from userspace.
A lot of DRM (especially game DRM) solutions don't care about reliability or stability.
If they figure a way that makes it harder to reverse or run or whatever, they will do it. Even if they are essentially rootkits bad enough someone really should try to see if CFAA would stick.
If they figure a way that makes it harder to reverse or run or whatever, they will do it. Even if they are essentially rootkits bad enough someone really should try to see if CFAA would stick.
In particular, Windows 9x has a completely different syscall interface (if you could even call it that) than the NT line; the majority of kernel32 functions seem to go through a far call to a fixed address:
https://news.ycombinator.com/item?id=13264007
The above-linked handful of pages may be the only detailed documentation of the 9x syscall interface that is still reachable by search engines today.
https://news.ycombinator.com/item?id=13264007
The above-linked handful of pages may be the only detailed documentation of the 9x syscall interface that is still reachable by search engines today.
Interestingly they started in more or less in alphabetical order. It certainly looks for the outside like there's a big file that gets updated wherever the dev adding the syscall thinks looks clean, and gets reorganized every now and then.
Combined with the obvious 'it's not a stable interface so we like changing it up regularly so people don't feel like they can depend on it'.
Here's the table itself for 32 bit x86 versions if anyone's curious: https://j00ru.vexillium.org/syscalls/nt/32/
Combined with the obvious 'it's not a stable interface so we like changing it up regularly so people don't feel like they can depend on it'.
Here's the table itself for 32 bit x86 versions if anyone's curious: https://j00ru.vexillium.org/syscalls/nt/32/
IIRC it's the other way around - it's Linux exposing syscalls for direct access that's unusual.
Yep; Darwin doesn't do raw syscalls, the BSDs don't (OpenBSD going so far as to block anything but libc from making syscalls, as a security feature), illumos doesn't... actually the only OS I can think of off-hand where applications regularly use syscalls is DOS, and even then it looks more like a side-effect of x86 architecture and the constraints that it was developed under. Go hit this particularly hard, having mostly grown up on Linux and then finding out the hard way that using libc isn't really optional on other systems.
On well designed systems (and I will count Windows in that group), libc is optional and syscalls are decoupled into another library. After all (unlike what some BSDs think) not everything is C, and forcing everyone to use the libc abstractions isn't very nice.
DOS syscalls are an outgrowth of the CP/M hardware abstraction. It isn't tied to x86.
I'm pretty sure Windows syscall ids are automatically generated in the build process, with no special effort to keep them consistent between releases. Since the code that receives syscalls (kernel) and the code that calls them (shared libraries like user32.dll or ntdll.dll) are shipped together that doesn't cause any problems, and is probably more convinient than manually mapping numbers to function calls.
Applications are not supposed to ever use them, but rather make use of the system libraries to access them. So having stable IDs is somewhere between not a priority and something to be avoided to stop developers ignoring that rule.
Imagine if WINE had become the defacto cross platform standard, instead of Electron. Or some modernizing shim on top of it.
The amount of memory and CPU we could have saved.
The amount of memory and CPU we could have saved.
I truly believe that the success of Electron is not limited to only being cross-platform. It is a very compelling motivation, really. But I think one of the key aspects is that the HTML/CSS/JS ecosystem is so huge for building UIs that is a lot simpler than to rely on the classic UI desktop frameworks
You’re correct. We need to solve the resource intensive issue while still allowing people to write apps with a web stack
Wine doesn’t help you on iOS or the web. It could never have been as cross platform as a browser
Yeah, I though about mobile as soon as I hit “add comment”. Still, there must be a better way. I'm rooting for all the lighter weight alternatives.
You're not trying hard enough.
> if you have the source code of a Windows program you can compile it against Wine's implementation of the Windows API and get a native executable out the other side
Interesting if anyone tried this on Notepad++.
Interesting if anyone tried this on Notepad++.
Good luck with this. Looks interesting.
This is also the approach of qemu user-space, which went unmentioned in the article. It can emulate another ISA and translate the syscalls to your host version of Linux. This way you can run x86 Wine on, say, ARM or RISC-V systems. From a user perspective, if everything is set up so, Windows applications for x86 will run just as well as ARM Linux applications.