Microsoft Has Manually Patched Their Equation Editor Executable(0patch.blogspot.com)
0patch.blogspot.com
Microsoft Has Manually Patched Their Equation Editor Executable
https://0patch.blogspot.com/2017/11/did-microsoft-just-manually-patch-their.html
164 comments
Modern compilers aren't that much better at code golf.
I tried equivalent C code and gcc-7.2 gets me 47 bytes, while clang-6.0 only manages 49 bytes (both with -m32 f.c -Os -fomit-frame-pointer).
I have a feeling that size optimization just isn't really important to (at least open source) compiler writers these days. There are more important things, like actual performance, standards compliance and nice diagnostics.
I tried equivalent C code and gcc-7.2 gets me 47 bytes, while clang-6.0 only manages 49 bytes (both with -m32 f.c -Os -fomit-frame-pointer).
I have a feeling that size optimization just isn't really important to (at least open source) compiler writers these days. There are more important things, like actual performance, standards compliance and nice diagnostics.
It’s intriguing to me that this is so, given that most of the point of writing native code these days (rather than targeting some managed-native system like the CLR) is to optimize hot loops, and one of the best optimizations for hot loops is to get them to fit entirely into a cache line. Do compilers for C/FORTRAN/etc. not have any mode or pragma to indicate that you’re attempting to get this performance benefit from a given function-and-its-dependencies?
How does this go with the often quoted mantra that you can only beat compilers today if you're an extremely skilled asm programmer? Or is the problem you describe just about executable size rather than speed?
Optimizing for size is easier because you only have exactly one metric to consider: how many bytes your instructions take.
When optimizing for speed you have to consider many factors like the relative speed of each instruction, cache behavior (including size of the cachelines, associativity, number of layers, relative speed of the layers...), pipelining, branch prediction, prefetching, whether moving your data to SIMD registers could be worth it, what to inline and what not to inline, what to unroll and what not to unroll, constraint solving to optimize things that can be computed or asserted statically etc...
When optimizing for speed you have to consider many factors like the relative speed of each instruction, cache behavior (including size of the cachelines, associativity, number of layers, relative speed of the layers...), pipelining, branch prediction, prefetching, whether moving your data to SIMD registers could be worth it, what to inline and what not to inline, what to unroll and what not to unroll, constraint solving to optimize things that can be computed or asserted statically etc...
And that's for a single processor! There's myriad CPUs the end user could be using.
x86 and x64 are the only two that someone could be using on desktop, right?
Timing rules can be very different even between different models of the same processor, let alone between different ranges (i3 vs i7) or generations (Skylake etc). An example: https://gmplib.org/~tege/x86-timing.pdf
Different microarchitectures can have big differences in performance between different instructions.
Instruction set makes much less of a difference than the actual microarchitecture. For an extreme example, see Pentium 4 vs Core. Something that runs fast on one could be dramatically different on the other.
The only time ISA really influences optimization is for unique ones like IA-64/Itanium. Otherwise, optimizing for e.g. a modern Xeon vs a POWER8 is not terribly different.
The only time ISA really influences optimization is for unique ones like IA-64/Itanium. Otherwise, optimizing for e.g. a modern Xeon vs a POWER8 is not terribly different.
How many cores? How big is each cache?
Well, the code wasn't compiled by today's compiler, it was compiled in late 2000. Visual Studio 6 maybe?
Even today compilers tend not to optimise the function preamble/postamble away. I'm only half in agreement with the mantra: you probably can beat the compiler, but is it worth it?
There are a few situations where it's genuinely a good idea to write in assembler to be explicit about predictable behaviour. Short security-critical constant-time functions are a good candidate.
Even today compilers tend not to optimise the function preamble/postamble away. I'm only half in agreement with the mantra: you probably can beat the compiler, but is it worth it?
There are a few situations where it's genuinely a good idea to write in assembler to be explicit about predictable behaviour. Short security-critical constant-time functions are a good candidate.
There are a lot of assembly language instructions that do slightly different things than standard C++ or C, but if the programmer is aware of them they can "handle" the differences.
For example, the xchg instruction doesn't have any C equivalent. (although it has a C++ equivalent: std::swap) The programmer may see:
---------------
Most of the low hanging fruit have been taken up for sure. Almost every "memcpy" turns into "rep stos" for example (which is the assembly-language equivalent to memcpy).
A high-level programmer may not know that "memcpy" turns into "rep stos" however, and may emit his own memory copying for-loop.
At very least, a good optimizing C / C++ programmer needs to know about these little things. They'll let the compiler turn "memcpy" into "rep stos" (for -Os) or AVX memory store instructions respectively instead of writing their own less efficient loops on the matter.
For example, the xchg instruction doesn't have any C equivalent. (although it has a C++ equivalent: std::swap) The programmer may see:
A ^= B; B ^= A; A ^= B
---------------
Most of the low hanging fruit have been taken up for sure. Almost every "memcpy" turns into "rep stos" for example (which is the assembly-language equivalent to memcpy).
A high-level programmer may not know that "memcpy" turns into "rep stos" however, and may emit his own memory copying for-loop.
At very least, a good optimizing C / C++ programmer needs to know about these little things. They'll let the compiler turn "memcpy" into "rep stos" (for -Os) or AVX memory store instructions respectively instead of writing their own less efficient loops on the matter.
Optimising for size is a relatively "obvious" goal, although it still takes a lot of skill to do it well. Optimising for speed is much less obvious however, the x86 architecture is incredibly complex when it comes to working out what code will be faster.
Well, it should also be noted that the responsible compiler in this case is at least 17 years old.
> How does this go with the often quoted mantra that you can only beat compilers today if you're an extremely skilled asm programmer? Or is the problem you describe just about executable size rather than speed?
Word 2000, so a 17+ year old compiler. Compilers have gotten a lot better since then.
Having worked on a compiler team back in the mid 2000s, even then I'd say it was easy for almost anyone to spot areas where a human could optimize more.
Now days, much less so.
Word 2000, so a 17+ year old compiler. Compilers have gotten a lot better since then.
Having worked on a compiler team back in the mid 2000s, even then I'd say it was easy for almost anyone to spot areas where a human could optimize more.
Now days, much less so.
This is a case of knowing the rules so well that you know when you can break them.
Its also a historical artifact from the days when many programmers wrote assembly yet compilers started getting good.
There’s also an element of avoiding premature optimization: don’t assume the compiler will product slower code or that if it does it will matter in your specific application.
At the very least you should give the compiler a chance, profile, then hand-tune after you’ve fixed all the low-hanging fruit.
Its also a historical artifact from the days when many programmers wrote assembly yet compilers started getting good.
There’s also an element of avoiding premature optimization: don’t assume the compiler will product slower code or that if it does it will matter in your specific application.
At the very least you should give the compiler a chance, profile, then hand-tune after you’ve fixed all the low-hanging fruit.
That mantra applies to "most" programmers.
I think he was talking mostly about size.
Odds are good most programmers tinkering in machine code won't beat the performance of the compiler. That takes experience. It is a good rule of thumb.
I think it is easier to write smaller (size) code than a compiler, but when you measure performance it will beat you often until you get good. Alignment, x86 tricks... It takes a bit of knowledge to do well.
I think he was talking mostly about size.
Odds are good most programmers tinkering in machine code won't beat the performance of the compiler. That takes experience. It is a good rule of thumb.
I think it is easier to write smaller (size) code than a compiler, but when you measure performance it will beat you often until you get good. Alignment, x86 tricks... It takes a bit of knowledge to do well.
simias has most of it but note also that that file appears to have last been compiled in the early 2000s. Compilers of that era were far less advanced, especially since many large companies were pretty conservative about the optimizations enabled (fixing a bug meant mailing CDs for many customers).
The general trend is that it's been getting harder and harder to do that easily, which means people want to be more focused — something like OpenSSL can still justify hand-tuned assembly for various processor families because it's a widespread hotspot but as compilers continue to improve the number of places where it's worth the maintenance cost is going to keep shrinking.
In the early 2000s, the scientific HPC programmers I worked with were careful to maintain a portable C implementation which they could use as a check both for correctness and for an optimization baseline — it wasn't uncommon for a new compiler and/or processor to substantially close the gap relative to a lot of hard manual work.
The general trend is that it's been getting harder and harder to do that easily, which means people want to be more focused — something like OpenSSL can still justify hand-tuned assembly for various processor families because it's a widespread hotspot but as compilers continue to improve the number of places where it's worth the maintenance cost is going to keep shrinking.
In the early 2000s, the scientific HPC programmers I worked with were careful to maintain a portable C implementation which they could use as a check both for correctness and for an optimization baseline — it wasn't uncommon for a new compiler and/or processor to substantially close the gap relative to a lot of hard manual work.
"Note that in 32-bit code, memcpy is typically implemented by first copying blocks of 4 bytes using the movsd (move double word) instruction, while any remaining bytes are then copied using movsb (move byte)."
Some software authors do not use memcpy().
https://marc.info/?l=djbdns&m=96477313901746&w=2
http://cr.yp.to/lib/byte.html
Some software authors do not use memcpy().
https://marc.info/?l=djbdns&m=96477313901746&w=2
http://cr.yp.to/lib/byte.html
#include "byte.h"
void byte_copy(to,n,from)
register char *to;
register unsigned int n;
register char *from;
{
for (;;) {
if (!n) return; *to++ = *from++; --n;
if (!n) return; *to++ = *from++; --n;
if (!n) return; *to++ = *from++; --n;
if (!n) return; *to++ = *from++; --n;
}
}In fact, REP MOVSD is harder to handle in hardware especially for unaligned locations because you can only interrupt on 4 byte boundries.
Binary hacking FTW.
The semi-official Debian server, alioth.debian.org, where a lot of random developer stuff is hosted, is stuck on Debian wheezy for various reasons. Most users, including myself (a Debian Developer) don't have root access to upgrade the server nor install new software.
The version of libapt-inst is too old to support Debian packages with control.tar.xz members (only control.tar.gz members). So we can't upload newer Debian packages to various custom APT repos that we host on that server.
I worked around this by looking at the libapt-inst source code, figuring out how to make it support control.tar.xz instead of control.tar.gz, and binary-patched libapt-inst.so to have this effect instead. It's actually fairly simple
1. there is a check for control.tar.gz, the failure branch prints an error and then returns. I overwrite this with NOP so it goes into the "success" branch.
2. then later it extracts the control.tar.gz member and pipes it through gzip. Luckily, nowhere else in the program uses the exact string "control.tar.gz" or "gzip" so I simply patch that string "control.tar.gz" -> "control.tar.xz" in the binary and also change "gzip" -> "xz\0\0".
(Actually given the change in (2), (1) is not necessary. But without it you get a bunch of spurious error messages.)
Applying this patch makes the resulting .so lose the ability of working with old control.tar.gz members (which is still needed of course). So my workaround does this:
LD_PRELOAD=libapt-inst.so.patched apt-ftparchive [..] && apt-ftparchive [..]
i.e. runs it once with the hack to pick up the new-style debs, and once again without the hack to pick up the old-style debs.
My motto is, "dirty solutions for dirty problems". :D :D :D
The semi-official Debian server, alioth.debian.org, where a lot of random developer stuff is hosted, is stuck on Debian wheezy for various reasons. Most users, including myself (a Debian Developer) don't have root access to upgrade the server nor install new software.
The version of libapt-inst is too old to support Debian packages with control.tar.xz members (only control.tar.gz members). So we can't upload newer Debian packages to various custom APT repos that we host on that server.
I worked around this by looking at the libapt-inst source code, figuring out how to make it support control.tar.xz instead of control.tar.gz, and binary-patched libapt-inst.so to have this effect instead. It's actually fairly simple
1. there is a check for control.tar.gz, the failure branch prints an error and then returns. I overwrite this with NOP so it goes into the "success" branch.
2. then later it extracts the control.tar.gz member and pipes it through gzip. Luckily, nowhere else in the program uses the exact string "control.tar.gz" or "gzip" so I simply patch that string "control.tar.gz" -> "control.tar.xz" in the binary and also change "gzip" -> "xz\0\0".
(Actually given the change in (2), (1) is not necessary. But without it you get a bunch of spurious error messages.)
Applying this patch makes the resulting .so lose the ability of working with old control.tar.gz members (which is still needed of course). So my workaround does this:
LD_PRELOAD=libapt-inst.so.patched apt-ftparchive [..] && apt-ftparchive [..]
i.e. runs it once with the hack to pick up the new-style debs, and once again without the hack to pick up the old-style debs.
My motto is, "dirty solutions for dirty problems". :D :D :D
dirty solutions for dirty problems :):)
> The semi-official Debian server, alioth.debian.org, where a lot of random developer stuff is hosted, is stuck on Debian wheezy for various reasons
Jeez. How is security maintained? That actually scares me a bit.
Jeez. How is security maintained? That actually scares me a bit.
Wheezy is supported until the end of May 2018, so it still gets security patches.
I'm surprised no one has noted the copyright is to Design Science - this is a small company in my hometown who are still around. I've spoken with their CEO a few times and I wouldn't be at all surprised if the source code was lost, or somehow at least wasn't being made available to Microsoft (I doubt it ever was). It's a really old school shop who seems to have largely been coasting on the licensing of this one component for the past couple decades and I wouldn't at all be shocked to find they no longer are capable of maintaining it themselves.
I noted it - thanks for the background info on the company! I also assume that either they are not able to maintain the software themselves, or they have lost the source code, but it might also be that setting up the toolchain to compile such an old piece of software is more effort than just patching the binary.
This is such an underappreciated aspect of code stewardship. There are powerful tools for source control and archiving. But ensuring that state of code could actually be built at an arbitrary date in the future is so much less assured.
I agree here.
I would guess the build environment involves lots of dependencies, lots of special config, lots of stuff which has to be the exact correct version, and all that knowledge has been lost as people have left the team and it wasn't properly documented.
Sure, you could spend a couple of weeks setting up a suitable environment again and relearning everything from scratch, but binary patching is probably easier.
I would guess the build environment involves lots of dependencies, lots of special config, lots of stuff which has to be the exact correct version, and all that knowledge has been lost as people have left the team and it wasn't properly documented.
Sure, you could spend a couple of weeks setting up a suitable environment again and relearning everything from scratch, but binary patching is probably easier.
According to an Ars comment: "I've got an older version of Mac Office (2011 I think), and there's a version of Equation Editor in there with a 1990-2010 design science copyright on it, so they have some version of newer code they could swap the old office one for."
Design Science still develops and sells MathType, the "pro" version of the Equation Editor licensed to Microsoft.
They also make other software meant to make math more accessible to people with various disabilities.
https://www.dessci.com/en/
They also make other software meant to make math more accessible to people with various disabilities.
https://www.dessci.com/en/
I once worked at a place which lost part of the source code for their giant mission-defining application. They spent a decade linking in object code for which there was no corresponding source code.
The build team was very proud when they announced that the application would finally start being built from the source code in version control.
Stuff happens!
The build team was very proud when they announced that the application would finally start being built from the source code in version control.
Stuff happens!
Stuff happens, indeed, and more often than most of us realise.
Getting on for a decade ago now I was working at Red Gate when they bought .NET Reflector - a decompiler for .NET code - from Lutz Roeder. After the acquisition we started asking people what they were using it for.
Turns out a significant minority of them were trying to recover lost source code, or source code they never had in the first place (e.g., where a supplier went out of business). I don't remember the exact figure but it might even have run into low double-digit percentages. Bear in mind this is a tool that was being downloaded tens of thousands of times every month by all manner of people working for all kinds of organisations of every size and you can see the scale of the problem.
There were a couple of Reflector add-ins that would allow you to take a .NET binary and generate a C# or VB.NET Visual Studio project with all source code from it. The source code was never perfect and wouldn't likely compile first time, but it was certainly better than starting from scratch. Not surprisingly these add-ins were among the most popular.
Granted, times have changed, and I think source control is probably the default for almost everyone these days - although I would have expected that even in 2008 - but, bottom line: I think this sort of thing happens a lot, for one reason or another.
Getting on for a decade ago now I was working at Red Gate when they bought .NET Reflector - a decompiler for .NET code - from Lutz Roeder. After the acquisition we started asking people what they were using it for.
Turns out a significant minority of them were trying to recover lost source code, or source code they never had in the first place (e.g., where a supplier went out of business). I don't remember the exact figure but it might even have run into low double-digit percentages. Bear in mind this is a tool that was being downloaded tens of thousands of times every month by all manner of people working for all kinds of organisations of every size and you can see the scale of the problem.
There were a couple of Reflector add-ins that would allow you to take a .NET binary and generate a C# or VB.NET Visual Studio project with all source code from it. The source code was never perfect and wouldn't likely compile first time, but it was certainly better than starting from scratch. Not surprisingly these add-ins were among the most popular.
Granted, times have changed, and I think source control is probably the default for almost everyone these days - although I would have expected that even in 2008 - but, bottom line: I think this sort of thing happens a lot, for one reason or another.
Heh can say I know some guys who have done this, and I myself have done this (and with similar but open source tools) there's also the "what is this sketchy .NET app really doing" moment where you want to know it's not doing anything "funny" to your system and you peek at the code.
I've done a significant amount of work with decompiling and rebuilding executables of crazy levels of complexity. It's definitely time consuming -- but not as bad as you'd think. Maybe 1-4 weeks with a dedicated team working through it and testing functionality. Definitely a viable solution if you've lost source code.
Yeah we used ILSpy for exactly this reason - a vendor went out of business and our client needed patches REAL QUICK.
A friend once told me a story about a software company that had offices in the World Trade Center in New York. Their offices were totally destroyed by 9/11; thankfully, all the staff got out alive, but it turned out they didn't have offsite backups of the source code repository, and it was lost completely. They found various bits of the source code floating around (e.g. some developers had bits of it on their home computers), but there were a few key components they could not locate any source for. Well, the customers still had the compiled binaries, so they got the binaries back from the customers, extracted the missing bits, ran them through a decompiler, and checked the result into the source code repository – since the application was written in Java, this actually worked quite well. Years later, new developers would find bits of obviously decompiled code still in the source repo (you can tell, it has a distinct look to it, e.g. variable names with numbers in them), and scratch their heads, and then get told the tale.
Stuff may happen but I would put replacing the module I lost to the source code to pretty high on the list of things that need fixing.
There is no way I could rely on something like that.
Am I the only one that thinks relying on something that has no source code is just asking for trouble and headaches in the future?
There is no way I could rely on something like that.
Am I the only one that thinks relying on something that has no source code is just asking for trouble and headaches in the future?
Then there is the situation where, one day, your source code control system tells you there are no files .... because someone in the org you are working at figured noone was using the server and wiped it. This really happened on a job I was on. Luckily we still had source dotted around a dozen or so machines, at various levels of up-to-datedness, so it was possible (with a load of scripting code generating some helpful timeline pix) to forensically reconstruct not just the tip but a decent portion of the history too. Fun times.
No, you're not the only one. But, at the same time, it isn't always the absolute top priority. Right now I'm working on a couple of systems that have had a few mystery DLLs in them. Not necessarily anything lacking source code - it's around "somewhere" - but certainly things we've lacked the immediate capability to rebuild.
But those DLLs aren't the only, or the biggest, problems with the systems. Hence finding the source and building them from it isn't necessarily the top priority, although we are progressively doing exactly this.
But those DLLs aren't the only, or the biggest, problems with the systems. Hence finding the source and building them from it isn't necessarily the top priority, although we are progressively doing exactly this.
> They spent a decade linking in object code for which there was no corresponding source code.
How would you go an entire decade without noticing this?
Wouldn't you have to use that missing source code for something within ten years?
How would you go an entire decade without noticing this?
Wouldn't you have to use that missing source code for something within ten years?
I only know details at the level of war story, secondhand. I may have overstated a bit. For sure there was a build process relinking in old object code for many years which no one knew how to reproduce. Possibly there was still associated source code, but the object bits had been declared "golden" and no one knew exactly what source version or build process had produced those "golden" bits.
I've had something similar before at a prior employer caused by bad linking of some C# to some C++. Someone thought it was a good idea to P/Invoke directly the mangled names. Needless to say that changed once I was finally able to get my hands on the source.
Oh, undoubtedly it was noticed, but an enterprise software company has a tremendous capacity to procrastinate on fixing things.
Likely the source was used to build an object library which then got linked in to form the executable. If the library Just Worked, there would be no reason to rebuild it.
There's only two reasons I can think of why they'd patch the binary directly: either they've lost the source-code, or they no longer have an environment they can build it in.
For ages Alan Kay has been claiming to know that MS has lost part of the Word codebase.
In this case they may never have had the source, the copyright is for Design Science, Inc which makes MathType a popular equation editor for Word. Depending on how this was licensed this may have even technically been illegal (due to copyright issues); although I suspect MS has a license that allows this.
I've never heard him claim that MS actually lost the source code to Word, only that they couldn't find the cause of a decades-old bug (related to selection and text-justification as I recall), even though it had thousands of bug reports, and that they spent years looking for it before giving up.
The point was that the code is too big and complicated, even for some of the best engineers in the world.
The point was that the code is too big and complicated, even for some of the best engineers in the world.
[deleted]
Microsoft has lost tons of code over the years. Even with the source, refactoring office which has people using file formats that are binary dumps of memory, is not trivial.
This was actually the Onus for the switch to the XML formats IIRC. Basically after the DOJ settlement they had the uneasy realization that the requirement to document the .doc format was going to be a nightmare because nobody had a complete spec of it. To make it worse the code wasn't very portable and customers were asking for x64 support pretty hard at the time.
Re: 64 bit is that about Excel? I found the idea of a Word document needing >4GB RAM somewhat absurd
Note that due to the way the 32-bit address space was laid out, a process could really use only about 2GB of RAM — the extra 2GB was reserved for the system based on old 386-era CPU limitations. Since that space had various things allocated in it, a normal program generally wouldn't get more than 1.7GB in a single allocation — e.g. according to https://support.microsoft.com/en-us/help/313275/-not-enough-... Excel 2003 had a heap limit of about 1GB.
The other thing to keep in mind is that Office documents are, in addition to being somewhat bloated internally, far more than just text. Think about all of the people writing things like documentation with hundreds of screenshots (all uncompressed BMPs), audio or video, etc. — and the people with those are far more likely to be at the kind of large corporations which expect value from their support contracts. There were multiple ways to deal with that problem but in general the easiest was switching to a 64-bit address space.
The other thing to keep in mind is that Office documents are, in addition to being somewhat bloated internally, far more than just text. Think about all of the people writing things like documentation with hundreds of screenshots (all uncompressed BMPs), audio or video, etc. — and the people with those are far more likely to be at the kind of large corporations which expect value from their support contracts. There were multiple ways to deal with that problem but in general the easiest was switching to a 64-bit address space.
Recently they had to enable /LARGEADDRESSAWARE in 32-bit Outlook.
If you want to use a 64-bit COM/OLE object, such as from a VBA script, then it's extremely difficult unless the host is 64 bit.
I can imagine users just pasting images without a 2nd thought about their sizes.
I routinely update single page Powerpoints with more than 4 MB, due to the background image or logos.
I routinely update single page Powerpoints with more than 4 MB, due to the background image or logos.
Maybe they just don't know how to run the same version of the compiler any more, Visual C++ 4.2 or whatever they had... I remember it stopped working with some newer version of Windows.
There'd be nothing extraordinary about that, either. I've never worked for a company that could build something today that they released twenty years ago.
25 years old and I got it to compile and run when I re-found it last year!
https://github.com/linker3000/Historic-code-PC-Pascal-and-AS...
https://github.com/linker3000/Historic-code-PC-Pascal-and-AS...
This do make a lot of sense. I don't think there is exe calls directly linked from other apps. If that were the case they would just update the address in the calling app. Much easier. Anyway, very nice piece of work. :)
Another reason could be that it has dependencies that link to specific addresses in the exe. It's very peculiar that they made the effort to keep all the original adresses.
It's not an effort, it's probably a side effect of simply patching the binary.
If you can't rebuild it, you have to manually edit it, and when you do that you can't really change the addresses, not without a lot of headache.
If you can't rebuild it, you have to manually edit it, and when you do that you can't really change the addresses, not without a lot of headache.
It's keeping the adresses that requires the least effort, changing them would have implied to also change all calls and jumps in the rest of the code, which could mean a very large number of addresses to change.
That's a really good point.
The most likely reason is there's a heavy handed government agency that relies on all the base addresses to be the same, who buys lots of Microsoft licenses.
Yes, the NSA is surveilling the populace 24/7 through Microsoft Equation Editor.
One other reason might be that they licensed part of the code from a third party and are no longer allowed to re-distribute it. I suppose patching out a few bytes might get them out of legal jail.
While this does suggest that they lost the source code for this program, it also shows an unbelievable amount of skill.
HN, where writing assembly shows an unbelievable amount of skill.
Well, when you have people calling Electron based apps native.....
Not just writing assembly, rewriting a compiled object file without letting any of the addresses change, without having the source to work with, and presumably with almost no documentation, to patch a program that has been left untouched for almost 20 years.
Those properties come by definition. Addresses don't change because you can't (realistically) change things any much and move things around.
Basically, think of it like this: you have an old book, written in some magic runes and is told that a certain (quite short) paragraph is wrong and can be badly misinterpreted due to poor wording. You know, the magic spell goes kaboom.
You have tools that can easily and painlessly: a) translate the runes into a text that mere mortal can reasonably easily read; b) scrub runes off the page and write any new ones over; c) translate your text into runes. There is a simple correspondence between text length and how many runes it would take. Now, all you need is to write the new text that must be no longer than old one was. Not a trivial task, but not something extraordinary. Just rare, because we don't deal with magic runes those days as familiars take orders and handle all the gory details.
Writing assembly requires a skill. So does reading old assembly code of a particular function and figuring out what it did. That's admirable, but not something unbelievable.
It's just rarer to see those days, but not a lost art or anything like that. Crackmes are still alive. On the game cheat forums such patching (albeit, for a different purposes) was the norm, and probably still is for the games that aren't protected. And many embedded developers have their fights for code size quite often.
Basically, think of it like this: you have an old book, written in some magic runes and is told that a certain (quite short) paragraph is wrong and can be badly misinterpreted due to poor wording. You know, the magic spell goes kaboom.
You have tools that can easily and painlessly: a) translate the runes into a text that mere mortal can reasonably easily read; b) scrub runes off the page and write any new ones over; c) translate your text into runes. There is a simple correspondence between text length and how many runes it would take. Now, all you need is to write the new text that must be no longer than old one was. Not a trivial task, but not something extraordinary. Just rare, because we don't deal with magic runes those days as familiars take orders and handle all the gory details.
Writing assembly requires a skill. So does reading old assembly code of a particular function and figuring out what it did. That's admirable, but not something unbelievable.
It's just rarer to see those days, but not a lost art or anything like that. Crackmes are still alive. On the game cheat forums such patching (albeit, for a different purposes) was the norm, and probably still is for the games that aren't protected. And many embedded developers have their fights for code size quite often.
It is easy to not let addresses change, because compilers without “-O2+“ do lots of extra stack ops. Documentation is not needed there, because it is overflow fix, it is catchable by debugger and both caller and callee are right in bt. And the fact that this program was not recompiled for 20 years actually adds to the possibility of what was done. Modern compilers are much less forgiving.
Anyway, your points are pretty weak and oh-magic-driven, and I don’t see any reason gp comment to be gray or work to be called stellar. Though of course it was done by asm-skilled person.
Anyway, your points are pretty weak and oh-magic-driven, and I don’t see any reason gp comment to be gray or work to be called stellar. Though of course it was done by asm-skilled person.
I think it's easy in all the superficial churn of frameworks and languages to forget how much depth there is in our field. To me, writing a compiler isn't that big a deal anymore; it's the kind of exercise I might use to try a new language out. But when I was a sophomore in college, even after a few years of computer usage, it would have been magic to me.
It takes a lot of work to get to the point of skill demonstrated in this article... but there's still a lot of skill runway beyond that level of skill, too. It's simultaneously true that this is an impressive amount of talent, and that there are people for whom this would be an entertaining momentary side diversion from their normal job.
It takes a lot of work to get to the point of skill demonstrated in this article... but there's still a lot of skill runway beyond that level of skill, too. It's simultaneously true that this is an impressive amount of talent, and that there are people for whom this would be an entertaining momentary side diversion from their normal job.
I kind of agree with the sentiment. It isn't that crazy.
We do this as a matter of course all the time. Patching a small handful of instructions is pretty easy. You could learn to do it on a week or less if you are a decent programmer.
Do it well? Do it quickly? Do it idiomatically and in a short amount of time.... Takes real skill.
We do this as a matter of course all the time. Patching a small handful of instructions is pretty easy. You could learn to do it on a week or less if you are a decent programmer.
Do it well? Do it quickly? Do it idiomatically and in a short amount of time.... Takes real skill.
I used to patch games for infinite-lives, or to allow my serial numbers to be accepted. Doing this wasn't hard, as somebody who grew up writing assembly language on 8-bit machines in the 80s.
One fun self-challenge was always to make my modifications as small as possible. e.g. one-byte changes were a lot more impressive than two-byte changes.
One fun self-challenge was always to make my modifications as small as possible. e.g. one-byte changes were a lot more impressive than two-byte changes.
Your own personal game genie.
It's interesting. I have observed if people learn on an 8 or 16 bit machine, like in Microcorruption, they tend to be able to pick up more complex ISAs much easier. It helps to know the first principles.
It's interesting. I have observed if people learn on an 8 or 16 bit machine, like in Microcorruption, they tend to be able to pick up more complex ISAs much easier. It helps to know the first principles.
It is indeed a lost art. I can count with just one hand the amount of colleagues that I know that are capable of doing this. Also this is not assembly, it is object code.
Also this is not assembly, it is object code.
Disassemblers exist. You can take the binary, generate the assembly code, fix it and then re-compile to find the needed changes. I cracked a few sharewares with OllyDbg this way (just for fun, never distributed), and I'm no "leet coder".
Disassemblers exist. You can take the binary, generate the assembly code, fix it and then re-compile to find the needed changes. I cracked a few sharewares with OllyDbg this way (just for fun, never distributed), and I'm no "leet coder".
Would the assembler maintain the same binary size and the exactly the same address module in the same order just like the article claims?
What they said is they found evidence that the binary was modified manually.
This is way more tedious that disassembling and reassembling a binary.
What they said is they found evidence that the binary was modified manually.
This is way more tedious that disassembling and reassembling a binary.
> This is way more tedious that disassembling and reassembling a binary.
It used to be stuff we did for fun.
Back in the day we might not even load the entire program into memory - I remember manually patching disk sectors on the C64 with tools that'd let me disassemble arbitrary content to see if it happened to match code.
I also spent a couple of years programming assembly directly in a "machine code monitor" - an assembler used to assemble/disassemble memory instruction by instruction rather than from a file.
This was something several members of my primary school class would do for entertainment.
The idea that this is particularly difficult just reflect that fewer developers have spent time getting familiar with assembly these days.
It used to be stuff we did for fun.
Back in the day we might not even load the entire program into memory - I remember manually patching disk sectors on the C64 with tools that'd let me disassemble arbitrary content to see if it happened to match code.
I also spent a couple of years programming assembly directly in a "machine code monitor" - an assembler used to assemble/disassemble memory instruction by instruction rather than from a file.
This was something several members of my primary school class would do for entertainment.
The idea that this is particularly difficult just reflect that fewer developers have spent time getting familiar with assembly these days.
> It used to be stuff we did for fun.
We still do! When I added Retroarch to my HTPC I wanted it to use the "ok" and "power" buttons on my remote instead of "enter" and "escape" which are only found on a keyboard. While I did contribute a patch to the Retroarch project, which I tested using a laptop, binary patching was much easier on the Raspberry Pi ARM binaries than figuring out the build system for LibreELEC (the binary patch drops support for enter/escape, so it's literally changing two bytes for the two keycodes).
We still do! When I added Retroarch to my HTPC I wanted it to use the "ok" and "power" buttons on my remote instead of "enter" and "escape" which are only found on a keyboard. While I did contribute a patch to the Retroarch project, which I tested using a laptop, binary patching was much easier on the Raspberry Pi ARM binaries than figuring out the build system for LibreELEC (the binary patch drops support for enter/escape, so it's literally changing two bytes for the two keycodes).
It stopped being fun for me when I moved to an x86 box, I'm afraid. Though I do get my share of asm thanks to my (very slow moving) Ruby compiler project, it's more painful than fun.
Yes, disassemblers will often write raw bytes directives (e.g. "db 72, 101, 108, 108, 111") if they can't disassemble the instruction, so you can get 1:1 by disassembling and reassembling; but I doubt this patch was done by doing that on the whole binary.
A tool designed for reverse engineering like OllyDbg will maintain everything, yes.
>This is way more tedious that dissembling and reassembling a binary.
That's not the case.
That's not the case.
To elaborate, you sketch out the assembly you need, assemble it and literally drop those new bytes in.
Tools like IDA Pro, Binary Ninja, and Hopper make this quite easy. A good hex editor and knowing the file offsets is also fine. This is seen as magic because it is a bit of a lost art, but it turns out to be easy to learn.
Check out "crackmes" if anyone has become interested in this topic of mangling binaries by hand. They are fun and you will get results quickly on the easier challenges.
Also check out Microcorruption CTF.
Tools like IDA Pro, Binary Ninja, and Hopper make this quite easy. A good hex editor and knowing the file offsets is also fine. This is seen as magic because it is a bit of a lost art, but it turns out to be easy to learn.
Check out "crackmes" if anyone has become interested in this topic of mangling binaries by hand. They are fun and you will get results quickly on the easier challenges.
Also check out Microcorruption CTF.
I wouldn't call it a lost art... Assembly is used many places, even for new projects. But it makes sense that assembly programming might seem impressive (or antiquated) to the HN crowd, which I have an impression is composed of a lot of newly grads, web developers, and comparatively few old hats.
>Also this is not assembly, it is object code.
It is a matter of hitting F4 in hiew.
It is a matter of hitting F4 in hiew.
I have no specific insight to this patch, but I do have personal experience binary patching a popular Microsoft product.
My patch was to the VC++ compiler nearly 20 years ago. We had source, and my fix was also applied to the source (which I'd imagine is still there today), but a binary patch also made sense in the short term.
The binary that I patched was used to build another important Microsoft product, and this bug was found late in the product cycle where any compiler change was risky.
We weren't 100% confident we had the exact sources used to build that version of the compiler (git would have been handy then), we only knew, plus or minus one day, what the sources where.
After carefully evaluating the binary patch versus the risk of building from uncertain source, the binary patch was taken to reduce risk.
I'm no reverse engineer, but this was a pretty interesting exercise in RE even though I had sources. I had no symbols, and the binary was optimized so that functions were not contiguous, cold paths were moved to the end of the binary. Just finding the code I needed to patch was not easy.
The code review was fun - a dozen or so compiler engineers reviewed the change on paper printouts - the most thorough review I've had in my career, and the only one that used paper.
To the best of my knowledge, this binary was never used to build anything other than that specific version of the product which I won't name - not that it matters really, the product is still in use, but that version is unlikely to be in use anywhere anymore.
My patch was to the VC++ compiler nearly 20 years ago. We had source, and my fix was also applied to the source (which I'd imagine is still there today), but a binary patch also made sense in the short term.
The binary that I patched was used to build another important Microsoft product, and this bug was found late in the product cycle where any compiler change was risky.
We weren't 100% confident we had the exact sources used to build that version of the compiler (git would have been handy then), we only knew, plus or minus one day, what the sources where.
After carefully evaluating the binary patch versus the risk of building from uncertain source, the binary patch was taken to reduce risk.
I'm no reverse engineer, but this was a pretty interesting exercise in RE even though I had sources. I had no symbols, and the binary was optimized so that functions were not contiguous, cold paths were moved to the end of the binary. Just finding the code I needed to patch was not easy.
The code review was fun - a dozen or so compiler engineers reviewed the change on paper printouts - the most thorough review I've had in my career, and the only one that used paper.
To the best of my knowledge, this binary was never used to build anything other than that specific version of the product which I won't name - not that it matters really, the product is still in use, but that version is unlikely to be in use anywhere anymore.
Thanks for sharing this. I suspected that "not being sure if you have the exactly right source code" could be a real world reason to patch a binary, and now I know.
That is both pretty impressive and horrific.
I wonder if they patched this way because they wanted to maintain as much binary compatibility as possible, or if they don't have the original source/couldn't reproduce the build process.
I wonder if they patched this way because they wanted to maintain as much binary compatibility as possible, or if they don't have the original source/couldn't reproduce the build process.
Horrific? This is what you do when you want to make sure you don't introduce any unintentional changes. Computers aren't magic, and there is nothing wrong about patching a binary.
Compiling the software with a modern compiler or linking to a modern runtime is very likely to bring obscure bugs in the codebase to the surface. It's pretty hard to replicate the entire build process that produced the original binary, even if they have the source code and everything else on hand.
Compiling the software with a modern compiler or linking to a modern runtime is very likely to bring obscure bugs in the codebase to the surface. It's pretty hard to replicate the entire build process that produced the original binary, even if they have the source code and everything else on hand.
> Horrific? This is what you do when you want to make sure you don't introduce any unintentional changes.
Horrific, because the average programmer would consider patching the binary a worst-case scenario.
> there is nothing wrong about patching a binary
I would only trust a skilled assembly programmer to do this task without creating other problems, and most businesses don't have those on retainer.
Horrific, because the average programmer would consider patching the binary a worst-case scenario.
> there is nothing wrong about patching a binary
I would only trust a skilled assembly programmer to do this task without creating other problems, and most businesses don't have those on retainer.
> Horrific, because the average programmer would consider patching the binary a worst-case scenario.
That says more about the average programmer than it does about the reasonableness of binary patches.
I used to table in MASM on the 8088 and have dabbled a little on microcontrollers (6800) as well. But the last time I looked at modern x86, I was pretty lost.
That says more about the average programmer than it does about the reasonableness of binary patches.
I used to table in MASM on the 8088 and have dabbled a little on microcontrollers (6800) as well. But the last time I looked at modern x86, I was pretty lost.
Admittedly, x86 assembler is a total clustefuck.
...and sometimes, compiling and linking takes even longer than just opening the binary with a hex editor and changing the right bytes. I've done this a few times with things like string constants that were slightly off, although after testing the binary, I change the source too.
>> It's pretty hard to replicate the entire build process that produced the original binary, even if they have the source code and everything else on hand.
I seem to recall Microsoft rebuilds everything from source daily (or weekly?). This is a common practice in companies with a large code base. "Nightly Builds" even come from Mozilla. The reason is simple - if you wait until it's time to release, you're gonna have problems. Even if a developer tests his changes locally, they need to be integrated in with everything else and retested.
If you're really serious about things you will have your tool chain under revision control. As a result you can reproduce binaries to the byte from source code, with the possible exception of embedded dates pulled in at compile time. This is actually a good fallback when someone finds a critical problem in old code - you start by finding the code and then verify by reproducing the affected binary from source. This is not technically hard, it requires discipline and best practices.
I seem to recall Microsoft rebuilds everything from source daily (or weekly?). This is a common practice in companies with a large code base. "Nightly Builds" even come from Mozilla. The reason is simple - if you wait until it's time to release, you're gonna have problems. Even if a developer tests his changes locally, they need to be integrated in with everything else and retested.
If you're really serious about things you will have your tool chain under revision control. As a result you can reproduce binaries to the byte from source code, with the possible exception of embedded dates pulled in at compile time. This is actually a good fallback when someone finds a critical problem in old code - you start by finding the code and then verify by reproducing the affected binary from source. This is not technically hard, it requires discipline and best practices.
Patching the binary manually means that now the source code and the binary are out of sync, and the changes will be lost if it is ever compiled from the source code again.
Its funny how very few people do fully self contained and reproducible builds. At least nixos tries hard.
Your QA process, including automates tests should ensure there were no unacceptable changes introduced.
Otherwise you ability to create and ship fixes in timely manner will be severely harmed by the fear of breaking things.
how many decades of software is your QA process, with its automated tests, covering?
Only 1 so far.
My point was mostly that binary patching is generally bad, and should be avoided. It could very well be the least-bad option in this particular case. Its certainly much better than leaving security holes unfixed.
They probably don't have the Office 97 or 2000 build pipeline around anymore. And back then for Office XP or 2003 copied the equation editor in binary form to the new repository.
Building a new binary means running a full QA against it, which is probably not cost-effective for such an old component. In contrast, this patch has exactly known impact.
I know it seems like magic to you lot, but it's a day's work if you've the right skills.
You have to know that the MSFT may not have the source code of Equation Editor, since it is a simplified version of MathType.
This ‘old’ equation editor is a limited version of MathType (https://en.wikipedia.org/wiki/MathType#Microsoft_Equation_Ed...) that has been supplanted by a built-in equation editor.
Chances are that Microsoft doesn’t have a license for bug fixes from Design Science (makers of MathType) anymore and isn’t willing to pay for this fix.
Alternatively, Design Science may not be able to deliver a version that, for maximum backwards compatibility, has only this fix (to minimize risks, they would have to have kept an environment around that hosts the compiler used back then)
Chances are that Microsoft doesn’t have a license for bug fixes from Design Science (makers of MathType) anymore and isn’t willing to pay for this fix.
Alternatively, Design Science may not be able to deliver a version that, for maximum backwards compatibility, has only this fix (to minimize risks, they would have to have kept an environment around that hosts the compiler used back then)
One reason for doing it this way is possibly this:
> Well, have you ever met a C/C++ compiler that would put all functions in a 500+ KB executable on exactly the same address in the module after rebuilding a modified source code, especially when these modifications changed the amount of code in several functions?
It's quite possible they are still contractually obligated to maintain some pretty old systems where changes to the .exe would produce unexpected behaviour. I had Access apps/databases crash on a system if they were built by a different version of Access.
> Well, have you ever met a C/C++ compiler that would put all functions in a 500+ KB executable on exactly the same address in the module after rebuilding a modified source code, especially when these modifications changed the amount of code in several functions?
It's quite possible they are still contractually obligated to maintain some pretty old systems where changes to the .exe would produce unexpected behaviour. I had Access apps/databases crash on a system if they were built by a different version of Access.
Slightly off-topic: what program is used to produce disassembly graphs as those in article?
https://www.hex-rays.com/products/ida/
IDA is widely regarded as the best disassembler and debugger out there. It comes with a price to match too though.
IDA is widely regarded as the best disassembler and debugger out there. It comes with a price to match too though.
If anyone wants to give it a go they can use v5.0, which is free for non-commercial use.
Otherwise: https://reverseengineering.stackexchange.com/questions/1817/...
Otherwise: https://reverseengineering.stackexchange.com/questions/1817/...
It is something of a rite of passage in the piracy community to crack your own copy of IDA Pro.
It's also a rite of passage to distribute cracked and boobytrapped copies on filesharing sites...
It's also a rite of passage to distribute cracked and boobytrapped copies on filesharing sites...
Oh, I remember the old "this will only work if your timezone is set to Moscow" version...
That plus softice.
x64dbg can also produce nice graphs and is open-source!
radare2 is another open source alternative, but it comes with quite a learning curve.
http://rada.re/r/
http://rada.re/r/
Ah, this brings up a lot of font memories of me in high school preparing presentations using this fine piece of software[0] before replacing it with a 1GB open source equation editor called LaTeX.
[0] It was actually quite usable once you got to know its warts.
[0] It was actually quite usable once you got to know its warts.
The article mentions that the timestamp of compilation gets embedded into the binary. When does this happen? I am used to having identical binaries when recompiling the same source code with same flags (and compiler and so on and so on)
Binary patching is a really common requirement in attack/defense CTF, and there are a few projects floating around to help with it.
Keypatch helps you do assembly overwrites in IDA Pro.
Binary Ninja lets you do assembly (and C shellcode!) overwrite patches, and even has undo.
I have my own project [1] for patching ELFs that relies on injecting additional segments and injecting a hook at any address, so as to not require in-place patches. It can also massage GCC/Clang output and inject that reliably into an existing binary.
[1] https://github.com/lunixbochs/patchkit
I have my own story about this as well. A few years ago I released a port of Uplink: Hacker Elite for the OpenPandora handheld with a few game engine patches, and some people were running into a bug: the game would enter the "new game" screen on every launch, even if you already had a save game to load.
I and couldn't find the exact source I'd used to build it and didn't want to spend time making sure I got all of my bugfixes into the vanilla repository, so... I went digging with IDA, found the topmost branch to the "new game" wizard, and patched the address to go to the main menu function instead. At that point you could still click "new game" from the menu and it wouldn't go through the patched address (so "new game" still worked), but you could also load an existing game, thus fixing the bug!
I still have nothing on Notaz, who statically recompiled StarCraft and Diablo for that community :)
Keypatch helps you do assembly overwrites in IDA Pro.
Binary Ninja lets you do assembly (and C shellcode!) overwrite patches, and even has undo.
I have my own project [1] for patching ELFs that relies on injecting additional segments and injecting a hook at any address, so as to not require in-place patches. It can also massage GCC/Clang output and inject that reliably into an existing binary.
[1] https://github.com/lunixbochs/patchkit
I have my own story about this as well. A few years ago I released a port of Uplink: Hacker Elite for the OpenPandora handheld with a few game engine patches, and some people were running into a bug: the game would enter the "new game" screen on every launch, even if you already had a save game to load.
I and couldn't find the exact source I'd used to build it and didn't want to spend time making sure I got all of my bugfixes into the vanilla repository, so... I went digging with IDA, found the topmost branch to the "new game" wizard, and patched the address to go to the main menu function instead. At that point you could still click "new game" from the menu and it wouldn't go through the patched address (so "new game" still worked), but you could also load an existing game, thus fixing the bug!
I still have nothing on Notaz, who statically recompiled StarCraft and Diablo for that community :)
It's an old program the source code for which may either not compile with the modern C++ compiler, or be lost. Back in 2000, Microsoft was using Visual Source Safe for managing its source code. I wouldn't be surprised if nobody can remember where the heck the VSS repository with that source code is located.
That leaves the binary monkey-patching as the only reasonable solution. I'm pretty sure Raymond Chen still works at Microsoft...
That leaves the binary monkey-patching as the only reasonable solution. I'm pretty sure Raymond Chen still works at Microsoft...
Binary patching is really only reasonable when the source code is indeed lost. If they had the code but simply needed a compiler that worked, they could have rebuilt it using the same toolchain and build environment it was built with to begin with. Old versions of Windows and MSVC are obviously still around.
Just a historical note: Patching used to be much more common. Back in the Vax VMS days the image file format (executables, not pictures) had a section for patches.
From the ANALYZE/IMAGE command…
Patch information --- Indicates whether the image has been patched (changed without having been recompiled or reassembled and relinked). If a patch is present, the actual patch code can be displayed. (VAX and Alpha only.)
From the ANALYZE/IMAGE command…
Patch information --- Indicates whether the image has been patched (changed without having been recompiled or reassembled and relinked). If a patch is present, the actual patch code can be displayed. (VAX and Alpha only.)
And here I am, manually patching Docker containers...
I suppose the fact that they have patched the binary means they can never again patch the source?
They'd have to re-implement the patch in source before doing anything else to it. I wonder if they are no longer able to build from source anymore... why else would they resort to this?
As explained in the article and in other comments, it's possible that there are dependencies that rely on address continuity of contents or file size continuity.
They probably just lost the ability to build it, or the source code can't be found. Happens quite often. 17 years is a /long/ time to maintain build systems and remember where you put the files.
I'm glad that most of the software development community seems to have settled on git. I get the feeling that I'll still have all of the source code for my projects in 20 years.
Redundant backups are especially important for software companies. It's scary to think how many startups give all cofounders and developers admin access to everything. It helps that git is distributed, but it's not hard to imagine a scenario where a ticked off former employee wipes everyone's laptops and deletes the hosted source code.
Even if you don't update the mirrors regularly, it's good to know that you have some copies of data in BitBucket/GitLab/Heroku/Google Drive.
Redundant backups are especially important for software companies. It's scary to think how many startups give all cofounders and developers admin access to everything. It helps that git is distributed, but it's not hard to imagine a scenario where a ticked off former employee wipes everyone's laptops and deletes the hosted source code.
Even if you don't update the mirrors regularly, it's good to know that you have some copies of data in BitBucket/GitLab/Heroku/Google Drive.
I don't know if I would hold your breath. 10 years ago I think most people hadn't even heard of git (it was ~2 years old) and Google Code was the hot new thing, and GitHub was a year or so away from creation. At the time most people seemed to be pretty content Subversion and hosting on Sourceforge (before it turned evil) or Google Code, but in the next ~5 years everything changed. Granted git, GitHub, etc. have far more momentum that anything that came before, but this is a field where it feels like the only constant is change.
If the thing is 17 years old an a replacement has existed since forever, what purpose does this file have today (Assuming I'm on a modern windows, I run either no ms office or a modern office version).
From the article:
> While Office has had a new Equation Editor integrated since at least version 2007, Microsoft can't simply remove EQNEDT32.EXE (the old Equation Editor) from Office as there are probably tons of old documents out there containing equations in this old format, which would then become un-editable.
> While Office has had a new Equation Editor integrated since at least version 2007, Microsoft can't simply remove EQNEDT32.EXE (the old Equation Editor) from Office as there are probably tons of old documents out there containing equations in this old format, which would then become un-editable.
Ah. missed that. But obviously I'd be very happy for this program to be patched by replacing it with this program:
Even better if they could at least render the old equation statically using the new office, but not edit it. Then it would be almost insanely rare that anyone needs the old editor.
MessageBox.Show("This document contains an old equation and you don't have the editor. Do you want to download the old editor?");
Even better if they could at least render the old equation statically using the new office, but not edit it. Then it would be almost insanely rare that anyone needs the old editor.
This is the kind of thing that can rapidly escalate to a CTO asking his Microsoft sales VP why he's spending $18M/year on upgrade and support contracts when a report that's worked "forever" can start talking back like Clippy.
Microsoft doesn't preserve backward compatibility because they're stubborn; it's a key part of their value proposition to some of their biggest clients.
Microsoft doesn't preserve backward compatibility because they're stubborn; it's a key part of their value proposition to some of their biggest clients.
This is the thing: I'm also a paying customer, I just don't pay as much. But I'd like to pay for more security/less compatibility, instead of the other way around.
This should also be very easy to do e.g. by noticing whether you are in a setting where there is any risk of the scenario you say. If it's a home machine for example, then don't worry about compatibility, focus on security.
This should also be very easy to do e.g. by noticing whether you are in a setting where there is any risk of the scenario you say. If it's a home machine for example, then don't worry about compatibility, focus on security.
We've seen where that leads to: there is plenty of software out there where you'd have to open every document you wrote with version N - 1 in version N to "convert it" to N's format, and version N + 1 can't read N - 1 files at all.
That can lead to a very ugly form of bitrot quickly. Do you convert every document you've ever touched every time, even ones you haven't needed in years, just in case? Do you worry that every time you convert a file it might corrupt the file in the process? Do you find some way to keep every version of the program available at all times and play try every version until it opens the file?
Backwards compatibility in general offers much greater means for archival.
In this specific example: losing backwards compatibility for ancient equations directly threatens the archival of math and science documents. That seems like it could have huge repercussions in some fields.
That can lead to a very ugly form of bitrot quickly. Do you convert every document you've ever touched every time, even ones you haven't needed in years, just in case? Do you worry that every time you convert a file it might corrupt the file in the process? Do you find some way to keep every version of the program available at all times and play try every version until it opens the file?
Backwards compatibility in general offers much greater means for archival.
In this specific example: losing backwards compatibility for ancient equations directly threatens the archival of math and science documents. That seems like it could have huge repercussions in some fields.
If I were designing the software, there would be a module which can upgrade a file format from version N to N+1.
That code is written in some kind of sandboxed VM/bytecode. You freeze the bytecode when you release a version.
When you release version 20 of the app, it has bytecode to convert 1 -> 2, 2->3, 3->4, etc, all the way to version 20. When it finds an old file, it runs all the updates as necessary.
If there is a bug in the updaters, it stays there forever.
If there is a security problem with the updaters, thats whats the sandbox is for.
That code is written in some kind of sandboxed VM/bytecode. You freeze the bytecode when you release a version.
When you release version 20 of the app, it has bytecode to convert 1 -> 2, 2->3, 3->4, etc, all the way to version 20. When it finds an old file, it runs all the updates as necessary.
If there is a bug in the updaters, it stays there forever.
If there is a security problem with the updaters, thats whats the sandbox is for.
Reminds me of that time Mark Watney used a similar method to patch his rover’s comms to connect to an old radio system.
I wonder how the checked in the fix to the source control system?
Probably:
git rm -r .
git add EQNEDT32.EXE
git commit -m "Fixed CVE-2017-11882"
git push
'FCIB', apparently: https://blogs.msdn.microsoft.com/oldnewthing/20171114-00/?p=...
> F-C-I-B or as a sort-of acronym eff-sib […] stands for "foreign checked-in binary" […] The term FCIB didn't originally mean "foreign checked-in binary". According to legend […] "Not another f—ing checked-in binary!"
I've done this in the past by checking in both a binary as well as a diff to the previous version. It's sometimes helpful to have both if your SCM doesn't handle binary diffs well.
[deleted]
There are plenty of companies hooking into private APIs within Word and Excel with their “productivity tools”. Probably an important MSFT customer was using one of these tools as a crucial part of their operations, so they convinced them not to break it.
Just like how Google had to put special cases in Android to keep compatibility with some hacks Facebook was using in their app.
I salute and respect the guy who did this while hoping I never have to do anything like this.
I was dreaming I’ll have to do something like that until it faded below the weight of modern <script src> programming. It is like dancing twist, rock and hardbass in the era of electronic arse shaking.
I noticed a 0F 1F NOP, which breaks older processors. This is in an update that goes back to Office 2007.
So Microsoft lost the source code? Or maybe the engineer couldn't be bothered to set up the old toolchain used to build this executable?
Manual patching is more of a bother, I imagine, so this wouldn't be due to a smug engineer getting his prima donna on.
Probably bought it in from a third party. Why'd you think Pinball went away?
Raymond Chen answered this question five years ago[0].
[0] https://blogs.msdn.microsoft.com/oldnewthing/20121218-00/?p=...
[0] https://blogs.msdn.microsoft.com/oldnewthing/20121218-00/?p=...
mariusmg(1)
This also compels me to "code-golf" the function even more:
Original: 58 bytes; patched: 44; mine: 30.
I've done plenty of patching like this, and indeed the relative "sparseness" of compiler output very often allows the more functional version to be smaller than the original. It's amazing how many instructions the original wastes --- notice how none of ebx, esi, or edi are used, yet they get needlessly pushed and popped; and despite saving those registers so they could be used locally, the compiler perplexingly decided to keep all the local variables on the stack instead. The "jump around a jump", with both of them being the "long" form (for destinations greater than 128 bytes away, not the case here) is equally horrible. This may actually be a case where today's compilers will generate smaller code for the same source.
Note that in 32-bit code, memcpy is typically implemented by first copying blocks of 4 bytes using the movsd (move double word) instruction, while any remaining bytes are then copied using movsb (move byte). This is efficient in terms of performance, but whoever was patching this noticed that some space can be freed by only using movsb, and perhaps sacrificing a nanosecond or two.
On older processors this was true, but since Ivy Bridge a REP MOVSB will essentially be as fast but smaller. Look up "enhanced REP MOVSB" for more information.