Why modern software is slow(randomascii.wordpress.com)
randomascii.wordpress.com
Why modern software is slow
https://randomascii.wordpress.com/2022/09/29/why-modern-software-is-slow-windows-voice-recorder/
822 comments
I'm a game developer and game performance is better than ever. 144 Hz monitor? 4k? We got you covered. Even ray tracing and VR is on the way.
Most games render frames of a 3D world in less than 17ms, but most websites take 3-7 seconds to load because of all the ads and bloat, and things shift around on you for another 20 seconds after that, so when you go to tap a link you accidentally tap an ad that finally loaded under your finger. If you optimize those websites, they run super fast though, but it's quite a pain to do with the dependency bloat in modern tech stacks...
(note: games lag as well when you drag in a million dependencies you don't need)
The thing is, most sites and web apps try to solve a user problem, and if you are the only company in town that solves that problem, then performance barely matters - what matters is solving the problem. The users will put up with some pain because the problem is even more painful.
With games, it's all about the experience of interacting with the software - so performance is (hopefully, depending on your team and budget) amazing.
That, and... performance tuning is hard work, and I think most people don't know much about it. It's a fractal rabbit hole. Cache misses, garbage collection, streaming, object pooling, dealing with stale pointers, etc. Even I have a ton to learn, and no matter how much I learn, I probably still will have a lot more to learn. It's easier for many teams to hand wave it I guess as long as they aren't losing too many customers because of it.
Most games render frames of a 3D world in less than 17ms, but most websites take 3-7 seconds to load because of all the ads and bloat, and things shift around on you for another 20 seconds after that, so when you go to tap a link you accidentally tap an ad that finally loaded under your finger. If you optimize those websites, they run super fast though, but it's quite a pain to do with the dependency bloat in modern tech stacks...
(note: games lag as well when you drag in a million dependencies you don't need)
The thing is, most sites and web apps try to solve a user problem, and if you are the only company in town that solves that problem, then performance barely matters - what matters is solving the problem. The users will put up with some pain because the problem is even more painful.
With games, it's all about the experience of interacting with the software - so performance is (hopefully, depending on your team and budget) amazing.
That, and... performance tuning is hard work, and I think most people don't know much about it. It's a fractal rabbit hole. Cache misses, garbage collection, streaming, object pooling, dealing with stale pointers, etc. Even I have a ton to learn, and no matter how much I learn, I probably still will have a lot more to learn. It's easier for many teams to hand wave it I guess as long as they aren't losing too many customers because of it.
Older readers will remember when you'd buy a computer magazine printed on paper and at least 80% of it was ads. Which often didn't change from month to month.
Massive slabs of wood pulp had to be printed and shipped to all to the stores that stocked them, at huge cost, just so you could manhandle one of them home.
And the content was mostly physical bloat.
That's the modern web. Except that you don't just get the ads, you get some of the machinery that serves them - at you, specifically, based on your browsing profile, which is used to decide which ads you see.
I always refuse all cookies. When I forget to do that for some reason it's obvious just how much slower the experience gets.
Massive slabs of wood pulp had to be printed and shipped to all to the stores that stocked them, at huge cost, just so you could manhandle one of them home.
And the content was mostly physical bloat.
That's the modern web. Except that you don't just get the ads, you get some of the machinery that serves them - at you, specifically, based on your browsing profile, which is used to decide which ads you see.
I always refuse all cookies. When I forget to do that for some reason it's obvious just how much slower the experience gets.
> Most games render frames of a 3D world in less than 17ms, but most websites take 3-7 seconds to load
Most (many?) games take minutes to load. Perf is nice once they're loaded (assuming you've got sufficient hardware), but loading is a drag. Modern load times are worse than all the fiddling it took to get NES games to start.
Most (many?) games take minutes to load. Perf is nice once they're loaded (assuming you've got sufficient hardware), but loading is a drag. Modern load times are worse than all the fiddling it took to get NES games to start.
> Most (many?) games take minutes to load.
What are you loading them off? HDD? The only time I experienced a load time longer than a minute for any game was on a PS2 game that was scratched to high hell. It's possible that I'm simply interested in games that don't have long load times but I've played enough of modern AAA on console and PC to never have experienced this first hand.
What are you loading them off? HDD? The only time I experienced a load time longer than a minute for any game was on a PS2 game that was scratched to high hell. It's possible that I'm simply interested in games that don't have long load times but I've played enough of modern AAA on console and PC to never have experienced this first hand.
F1 2020 on Xbox is horrible, loading all the time. Loading once in the beginning would be ok, but it’s loading between every step. Sometimes it feels you spend more time waiting for the loading than actual racing.
That's bad design then. The Witcher 3 has no loads and rhe world is massive. As OP said, priorities.
A minute seems like a long wait for software that brags about its performance
It does depend on the game, but it's a good point. We do this 'cache everything at the start and avoid creation/destruction' during runtime thing that can lead to slower startup times.
After initial startup though, it depends a lot on the game itself how much loading you will wait for. I think SPAs are sort of the equivalent of zero-loading-screen games in some ways: both are trying to eliminate that loading pause.
After initial startup though, it depends a lot on the game itself how much loading you will wait for. I think SPAs are sort of the equivalent of zero-loading-screen games in some ways: both are trying to eliminate that loading pause.
Who is we? BR games have one map and reload it every time they start and even loading the menu takes extremely long. Then they go on top make the time between matches solid minutes to wait for people to be pointlessly loading the map. Fortnite for example (haven't played it in years but it was like that in the first 2-3 years). There's literally no excuse. Now that I got myself ranting on BR games I'll note that their quality is an order of magnitude worse (bugs, performance) than games with 5 minute rounds, while they have 30 minute rounds. Meh anyway I guess I'm attacking your random thought too hard.
That has never been different. Games push the envelope a lot. Especially if you don't have the best of everything.
Some awesome games were small. Other used a full 1.44" FDD. Some used 4 FDDs. Then the same happened with CDs. Finally only one CD needed. But there's space left over so later games made full use of it and soon there were games that needed 2 or more CDs. Lots of loading even going from one place to another. Decisions like "do I really want to go there now, I'll have to put in CD3 again". Or you bought the bigger HDD for much more $$$ and could install all 6 CDs onto that instead.
Can't run "Apache Longbow", need more RAM. But I can either buy the RAM or the game with the money I have. Damn!
What's been a blessing for me is not having the time to actually keep up with any of this time wise. Not enough time to really warrant spending money on new games and new hardware. Now I play games that are years and years old, I can get for cheap on GOG and that run fine on Intel Integrated graphics chips. Coincidentally those games usually load just fine in almost no time on modern SSDs as well and since I can't crank up the graphics full throttle, loading all that stuff into memory is relatively fast too etc.
Some awesome games were small. Other used a full 1.44" FDD. Some used 4 FDDs. Then the same happened with CDs. Finally only one CD needed. But there's space left over so later games made full use of it and soon there were games that needed 2 or more CDs. Lots of loading even going from one place to another. Decisions like "do I really want to go there now, I'll have to put in CD3 again". Or you bought the bigger HDD for much more $$$ and could install all 6 CDs onto that instead.
Can't run "Apache Longbow", need more RAM. But I can either buy the RAM or the game with the money I have. Damn!
What's been a blessing for me is not having the time to actually keep up with any of this time wise. Not enough time to really warrant spending money on new games and new hardware. Now I play games that are years and years old, I can get for cheap on GOG and that run fine on Intel Integrated graphics chips. Coincidentally those games usually load just fine in almost no time on modern SSDs as well and since I can't crank up the graphics full throttle, loading all that stuff into memory is relatively fast too etc.
Maybe on consoles. The only PC games I've played in the last decade that took minutes have been GTA5, Destiny 2, and heavily modded Minecraft. Most games load in under 15 seconds.
If anything, it's the other way around. PS5 games often don't even have loading bars, since they assume almost-instant streaming of assets from the SSD.
I like how 15 seconds is considered a decent bar.
15 seconds is my upper bar here, and it includes known, very poorly optimized, alpha games. Not once did anyone use the word decent here, you are imagining unsaid things.
Just check how much GPU memory games are using, it take a while to load a process the assets before they are ready to use, so the bottleneck is mostly IO I believe, AMD and Nvidia were working on improving that by loading assets directly into the GPU instead of first go to the CPU+RAM and then the GPU.
IO is also the bottleneck on the web, unsurprisingly.
I don’t know, is it? I get it, sure data has to be loaded, but the problematic part is the work being done on that data, this is what takes much more time than it should reasonably take.
Just take the example before, if twitter were server-side rendered, it would likely load in basically a blink of an eye.
The problem is code, which will later do more network calls which again have to be waited on, this is multiple round trips. Also, the fault lies probably a bit on web tech as well, html+css is just a terrible abstraction, layouting not being properly solved.
Just take the example before, if twitter were server-side rendered, it would likely load in basically a blink of an eye.
The problem is code, which will later do more network calls which again have to be waited on, this is multiple round trips. Also, the fault lies probably a bit on web tech as well, html+css is just a terrible abstraction, layouting not being properly solved.
Maybe a partial solution will be to use qwick approach, load js on demand when needed, instead of all at once, while ideally will be use strictly the necessary js (that in some cases could be none).
Network calls are IO reducing them greatly speeds up websites.
A network call waiting on the result of another call is still two times as long as it should/could be, no matter how efficient you make it.
Optimizing your IO is indeed a good idea. Back in the day we used to order game data on CD/DVD for efficient streaming.
I don't think so.
My Internet connection saturates my PLC at 10MiB/s, my ping is under the 15ms, and Twitter still takes ~2.4s to load and render a random tweet.
My Internet connection saturates my PLC at 10MiB/s, my ping is under the 15ms, and Twitter still takes ~2.4s to load and render a random tweet.
Twitter is a terrible piece of garbage website, isn't that why everyone uses it as the ultimate example of the web sucking?
Twitter wants you to download their app and sign up so they intentionally neutered their web experience. It would probably require 0.1% of their 7500 employees to spend 3 months to fix it entirely, but it's business that drives the bad tech, not the developers
I build internal apps that load collaborative an entire app with MBSE capabilities that have a time to interactivity of < 10s. Google docs takes like 7s to spin up a new doc and interact with it. I can promise you Twitter could reduce rendering some stylish HTML/CSS and a 280 character tweet to a single round trip and see maximum 0.5s browser parse & paint if they wanted to
Twitter wants you to download their app and sign up so they intentionally neutered their web experience. It would probably require 0.1% of their 7500 employees to spend 3 months to fix it entirely, but it's business that drives the bad tech, not the developers
I build internal apps that load collaborative an entire app with MBSE capabilities that have a time to interactivity of < 10s. Google docs takes like 7s to spin up a new doc and interact with it. I can promise you Twitter could reduce rendering some stylish HTML/CSS and a 280 character tweet to a single round trip and see maximum 0.5s browser parse & paint if they wanted to
Taking a quick look at a random tweet through the network tab, about 2/3rds of the time spent is on downloading files. Now should Twitter preload a bunch of videos from recommended tweets it shows below the tweet? Probably not. Does Twitter need megabytes of JavaScript just to show a tweet? I'd hope not. Those things do appear to be the main bottleneck however.
That's not true.
I dunno it definitely seems that way. In our own product reducing IO significantly sped up time to live.
And for the love of God, if it's a single player game let me pause while it's loading (which shouldn't interrupt the loading, but should make it start in a paused state).
Definitely correct me if I’m wrong, but games have thousands of little cores generating those frames every 17ms, but websites usually rely on a big slow cpu to direct the workflow of tiny packets from some far off network, no?
I haven't noticed loading times in many years, basically since SSD replaced HDD.
I wish more game developers optimized for space. I stopped playing video games almost altogether because I'm not going to download 70 GB every time I want to play something. The size has gotten absurd.
Textures are big... bigbly big! And to be fair, game devs even have middleware solely dedicated to compression (such as Oodle http://www.radgametools.com/oodle.htm). I am sure that some % of that space can still be recovered, but it's definitely not significant. I'm sure you can't drop to half and maintain the ultra settings.
You would be surprised, actually. This [0] shows an example of some excellent space savings. I used to work for Epic and the results we got on other projects were in this ballpark too.
[0] https://dev.epicgames.com/community/learning/tutorials/ry2D/...
[0] https://dev.epicgames.com/community/learning/tutorials/ry2D/...
My assumption was that you get down to 70gb packages after using tools like oodle. But hell, I may be wrong there and may be giving most AAA devs too much credit.
> But hell, I may be wrong there and may be giving most AAA devs too much credit.
I don't think this is a fair comment. Theres an enormous tradeoff to make when compressing game assets, and uncompressed game assets are _absolutely enormous_. My current project has about 30 people at the moment. The game content is ~50GB, the client itself is ~4GB but the source substance files/fbx files/photogrammetry assets are tipping on 500GB. We're a _small_ AAA game. My previous project was closer to 500GB for the game content, and the source assets were multiple TB.
Tools like Oodle are third party licenses that need to be considered. Most games are compressing but switching to better compression methods can make the differences shown. Theres still huge tradeoffs to be made though. Most GPU formats for consoles require platform specific texture formats, and some of them don't compress well (hello dxt). You might have to trade a 20% file size reduction for a 30 second increase load time. Is that worth it? Maybe.
Either way, it's not just "dumb Devs". We care, we work hard on this.
I don't think this is a fair comment. Theres an enormous tradeoff to make when compressing game assets, and uncompressed game assets are _absolutely enormous_. My current project has about 30 people at the moment. The game content is ~50GB, the client itself is ~4GB but the source substance files/fbx files/photogrammetry assets are tipping on 500GB. We're a _small_ AAA game. My previous project was closer to 500GB for the game content, and the source assets were multiple TB.
Tools like Oodle are third party licenses that need to be considered. Most games are compressing but switching to better compression methods can make the differences shown. Theres still huge tradeoffs to be made though. Most GPU formats for consoles require platform specific texture formats, and some of them don't compress well (hello dxt). You might have to trade a 20% file size reduction for a 30 second increase load time. Is that worth it? Maybe.
Either way, it's not just "dumb Devs". We care, we work hard on this.
Question: why can you not separate the different graphics tiers as optional addons? That way, I only have to download high resolution textures if my PC can actually handle them.
Edit: never mind. Saw this being discussed elsewhere in this thread.
Edit: never mind. Saw this being discussed elsewhere in this thread.
> it's not just dumb devs
I may have come too rough in my comments and lost the original intent. I was hoping to defend that devs _already have_ access to good compression tools and that they already use them, and 70gb games are that big after compression is applied. That's where my assumption started.
I understand well that there are custom formats for different platforms, and that the whole reason these tools exist is not to squeeze an extra 10% smaller files, but to get fast load times while maintaining small file sizes.
> My previous project was 500gb
That wouldn't surprise me to be honest. It makes perfect sense.
I may have come too rough in my comments and lost the original intent. I was hoping to defend that devs _already have_ access to good compression tools and that they already use them, and 70gb games are that big after compression is applied. That's where my assumption started.
I understand well that there are custom formats for different platforms, and that the whole reason these tools exist is not to squeeze an extra 10% smaller files, but to get fast load times while maintaining small file sizes.
> My previous project was 500gb
That wouldn't surprise me to be honest. It makes perfect sense.
But that's kinda the point, when you are on a low to mid tier setup you now have to download 50GB of 4k textures you're never gonna see.
For the steam deck valve added support for games to only include the relevant texture quality tier to save space. Hopefully that expands and rolls out more broadly, but it's also a bit awkward to specify your game quality settings before launching the game.
Both the current Xbox and PS generations support this in a way as well. The developers need to do some work to make use of the features, but they are available.
Just get up close to a model, and you can still see the difference.
And there are YT videos that call developers out for mushy textures on every new game release.
And there are YT videos that call developers out for mushy textures on every new game release.
I looked through the generated binaries from unreal engine, and they inline sort algorithms almost everywhere in their code, so even their binaries becomes huge not just the assets. Inlining sort algorithms is in my experience a very common cause to binary bloat, and not at all helpful for performance, forcing the cpu to load that much code everywhere hurts performance. It will look good in local benchmarks though since then the CPU can keep all the code in its cache.
My first computer had a 7GB hard drive. At that time we had CDs with ~700mb capacity. Now we routinely have 1TB storage and games are 70GB. As a percentage of total storage these are similar. I'm not sure how the download rate of 70GB compares to the read speed of a CD but installs took a while back in the day as well.
I had a cdrom drive in a 486 with 8mb of RAM and ~200MB of disk. I could either have a game or two installed (which at the time invariably ran in DOS) or Windows 3.1 installed. Not both.
At the time it was like… wait I just bought this fancy CD-ROM… why do I need to install 30 or 40 whole megabytes to disk to play it?
At that time it was pretty common to actually install all the game files to disk (drives were slow… I think it was 4x read speed) and then the music was encoded in the cd as additional tracks of plain CD audio.
At the time it was like… wait I just bought this fancy CD-ROM… why do I need to install 30 or 40 whole megabytes to disk to play it?
At that time it was pretty common to actually install all the game files to disk (drives were slow… I think it was 4x read speed) and then the music was encoded in the cd as additional tracks of plain CD audio.
I don't have the fastest internet connection but downloads from steam sit happily at 30MB/s so in an hour I can download 108GB of data.
The earliest cdroms that I remember were 1x drives - so it took an hour to read that 700MB. Obviously drive speeds increased tremendously over the lifespan of cdrom (did it reach 48x?), but compared to the earliest generation it is about the same.
The earliest cdroms that I remember were 1x drives - so it took an hour to read that 700MB. Obviously drive speeds increased tremendously over the lifespan of cdrom (did it reach 48x?), but compared to the earliest generation it is about the same.
The limiting factor of the CD/DVD drives was seek time, not read speed. Seeking a spot on the disk was on the order of 150ms. You basically got 5-7 read OPS. The best drive I remember stood out at 120ms seek, some would be over 200ms seek. Sometimes this was a hard number to track down. Even with a 48x drive, installation could easily still take 30+ minutes particularly if the installer wasn't optimized for sequential read off a flat file.
Sometimes you got a really bad installer, and it could take a pretty grueling 2+ hours. Some companies chose to optimize for space (thus cost..) rather than user experience.
As space got more plentiful it would sometimes be easier to copy the disk and remount it and then install.
Sometimes you got a really bad installer, and it could take a pretty grueling 2+ hours. Some companies chose to optimize for space (thus cost..) rather than user experience.
As space got more plentiful it would sometimes be easier to copy the disk and remount it and then install.
"Patch day, no play!"
> I stopped playing video games almost altogether because I'm not going to download 70 GB
This is why some people love game streaming. Esp true if you only play a game once in a while.
This is why some people love game streaming. Esp true if you only play a game once in a while.
Game streaming presumably needs a high bandwidth, low latency internet connection? If so that's exactly the connection that makes a 70GB download something that can happen in the background while you start playing
Or why others still love FITGIRL repacks.
> That, and... performance tuning is hard work
Most 'performance tuning' out there is eliminating quadratic functions and putting stuff into simple dictionary caches. This should be easily teachable to any competent developer.
The problem isn't 'hard work', it's nobody cares about this.
Most 'performance tuning' out there is eliminating quadratic functions and putting stuff into simple dictionary caches. This should be easily teachable to any competent developer.
The problem isn't 'hard work', it's nobody cares about this.
That is not the lowest hanging fruit (where I would put trivial changes like inlining functions or switching an of-the-shelf library by a faster alternative). But it is medium-low hanging fruit.
Getting measurements that help you identify where the bottlenecks really are can take time. Often these have to be retrofitted into existing complex system, when the project is already underway.
Sometimes the only way forward is replacing the current way of doing things with a more efficient way of doing things. This requires understanding the current way well (it can be the result of many teams working together for months) as well as coming up with a new design. Both of these things take time.
Finally, there's bug hunting, where you get into rabbit holes. "There's a slowdown when we're on this particular step. You investigate for days, until you realize that the slowdown only happens when the user mouse is near a section of the screen. It turns out that there's a non-visible UI element that should be inactive but it isn't because it is a custom element instead of one provided by the UI library, and it is attempting to render a special effect but finds no graphical context, so it request for a new context once per frame." And one week of dev time has passed.
Getting measurements that help you identify where the bottlenecks really are can take time. Often these have to be retrofitted into existing complex system, when the project is already underway.
Sometimes the only way forward is replacing the current way of doing things with a more efficient way of doing things. This requires understanding the current way well (it can be the result of many teams working together for months) as well as coming up with a new design. Both of these things take time.
Finally, there's bug hunting, where you get into rabbit holes. "There's a slowdown when we're on this particular step. You investigate for days, until you realize that the slowdown only happens when the user mouse is near a section of the screen. It turns out that there's a non-visible UI element that should be inactive but it isn't because it is a custom element instead of one provided by the UI library, and it is attempting to render a special effect but finds no graphical context, so it request for a new context once per frame." And one week of dev time has passed.
Well, for starters... dictionary hashing is too slow to be ideal. We don't swear off dictionaries, but we try to get our constant-time lookups without needing a hashing step when possible. The dictionaries can also lead to extra cache misses, since the memory often won't be compacted as you step through your data.
In most games this doesn't matter, but occasionally you have a frame time target and you need to take extra steps like that. Those extra steps can be hard work when they come up!
In most games this doesn't matter, but occasionally you have a frame time target and you need to take extra steps like that. Those extra steps can be hard work when they come up!
> dictionary hashing is too slow to be ideal
Even when you use SIMD intrinsics?
For example: https://en.wikipedia.org/wiki/Intel_SHA_extensions
A repo I just found illustrating those instructions' usage: https://github.com/noloader/SHA-Intrinsics
Even when you use SIMD intrinsics?
For example: https://en.wikipedia.org/wiki/Intel_SHA_extensions
A repo I just found illustrating those instructions' usage: https://github.com/noloader/SHA-Intrinsics
Not the OP, but those extensions implements cryptographic hash functions, they are not very suitable (too slow) for hash-tables and other hash based data structures.
4 CPU cycles is too slow? Because SHA256RNDS2 is 4 clock cycles on Zen3 CPUs. 6 clock cycles on Intel Alder Lake (12th gen).
Source is the measurements here: https://uops.info/table.html
Source is the measurements here: https://uops.info/table.html
Yes, because SHA256RNDS2 aid the implementation of the update function of SHA256. We should invoke it multiple times on the message blocks, it performs two rounds out of the 64 SHA256 rounds.
> We should invoke it multiple times
That'd make SHA256RNDS2 even faster because of the throughput numbers.
I'm curious, what's the faster algorithm for hashing you use? I don't want to waste time on intrinsics and inflict complexity on my colleagues if there are alternatives I can use!
That'd make SHA256RNDS2 even faster because of the throughput numbers.
I'm curious, what's the faster algorithm for hashing you use? I don't want to waste time on intrinsics and inflict complexity on my colleagues if there are alternatives I can use!
I’ll just use xxHash family, murmur hash family or even FNV.
For crypto, BLAKE2 is a cryptographic hash that’s faster than SHA,MD5, etc and is at least as secure as the SHA-3.
That’s just good software development practices. Game developers and scientific software developers borderline abuse the software (language, drivers, etc.) and hardware to get these performance numbers.
For example, I need to reorder a matrix and change my access pattern to it to extract more performance and reduce cache trashing by exploiting cache prefetch dynamics of a CPU.
I need to test this with every CPU generation and test whether it works as expected performance-wise.
To achieve this, I need to rewrite half of the said software.
The reason I’m not doing this is it’s still way faster than other software I’ve seen in that category, and extreme performance tuning can hurt flexibility of the code down the road.
For example, I need to reorder a matrix and change my access pattern to it to extract more performance and reduce cache trashing by exploiting cache prefetch dynamics of a CPU.
I need to test this with every CPU generation and test whether it works as expected performance-wise.
To achieve this, I need to rewrite half of the said software.
The reason I’m not doing this is it’s still way faster than other software I’ve seen in that category, and extreme performance tuning can hurt flexibility of the code down the road.
> Most 'performance tuning' out there is eliminating quadratic functions and putting stuff into simple dictionary caches. This should be easily teachable to any competent developer.
That does happen, but some of the more severe performance issues I've seen are memory related. Every byte allocated has to be freed. If you lower the memory profile of your function then you reduce the overhead of running the code significantly.
I recall a web service that ground to a halt and would sit at 100% CPU, after I put it in a profiler 97% of the cycles were in the garbage collector. Dig a little deeper and I found that the caching library used was cloning the data before sending it back. Ahh, junior devs.
I rewrote a library with the same interface that did not clone the data, this was safe as the original data wasn't being changed, 15x speedup from 10 req/s to 150 req/s.
Then I dug deeper and changed which data was getting cached. Cache post-render and post-compression instead of the pre-render database calls. 150x speedup from the original or 10x from the previous point. The system went from being able to serve 10 requests/s to 1500 req/s. This change also was safer since nobody is going to ever try to alter compressed data, no such guarantees from the cached database calls.
That does happen, but some of the more severe performance issues I've seen are memory related. Every byte allocated has to be freed. If you lower the memory profile of your function then you reduce the overhead of running the code significantly.
I recall a web service that ground to a halt and would sit at 100% CPU, after I put it in a profiler 97% of the cycles were in the garbage collector. Dig a little deeper and I found that the caching library used was cloning the data before sending it back. Ahh, junior devs.
I rewrote a library with the same interface that did not clone the data, this was safe as the original data wasn't being changed, 15x speedup from 10 req/s to 150 req/s.
Then I dug deeper and changed which data was getting cached. Cache post-render and post-compression instead of the pre-render database calls. 150x speedup from the original or 10x from the previous point. The system went from being able to serve 10 requests/s to 1500 req/s. This change also was safer since nobody is going to ever try to alter compressed data, no such guarantees from the cached database calls.
GC has not been an issue in Java for years. I haven't seen memory issues, except for services running in ridiculously CPU or memory starved kubernetes containers.
Isn't it another example of misaligned incentives? Your website renders on my computer, making rendering performance mostly my problem. Your website is only financially viable because it includes Google adverts and tracking, and Google again isn't paying to run those on my computer.
Backend developer here.
Most (non FAANG) sites are a mess because they're made by inexperienced developers turning everything they touch into a tangled slow mess.
The process goes like this:
- Devs ditch the legacy (often monolithic) code
- They start creating microservices fron scratch
- They don't see the need to optimize or even make readable code. It's a clean slate! the project is small, the new code is just whatever they write.
- They do that with the idea that the legacy code is "bad" and the new code is "good"
- Requirements either scale or downright change
- To the code made before, they add a "tiny" special case and patch here and there. After all, the code is "good"
Repeat
Now you have a new messy project. The dev will either quit or work in a new project. The next dev will do the same. And the new team will probably push for another replacement from scratch, so they can feel that they re creating "good" code again
Most (non FAANG) sites are a mess because they're made by inexperienced developers turning everything they touch into a tangled slow mess.
The process goes like this:
- Devs ditch the legacy (often monolithic) code
- They start creating microservices fron scratch
- They don't see the need to optimize or even make readable code. It's a clean slate! the project is small, the new code is just whatever they write.
- They do that with the idea that the legacy code is "bad" and the new code is "good"
- Requirements either scale or downright change
- To the code made before, they add a "tiny" special case and patch here and there. After all, the code is "good"
Repeat
Now you have a new messy project. The dev will either quit or work in a new project. The next dev will do the same. And the new team will probably push for another replacement from scratch, so they can feel that they re creating "good" code again
Yes, last part of performance tuning is relatively simple process as you described, but that's only because apps where performance matters are developed from the ground app with performance in mind, so entire architecture is designed in a way that supports performance.
Now try to take some React developer that has been working for years in completely different abstraction level, and help him design more performant code base.
You really think all you will talk about are cache misses?
Now try to take some React developer that has been working for years in completely different abstraction level, and help him design more performant code base.
You really think all you will talk about are cache misses?
no, but you can definitely talk about it.
i also had a chat with a C++ dev that created a "Database"-class abstraction for our MongoDB instance and implemented methods to access each field of every documents separately - didn't care about side-effects, didn't know about ACID transactions and didn't care.
finding a good abstraction that allows for performance tuning later on is hard, no matter the language, and react devs should make use of good abstractions as well.
i also had a chat with a C++ dev that created a "Database"-class abstraction for our MongoDB instance and implemented methods to access each field of every documents separately - didn't care about side-effects, didn't know about ACID transactions and didn't care.
finding a good abstraction that allows for performance tuning later on is hard, no matter the language, and react devs should make use of good abstractions as well.
Optimizing Websites is easy: turn on an ad locker.
I was surprised how my fans of my pc went up once I turned off uBlock Origin once
True, although even without ads nor a bloated stack, you start with network latency, code running in a VM that has to use hardware with terrible embedded gpu, accessibility requirements, seo, and so the obligation to use the dom api, the latter not being accessible throught a compiled language.
Also, the website teams is usually smaller, and have a 10th of the time to ship.
But it doesn't excuse everything, for sure. Most websites should be fast given how little they do.
Also, the website teams is usually smaller, and have a 10th of the time to ship.
But it doesn't excuse everything, for sure. Most websites should be fast given how little they do.
Figma is an excellent example of performance being a top priority to set themselves apart, since they weren't the only player in town at the time.
We had Adobe Illustrator, XD was still in beta phases, and Sketch for macOS was showing signs of bloat and major performance hangups in any considerably realistic document. Affinity Designer was also coming into the scene with stellar performance, but had a high learning curve and wasn't well suited for interactive prototyping.
Figma swooped in and solved 3 problems:
1. Availability in the browser on any platform, where native apps can't reach (and thus 10x easier sharing of documents). 2. Incredible performance on first release through their WebGL renderer, rivaling all of the native apps listed above. 3. Stayed flexible, and yet, barebones enough to get at least 80% of a designer's needs covered initially.
Performance (and stability) were primarily what won me over to it, and I'd argue probably the same for many who switched over.
We had Adobe Illustrator, XD was still in beta phases, and Sketch for macOS was showing signs of bloat and major performance hangups in any considerably realistic document. Affinity Designer was also coming into the scene with stellar performance, but had a high learning curve and wasn't well suited for interactive prototyping.
Figma swooped in and solved 3 problems:
1. Availability in the browser on any platform, where native apps can't reach (and thus 10x easier sharing of documents). 2. Incredible performance on first release through their WebGL renderer, rivaling all of the native apps listed above. 3. Stayed flexible, and yet, barebones enough to get at least 80% of a designer's needs covered initially.
Performance (and stability) were primarily what won me over to it, and I'd argue probably the same for many who switched over.
Illustrator's bad performance amazes me. I don't know about newer releases, but CC 2020 is basically single-threaded for everything. It's crazy to see it wait for 8 seconds to make a png vectorial, while using 5% CPU on a 16 core 3950x.
Bingo! That about sums up my memories of using it every since it went CC-anything. I think CS5 was the last version I used that felt pleasantly performant -- and on older machines, no less (2013 Macbook Air w/ i7, 16GB).
The 3d world has long been a curious space to me. Even in late 90s, and early GPUs, you knew how responsive things were compared to 2d applications. It was hard to reconcile in my head.
In my opinion it's because games are art, and art is about feeling. I made a browser game, and recently optimized it heavily. Now it loads a game and huge 3D area in 2-3 seconds, faster than most mainstream websites. Like grandparent said, it is possible...and in art, how something feels is a priority.
> and things shift around on you for another 20 seconds after that, so when you go to tap a link you accidentally tap an ad that finally loaded under your finger.
Why don’t browsers mask out events during render of page regions or note their time and bounding boxes so click events reach the correct element?
Steps to reproduce this annoyance:
- visit Twitter mobile site: mobile.twitter.com in Safari on a slow iPhone (eg iPhone 6 stuck on iOS 12.X). - scroll and view tweets for a while soaking up memory. - visit a tweet - press tweet share icon - popup menu takes forever to render fully to include the Cancel button. - clicking Bookmark tweet menu item will trigger item after that, Copy Link to Tweet, when it finally loads and moves that menu item under your finger.
Why don’t browsers mask out events during render of page regions or note their time and bounding boxes so click events reach the correct element?
Steps to reproduce this annoyance:
- visit Twitter mobile site: mobile.twitter.com in Safari on a slow iPhone (eg iPhone 6 stuck on iOS 12.X). - scroll and view tweets for a while soaking up memory. - visit a tweet - press tweet share icon - popup menu takes forever to render fully to include the Cancel button. - clicking Bookmark tweet menu item will trigger item after that, Copy Link to Tweet, when it finally loads and moves that menu item under your finger.
> Most games render frames of a 3D world in less than 17ms, but most websites take 3-7 seconds to load because of all the ads and bloat
Well now, "most games" will take longer than 3-7 seconds to load. That's fair - they're doing a lot more than websites!
Most websites also render frames in less than 17ms.
Well now, "most games" will take longer than 3-7 seconds to load. That's fair - they're doing a lot more than websites!
Most websites also render frames in less than 17ms.
When I'm working on a web project, like my "PlotBinder" WIP, I spend way more time trying to hit 60fps than I do in my game(s). Adding little css animations or transparent overlays cut the framerate down to 40fps on my old laptop until I tinkered for a while with it and made some compromises. The way pages render on the web is finnicky to optimize for.
most websites take 3-7 seconds to load because of all the ads and bloat, and things shift around on you for another 20 seconds after that, so when you go to tap a link you accidentally tap an ad that finally loaded under your finger
The problem with remarks like this is that if this was really your experience of using the web you'd have installed an ad blocker years ago, and you wouldn't make that argument now. Consequently either it's easily dismiasable as hyperbole or you're a masochist who enjoys terrible websites.
There are bad websites. It isn't "most websites" though. Just like some games drop frames horribly, but not "most games".
The problem with remarks like this is that if this was really your experience of using the web you'd have installed an ad blocker years ago, and you wouldn't make that argument now. Consequently either it's easily dismiasable as hyperbole or you're a masochist who enjoys terrible websites.
There are bad websites. It isn't "most websites" though. Just like some games drop frames horribly, but not "most games".
I'm using an ad blocker, and with a cold cache, clicking a Twitter link shows a blue bird for a second, then two blue spinners (center and top) and then after 3 or 4 more seconds I can finally see a message that's a maximum of 280 characters long. News sites don't load much faster. Pages like HN are the exception, and mostly come from the tech corner. I've been on some brand's online store a few days ago and it felt like they want to drive customers away.
But fair enough, it might not actually be most websites, but enough to not be a rare experience either.
But fair enough, it might not actually be most websites, but enough to not be a rare experience either.
I just checked twitter's network log on a single tweet by John Carmack.
- 131 network calls
- 280 kilobytes of data received
- 4500ms total network time
Thankfully, the vast majority of other websites aren't nearly this bad, but we're trending towards twitter as an industry, not away from it.
I try to write API calls that provide broadband utility and return lightning fast by always making sure my critical path gets a fair shake for optimization, but I know there exist calls out there in react land that take my 220ms response and spend an extra 7 seconds transforming it to populate a (paginated) table on the UI.
- 131 network calls
- 280 kilobytes of data received
- 4500ms total network time
Thankfully, the vast majority of other websites aren't nearly this bad, but we're trending towards twitter as an industry, not away from it.
I try to write API calls that provide broadband utility and return lightning fast by always making sure my critical path gets a fair shake for optimization, but I know there exist calls out there in react land that take my 220ms response and spend an extra 7 seconds transforming it to populate a (paginated) table on the UI.
Amount of data transmitted seems fine...but 131 network calls? No wonder it always feels so slow to use Twitter.
Keyword and. Ads and bloat. Adblocking doesn't get rid of all bloat.
And GP's point still stands. 17ms vs the typical 2s minimum a site takes to load is massive given the differences in what is demanded. Games actively got faster despite being more intense, websites active got slower despite relatively similar output.
Webdev's performance is shocklingly bad relative to most other subfields.
And GP's point still stands. 17ms vs the typical 2s minimum a site takes to load is massive given the differences in what is demanded. Games actively got faster despite being more intense, websites active got slower despite relatively similar output.
Webdev's performance is shocklingly bad relative to most other subfields.
GP's point ignores the load times of a game (the time it takes to have a usable game-no progress bar), and instead compares FRAME time (17 ms to render a single frame) with the LOAD TIME of a website.
You too ignore this.
You too ignore this.
Comparing the loading time of a website to the frame render time of a console game hardly seems fair.
Plenty of websites scroll at 60fps or 120fps just fine (and those that don’t definitely deserve to be criticised).
How many console games can load from scratch in only a few seconds?
Plenty of websites scroll at 60fps or 120fps just fine (and those that don’t definitely deserve to be criticised).
How many console games can load from scratch in only a few seconds?
This is a reasonable point, _but_ how much data do games need to load versus a website. Using Twitter as an example, when I open it I can see 4 tweets. Assuming each tweet is 1MB (for easy math), I have a 500Mb WiFi connection, I should be be able to do 100 tweets in under a second. Meanwhile most/many games are pulling in gigabytes of data in textures and audio data. Spiderman on PS5 is under 15 seconds from the home screen to being in game, playing which is about how long it takes for the Facebook mobile website to load.
Comparing latency and throughput of local and remote assets says nothing.
Your comparing gigabytes of data to a few MB. Now that hardly seems fair.
You might know this already but in case you didn't: ad-blockers are the reason browsers are considered greedy pigs when it comes to resources.
It's no longer a universal truth that a adblocker will save you memory or CPU, ad-blockers contribute greatly to the per-tab memory cost. They also slow down page rendering greatly.
But, hey, the web is an unusable mess without them so we all use one and it's considered the cost of doing business.
It's no longer a universal truth that a adblocker will save you memory or CPU, ad-blockers contribute greatly to the per-tab memory cost. They also slow down page rendering greatly.
But, hey, the web is an unusable mess without them so we all use one and it's considered the cost of doing business.
noscript can block most ads without much memory overhead
[deleted]
Websites can def be fast, but to your point it’s a priority problem.
This comparison I made at https://legiblenews.com/speed that got upvoted here a while back shows just that—most news websites care about ads first before their users speed experience. Only Legible News, USA Today, and Financial Times have a reasonable speed score, which is kind of depressing.
This comparison I made at https://legiblenews.com/speed that got upvoted here a while back shows just that—most news websites care about ads first before their users speed experience. Only Legible News, USA Today, and Financial Times have a reasonable speed score, which is kind of depressing.
> I'm a game developer and game performance is better than ever.
I'm a game developer (and player) and I strongly disagree. What game can even do P95 144FPS on affordable hardware? On lowest graphics, with populated lobbies (there also can't be a that ONE map where this all goes to shit). And are we talking about AAA games or not? Because non-AAA means a minecraft knockoff cannot meet 60FPS.
I'm a game developer (and player) and I strongly disagree. What game can even do P95 144FPS on affordable hardware? On lowest graphics, with populated lobbies (there also can't be a that ONE map where this all goes to shit). And are we talking about AAA games or not? Because non-AAA means a minecraft knockoff cannot meet 60FPS.
> I'm a game developer and game performance is better than ever.
It could be, if users would have more control over the graphics settings of the games that they want to play and they were allowed to scale back further to accommodate older hardware.
For example, the Unity engine by default has a setting to allow downscaling the texture resolution that the game will use 2X, 4X and 8X, yet many games out there actually disable that. Same for some options menus not having framerate limits, dynamic render resolutions (though most engines support that functionality in one way or the other), particle density/soft particle options, options to disable SSAO/HBAO or other post processing like that, as well as enabling/disabling tessellation.
The end result is that many games that could run passably on integrated graphics or hardware from a few generations ago (e.g. GTX 650 Ti) instead struggle greatly, because the people behind the game either didn't care or didn't want to allow it to ever look "bad" in their pursuit of mostly consistent graphical quality (and thus how the game will look in the videos/screenshots out there).
The only real exception to this are e-sports titles, something like CS:GO is optimized really well for performing across a variety of hardware, while also giving the user the controls over how the game will look (and run). Games like DOOM are also a good example, but they're generally the exception, because most don't care about such technical excellence (though it's useful when you try porting the game to something like Nintendo Switch).
Most other games don't give you that ability, just because they try to always do more stuff, which isn't that different from Wirth's law (software gets slower as hardware gets faster). Of course, this is also prevalent in indie titles, many of which don't even have proper LOD setups, because engines like Unity don't automatically generate LOD models and something like Godot 3 didn't even have any sort of LOD functionality out of the box.
Engines like Unreal might make this better with Nanite, except that most people will use it for shoving more details into the games (bloating install sizes a bit), instead of as a really good LOD solution. That said, Godot 4 is also headed in the right direction and even for Godot 3 there are plugins (even though it's just like Unity, where you still need to make the models yourself), for which I actually ported the LOD plugin from GDScript to C#: https://blog.kronis.dev/articles/porting-the-godot-lod-plugi...
It could be, if users would have more control over the graphics settings of the games that they want to play and they were allowed to scale back further to accommodate older hardware.
For example, the Unity engine by default has a setting to allow downscaling the texture resolution that the game will use 2X, 4X and 8X, yet many games out there actually disable that. Same for some options menus not having framerate limits, dynamic render resolutions (though most engines support that functionality in one way or the other), particle density/soft particle options, options to disable SSAO/HBAO or other post processing like that, as well as enabling/disabling tessellation.
The end result is that many games that could run passably on integrated graphics or hardware from a few generations ago (e.g. GTX 650 Ti) instead struggle greatly, because the people behind the game either didn't care or didn't want to allow it to ever look "bad" in their pursuit of mostly consistent graphical quality (and thus how the game will look in the videos/screenshots out there).
The only real exception to this are e-sports titles, something like CS:GO is optimized really well for performing across a variety of hardware, while also giving the user the controls over how the game will look (and run). Games like DOOM are also a good example, but they're generally the exception, because most don't care about such technical excellence (though it's useful when you try porting the game to something like Nintendo Switch).
Most other games don't give you that ability, just because they try to always do more stuff, which isn't that different from Wirth's law (software gets slower as hardware gets faster). Of course, this is also prevalent in indie titles, many of which don't even have proper LOD setups, because engines like Unity don't automatically generate LOD models and something like Godot 3 didn't even have any sort of LOD functionality out of the box.
Engines like Unreal might make this better with Nanite, except that most people will use it for shoving more details into the games (bloating install sizes a bit), instead of as a really good LOD solution. That said, Godot 4 is also headed in the right direction and even for Godot 3 there are plugins (even though it's just like Unity, where you still need to make the models yourself), for which I actually ported the LOD plugin from GDScript to C#: https://blog.kronis.dev/articles/porting-the-godot-lod-plugi...
How prevalent is a lack of graphics options really? To me it’s an expected piece of functionality in a PC build and in most games the first place I go to and turn DoF and motion blur off!
It's extremely common in PC ports of console exclusives/console first games.
If the game is popular then someone usually comes along with a DLL that allows you to tweak at least some of the settings that weren't exposed.
If the game is popular then someone usually comes along with a DLL that allows you to tweak at least some of the settings that weren't exposed.
I absolutely hate motion blur and depth of field, even if my computer is fast enough to render them.
You'd be surprised. Here's a Wiki that attempts to document various configuration options, tricks and utilities for many games and they have tables with what features are supported in the games that are featured on the site: https://www.pcgamingwiki.com/wiki/List_of_lists#Video
While it's not an exhaustive list, you can open any of those features and sort the table by whether the game supports the functionality itself, needs some sort of a config/tool fix/hack or does not support in any known way.
(the list is so long that you might have to view it with 500 items per page and even then jump through multiple pages, I wonder whether the dataset can have aggregate queries against it done)
While it's not an exhaustive list, you can open any of those features and sort the table by whether the game supports the functionality itself, needs some sort of a config/tool fix/hack or does not support in any known way.
(the list is so long that you might have to view it with 500 items per page and even then jump through multiple pages, I wonder whether the dataset can have aggregate queries against it done)
> Most games render frames of a 3D world in less than 17ms, but most websites take 3-7 seconds to load because of all the ads and bloat, and things shift around on you for another 20 seconds after that
I know what you want to say (and I agree), but... website rendering depends on the network mostly, 16ms latency is already the top 0.5% of fiber users and you have to add that on top of every new connection... GPU rendering happens on a bus that is thousands of times faster, and it needs to cover a minuscule distance. You can't really compare the two.
I know what you want to say (and I agree), but... website rendering depends on the network mostly, 16ms latency is already the top 0.5% of fiber users and you have to add that on top of every new connection... GPU rendering happens on a bus that is thousands of times faster, and it needs to cover a minuscule distance. You can't really compare the two.
I have 1 ms latency to 1.1.1.1 and a 10 gbps connection - faster than a SATA SSD - and most websites are still slow.
Agreed. Why is it taking my garden hose so long to fill my swimming pool when tsunamis can do it in a matter of milliseconds?
Differences in order of magnitude.
That said, websites and even web apps really should be faster.
Differences in order of magnitude.
That said, websites and even web apps really should be faster.
> (note: games lag as well when you drag in a million dependencies you don't need)
No they don't. Your compiler will eliminate dead code.
No they don't. Your compiler will eliminate dead code.
It think the implication was that the dep is used but not actually needed.
That is a flattering interpretation.
Parent did write that "game lags as well when you drag in a million dependencies".
That is hard for me to interpret like you did.
Parent did write that "game lags as well when you drag in a million dependencies".
That is hard for me to interpret like you did.
> For most software, functionality, signing up for a subscription, platform availability etc are usually prioritized higher than response times and keyboard shortcuts.
This is why I love working in fintech. The engineering is paramount. Customers will not accept slow or buggy software.
I get to solve hard problems, and really build systems from the ground up. My managers understand that it is better to push back a deadline than to ship something that isn't up to standard.
This is why I love working in fintech. The engineering is paramount. Customers will not accept slow or buggy software.
I get to solve hard problems, and really build systems from the ground up. My managers understand that it is better to push back a deadline than to ship something that isn't up to standard.
Yes, but all in service of what amounts to automated stealing (cough) supplying liquidity.
Better than adtech, anyway. Or nukes. Lots of things, really. (I would have said weapons, last year.)
The only really defensible tech activity these days is things to help get off carbon-emitting processes. Making factories to make electrolysers. Making wind turbines better. Adapting airliners to carry liquid hydrogen in underwing nacelles. Making robots to put up solar fences on farms and pastures. Banking energy for nighttime without lithium. Making ammonia on tropical solar farms for export to high latitudes.
It's even money whether we can get it done before civilization collapses. I guess we will need plenty of liquidity...
Better than adtech, anyway. Or nukes. Lots of things, really. (I would have said weapons, last year.)
The only really defensible tech activity these days is things to help get off carbon-emitting processes. Making factories to make electrolysers. Making wind turbines better. Adapting airliners to carry liquid hydrogen in underwing nacelles. Making robots to put up solar fences on farms and pastures. Banking energy for nighttime without lithium. Making ammonia on tropical solar farms for export to high latitudes.
It's even money whether we can get it done before civilization collapses. I guess we will need plenty of liquidity...
Those are worthwhile problems to solve, but they seem well on their way to getting solved, though it's not guaranteed. Your civilization is collapsing because you're fighting each other, not because of climate change. Fighting each other is also why it's not guaranteed.
Other problems are also worthwhile to solve. I agree that zero-sum HFT and negative-sum adtech and arms races are not the most prominent ones.
Other problems are also worthwhile to solve. I agree that zero-sum HFT and negative-sum adtech and arms races are not the most prominent ones.
> Your civilization is collapsing because you're fighting each other, not because of climate change
10% of all deaths each year are due to air pollution. There's pretty much no place on earth where unfiltered rainwater is still safe to drink. Ocean acidification is accelerating and killing off the organisms that are both the biggest producers of oxygen and the biggest sequesterers of carbon.
10% of all deaths each year are due to air pollution. There's pretty much no place on earth where unfiltered rainwater is still safe to drink. Ocean acidification is accelerating and killing off the organisms that are both the biggest producers of oxygen and the biggest sequesterers of carbon.
Air pollution is not due to climate change, unless you count CO₂ as air pollution, but that's eargrayish misreasoning — that's not the kind of air pollution that causes 10% of all deaths. The sort of local air pollution we're talking about amounts to factories strip-mining local children's lungs for profit; it's a form of fighting each other.
Rainwater harvesting risks seem to be primarily due to birds pooping on your roof and other fecal contamination, and they can be adequately mitigated by boiling or chemical sterilization: https://www.nature.com/articles/s41545-019-0030-5.
Climate change very likely would collapse your civilization, given enough time, but that isn't what is happening now.
Rainwater harvesting risks seem to be primarily due to birds pooping on your roof and other fecal contamination, and they can be adequately mitigated by boiling or chemical sterilization: https://www.nature.com/articles/s41545-019-0030-5.
Climate change very likely would collapse your civilization, given enough time, but that isn't what is happening now.
If we spent on those what we spend instead on adtech etc., it really would be guaranteed. Even with fighting invaders. Albeit probably not invaders using nukes.
I think 228 GWp of solar power generation capacity is being installed in 02022, which is about US$50 billion for the solar modules and another US$50 billion for balance of plant, not counting things like battery storage, pumped storage, transmission, and Fischer-Tropsch. Wind energy generation capacity is getting installed at a similar pace, in terms of peak gigawatts per year, but I have the impression that it is significantly cheaper. Tesla's revenues are US$53.8 billion per year, of which about US$0.07 billion is solar panels and therefore getting double counted. I don't know how to measure other automakers' EV production, housing switching to heat pumps, housing adding insulation, and R&D, so I'm just not counting them.
Alphabet's yearly revenue is US$258 billion, some of which is earned from things like Google Play, Google Cloud, and Google Workspace, and some of which has been spent on things like Makani, Verily Life Sciences, DeepMind, and Google Fiber. Meta's revenue is US$118 billion.
https://www.globaldata.com/data-insights/internet-services-t... says, "Google earned nearly 81% of its total revenues in 2021 from advertisements," and it's talking about that US$258B number, not some smaller number that excludes other Alphabet companies.
We can maybe estimate the whole adtech spend, generously, at US$400 billion per year [edited: was US$300 billion]: most of Alphabet, all of Meta, plus a handful of smaller fish. (I say this is "generous" because it's counting all the effort that goes into running Google Search, YouTube, Instagram, and Android development as "adtech", because it's monetized through ads, even though most of the people working on those projects are not directly concerned with ads; if you omit that, you might size adtech at only US$50 billion.) And maybe US$200 billion per year goes into the renewables transition. I'm not convinced that doubling that to US$400 billion per year would guarantee a win, particularly when the humans are going around bombing each other's power plants, snarling up power plant construction and energy-efficient housing in red tape, and publishing disinformation that "debunks" global warming.
These are of course very rough estimates, but how would you estimate these numbers?
As far as I know, there aren't any invaders the humans are fighting; they're only fighting other humans, plus the occasional rogue elephant.
Alphabet's yearly revenue is US$258 billion, some of which is earned from things like Google Play, Google Cloud, and Google Workspace, and some of which has been spent on things like Makani, Verily Life Sciences, DeepMind, and Google Fiber. Meta's revenue is US$118 billion.
https://www.globaldata.com/data-insights/internet-services-t... says, "Google earned nearly 81% of its total revenues in 2021 from advertisements," and it's talking about that US$258B number, not some smaller number that excludes other Alphabet companies.
We can maybe estimate the whole adtech spend, generously, at US$400 billion per year [edited: was US$300 billion]: most of Alphabet, all of Meta, plus a handful of smaller fish. (I say this is "generous" because it's counting all the effort that goes into running Google Search, YouTube, Instagram, and Android development as "adtech", because it's monetized through ads, even though most of the people working on those projects are not directly concerned with ads; if you omit that, you might size adtech at only US$50 billion.) And maybe US$200 billion per year goes into the renewables transition. I'm not convinced that doubling that to US$400 billion per year would guarantee a win, particularly when the humans are going around bombing each other's power plants, snarling up power plant construction and energy-efficient housing in red tape, and publishing disinformation that "debunks" global warming.
These are of course very rough estimates, but how would you estimate these numbers?
As far as I know, there aren't any invaders the humans are fighting; they're only fighting other humans, plus the occasional rogue elephant.
Maybe boosting $200B to $500B would provide a win, but ok, maybe not.
Invasion is an activity, not an identity. Those so engaged could stop anytime. Many will.
Invasion is an activity, not an identity. Those so engaged could stop anytime. Many will.
Nobody is, as far as I know, invading the humans' territory from someplace else, such as Delta Centauri. Humans invading other humans' territory is just a sort of humans fighting among themselves, but not, currently, the sort that is the biggest obstacle to the renewable transition.
Well, there's education - that's net good. And I think entertainment is pretty neutral (even slightly positive) if it doesn't involve gambling or rely on player addiction. Fintech and adtech are where people bright enough to contribute positively to society go to make money instead, kind like the oil industry. All IMHO, YMMV, etc.
Who would they be stealing from?
The body politic.
People program computers to exchange messages furiously, to no external effect besides power consumed. Then they go buy yachts. It's all perfectly legal, of course.
People program computers to exchange messages furiously, to no external effect besides power consumed. Then they go buy yachts. It's all perfectly legal, of course.
I'm not sure how this 'stealing' is supposed to work.
The stock market is a zero-sum game. Every single penny taken out was a penny put in. Money going in is people's pensions, largely.
As a society, we like to reward people who do things that are useful: feed, house, clothe, warm, transport, educate, protect, medicate, lately amuse people, and get money that may be exchanged for those services, or yachts.
HFT traders provide none of those, nor anything comparable, but get yachts anyway.
As a society, we like to reward people who do things that are useful: feed, house, clothe, warm, transport, educate, protect, medicate, lately amuse people, and get money that may be exchanged for those services, or yachts.
HFT traders provide none of those, nor anything comparable, but get yachts anyway.
>The stock market is a zero-sum game. Every single penny taken out was a penny put in. Money going in is people's pensions, largely.
Ah, so you lack a basic understanding of how the stock market works, but think you're qualified to make very strong statements about it.
Ah, so you lack a basic understanding of how the stock market works, but think you're qualified to make very strong statements about it.
Trades batched by second would provide as much liquidity as anybody could use, without the yachts.
You can start a new exchange that does this batching.
Btw, you need to be careful, because people might still race to be the last (or first) to contribute to a batch.
They'd race to be the first, if tied prices within a batch are resolved in favour of the earlier bird. Otherwise, they would race to be the last, so they have the most information available when deciding on their price (and so that they can change their mind until the last nano-second.)
Btw, you need to be careful, because people might still race to be the last (or first) to contribute to a batch.
They'd race to be the first, if tied prices within a batch are resolved in favour of the earlier bird. Otherwise, they would race to be the last, so they have the most information available when deciding on their price (and so that they can change their mind until the last nano-second.)
And if ties within that second were resolved randomly, what perverse optimizing would you predict?
Well traders do provide liquidity which saves pensioners enormous sums compared to selling retirement assets at will into illiquid markets
Granted traders can play other games, but their primary benefit to others is significant
Granted traders can play other games, but their primary benefit to others is significant
> The engineering is paramount.
You must be lucky, not everywhere in the sector that’d be remotely close to true unfortunately…
You must be lucky, not everywhere in the sector that’d be remotely close to true unfortunately…
He's probably referring to the trading desk or similar teams.
Stock traders forced a lot of advances onto software. Random examples of high-perf stuff from that space include the new garbage collector in the JVM with a minimal pause time and LMAX Disruptor. Multi-threaded GUIs are relatively common in that space as well, to ensure that one hung control or window won't stop anything else.
Stock traders forced a lot of advances onto software. Random examples of high-perf stuff from that space include the new garbage collector in the JVM with a minimal pause time and LMAX Disruptor. Multi-threaded GUIs are relatively common in that space as well, to ensure that one hung control or window won't stop anything else.
I've known a handful of software engineers who have stopped through The Trade Desk for a year or less. It doesn't sound like a great place to work to me, despite the high throughput their software demands.
> This is why I love working in fintech.
The prototypical nerd would happily work on building a nuke capable of destroying a continent if that entailed letting him work on “hard problems”.
The prototypical nerd would happily work on building a nuke capable of destroying a continent if that entailed letting him work on “hard problems”.
I wouldn’t work on a continent-destroying nuke unless the project used Rust. Wouldn’t want it to be unsafe.
Obligatory - https://devblogs.microsoft.com/oldnewthing/20180228-00/?p=98...
Missiles don't really need GC or RAII. They have their own special resource clean up.
Missiles don't really need GC or RAII. They have their own special resource clean up.
Oh you mean the Manhattan project?
I don't know what you mean by "fintech" but from my experience, bank and other finance apps are usually not that great, neither are the websites. So maybe some parts of fintech are nice and clean, but the part that the end user faces, not so much.
I mean the actual tech than enables financial markets (fintech) - high frequency trading, matching engines, market data, order execution, etc.
Yeah I work in finance too and it's a mixed bag. The stuff I work on right now has a very high focus on correctness and stability which is nice.
But I've also seen productionized jupyter notebooks written by just out of school data scientists and 1000 lines long SQL queries that encoded important business logic and weren't version controlled.
But I've also seen productionized jupyter notebooks written by just out of school data scientists and 1000 lines long SQL queries that encoded important business logic and weren't version controlled.
If I am not mistaken, fintech refers to the software/hardware surrounding trade execution on the various exchanges (stocks, bonds, commodities, etc.) where speed is of primary importance.
Now I want to work in a fintech
Sounds like places Id love. May I ask what company youre working for?
This has pretty much been the exact opposite of my experience in fintech. But it's a startup so maybe that's the difference
[deleted]
The way that game developers get their performance is more or less orthogonal to the way many other applications are expected to function.
It is impressive that they can draw so much stuff so fast, but there are actually very few objects on the screen that the user can directly interact with.
A specific example: in a DAW, you might have tens or even hundreds of thousands of MIDI notes on the screen. These look like (typically) little rectangles which are precisely the short of thing that games can draw at unbelievable speed. But in a DAW (and most design / creation applications), every single one of them is potentially the site of user interaction with some backend model object.
All those complex surfaces you see in contemporary games? Very nice. But the user cannot point at an arbitrary part of a rock wall and say "move this over a bit and make it bigger".
Consequently, the entire way that you design and implement the GUI is different, and the lessons learned in one domain do not map very easily to the other.
It is impressive that they can draw so much stuff so fast, but there are actually very few objects on the screen that the user can directly interact with.
A specific example: in a DAW, you might have tens or even hundreds of thousands of MIDI notes on the screen. These look like (typically) little rectangles which are precisely the short of thing that games can draw at unbelievable speed. But in a DAW (and most design / creation applications), every single one of them is potentially the site of user interaction with some backend model object.
All those complex surfaces you see in contemporary games? Very nice. But the user cannot point at an arbitrary part of a rock wall and say "move this over a bit and make it bigger".
Consequently, the entire way that you design and implement the GUI is different, and the lessons learned in one domain do not map very easily to the other.
A user of Blender can absolutely point at an arbitrary part of a rock wall and say "move this over and make it bigger". Blender sacrifices rendering quality to make sure that interaction is reliably responsive. Cube/Sauerbraten forgoes some of the rendering optimizations provided by some other 3-D game engines to make sure you can always edit any part of the environment at any time, but it was already delivering interactive frame rates 20 years ago. And of course Minetest has very little trouble with arbitrary sets of nodes appearing and disappearing from one frame to the next, but Minetest isn't that great at performance, and its expressiveness is a bit limited compared to Cube and Blender, so maybe it's a less compelling example.
As long as it doesn't cause a glitch in playback, it's acceptable for your DAW to delay 10 milliseconds to figure out which note you clicked on. That's about 100 million instructions on one core of the obsolete laptop I'm typing this on. As you obviously know, that's plenty of time to literally iterate over your hundreds of thousands of little rectangles one by one, in a single thread, testing each one to see if it includes the click position.
But (again, as you obviously know) you don't have to do that; for example, you can divide the screen into 32×32 tiles, maybe 8192 of them, and store an array of click targets for each tile, maybe up to 2048 of them, but on average maybe 64 of them, sorted by z-index. If a click target overlaps more than one tile, you just store it in more than one tile. When you have a click, you bit-shift the mouse coordinates and combine them to index the tile array, then iterate over the click targets in the array until you find a hit. This is thousands of times faster than the stupid approach and we haven't even gotten to quadtrees.
A different stupid approach is to assign each clickable object a unique z-coordinate and just index into the z-buffer to instantly find out what the person clicked. This requires at least a 24-bit-deep z-buffer if you have potentially hundreds of thousands of MIDI notes. But that's fine these days, and it's been fine for 25 years if you were rendering the display in software.
As long as it doesn't cause a glitch in playback, it's acceptable for your DAW to delay 10 milliseconds to figure out which note you clicked on. That's about 100 million instructions on one core of the obsolete laptop I'm typing this on. As you obviously know, that's plenty of time to literally iterate over your hundreds of thousands of little rectangles one by one, in a single thread, testing each one to see if it includes the click position.
But (again, as you obviously know) you don't have to do that; for example, you can divide the screen into 32×32 tiles, maybe 8192 of them, and store an array of click targets for each tile, maybe up to 2048 of them, but on average maybe 64 of them, sorted by z-index. If a click target overlaps more than one tile, you just store it in more than one tile. When you have a click, you bit-shift the mouse coordinates and combine them to index the tile array, then iterate over the click targets in the array until you find a hit. This is thousands of times faster than the stupid approach and we haven't even gotten to quadtrees.
A different stupid approach is to assign each clickable object a unique z-coordinate and just index into the z-buffer to instantly find out what the person clicked. This requires at least a 24-bit-deep z-buffer if you have potentially hundreds of thousands of MIDI notes. But that's fine these days, and it's been fine for 25 years if you were rendering the display in software.
Blender, as a design/creation tool, is something I would classify as closer to a DAW than most contemporary video games. The fact that you can use it to create games isn't central.
Its performance is very gamelike, I think, and your point seemed to be that design/creation tools like DAWs can't get gamelike performance, so I was offering it as a counterexample. I agree that the fact that you can use it to create games is irrelevant, and that wasn't a fact I was attempting to focus on.
With a modern data driven or ECS architecture, you actually can do that. There are demos out there with hundreds of interactive things on screen at one time. It's kind of amazing.
It's not like games don't have complex UIs either, where each button or field has a lot of logic to them. Many games have very simple UIs, but others get more complex than many complex web apps. Some are even multiplayer, and the server code... it's nuts how much work goes into this. The number of updates you send each second to keep players in a game in sync compared to what is needed to keep a chat app in sync is really impressive to me.
From a technical point of view, games are really cool!
It's not like games don't have complex UIs either, where each button or field has a lot of logic to them. Many games have very simple UIs, but others get more complex than many complex web apps. Some are even multiplayer, and the server code... it's nuts how much work goes into this. The number of updates you send each second to keep players in a game in sync compared to what is needed to keep a chat app in sync is really impressive to me.
From a technical point of view, games are really cool!
I feel like the reason a lot of games are slow right now is because they are built out of these huge abstract "everything-doers". For example, each button can be configured in millions of different ways even though the game usually only has a few different types of buttons. It's not an optimal data structure by any means.
It's not like in the olden days where you would blit this stuff out to the screen using specialized routines that scream along as fast as the CPU will let you. Now, everything is dynamic and multipurpose and configured using properties on the object... engines like Unity get away with C# and way too much runtime reflection because it's convenient in the editor. Developers like it, it's less difficult than writing specialized routines.
It's not like in the olden days where you would blit this stuff out to the screen using specialized routines that scream along as fast as the CPU will let you. Now, everything is dynamic and multipurpose and configured using properties on the object... engines like Unity get away with C# and way too much runtime reflection because it's convenient in the editor. Developers like it, it's less difficult than writing specialized routines.
> It is impressive that they can draw so much stuff so fast, but there are actually very few objects on the screen that the user can directly interact with.
> > With a modern data driven or ECS architecture, you actually can do that.
At some point people forgot that there are synonyms and different ways to phrase “actually”.
> > With a modern data driven or ECS architecture, you actually can do that.
At some point people forgot that there are synonyms and different ways to phrase “actually”.
>All those complex surfaces you see in contemporary games? Very nice. But the user cannot point at an arbitrary part of a rock wall and say "move this over a bit and make it bigger".
The Red Faction series, building games like Minecraft and 7 Days to Die, and games like Factorio are some pretty obvious examples where you're completely and utterly wrong, so I'm not really sure why I should trust anything you said in the rest of your comment.
The Red Faction series, building games like Minecraft and 7 Days to Die, and games like Factorio are some pretty obvious examples where you're completely and utterly wrong, so I'm not really sure why I should trust anything you said in the rest of your comment.
He's a pretty experienced and expert programmer, but I don't think he's much of a gamer.
You might be underestimating modern games a bit I feel, but as well game engines usually have editors just as complex as a DAW. Yet both games, game engine editors, DAWs and all the examples you mentioned all run vastly more performantly than many modern simple pieces of software. Which tells us that performance is possible regardless of domain if we build properly.
Yes, but the editors are (to the best of my understanding) not written using the same software techniques as the games themselves.
Don't know about the others, but I think Godot's editor is built using its own in-game UI system.
And Godot's editor is very snappy and responsive
UE5's editor is completely built in the engine's native UI toolkit, Slate.
No not all of them. The godot game engine editor is built in the godot game engine though so that is interesting.
There are lots of games with destructible environments and thousands or tens of thousands of interactive objects moving around, the static world you're describing hasn't been the Only Way for quite a while. Ignoring the obvious cases like Minecraft, Red Faction and Fortnite, even games where it isn't relevant to the gameplay still implement it sometimes - for example, The Division (2016) had fully destructible walls in many locations and if you fired enough rounds into a concrete wall you could punch a big hole in it that could be seen through and fired through to attack enemies or heal allies. This sort of thing doesn't have to come at the expense of visuals either, modern rendering tech along the lines of Unreal's Lumen & Nanite can adapt to changes in the environment and handle massive dynamic crowds of people or groups of cars.
You can quite literally point at an arbitrary part of a rock wall and move it over a bit then make it bigger depending on the engine you're using. It's why Unreal Engine has got a foothold in TV production.
You can quite literally point at an arbitrary part of a rock wall and move it over a bit then make it bigger depending on the engine you're using. It's why Unreal Engine has got a foothold in TV production.
What? What are you talking about? Games handle thousands of colliders and raycasts just fine. I thought you would mention that games get to use dedicated GPU hardware and apps might not be hardware accelerated but colliders? Very odd take.
Eh, the actual number of interactive items in a modern game is significantly less than thousands. Probably more like a few dozen when all is said and done.
You've missed my point entirely.
Games (generally speaking) do not involve allowing the user to interact in arbitrary user-defined ways with every object represented on the screen. The interactions that do happen are part of the physics engine not the "user clicked that and dragged it to the left" engine.
Games (generally speaking) do not involve allowing the user to interact in arbitrary user-defined ways with every object represented on the screen. The interactions that do happen are part of the physics engine not the "user clicked that and dragged it to the left" engine.
No I think klabb3 has the right of it. They're really not that different and it does just boil down to priority and what is fine to be outsourced to a huge stack of slow libraries and what is prioritized as a core competency.
Yeah, go check out some Factorio screenshots. Thousands of interactive objects animating on screen at once. Not only that, there is a complex sim running in the background. On my machine it runs well past 100fps.
Or 2d strategy games. Which had very decent number of animated units each with their own actions and which you could click and select...
It took the devs years to optimize it to that point though. And I'm quite sure that a screen full of trees or smoke emitting entities (thankfully you can turn smoke/steam off in settings) still makes it go below 60fps on my laptop.
Optimize, yes. But then... It's doing several (hundreds) of orders of magnitudes more than a typical app.
Several hundreds of orders of magnitude more? That doesn't sound possible.
That was an exaggeration but only slightly I guess :)
Considering how many objects and interactions Factorio tracks and at the same time lets you move around, inspect any of those things etc.
A typical app on the other hand does a CRUD operation every once in a blue moon.
Considering how many objects and interactions Factorio tracks and at the same time lets you move around, inspect any of those things etc.
A typical app on the other hand does a CRUD operation every once in a blue moon.
This is nonsense. First, no one can interact in "arbitrary user-defined ways" with anything. Second, anything you can imagine on the web and more has already been done in games. Please don't tell people that running a 3d strategy game or a AAA multiplayer first-person shooter is somehow less complex than scrolling an effing window with a bunch of text and forms in it.
> Games (generally speaking) do not involve allowing the user to interact in arbitrary user-defined ways with every object represented on the screen.
Ah yes, and which apps actually allow you interaction in arbitrary user defined ways?
Ah yes, and which apps actually allow you interaction in arbitrary user defined ways?
IIUC, in a game, let's suppose you pick up an object. We'll ignore the fact that you probably provide some sort of fairly broad movement to indicate that you want to pick it up - the game just notices this, and your on-screen avatar takes care of the rest. But now that you have it in your hand, you do not actually have control over in a direct sense. You may indicate that you want to lift your arm - all fine, but that information will be fed into a physics engine that will work on actual speed & trajectory, obstacles and collisions etc. What is shown on screen is not the result of you dragging a GUI object across the screen, but the indirect result of your indicating "lift my right arm" etc.
By contrast, most design/creation applications give you the ability to point with the mouse (or touchpad or whatever) at some object on the screen and then move it (sometimes along 3 axes). The speed of the motion, the axial extents and the destination come (at least initially) from you. This is true of drawing applications, design applications, DAWs and lots more.
By contrast, most design/creation applications give you the ability to point with the mouse (or touchpad or whatever) at some object on the screen and then move it (sometimes along 3 axes). The speed of the motion, the axial extents and the destination come (at least initially) from you. This is true of drawing applications, design applications, DAWs and lots more.
Games have come a long way since Space Invaders and Pac Man.
They routinely do have hundreds, or thousands, of interactive things. Especially things like RTS games. But also, even if you look at turn-based strategy games, which have a much more application-like interface. Every hexagon, terrain feature, and unit is interactive, along with a full application-like menu, statusbar, and UI system.
They routinely do have hundreds, or thousands, of interactive things. Especially things like RTS games. But also, even if you look at turn-based strategy games, which have a much more application-like interface. Every hexagon, terrain feature, and unit is interactive, along with a full application-like menu, statusbar, and UI system.
> But the user cannot point at an arbitrary part of a rock wall and say "move this over a bit and make it bigger".
In some games, you can.
In some games, you can.
> Alternative hypothesis: (brace yourselves) people don't care enough. Any vendor will prioritize requirements, if performance is not in there, that CPU and memory is going to be used if in any way it helps the developers. Conversely, by looking at a system you can infer its requirements.
I think most of all, it isn't sufficiently visible. Most development is done on high powered hardware that makes slow code very difficult to distinguish from fast code, even though you can often get 10x performance improvements without sacrificing readability or development effort.
Individually it's just a millisecond wasted here and there, but all these small inefficiencies add up across the execution path.
Here's a fun benchmark to illustrate how incomplete beliefs like "compilers are smart enough to magically make this not matter" can be:
https://memex.marginalia.nu//junk/DedupTest.gmi
It's a 50x difference between the common idiomatic approach and the somewhat optimized greybeard solution, with a wide spectrum of both readability and performance in-between.
If you put zero thought toward this, your modern code will make your modern computer run as though it was a computer from the late '90s.
I think most of all, it isn't sufficiently visible. Most development is done on high powered hardware that makes slow code very difficult to distinguish from fast code, even though you can often get 10x performance improvements without sacrificing readability or development effort.
Individually it's just a millisecond wasted here and there, but all these small inefficiencies add up across the execution path.
Here's a fun benchmark to illustrate how incomplete beliefs like "compilers are smart enough to magically make this not matter" can be:
https://memex.marginalia.nu//junk/DedupTest.gmi
It's a 50x difference between the common idiomatic approach and the somewhat optimized greybeard solution, with a wide spectrum of both readability and performance in-between.
If you put zero thought toward this, your modern code will make your modern computer run as though it was a computer from the late '90s.
The sad thing is that a computer from the late '90s running software from the late '90s is extremely fast and snappy basically all the time. Old computers are just so much more responsive.
Honestly, what I want is for more developers to test and optimize their software for low-power machines. Say, a cheap netbook for example (like a Chromebook). I've heard that if you do that, you will be faster than basically every other piece of software on the system. And that speed will persist (multiply, even) on any more-powerful computer.
I've heard of one person who does that for their Quake-engine game/implementation (don't remember which). They get thousands of frames per second on a modern machine. I am guilty of not doing that myself, though. Might pick up a cheap netbook from eBay for around $30.
Honestly, what I want is for more developers to test and optimize their software for low-power machines. Say, a cheap netbook for example (like a Chromebook). I've heard that if you do that, you will be faster than basically every other piece of software on the system. And that speed will persist (multiply, even) on any more-powerful computer.
I've heard of one person who does that for their Quake-engine game/implementation (don't remember which). They get thousands of frames per second on a modern machine. I am guilty of not doing that myself, though. Might pick up a cheap netbook from eBay for around $30.
> Old computers are just so much more responsive.
Because the software on them predates the explosion of GCs, JITs, and hundreds of layers of abstractions & dependencies. All those techniques that improve developer iteration & velocity came at the cost of runtime performance.
Because the software on them predates the explosion of GCs, JITs, and hundreds of layers of abstractions & dependencies. All those techniques that improve developer iteration & velocity came at the cost of runtime performance.
Yet the only reason those things exist and are allowed to exist is because we have been training users that snappiness is not to be expected. We have been training them to wait, because their computer is busy and has a lot of work to do. The only reason these things are socially acceptable is because nobody is doing anything about them, and the average user does not care.
Also, improving developer iteration and velocity is possible without doing this.
Also, improving developer iteration and velocity is possible without doing this.
And on the mobile side you cjisyvthroe your hands in the air and claim network latency is the root cause of your performance problems. I spent some time with a team that did this, and what I saw, and tried but failed to fix, was that the problem is usually an abuse of ORMs and a complete and utter lack of software design.
GCs and JITs are absolutely not at blame here. You could in general write just as snappy programs with them than you could with assembly in the 90s. Hell, a hot loop in Java can be as fast as C code, and it has support for vector instructions as well if you want to write even faster (and platform-independent) code than you could with only C.
Bloat usually happens at the top 2 layers (actual program code and its dependencies), not from the runtime/OS/hardware.
Bloat usually happens at the top 2 layers (actual program code and its dependencies), not from the runtime/OS/hardware.
JIT runtimes usually result in (very) slow first execution, and the post in question is specifically about app launch. Responsiveness is not isolated to only after the app has been chugging along for a while.
> Hell, a hot loop in Java can be as fast as C code
Only if the Java code in question is written like C code and not like Java code. Which is sometimes possible, yes, but very very rare, and almost never done.
> and it has support for vector instructions as well if you want to write even faster (and platform-independent) code than you could with only C.
Absolutely nonsense. C compilers are more than capable of cpu-specific vectorization, too. CPU detection at runtime is ancient technology and perfectly compatible with AOT languages. And since offline compilers have the benefit of getting to spend all the time in the world on it, regularly and consistently have much better auto vectorization outputs.
JITs are normally under such a time crunch that it's common for their initial output to be quite mediocre, hence why you end up with multi-level JITs. Which then take even longer before the code stops being slow. Great for benchmarks, but pretty shit in the real world for general app performance.
Now you can sometimes have your cake and eat it too here, such as what Google is doing with cloud profiles on Android to basically run the JIT ahead of time. But that is very far from the norm for JIT'd things.
> Hell, a hot loop in Java can be as fast as C code
Only if the Java code in question is written like C code and not like Java code. Which is sometimes possible, yes, but very very rare, and almost never done.
> and it has support for vector instructions as well if you want to write even faster (and platform-independent) code than you could with only C.
Absolutely nonsense. C compilers are more than capable of cpu-specific vectorization, too. CPU detection at runtime is ancient technology and perfectly compatible with AOT languages. And since offline compilers have the benefit of getting to spend all the time in the world on it, regularly and consistently have much better auto vectorization outputs.
JITs are normally under such a time crunch that it's common for their initial output to be quite mediocre, hence why you end up with multi-level JITs. Which then take even longer before the code stops being slow. Great for benchmarks, but pretty shit in the real world for general app performance.
Now you can sometimes have your cake and eat it too here, such as what Google is doing with cloud profiles on Android to basically run the JIT ahead of time. But that is very far from the norm for JIT'd things.
I’m not talking about autovectorization, but explicit one. Autovectorization simply doesn’t work reliably enough no matter how much compiler time you spend on it, and leaves behind huge performance gains. With C, your best bet is compiler-specific intrinsics, or inline assembly, while java - strangely - has an explicit, cross-architecture Vector API, that can reliably compile down to the architecture’s SIMD instructions.
You mean the brand new Vector API? The thing that's only, oh, 25 years late? That's your example of JITs not being a bottleneck?
And it's not like the JIT actually provides any advantage here anyway. There's no shortage of similarly high level SIMD libraries for the AOT-world.
And it's not like the JIT actually provides any advantage here anyway. There's no shortage of similarly high level SIMD libraries for the AOT-world.
Many of the game studios near me (I live in Irvine, there's quite a few) test on older hardware as part of their QA process. I don't know what their limits are these days but I remember being impressed by Blizzard when I was visiting their campus once with a friend who worked there. It was fairly comprehensive at the time. According to him, anyway, many of the studios around here had similar practices.
Frankly I'm shocked that deduplicateTree is two orders of magnitude slower than deduplicateGreybeard, despite having asymptotically better-or-equal performance (TreeSet runtime is O(output log output), whereas sorting an array is O(input log input)). I'd say that perhaps boxing the integers is slowing the algorithm down, but deduplicateGreybeardCollections has only a <3x performance hit rather than >100x. Is the problem due to allocating large amounts of internal memory and branching heavily (binary trees as opposed to B-trees), or less predictable/vectorizable code execution than array traversal, or virtual comparison dispatch slower than `items.sort(Comparator.naturalOrder())`, or some TreeSet-specific slowdown?
Very good question. I'm away from my work station so I can't do additional profiling, but my overall experience is that TreeSet is hella slow in almost every case.
My hunch is that the extreme indirection makes mince out of data locality during traversal, since TreeSet is backed by a red black tree, so conversion to list is O(n) cache misses, whereas the sorting algorithms are reasonably well behaved from a cache perspective (Java uses a dual pivot quicksort for primitives and TimSort for objects IIRC).
My hunch is that the extreme indirection makes mince out of data locality during traversal, since TreeSet is backed by a red black tree, so conversion to list is O(n) cache misses, whereas the sorting algorithms are reasonably well behaved from a cache perspective (Java uses a dual pivot quicksort for primitives and TimSort for objects IIRC).
Correction: Performing O(input) failed insertions into a TreeSet might be O(input log output) runtime, which is slower than what I stated above if input > output.
> Most development is done on high powered hardware that makes slow code very difficult to distinguish from fast code, even though you can often get 10x performance improvements without sacrificing readability or development effort.
I looked at this source when you shared it and had planned to show it to a few engineers on my team. Now, sadly, the file is gone. Any chance you can put it back up?
> This proves (anecdotally) that performance isn't unacheivable at all, but rather deprioritized. Nobody wants slow apps but it's just that developer velocity, metrics, ads etc etc are higher priorities, and that comes with a cpu and memory cost that the vendor doesn't care about.
I saw a project that had a very clear N+1 problem in its database queries and yet nobody seemed to care, because they liked doing nested service calls, instead of writing more complicated SQL queries for fetching the data (or even using views, to abstract the complexity away from the app) and the performance was "good enough".
Then end result was that an application page that should have taken less than 1 second to load now took around 7-8 because of hundreds if not thousands of DB calls to populate a few tables. Because it was a mostly internal application, that was deemed good enough. Only when those times hit around 20-30 seconds, I was called in to help.
At that point rewriting dozens of related service calls was no longer viable (in the time frame that a fix was expected in), so I could "fix" the problem with in memory caching because about 70% of the DB calls requested the very same data, just in different nested loop iterations. Of course, this fix was subjectively bad, given the cache invalidation problems that it brought (or at least would bring in the future if the data would ever change during cache lifetime).
What's even more "fun" was the fact that the DB wasn't usable through a remote connection (say, using a VPN during COVID) because of all the calls - imagine waiting for a 1000 DB calls to complete sequentially, with the full network round trip between those. And of course, launching a local database wasn't in the plans, so me undertaking that initiative also needed days of work (versus something more convenient like MySQL/MariaDB/PostgreSQL being used, which would have made it a job for a few hours, no more; as opposed to the "enterprise" database).
In my eyes, it's all about caring about software development: either you do, or you don't. Sometimes you're paid to care, other times everyone assumes that things are "good enough" without paying attention to performance, testing, readability, documentation, discoverability etc. I'm pretty sure that as long as you're allowed to ship mediocre software, exactly that will be done.
I saw a project that had a very clear N+1 problem in its database queries and yet nobody seemed to care, because they liked doing nested service calls, instead of writing more complicated SQL queries for fetching the data (or even using views, to abstract the complexity away from the app) and the performance was "good enough".
Then end result was that an application page that should have taken less than 1 second to load now took around 7-8 because of hundreds if not thousands of DB calls to populate a few tables. Because it was a mostly internal application, that was deemed good enough. Only when those times hit around 20-30 seconds, I was called in to help.
At that point rewriting dozens of related service calls was no longer viable (in the time frame that a fix was expected in), so I could "fix" the problem with in memory caching because about 70% of the DB calls requested the very same data, just in different nested loop iterations. Of course, this fix was subjectively bad, given the cache invalidation problems that it brought (or at least would bring in the future if the data would ever change during cache lifetime).
What's even more "fun" was the fact that the DB wasn't usable through a remote connection (say, using a VPN during COVID) because of all the calls - imagine waiting for a 1000 DB calls to complete sequentially, with the full network round trip between those. And of course, launching a local database wasn't in the plans, so me undertaking that initiative also needed days of work (versus something more convenient like MySQL/MariaDB/PostgreSQL being used, which would have made it a job for a few hours, no more; as opposed to the "enterprise" database).
In my eyes, it's all about caring about software development: either you do, or you don't. Sometimes you're paid to care, other times everyone assumes that things are "good enough" without paying attention to performance, testing, readability, documentation, discoverability etc. I'm pretty sure that as long as you're allowed to ship mediocre software, exactly that will be done.
> In my eyes, it's all about caring about software development: either you do, or you don't.
Yes, that's the difference. I'm lucky enough to work mostly with colleagues that share a certain level of craftsmanship about the software we ship. We all care about the quality of the product, although we may not always agree on what an ideal solution should look like.
Yes, that's the difference. I'm lucky enough to work mostly with colleagues that share a certain level of craftsmanship about the software we ship. We all care about the quality of the product, although we may not always agree on what an ideal solution should look like.
I think people care but there are no alternatives. My 3 month old Windows 11 machine with a Ryzen 5700 and an RTX 3080 regularly starts chugging on stupid things like moving a window, chrome will randomly just peg a cpu with some sort of "report" process, and visual studio locks up when the laptop awakes from sleeping. Id drop it in a second but for what? Mac sucks for game development, has the same issues with too many weird background processes, and linux distros never seem to work on laptops well, even ones designed for it (system 76 for instance). The aren't real alternatives.
(I know that this is not the ideal solution, but) I had sucesss by installing a trimmed-down 'gamer' version of Windows 11 on a similarly high-specced PC, and it's still blazingly fast nearly a year later.
(If you go this route, you've got to be careful what's been trimmeed, mind, if you want to use it for general purposes - mine doesn't have WSL and seems to have damaged printing support, which is obviously suboptimal depending on your needs.)
(If you go this route, you've got to be careful what's been trimmeed, mind, if you want to use it for general purposes - mine doesn't have WSL and seems to have damaged printing support, which is obviously suboptimal depending on your needs.)
How does one find such an edition?
Ameliorated Windows 10 is one "distro" I use and (blindly) trust. It lacks store app support (not a problem), .appx package support (annoying when you want Windows Terminal), creating new user accounts (unacceptable on any machine used by multiple people), and has questionable "security" decisions by default like requiring a separate admin password for elevation, disabling VBS scripts, and an ugly logon screen wallpaper that requires downloading and blindly trusting a PowerShell script from the Internet to change (https://git.ameliorated.info/Joe/amecs). Frankly... if you're using it for situations that need security, you're better off running the (documented) amelioration steps yourself than downloading shady ISOs and wallpaper changers from online.
Torrent sites, ideally combined wiith a legitimate serial #.
By people I think he meant "product managers", not users.
One of the reasons users don't care is that they have no expectations of quality. It's painful to watch non-tech people use software written by incompetent developers, like Teams. You'd literally see someone type, then Teams UI would hand. They would pause typing and patiently stare, hands on keyboard. UI would unfreeze 3 seconds later, they'd keep typing. To me this is insane, but people are used to it / have been trained to expect it and consider it normal, apparently. Just like (a somewhat more reasonable) expectation that things need to be restarted in an escalating sequence if something is not working.
If I could go back in time and make one change to the history of computing, I'd add code to every consumer-facing GUI OS that would kill-dash-nine any application blocking UI thread for more than say 200ms. And somehow make it illegal to override for any consumer software, or if I were God make it so any developer trying to override it to let their software be slow gets a brain aneurysm rendering them (even more) incapable of writing software and has to find work digging ditches.
If I could go back in time and make one change to the history of computing, I'd add code to every consumer-facing GUI OS that would kill-dash-nine any application blocking UI thread for more than say 200ms. And somehow make it illegal to override for any consumer software, or if I were God make it so any developer trying to override it to let their software be slow gets a brain aneurysm rendering them (even more) incapable of writing software and has to find work digging ditches.
Bad alternative hypothesis. The implemented approach with recursive search is more complicated than just enumerating the files in the Sound recordings directory. A singular focus on requirements would have resulted in the easy solution that's also more efficient.
This is sheer stupidity: a more complicated, less efficient solution to a simple problem.
This is sheer stupidity: a more complicated, less efficient solution to a simple problem.
Why is it more complicated?
They presumably wanted to recursively enumerate all the files under the Sound recordings directory?
(In any case, for the application programmer the difference between recursive enumeration of files vs only what directly in a specific directly is likely only one flag or so.)
They presumably wanted to recursively enumerate all the files under the Sound recordings directory?
(In any case, for the application programmer the difference between recursive enumeration of files vs only what directly in a specific directly is likely only one flag or so.)
This is why I think it's so important that tool makers take the responsibility of performance seriously.
My old company used a proprietary VPN software (Appgate SPD) that used an electron front end - why? Well the front end could be reused on Windows, Linux and MacOS. Fair enough but it was very bloated and lacked features. Compound that to half a dozen electron apps and suddenly my laptop battery dies in half the time.
We can't guarantee or mandate that every project will be written in Rust, multiple times for every platform using their native APIs - so it's on the tool makers to bridge that gap.
Electron shows that we need a robust, efficient, cross-platform, cross-language GUI API.
Would be nice to be able to use Go, Rust, Python, JavaScript or whatever and interact with an GUI interpreter that translates a familiar API to native widgets.
My old company used a proprietary VPN software (Appgate SPD) that used an electron front end - why? Well the front end could be reused on Windows, Linux and MacOS. Fair enough but it was very bloated and lacked features. Compound that to half a dozen electron apps and suddenly my laptop battery dies in half the time.
We can't guarantee or mandate that every project will be written in Rust, multiple times for every platform using their native APIs - so it's on the tool makers to bridge that gap.
Electron shows that we need a robust, efficient, cross-platform, cross-language GUI API.
Would be nice to be able to use Go, Rust, Python, JavaScript or whatever and interact with an GUI interpreter that translates a familiar API to native widgets.
GTK can run on Windows, Linux and Mac. It's open source (LGPL), fast enough and the ecosystem is fine. QT is another one.
The truth is, Electron is easy to use, quick to hire and cheap to build on, that makes it the best choice for many companies, even when the user might (highly-likely) hate it.
The truth is, Electron is easy to use, quick to hire and cheap to build on, that makes it the best choice for many companies, even when the user might (highly-likely) hate it.
Except GTK looks like shit on Windows and Mac, so nobody wants to use it for those platforms. Most developers that create software with these bloat issues would not accept GTK.
Performance should be part of the requirements!
Not in general, no. Performance is one thing that a user might care about, but there are dozens of others. Cost, how fast to build the software, etc.
Telling users what they ought to want is not a very good idea.
Telling users what they ought to want is not a very good idea.
No, no, performance should be part of the requirements.
Not in the sense that every project needs awesome performance. But in the sense that requirements should not what performance goals are required.
So you should note down target throughput and latency for different scenarios in your requirements.
That can be as simple as saying that any user interaction should give feedback within one second. (Be that either doing what the user requested within that one second, or giving some indication that the program is working on it.)
Not in the sense that every project needs awesome performance. But in the sense that requirements should not what performance goals are required.
So you should note down target throughput and latency for different scenarios in your requirements.
That can be as simple as saying that any user interaction should give feedback within one second. (Be that either doing what the user requested within that one second, or giving some indication that the program is working on it.)
Yeah with that I agree, part of the requirements can definitely be what performance is required.
The way that GP comment was written though made it sound like good performance should be part of the requirements for all software, which is what I disagree with.
The way that GP comment was written though made it sound like good performance should be part of the requirements for all software, which is what I disagree with.
Why would a user care about how fast it was to build the software?
That depends on the user. As Joel Spolsky famously said, there are different spheres of software. Things like Word or Windows, which are general purpose and go out to millions at once, are one kind.
On the other hand, lots of software is custom-built for a specific end-user. E.g. an organization's internal tooling. There, you can often sacrifice a lot of things for it to be done faster, depending on the use case. In that case, I consider the user to be the "organization" or specific people/teams within that organization.
On the other hand, lots of software is custom-built for a specific end-user. E.g. an organization's internal tooling. There, you can often sacrifice a lot of things for it to be done faster, depending on the use case. In that case, I consider the user to be the "organization" or specific people/teams within that organization.
I agree. I'm claiming that most customers and/or decision makers do not. We should recognize that and focus on a solution instead of blaming engineers and local technical decision making. A first step is to avoid slow products, or influence those that buy products for us.
It is... fintech normally choses good and fast (in this case fast == fast execution, not necessarily quickly delivered).
That is to say -- not cheap.
That is to say -- not cheap.
When your business is in the middle of financial transactions you have the privilege of picking fast, good, and expensive.
‘Should’ doesn’t pay the rent
I guess, but there does seem to be a lot of things companies don't actually care about - performance, security, documentation.. there seems to be a real list of things that regularly show up with people complaining about it and the answer is nobody cares about that stuff.
That said I did work at a major media company and the performance sucked (in the help section) and I said I want to improve it and the PM said "nobody cares" but I think she meant the business didn't care. So there is one anecdote supporting your claim!
That said I did work at a major media company and the performance sucked (in the help section) and I said I want to improve it and the PM said "nobody cares" but I think she meant the business didn't care. So there is one anecdote supporting your claim!
I think you are very right about it, but for certain things speed needs to be a requirement. I have tried both Evernote and notion plus onenote.
They are all much more powerful than Apple notes but they are so slow by the time they are ready for my input I have forgotten what I was trying to note down.
They are all much more powerful than Apple notes but they are so slow by the time they are ready for my input I have forgotten what I was trying to note down.
We do live in this paradigm of everything being in the cloud and clients are stupid and constantly need to sync, often times in a "stop the world" fashion. Last time I used Twitter web I saw three spinners in the same screen. So that's a big factor as well. It's incredibly poor engineering culture imo, but again those are there because that's what they wanted to build.
Yeah but that’s twitter so it doesn’t matter, you just open it as needed. Evernote works offline and is an actual tool, so there is no reason it’s so slow
[deleted]
I'd say it's also instructive to consider different aspects of performance. For example, many games start up unnecessarily slowly because nobody bothered to optimise it. Users will load the game, then spend a fair while playing it, so the slow startup time doesn't hugely impact their experience. Hence, improving frame rate is a much better investment
True. I don't think users care either. Typically I get lambasted when I bring up performance issues on vendor or user group forums.
Visual Studio and Adobe Creative Suite are examples of the two most egregious offenders. They're both awfully slow and Creative Suite is extremely buggy as well. Visual Studio is nearly 30 years old, and Photoshop is even older. That's a lot of features to have to carry forward over the years.
It would be neat if during setup, they tried to figure out what your goals were as a user and turned stuff off you didn't need. My pet peeve about Visual Studio is whenever I upgrade to a new version, my default setting overrides are not carried over.
Visual Studio and Adobe Creative Suite are examples of the two most egregious offenders. They're both awfully slow and Creative Suite is extremely buggy as well. Visual Studio is nearly 30 years old, and Photoshop is even older. That's a lot of features to have to carry forward over the years.
It would be neat if during setup, they tried to figure out what your goals were as a user and turned stuff off you didn't need. My pet peeve about Visual Studio is whenever I upgrade to a new version, my default setting overrides are not carried over.
Isn't this thread full of evidence that people do care? Most of us, talking as end-users of software, seem to care. It's annoying when notepad.exe is slow.
Most people don't know what notepad.exe means. We're an extremely narrow set of people.
Also, it's about caring enough to actually stop using it. Have you ever changed software to a competitor with better performance? I don't know if I have..
Also, it's about caring enough to actually stop using it. Have you ever changed software to a competitor with better performance? I don't know if I have..
I guess it’s more that what is developed is lead by product people and those don’t know how to plan for performance issues and instead inadvertently push for more features in a shorter time.
As well: Companies pay more for dev time than they ever have in the past. There are tonnes of stories of companies buying devs the best hardware so that they can program more effectively, and of course devs with the best hardware don’t even see a lot of performance issues.
Then on top of that, companies are making choices like “we could optimize X and Y, but that would take too much dev time. Let’s just (compromise somewhere/bump up the requirements)”
Then on top of that, companies are making choices like “we could optimize X and Y, but that would take too much dev time. Let’s just (compromise somewhere/bump up the requirements)”
People do care, just everything is silod and monopolised/duopolies with no where else to go.
We’re stuck and getting stucker.
We’re stuck and getting stucker.
This brings to mind Leonard's law, which is that software will be as bad as it's users can tolerate.
This kind of idiocracy reminds me of the GTA json thing.. It's kind of hard to attribute these things to ignorance, but there's been a notable shift in perception among developers.. It used to always be about "tricking" the result out of the computer by applying deep understanding of both system and problem to find the fewest, most performant steps required to yield the result.
30 years ago, nobody in their right mind would have thought "du'h, I'll just recursively scan whatever directory THIS environment variable points at and then apply whatever this library does to each file in it to get some list and then extract the files I need from that list"..
Even IF you had to do such a terrible and expensive operation, you'd at least have the good sense to save your result so that you can do it more efficiently the next time..
30 years ago, nobody in their right mind would have thought "du'h, I'll just recursively scan whatever directory THIS environment variable points at and then apply whatever this library does to each file in it to get some list and then extract the files I need from that list"..
Even IF you had to do such a terrible and expensive operation, you'd at least have the good sense to save your result so that you can do it more efficiently the next time..
> 30 years ago, nobody in their right mind would have thought "du'h, I'll just recursively scan whatever directory THIS environment variable points at and then apply whatever this library does to each file in it to get some list and then extract the files I need from that list"..
I think plenty of people would have made the error of filtering after applying an expensive operation than before. This seems like a classic rookie mistake, not an indication of some paradigm shift in thinking. What was impressive was that no one at Rockstar thought of improving the loading performance for so long that a use decided to reverse-engineer it.
I think plenty of people would have made the error of filtering after applying an expensive operation than before. This seems like a classic rookie mistake, not an indication of some paradigm shift in thinking. What was impressive was that no one at Rockstar thought of improving the loading performance for so long that a use decided to reverse-engineer it.
I think some of these issues should be looked through a lens of incentives though as well. At my previous job, management didn't care much about performance improvements or bug fixes, they care more about delivering projects or big features. Fixing stuff like this wouldn't help you get promoted.
Absolutely, from a local perspective, decisions (or lack thereof) in regards to performance and efficiency make perfect sense, they simply shift the cost to the customer..
Pay less for development at the cost of performance which the customer will pay for with more computing capacity, electricity and time.
It scales terribly though, something that may cost an additional $20000 in development cost, may cost each customer a few tens of dollars over the lifetime of the product (not including putting a price on frustration and quality of life).. But if they have a few million customers using that feature, it's terrible for society and the greater economy/ecology.
Not only due to electricity, but also the high rate of replacing perfectly working computing equipment for higher-performing, only to spend any additional capacity running increasingly crappy and demanding software.
Not only due to electricity, but also the high rate of replacing perfectly working computing equipment for higher-performing, only to spend any additional capacity running increasingly crappy and demanding software.
Excuse my ignorance as I work in infrastructure, but do "pure" software engineers never have any time to do work outside of their assignments? Is it constant agile crunch to smash as many story points as possible into the current sprint?
Almost every developer can fit a bit of discretionary work into their existing stories. You can polish up the performance when you first code it.
Almost every employer also has a bunch of things they'd like you to do outside of your stories - mentoring junior developers, recruiting, planning, team building, professional development.
Some employers have SOC2 compliance policies that all changes should be traceable to a documented requirement, so fixes after the change is complete can be a bit more bureaucratic. If the project is behind schedule, the bosses might want you to work on something else instead.
Many developers are given pretty high-spec machines, so they can compile quickly and run weighty IDEs and so on - meaning an issue that's painful for users with 2GB of RAM and 2 CPU cores might be barely noticeable to users with 32GB of RAM and 8 cores.
And of course, some workplace cultures would say if you've 'got your work done' you can do discretionary performance work or you can leave early - and who'd choose performance work over spending time with their loved ones?
Almost every employer also has a bunch of things they'd like you to do outside of your stories - mentoring junior developers, recruiting, planning, team building, professional development.
Some employers have SOC2 compliance policies that all changes should be traceable to a documented requirement, so fixes after the change is complete can be a bit more bureaucratic. If the project is behind schedule, the bosses might want you to work on something else instead.
Many developers are given pretty high-spec machines, so they can compile quickly and run weighty IDEs and so on - meaning an issue that's painful for users with 2GB of RAM and 2 CPU cores might be barely noticeable to users with 32GB of RAM and 8 cores.
And of course, some workplace cultures would say if you've 'got your work done' you can do discretionary performance work or you can leave early - and who'd choose performance work over spending time with their loved ones?
A programmer? The gp talked about promotion as an incentive. 15 or maybe 20 years ago people didn't go into programming as a profession so much as they got paid to do what they would have done anyway. That was after the first dotcom bust, so maybe some people before that were in it for the money. But the number of times I have had to apologize to the wife because I got lost in a problem and was late getting home is large. Heck on /. they would have questioned that I actually had a wife, because programmers don't date. The motivation of the people doing the job has changed now that it pays well. But also, the culture of "if you aren't working your backlog item you aren't working" is new. The scrum kanban thing is somewhat to blame. Waterfall was terrible, but everyone on a good team knew that and just did what made sense, which is kind of what the agile manifesto was about.
Omg I’m a younger gen it sounds like, but I’m constantly getting in trouble with my wife for getting too lost in a problem. She hates, too, that I can continue to be lost in it even as I’m eating dinner or walking with her in the park or whatever. As for where that energy goes…when an employer tells me that in order to do XYZ I need to open tickets and write stories and and and, even though I agree those are the right things to do, my brain is more likely to the attack the problems where all that distracting bullshit is unnecessary.
> Excuse my ignorance as I work in infrastructure, but do "pure" software engineers never have any time to do work outside of their assignments?
A valid point, but that just kicks the same question down the road: why was no engineer assigned to investigate loading times? That might be indicative of a culture that is biased towards projects with an outcome decided a priori. A lot of managers don't want to assign tasks where the definition of "done" is ambiguous.
A valid point, but that just kicks the same question down the road: why was no engineer assigned to investigate loading times? That might be indicative of a culture that is biased towards projects with an outcome decided a priori. A lot of managers don't want to assign tasks where the definition of "done" is ambiguous.
In some places, such as the one I'm working in right now, explicitly discourage working on anything but what you've been assigned. I've even been scolded from taking items from the next sprint when I was done with the current (and in no position to really help my colleagues).
(others) YMMV but I usually don't. It's the complexity of the problems which I need to tackle and for which I have no playbook to follow and it will mostly involve deep understanding of HW, operating systems and algorithmic design. There're also compilers and programming languages which do require a non-trivial amount of time to understand but also keeping the pace with it because they're constantly evolving. Pick any of those to study and I guarantee that you can spend your lifetime doing so without having a single second to spare. However, to provide an actual business value to your company, you obviously cannot keep running through this rabbit hole on and on but by the end of the day you have to learn how to pick the right balance. In the end, I'd say that the problem space is simply too huge and I personally haven't found myself idling in my 10+ year career other than in the moments when I became too frustrated with the problem at hand. That's also one of the reasons I am reconsidering what I want to do in next 5-10 years :)
In my experience, pretty close to constantly working on whatever has been prioritized from above and needing actively to push back and have exceptionally good arguments for improving things that are not in scope.
In an agency setting anyways the percentage of billable time is usually a kpi
Unless it can be made billable it is usually frowned upon.
Eventually, this attitude develops software that takes all day to generate erroneous results and nobody really understands the code because it was all added quickly without refactoring. It's the dark side of "Do the Simplest Thing that could Possibly Work" which is probably the most misunderstood practice in XP.
At that point, every developer on the team becomes an interchangeable useless warm body and the promotion goes to the "rock star" from outside who comes in and rewrites it or surgically fixes the top performance issues.
At that point, every developer on the team becomes an interchangeable useless warm body and the promotion goes to the "rock star" from outside who comes in and rewrites it or surgically fixes the top performance issues.
This is the exact issue I face at my present job. Our clients have low budgets for hosting, and that means code needs to be as optimized as possible.
Sadly, the people I report to don't care enough about it. All they want is to push through the visible changes as fast as possible. And every single time these performance bugs come to bite us.
Our solution? Scale the system resources.
I'm not particularly satisfied with the work. But as an engineer I don't get to call the shots.
Sadly, the people I report to don't care enough about it. All they want is to push through the visible changes as fast as possible. And every single time these performance bugs come to bite us.
Our solution? Scale the system resources.
I'm not particularly satisfied with the work. But as an engineer I don't get to call the shots.
Plenty of people would have made the mistake, few would have left it there for years after performance was absymal.
Alternative opinion here, 30 years ago games/software were much simpler. I assume we did not have conglomerates like Rockstar and EA who pushed their employees beyond their limits. Game industry is notoriously known for exploiting programmers.
Imagine being a SWE working 12+ hours a day 6 days a week getting no rest. I might have accidentally coded something like this so that I could go home a couple of hours earlier.
Imagine being a SWE working 12+ hours a day 6 days a week getting no rest. I might have accidentally coded something like this so that I could go home a couple of hours earlier.
> Alternative opinion here, 30 years ago games/software were much simpler.
Funny coincidence, yesterday I posted a link which supports exactly your statement (even the 30 years): https://news.ycombinator.com/item?id=33018669
Funny coincidence, yesterday I posted a link which supports exactly your statement (even the 30 years): https://news.ycombinator.com/item?id=33018669
Certainly, I see I'm coming across as bashing the devs, that's not my intention.. It is my personal experience that cost of implementation is almost always the main restraint, it sounds rational when we put it this way.. Don't spend more on implementation than is needed.. Defining what's needed is more difficult.
> Game industry is notoriously known for exploiting programmers.
I'd be wary with these kinds of words.
The thing is that people seem really, really eager to work in the video games industry. (Just like people are really eager to be artists or musicians or pet vets or teachers etc.)
Approximately everyone who programs for the game industry could also land a cushy job writing boring CRUD software.
I do admit that it's tempting to show compassion for people who put up with crappy working conditions and low pay in order to work in their beloved field.
(Especially if that field sounds socially desirable like teaching.)
But unless you severely restrict entry to the beloved industry, conditions and pay will always be lousy compared to what similar skills could get you in an unglamorous field. That's just supply and demand.
> Imagine being a SWE working 12+ hours a day 6 days a week getting no rest. I might have accidentally coded something like this so that I could go home a couple of hours earlier.
In the case of the game industry, more likely so that you could work on the next ticket during the eternal death march.
I'd be wary with these kinds of words.
The thing is that people seem really, really eager to work in the video games industry. (Just like people are really eager to be artists or musicians or pet vets or teachers etc.)
Approximately everyone who programs for the game industry could also land a cushy job writing boring CRUD software.
I do admit that it's tempting to show compassion for people who put up with crappy working conditions and low pay in order to work in their beloved field.
(Especially if that field sounds socially desirable like teaching.)
But unless you severely restrict entry to the beloved industry, conditions and pay will always be lousy compared to what similar skills could get you in an unglamorous field. That's just supply and demand.
> Imagine being a SWE working 12+ hours a day 6 days a week getting no rest. I might have accidentally coded something like this so that I could go home a couple of hours earlier.
In the case of the game industry, more likely so that you could work on the next ticket during the eternal death march.
Isn't it exploitation to benefit from programmers'/designers' love of gaming? Labour laws are there for a reason. I believe it is a basic human right to work reasonable hours and rest.
Game development companies are making millions in profits, what stops them from hiring 2x the workforce so that each can work 8 hours a day 5 days a week instead of 12 hours x 6? The demand in the field is obvious.
I personally would prefer ethically coded games if such a category existed. I get no fun from playing a game from a company that I know exploits people. Maybe that is the reason why indie games are so popular.
Game development companies are making millions in profits, what stops them from hiring 2x the workforce so that each can work 8 hours a day 5 days a week instead of 12 hours x 6? The demand in the field is obvious.
I personally would prefer ethically coded games if such a category existed. I get no fun from playing a game from a company that I know exploits people. Maybe that is the reason why indie games are so popular.
As I said, programmers can already find relatively cushy CRUD jobs. So any 'exploitation' is in some sense voluntary.
Would you say that Bundesliga footballers are exploited? They physically work much harder than programmers in the game industry.
> Game development companies are making millions in profits, what stops them from hiring 2x the workforce so that each can work 8 hours a day 5 days a week instead of 12 hours x 6? The demand in the field is obvious.
You can have a look at video game industry stock indexes. See eg https://etfdb.com/etfs/industry/gaming/ or https://www.investopedia.com/articles/investing/010616/top-4...
If there's evil exploitation and excess profit, they don't seem show up in returns to shareholders.
At least no better returns than any other industry. Including those with the cushy CRUD dev jobs.
Would you say that Bundesliga footballers are exploited? They physically work much harder than programmers in the game industry.
> Game development companies are making millions in profits, what stops them from hiring 2x the workforce so that each can work 8 hours a day 5 days a week instead of 12 hours x 6? The demand in the field is obvious.
You can have a look at video game industry stock indexes. See eg https://etfdb.com/etfs/industry/gaming/ or https://www.investopedia.com/articles/investing/010616/top-4...
If there's evil exploitation and excess profit, they don't seem show up in returns to shareholders.
At least no better returns than any other industry. Including those with the cushy CRUD dev jobs.
Sorry, I think you are mistaking stock price and profits. ETFS' performance is based on stock prices. Game companies still continue to make millions in profits. Stock price is not a reflection of profits.
Hiring a recent college grad and working him 12+ hours a day with no rest is IMHO exploitation.
This is why we have labour laws in the first place. Employers have too much power and influence. Offer a college grad the chance to work on games and above average pay, you get him on the hook. What you are not saying to him is that you'll overwork him breaching labour laws. I gave him a job, he wanted it, if he doesn't like it, he can leave is not sufficient to make it ethical nor legal.
Please read this insider story on how brutal the working conditions are: https://ea-spouse.livejournal.com/274.html
I cannot comment on the comparison with Bundesliga footballers, because I have no experience in working a physical job. I am a knowledge worker.
Hiring a recent college grad and working him 12+ hours a day with no rest is IMHO exploitation.
This is why we have labour laws in the first place. Employers have too much power and influence. Offer a college grad the chance to work on games and above average pay, you get him on the hook. What you are not saying to him is that you'll overwork him breaching labour laws. I gave him a job, he wanted it, if he doesn't like it, he can leave is not sufficient to make it ethical nor legal.
Please read this insider story on how brutal the working conditions are: https://ea-spouse.livejournal.com/274.html
I cannot comment on the comparison with Bundesliga footballers, because I have no experience in working a physical job. I am a knowledge worker.
So who is supposed to be benefiting from the 'exploitation'?
Management gets fat bonuses for reaching profitability targets, investors are happy.
I mentioned that investors don't seem to be getting any special benefit, if you look at the stock (including re-invested dividends).
Public investors may not be getting any benefit, because the private investors have already IPOd at a ridiculous stock price. Private investors usually have non-trading shares. The way the scheme works is that if they find the stock price to be right, they convert their non-trading shares into trading shares and profit more. This way the stock price doesn't increase much. They basically dilute the amount of public shares for profit.
They don't give any thought to public investors such as you and me.
They don't give any thought to public investors such as you and me.
Yeah, I don't buy it. I've been programming for more than 20 years, and I'm pretty sure people were making the same complaints 20 years ago that this article and you are making now.
For some software, performance is important. For others, it isn't. Some developers are good, at least at making performant software, some are worse. That's always been true and will probably always be true.
For some software, performance is important. For others, it isn't. Some developers are good, at least at making performant software, some are worse. That's always been true and will probably always be true.
Still, software is slowing down. Anecdotal example: my phone. It used to run fine at the beginning then now a few years later everything is slow: Google maps, YouTube, Signal, the pin-code screen… I barely have installed any apps, the only thing that changed is the number of messages I have and the updates.
Perhaps we could justify it with new features at least? Perhaps my phone is slower now because the things in there are so much more useful now? Well, no. I got maybe a couple cosmetic changes, but functionality is exactly the same. My phone got slower for no benefit.
Perhaps we could justify it with new features at least? Perhaps my phone is slower now because the things in there are so much more useful now? Well, no. I got maybe a couple cosmetic changes, but functionality is exactly the same. My phone got slower for no benefit.
My phone is about 6 years old. It's not noticeably slower than when I got it and it's running a lot more in the background than it used to.
My original SE is definitely slower.
If you’ve updated your phone’s OS since then, you now have additional services running in the background. Things like on-device face detection for photos, “suggest apps” functionality, and others.
Perhaps, but I still get zero benefit from any of those.
Indeed. Personally I think that this kind of “helpful service bloat” is part of how companies are getting away with slowing phones down over time.
Apple for example barely mentions the fact that these services are running, and certainly does not give you any way to turn them off.
Apple for example barely mentions the fact that these services are running, and certainly does not give you any way to turn them off.
But the weather app now has lightning animations when there's a thunderstorm! Innovation!
Well, it may be just planned obsolescence, or a system-wide bug. Chances are it would be mostly as snappy as new, if you were to reinstall it and start anew, for example. In my opinion the problematic layer is usually the highest one, the kernel/low-level OS layer doesn’t have too many performance problems.
It's not even a good vs bad dev issue most of the time, I can imagine lots of devs at Rockstar who hate working there and are under strict deadlines from managers so they just make it work while they leetcode and apply to faang for better salaries where they'll "finally" optimize things, etc.
There's a big difference between latency vs throughput.
We had lots of 'free' improvements in throughput. But latency still requires careful attention.
We had lots of 'free' improvements in throughput. But latency still requires careful attention.
Funny enough, if you read the guidelines for Metro apps (or Modern apps, or UWP apps, or whatever the hell they are called now), Microsoft specifically says that you should offload all expensive operations to an async thread and keep the UI thread free so the interface can be quick and responsive. When I was making my first apps in university, they would routinely reject them from their app store saying the UI is not responsive enough. You'd think they actually care about efficiency and responsiveness.
But then they hardcode a whole Documents directory scan right in the UI thread in their own damn voice recorder app. Unbelievable.
But then they hardcode a whole Documents directory scan right in the UI thread in their own damn voice recorder app. Unbelievable.
I wonder if they only ever tested the UI when there were less than 3 files on that folder.
“Looks fast to me! Ship it.”
“Looks fast to me too! Approve it!”
“Looks fast to me! Ship it.”
“Looks fast to me too! Approve it!”
"ship as much as possible as fast as possible" can lead to that kind of result (i've seen it over and over...)
They were very strict on manual reviews but when numbers became more important they let their guidelines slide.
> It used to always be about "tricking" the result out of the computer by applying deep understanding of both system and problem to find the fewest, most performant steps required to yield the result.
We didn't used to have product teams as large as engineering teams. You can still engineer software the way you want, you're just beholden to some product persons idea about deadlines now. If you can turn your little unit of work around quickly no one will bat an eye (of course, we all know that's not realistic most of the time). If you dare trying to take longer than their perception of how long it ought to take then you better be prepared to discuss the "value add" of your performance minded solution as nauseum.
We didn't used to have product teams as large as engineering teams. You can still engineer software the way you want, you're just beholden to some product persons idea about deadlines now. If you can turn your little unit of work around quickly no one will bat an eye (of course, we all know that's not realistic most of the time). If you dare trying to take longer than their perception of how long it ought to take then you better be prepared to discuss the "value add" of your performance minded solution as nauseum.
We went from ”premature optimization is evil” to ”optimization is evil”
An alternate view is that the state of program optimization is terrible.
Well, optimizations need good abstractions to work with. The reason why a SQL query can run much faster than a naive for loop would is precisely due to SQL being very removed from lower-level details.
So in extension, optimization is terrible because we might be using the wrong abstraction level? E.g. it is hard to optimize a larger JS program, because if the developer specified some sort of detail it might throw out an entire class of possible optimizations.
There is actually an interesting presentation on the topic here: https://news.ycombinator.com/item?id=9396950 , which discusses among others whether a two-way communication between compiler and developer may help.
So in extension, optimization is terrible because we might be using the wrong abstraction level? E.g. it is hard to optimize a larger JS program, because if the developer specified some sort of detail it might throw out an entire class of possible optimizations.
There is actually an interesting presentation on the topic here: https://news.ycombinator.com/item?id=9396950 , which discusses among others whether a two-way communication between compiler and developer may help.
I don't think either is true. I think it's a case of "development by committee" - many different people work on a codebase, but nobody is critical enough of the whole product.
I can imagine the audio recorder is a low priority minor side project that gets handed from one guy (or team) to the next every couple of years, with the emphasis on "don't spend too much time on it".
The other issue is that performance is taken for granted; if something is slow, it's not immediately obvious.
I can imagine the audio recorder is a low priority minor side project that gets handed from one guy (or team) to the next every couple of years, with the emphasis on "don't spend too much time on it".
The other issue is that performance is taken for granted; if something is slow, it's not immediately obvious.
For sure. But not just "development by committee" also "development by hundreds of unrelated committees". By which of course I mean the full stack of dependencies that everything relies upon now, along with all the glue code in between that has to repeatedly transform data as it gets passed around through the system.
I've often seen some dependency module be the bottleneck, and an in-house replacement that does just what we need can perform an order of magnitude or two better than a generic 'all things for all people' module.
Also there are things that work well for dozens or hundreds of data pieces but become abysmal when you eventually scale to hundreds of thousands. But no one else has that use case, so it works fine for them. And for those that do, by then the frog is boiled and they're used to it being slow.
I've often seen some dependency module be the bottleneck, and an in-house replacement that does just what we need can perform an order of magnitude or two better than a generic 'all things for all people' module.
Also there are things that work well for dozens or hundreds of data pieces but become abysmal when you eventually scale to hundreds of thousands. But no one else has that use case, so it works fine for them. And for those that do, by then the frog is boiled and they're used to it being slow.
Having a list of all audio recordings always shown on the right pane is kinda an optimization (VoiceRecorder.exe). The usefulness is limited, because VoiceRecorder can't edit them, only play back and set markings.
yea, we went from being burned by trying to optimize before we knew the problem, to being burned by not optimizing because we knew the problem but didn't optimize.
30 years ago, much (UNIX) system software was built out of calls to system(), ie it would have shell commands do most of the steps. Partly this is because it is hard to do everything in C and partly it is just laziness. This is partly the reason for terrible security (the IFS environment variable could be changed to change the meaning of the system() calls and trivially make a suid program do bad things).
I think you’re giving programmers of the past too much credit.
I think you’re giving programmers of the past too much credit.
I think you're not giving them enough. They managed to have a glaring security hole to be exploited, but no one bothered to. It took the bunch of degenerates we have around today with no respect for the sanctity of someone else's machine for this type of practice to even become a problem.
You'll be amazed how many seemingly intractable technical issues aren't due to having been solved at higher layers of abstraction. Nowadays with things like clear cut ownership of post first sale ownership breaking down, we're actually getting bitten by having to try to technically solve problems, previously made tractable by the social consensus between professionals.
As a consequence, here we are with a now crappy, pathological Notepad.
You'll be amazed how many seemingly intractable technical issues aren't due to having been solved at higher layers of abstraction. Nowadays with things like clear cut ownership of post first sale ownership breaking down, we're actually getting bitten by having to try to technically solve problems, previously made tractable by the social consensus between professionals.
As a consequence, here we are with a now crappy, pathological Notepad.
We must be talking about a different 1990s. It’s true that before then it was easier to get away with bad security due to few computers being networked, and it took a while for eg buffer overflows to be exploited (smashing the stack for fun and profit was November ’96). But in the first half of the decade or so there were lots of vulnerabilities found in sendmail and suchlike.
I disagree, software has always sucked. We can remember lots of the good software, and back how things used to be but the reality is this is nostalgia speaking.
A perfect example is windows 95 - wiggling the mouse causes a huge speedup in long running applications [0].
> used to always be about "tricking" the result out of the computer by applying deep understanding of both system and problem to find the fewest, most performant steps required to yield the result.
That "tricking" came with lots of issues. Massive security holes, significant app stability issues, and hugely increased developer time to implement things. I can get a hardware accelerated Sha1 hash of a buffer in one line of c++ today, and I know it will be correct.
People who do that work still exist. Computers are fast enough that it's worth not spending hours optimising for every allocation when you're displaying half a dozen strings on screen, but people like Casey Muratori, Bruce Dawson, still exist and still write code.
[0] https://retrocomputing.stackexchange.com/questions/11533/why...
A perfect example is windows 95 - wiggling the mouse causes a huge speedup in long running applications [0].
> used to always be about "tricking" the result out of the computer by applying deep understanding of both system and problem to find the fewest, most performant steps required to yield the result.
That "tricking" came with lots of issues. Massive security holes, significant app stability issues, and hugely increased developer time to implement things. I can get a hardware accelerated Sha1 hash of a buffer in one line of c++ today, and I know it will be correct.
People who do that work still exist. Computers are fast enough that it's worth not spending hours optimising for every allocation when you're displaying half a dozen strings on screen, but people like Casey Muratori, Bruce Dawson, still exist and still write code.
[0] https://retrocomputing.stackexchange.com/questions/11533/why...
> Computers are fast enough that it's worth not spending hours optimising for every allocation when you're displaying half a dozen strings on screen,
And this attitude is what gives us such gems as the one mentioned in the OP article.
And this attitude is what gives us such gems as the one mentioned in the OP article.
Especially when we as modern users run multiple applications, each of them deciding optimization is not worth bothering with.
That's not what I said, at all. Optimisation where it matters is important.
I’m not sure if you meant to reply to me. I was commenting on what dusted said
No, it's not. Not at all. I care about performance in my architecture, design, and code. However I _don't_ care about eeking out every %ge in every scenario.
The difference is caring where it matters. Bugs and mistakes happen. The measure of someone who cares and someone who is improving is how they handle it when these issues happen.
The difference is caring where it matters. Bugs and mistakes happen. The measure of someone who cares and someone who is improving is how they handle it when these issues happen.
too many developers means they switched from trying to emulate the elites to trying to follow the trend of the day, because otherwise you get ostracized by the crowd
I personally blame the “premature optimization is root of all evil” quote which most of the time is taken out of context but formed a lasting impact in many tech leaders’ minds
I was writing an NMEA parser today. (NMEA 0183 is a common serial GPS data protocol) My serial port reads returned an IOvec-like type which exposed limited operations. I spent probably an hour making helper functions and doing various verification and parsing operations on the IOvec type because it saved having to copy the data into a contiguous buffer. Midway through I stopped and realized that I would only need to copy when the ~50B message straddled the end of the ~64kB buffer. It takes a few nanoseconds to copy 50B. I spent an hour on an optimization that would never do anything but complicate the code. If I had just written the simple and obvious code, and spent that hour profiling the application to optimize what would actually make a difference, I'd be much better off.
That is what the quote is talking about. I don't think you can blame the quote if people are just ignoring the first word, and I'm not convinced that's actually a common thing.
That is what the quote is talking about. I don't think you can blame the quote if people are just ignoring the first word, and I'm not convinced that's actually a common thing.
IMHO that's better called "premature complexity".
You shouldn't even need to profile to realise that if you're using a serial port for I/O, unless your hardware is something embedded and in the low-MHz, range, you'll spend most of your time waiting for I/O unless you massively overcomplicate your solution.
You shouldn't even need to profile to realise that if you're using a serial port for I/O, unless your hardware is something embedded and in the low-MHz, range, you'll spend most of your time waiting for I/O unless you massively overcomplicate your solution.
New quote just dropped:
Premature complexity is the root of all evil.
Premature complexity is the root of all evil.
Maybe you can't blame the quote author but the quote itself is definitely a problem now.
In my experience it is mostly used as an excuse not to think about performance at all. "The quote says we can worry about it later!"
That's why the quote needs to die.
In my experience it is mostly used as an excuse not to think about performance at all. "The quote says we can worry about it later!"
That's why the quote needs to die.
Or extended with the context, like “Yet we should not pass up our opportunities in that critical [hot loop]”
The "hot loop" is also a myth. Very few programs actually have a hot loop with larger-than-average returns on optimizations. Heck, the blog post itself is an example of that not being the case - it's startup that's slow, not the program's steady state hot loop. For something you actually spend a significant amount of time using continuously, yeah maybe it has a hot loop. But even then, probably more like hundreds or thousands of hot loops depending on the features being used, what stage the program is in (eg, blender while editing the scene is very very different from blender while exporting a rendering).
In other words: first, understand your data.
That's what Mike Acton says before he shakes your hand in the morning.
That's what Mike Acton says before he shakes your hand in the morning.
Virtually every single performance thread on stack exchange has that one guy that says regurgitates something about premature optimizations and maintainability eg
Q "I'm working on high throughput data science thing, is branchless math better than conditional instructions here, because this thing needs to take less than a lifetime to finish"
A "You should focus on code readability, it's way more important ..yadeyadaya ... i work on python and websites"
I blame slipping standards in CS programs and the entire concept of "information science"
Q "I'm working on high throughput data science thing, is branchless math better than conditional instructions here, because this thing needs to take less than a lifetime to finish"
A "You should focus on code readability, it's way more important ..yadeyadaya ... i work on python and websites"
I blame slipping standards in CS programs and the entire concept of "information science"
SX/SO are full of such anti-intellectual cargo-culting. I've avoided them for anything but the shortest of answers.
A good mentor told me once not to let 'perfect' be the enemy of 'good'. At the time I thought it was clever but it didn't really hit home until year 10 or 15. I did a lot of things that made no difference to the user, but to me was finesse.
It's definitely a good thing to keep in mind. Perfectionism is a real danger. However, do note that there are entire categories of code quality worth keeping up that make no difference to users now. A simple example would be maintainability (whatever that means in each case).
Tbh, most performance questions I see on SE are bad too. The necessary question here is: what's the profile of the whole thing - does this line of math even matter? The questions that are already researched to the level where the original question makes sense are extremely rare - and I do enjoy those.
Also the "developer's time is expensive" belief pushes developers to reuse layers and layers of packages and libraries whose behaviors are not well aligned with the goal of application, but hey, it's cheaper to just slap something on the existing packages and ship it. The "expensive parts" are pushed to the end users.
The funny part is that often interviewing is about knowing algorithms. But then it's frowned upon if you actually implement one.
"Don't roll your own… anything"
While you are right to an extent, I'd rather have someone that can check if the implementation is correct for the use case, wether it's theirs or someone's else, and settles for someone's in first when it fits.
"Dammit, I went to school for the whole algorithm, I'm gonna implement the whole algorithm!"
> Also the "developer's time is expensive"
Agreed. I really dislike this excuse for sloppy slow code.
Yes, developer time is expensive, need to keep ROI in mind.
But if the expensive developer spents a week optimizing the function to make it 1 second faster, is that worth it? Too many people will respond "developer's time is expensive" without thinking beyond that. But of course, it depends.
If that function is being invoked millions of times per day by hundreds of thousands of people, it very quickly becomes totally worth it.
Agreed. I really dislike this excuse for sloppy slow code.
Yes, developer time is expensive, need to keep ROI in mind.
But if the expensive developer spents a week optimizing the function to make it 1 second faster, is that worth it? Too many people will respond "developer's time is expensive" without thinking beyond that. But of course, it depends.
If that function is being invoked millions of times per day by hundreds of thousands of people, it very quickly becomes totally worth it.
Adding on to that, “QA time is expensive” too. I’ve learned from experience that trying to tweak well established repos/libraries rarely gets management’s approval because the extra time it would take to QA and regression test is considered pure loss. Better to leave it be and force the new code to work with the old stuff. That way it is at least billable.
> developers to reuse layers and layers of packages and libraries whose behaviors are not well aligned with the goal of application, but hey, it's cheaper to just slap something on the existing packages
I don’t know, should I really reimplement that graph lib in a buggier, slower way, over choosing a battle-tested one that may also contain some additional features I don’t need?
According to Brook’s famous paper, code reuse is the only way to significantly improve programmer productivity. No new language will suddenly make you 10x faster, and even that would be meaningless against all the quality code bases out there.
According to Brook’s famous paper, code reuse is the only way to significantly improve programmer productivity. No new language will suddenly make you 10x faster, and even that would be meaningless against all the quality code bases out there.
It's ironic when the end users of the software are themselves developers. "Reap what you sow."
Yup. There surely is such thing as over-optimized, early-optimized, over-clever, over-terse, over-engineered, over-abstracted, etc. But in terms of actual impact in the software ecosystem, the simple problem of not-good-at-programming still blows them all out of the water, combined.
It's important to be able to recognize which of the two echelons of problem-space one's codebase inhabits.
It's important to be able to recognize which of the two echelons of problem-space one's codebase inhabits.
It's a great maxim but it's misapplied.
What's ironic is premature optimization is not just code, using K8s or expensive cloud solutions when a bare metal dedi will suffice. etc.
Instead it's just parroted by people who want to churn out MVP quality code in prod.
What's ironic is premature optimization is not just code, using K8s or expensive cloud solutions when a bare metal dedi will suffice. etc.
Instead it's just parroted by people who want to churn out MVP quality code in prod.
For many readers that quote is the only part of The Art of Computer Programming they can comprehend. No wonder they bikeshed it to hell and back.
Well it's the art of computer programming, not the art of shipping performant software. People just need a more rounded education.
Well, modern programmers more often think "optimization is the root of all evil". I read an article that literally recommends not to optimize until the end.
I don't encounter mention of "the hot path" anywhere near often enough.
Defend Your Hot Path!
Defend Your Hot Path!
Nobody wants to do post-mature optimizations, as in never touch a running system.
No one will say thank you for fixing something, but everybody will blame you if you fail trying.
No one will say thank you for fixing something, but everybody will blame you if you fail trying.
As far as I can tell, it's quite common to do work to improve the performance of existing systems?
Probably, but it is a good point: premature optimization is easy and fun because you're doing it while writing the code and it's already in your head. Optimizing a large, working, congealed codebase that you didn't write is often hard and feels risky, even with good test coverage. Very local optimizations, fine, but those requiring refactoring? Less likely to happen.
Maybe there's a golden opportunity somewhere inbetween where optimization is not premature but also not painful to do.
Maybe there's a golden opportunity somewhere inbetween where optimization is not premature but also not painful to do.
Though it does ultimately boil down to no “fearless” refactoring being possible in real life. Unfortunately we seldom have such systems and test coverage/quality.
Yep, I think baseline optimize should be just build into the core of framework or language. Specified optimization could be applied later but not these. Basic caches that don't affect performance or memory should just be enabled by default instead of some "let's just don't optimize it at all until it lags like hell" shit like react.(Spoiler: nobody actually optimize it in the end) I hate websites written by react so much because of this.
Another one that's been abused and thoroughly misunderstood is "correlation does not mean causation".
You could come up with data showing an R^2 of like 0.99 and there's always a smartass that immediately parrots "but remember that correlation does not mean causation". Ugh.
You could come up with data showing an R^2 of like 0.99 and there's always a smartass that immediately parrots "but remember that correlation does not mean causation". Ugh.
And how is that being abused and misunderstood there?
The quote implies optimization at a later date.
Why blame a quote, an inanimate thing, and not the foolish programmer that scans the entire `Documents` directory just to ignore most of it and only keep the results of the `Sound Recordings` subdirectory? (Among other apparent issues with this program.)
I think that quote can often be the source of misguided beliefs like "it's OK to scan the whole Documents directory".
'Cuz why waste time doing something harder? Premature optimization and all that.
Honestly, in my experience, basically the opposite of the quote is true. If you want your software to be fast, you need to be thinking about speed in the design from the very beginning. Very hard, perhaps impossible, to make something not designed for speed from the beginning be as fast as something that was.
Of course there's a difference between having a holistic design that you know can go fast and immediately wasting time eeking out x% of speed in some random sub-fn that will only run y% of the time, when x and y are small.
But really performance does need to be built into the design.
(As an example, look at the heroic steps that are being taken to try to make the Rust compiler faster. It's slowly getting better, but it will never be fast. On the other hand, I would predict that if you started making a Rust-like-language from day 0 with a fast compiler as a goal, you could get something much faster.)
'Cuz why waste time doing something harder? Premature optimization and all that.
Honestly, in my experience, basically the opposite of the quote is true. If you want your software to be fast, you need to be thinking about speed in the design from the very beginning. Very hard, perhaps impossible, to make something not designed for speed from the beginning be as fast as something that was.
Of course there's a difference between having a holistic design that you know can go fast and immediately wasting time eeking out x% of speed in some random sub-fn that will only run y% of the time, when x and y are small.
But really performance does need to be built into the design.
(As an example, look at the heroic steps that are being taken to try to make the Rust compiler faster. It's slowly getting better, but it will never be fast. On the other hand, I would predict that if you started making a Rust-like-language from day 0 with a fast compiler as a goal, you could get something much faster.)
The quote is talking about micro optimization, but people think they should apply it st design time. That is wrong, the right data structures up front make mmicro optimizations mostly unnecessary (assuming a modern optimizing compiler) while if you get that wrong nothing will fix performance but a major rewrite
> the right data structures
Surely no one thinks that "when searching a directory, specify the full path of the directory you want to search" is premature optimization...?
Isn't that just... doing the thing you intended to do, in the most straightforward way possible?
Who says "it's too much trouble getting the files in the directory I want, I'd rather search the entire computer and filter that list down to the files in that directory"?
This is like saying "it's too much effort to drive to San Francisco, so I'm going to drive to cities in California until I happen to arrive in San Francisco".
Isn't that just... doing the thing you intended to do, in the most straightforward way possible?
Who says "it's too much trouble getting the files in the directory I want, I'd rather search the entire computer and filter that list down to the files in that directory"?
This is like saying "it's too much effort to drive to San Francisco, so I'm going to drive to cities in California until I happen to arrive in San Francisco".
>If you want your software to be fast, you need to be thinking about speed in the design from the very beginning. Very hard, perhaps impossible, to make something not designed for speed from the beginning be as fast as something that was.
This is absolutely true in low latency finance.
This is absolutely true in low latency finance.
Latency, power consumption, and safety have the problem that if they aren't a first order requirement they're expensive/impossible to bolt on after.
Reminiscent of the "simplify and then add lightness" quote from Colin Chapman (of Lotus cars fame)
Yeah one should consider whether a choice bakes in something against the ultimate requirements or is just a trivially replaceable stop gap. Bonus sometimes you find the stop gap is actually acceptable.
> As an example, look at the heroic steps that are being taken to try to make the Rust compiler faster. It's slowly getting better, but it will never be fast.
Eh. It's fast enough. And anything with overhead of borrow checker and focus on runtime performing, wouldn't be fast to compile either.
Biggest win could be getting rid of LLVM but that would have crippled its development speed.
Eh. It's fast enough. And anything with overhead of borrow checker and focus on runtime performing, wouldn't be fast to compile either.
Biggest win could be getting rid of LLVM but that would have crippled its development speed.
Nobody is blaming the quote, a quote is ink on paper. People are blaming the programmers and management who cite the quote in their reasoning.
It’s just an expression ofc i blame the engineers (actually mostly tech leads and management)
Counterintuitively, software performance as a whole is not a function of hardware, it's a function of human tolerance. Apps are as slow as we humans are willing to tolerate.
As hardware gets better, we're finding ways to use it less efficiently. So if you want apps to become faster, you shouldn't be building better hardware. You should be changing human tolerance thresholds.
Which is almost impossible to pull off. Apple did this when they released the iPhone. Remember mobile apps before iOS? Mobile browsers before Safari?
As hardware gets better, we're finding ways to use it less efficiently. So if you want apps to become faster, you shouldn't be building better hardware. You should be changing human tolerance thresholds.
Which is almost impossible to pull off. Apple did this when they released the iPhone. Remember mobile apps before iOS? Mobile browsers before Safari?
I like your take that human tolerance is the limiting factor.
I'd agree that is a major constraint. The rest of it is just economics.
When I started programming, people I learned from would optimise code in assembler, use clever algorithms and write highly unreadable code that was impossible to debug. One bit of code I remember interleaved various operations so that it's access would line up with disk reads. I rewrote it to just hold the data on memory. It wasn't much faster than his disk based code in practice.
But the only reason to go to that kind of effort was the severe constraints hardware placed on software. When those are largely removed, the imperative is to write code fast that other people can still understand.
When I started programming, people I learned from would optimise code in assembler, use clever algorithms and write highly unreadable code that was impossible to debug. One bit of code I remember interleaved various operations so that it's access would line up with disk reads. I rewrote it to just hold the data on memory. It wasn't much faster than his disk based code in practice.
But the only reason to go to that kind of effort was the severe constraints hardware placed on software. When those are largely removed, the imperative is to write code fast that other people can still understand.
I think part of it is the terrible software people are forced to use at work, which is bought by people who only look at feature lists and price. This stuff eats away at their expectations until they are OK with bloated stuff like discord for private use too.
Speaking of Windows: the software gets slower faster than the hardware gets faster.
In around 1990, I gave my dad my old 286 machine, with Win 3.1. He never upgraded it.
About 2000, when I was accustomed to much newer software & hardware, I used his old machine. I was amazed how much faster everything was.
In around 1990, I gave my dad my old 286 machine, with Win 3.1. He never upgraded it.
About 2000, when I was accustomed to much newer software & hardware, I used his old machine. I was amazed how much faster everything was.
I have almost 15 year old CPU with 2 GB RAM and Intel Core 2 Duo processor. It has Windows 7 installed. I recently opened it again and was amazed to see that it actually works and it is also very snappy by current standards.
My Core 2 Duo Merom laptop was sadly unbearably sluggish on Windows 7 with a mechanical drive even with indexing and background services disabled, even at browsing files and such (and merely laggy with a SSHD mechanical drive with flash cache), though I hear Penryn CPUs are faster (and perhaps desktops are faster than laptops). The computer performs much better on a non-JS-based Linux DE like MATE (as opposed to GNOME Shell and plasmashell), though taskbar window previews are black (probably it doesn't work on an anemic GMA 965). I swear Xfce on a 2-core Ivy Bridge laptop with iGPU and SATA SSD, feels just as fast as KDE on a 6-core Zen 3 desktop with non-Nvidia GPU, at file management, Konsole, Telegram/nheko chatting, and PDF reading, until you open a browser or Electron app or run a modern optimizing compiler.
Oddly, with Windows 7 on the Core 2 Duo, the taskbar would randomly stop updating (I think it stopped happening after moving the same hard drive into a newer laptop?), and I got no end of driver issues: the touchpad would stop responding on the lock screen or logged in after sleep-wake, and audio would randomly stop playing until I restarted audiodg.exe. As far as I can remember, none of these issues happen on my Ivy Bridge laptop, where I'm clinging for life onto Windows 7 (the last version with a soul) for as long as I can keep apps running... though I'm getting rather tired of Tailscale creating a new network adapter and asking for home/work/public on every reboot.
Oddly, with Windows 7 on the Core 2 Duo, the taskbar would randomly stop updating (I think it stopped happening after moving the same hard drive into a newer laptop?), and I got no end of driver issues: the touchpad would stop responding on the lock screen or logged in after sleep-wake, and audio would randomly stop playing until I restarted audiodg.exe. As far as I can remember, none of these issues happen on my Ivy Bridge laptop, where I'm clinging for life onto Windows 7 (the last version with a soul) for as long as I can keep apps running... though I'm getting rather tired of Tailscale creating a new network adapter and asking for home/work/public on every reboot.
I had the same experience with a 900 Mhz Pentium III machine, and Windows XP SP3. It was lightning, blazing fast. It geniunely felt like the reactions for my mouse clicks were faster than I could finish the clicking motion.
This is why some people want to still use DOS and Win 3.1 because it is faster.
I run Win95 and Win2000Pro in a virtal machine with old software and it runs faster than the most recent versions.
I put Ubuntu on a Lenovo PC and it runs faster than Windows 10 did.
I run Win95 and Win2000Pro in a virtal machine with old software and it runs faster than the most recent versions.
I put Ubuntu on a Lenovo PC and it runs faster than Windows 10 did.
I switched to Linux Mint as my daily driver and I'm amazed by its overall performance.
There's still a couple quirks, but Windows 10 had its fair number of quirks too...
There's still a couple quirks, but Windows 10 had its fair number of quirks too...
I had a experiences both with a very very old win95 box and an old linux 2.4 kali usb key.
The linux one was strange because my arch/systemd/ssd/i3 setup is lean, you get parallel boot, no heavy DE no bloat.. but everything felt lighter and faster on kali and I had a physical reaction saying "I don't miss anything from my current laptop, that old thing was closer to my needs".
Maybe a part of our brain that doesn't care about typography or visual effects much and prefer good old crude solid lag free tools.
The linux one was strange because my arch/systemd/ssd/i3 setup is lean, you get parallel boot, no heavy DE no bloat.. but everything felt lighter and faster on kali and I had a physical reaction saying "I don't miss anything from my current laptop, that old thing was closer to my needs".
Maybe a part of our brain that doesn't care about typography or visual effects much and prefer good old crude solid lag free tools.
I recently had the pleasure of installing OS9 on a classic 2001 iMac – the "lampshade" one, though I don't think that nickname does it justice!
I was, and am, blown away by how responsive it is. Plus, the UI sound effects add to the experience in a great way. You can HEAR when you've done something "in the computer." Just a bunch of clicks and boops. It's fantastic. Makes you think about what we've just... gotten used to.
I was, and am, blown away by how responsive it is. Plus, the UI sound effects add to the experience in a great way. You can HEAR when you've done something "in the computer." Just a bunch of clicks and boops. It's fantastic. Makes you think about what we've just... gotten used to.
Speaking of Windows: Microsoft PMs stupidity can never be matched by declining SW development efforts, even with better HW.
Here you cannot really blame the poor developer, when the PM demands to scan all Documents, not just some relevant sound recordings. It's so obviously insane, that only PM's get away with this. Of course the dev matched our PM here, with his insane directory iters, but alone the idea must be appreciated first.
Here you cannot really blame the poor developer, when the PM demands to scan all Documents, not just some relevant sound recordings. It's so obviously insane, that only PM's get away with this. Of course the dev matched our PM here, with his insane directory iters, but alone the idea must be appreciated first.
PMs don't get to order devs around at Microsoft (or any sane organization). Devs can tell their PM counterparts to stuff it if they try to demand stuff. If you've worked at Microsoft, or met someone who has, you'd know this.
That stack trace says it all --- the bloat known as COM rears its ugly head again. I also suspect there's a quadratic or even worse algorithm in there, hidden away amongst all that useless abstraction. The fundamental way of scanning for files with the Win32 API is an iterator (FindFirstFile/FindNextFile). It looks like they put some sort of array/indexing abstraction on top of that ("GetRowsAt", "FetchResultAt") which probably gets the N'th entry by stepping the iterator N times, and from there it's not hard to imagine someone doing
for(int i = 0; i < obj->GetNumItems(); i++)
doSomething(obj->GetItemAt(i));I think the issue is actually the sandboxing and other stuff. WinRT StorageProvider API is known to be extremely slow and NTFS / Windows IO subsystem is itself already quite slow compared to UNIX. The issue IIRC is that StorageProvider is designed for sandboxing and the way they implemented that involves doing RPCs to other processes. So there's probably some heavy context switching involved, but it's architectural, and so the voice recorder was never fixed.
https://github.com/microsoft/WindowsAppSDK/issues/8
"The challenge here is simply that listing the StorageItems in a StorageFolder is incredibly slow and resource intensive compared to using the standard .Net/Win32 API to list the file paths. A quick test showed that in .Net it takes about 0.003ms per file, whereas with UWP StorageItems it takes about 2ms per file, which is around 700 times slower. Also, looking at the Runtime Broker, UWP uses about 70kB memory per file, which is a huge cost when loading 10,000+ files, whereas .Net uses around 0.1kB per file (that’s a very rough estimate)."
Somewhere else, someone proposes a theory that due to API mismatches/design issues in Win32 the RuntimeBroker is trying to use a design that isn't a great fit in order to try and provide certain security guarantees, and this requires pre-fetching a lot of data up front in case the app requests it. But NTFS is slow, so, all this additional traffic makes opening files via the Storage API really really slow.
The problem here isn't really "modern software", it's that Microsoft wrote a lot of new APIs over time with the intention of replacing Win32 (UWP/WinRT) but they aren't dogfooding them all that effectively, and they're using C++ for everything, so there are problems that don't get fixed for years.
https://github.com/microsoft/WindowsAppSDK/issues/8
"The challenge here is simply that listing the StorageItems in a StorageFolder is incredibly slow and resource intensive compared to using the standard .Net/Win32 API to list the file paths. A quick test showed that in .Net it takes about 0.003ms per file, whereas with UWP StorageItems it takes about 2ms per file, which is around 700 times slower. Also, looking at the Runtime Broker, UWP uses about 70kB memory per file, which is a huge cost when loading 10,000+ files, whereas .Net uses around 0.1kB per file (that’s a very rough estimate)."
Somewhere else, someone proposes a theory that due to API mismatches/design issues in Win32 the RuntimeBroker is trying to use a design that isn't a great fit in order to try and provide certain security guarantees, and this requires pre-fetching a lot of data up front in case the app requests it. But NTFS is slow, so, all this additional traffic makes opening files via the Storage API really really slow.
The problem here isn't really "modern software", it's that Microsoft wrote a lot of new APIs over time with the intention of replacing Win32 (UWP/WinRT) but they aren't dogfooding them all that effectively, and they're using C++ for everything, so there are problems that don't get fixed for years.
This was exactly the problem with CORBA in the 90s. It made remote calls as easy to use as local calls. Once they looked the same people would make RPCs in loops like that.
Imagine the cost of a non obvious RPC call in a nested for loop? Just not funny.
Imagine the cost of a non obvious RPC call in a nested for loop? Just not funny.
Odds are someone who writes an app is doing it in a sterile environment, where there are only going to be a handful of test files in any folders.
If they do try it on their own system, they'll likely discount any performance hits as due to their own hording of files, and not consider the user likely to have just as many, if not more files. Thus, its not likely the programmer will notice it.
It's only if some exogenous event causes a programmer to consider run-time performance that it will be measured, and then optimized.
If they do try it on their own system, they'll likely discount any performance hits as due to their own hording of files, and not consider the user likely to have just as many, if not more files. Thus, its not likely the programmer will notice it.
It's only if some exogenous event causes a programmer to consider run-time performance that it will be measured, and then optimized.
Another reason could also be the fact that devs usually only ever run development builds of the software.
So you end up dismissing slowness due to this ("I am sure this is faster when compiled in Release!").
I've made that mistake before until it was so slow that I decided to compile a Release build which was just as slow and found out that a regression was introduced.
Automatic performance monitoring for 'tasks'/services/computations is relatively straightforward but not quite as easy for UI interactions so these often get ignored.
So you end up dismissing slowness due to this ("I am sure this is faster when compiled in Release!").
I've made that mistake before until it was so slow that I decided to compile a Release build which was just as slow and found out that a regression was introduced.
Automatic performance monitoring for 'tasks'/services/computations is relatively straightforward but not quite as easy for UI interactions so these often get ignored.
> there are only going to be a handful of test files
Bingo. This used to be a well-known benchmarking cheat, because performance on a full (or even once-full) filesystem can be very different than on a freshly made one. Almost as common as short-stroking, if you remember that. Anybody who wanted realistic results would run tests on a pre-aged filesystem. Cache warming is another example of a similar effect, and there are probably others across every area of computing. It's always really tempting to run tests in a maximally clean environment to get maximally consistent results, but that just doesn't reflect real-world usage.
Bingo. This used to be a well-known benchmarking cheat, because performance on a full (or even once-full) filesystem can be very different than on a freshly made one. Almost as common as short-stroking, if you remember that. Anybody who wanted realistic results would run tests on a pre-aged filesystem. Cache warming is another example of a similar effect, and there are probably others across every area of computing. It's always really tempting to run tests in a maximally clean environment to get maximally consistent results, but that just doesn't reflect real-world usage.
The actual capabilities of a modern AMD or Intel x86 chip are staggering compared to the zombie-like procession of application experiences they are forced to support every day.
In the 90s, we had shooters running at playable frame rates using software rasterization on a single thread. Now from this vantage point, think about the "what if you had a 1000x faster CPU" thought experiment that was posted to HN recently. Except, make it more like 100,000x faster...
If we had taken SIMD a bit further on the CPU and kept our extremely scrappy software engineering practices, I think it's possible the GPU might not have ever really emerged as a major consumer product. Recent game engine tech, such as Nanite's software rasterizer, is starting to shift ideologies back the other way now too.
In the 90s, we had shooters running at playable frame rates using software rasterization on a single thread. Now from this vantage point, think about the "what if you had a 1000x faster CPU" thought experiment that was posted to HN recently. Except, make it more like 100,000x faster...
If we had taken SIMD a bit further on the CPU and kept our extremely scrappy software engineering practices, I think it's possible the GPU might not have ever really emerged as a major consumer product. Recent game engine tech, such as Nanite's software rasterizer, is starting to shift ideologies back the other way now too.
> it's possible the GPU might not have ever really emerged as a major consumer product
I’m really skeptical about this take, the gains from the GPU architecture are too large to pass up on
I’m really skeptical about this take, the gains from the GPU architecture are too large to pass up on
Parent poster seems to be from Intel. They argued very similar about specialized CPUs back then. "You only need one chip"
Then came 3dfx, which was a blast.
You can even call the Amiga Blitter as one of the first GPUs, at least a specialized graphics chip. Same goes for coprocessors, like a math unit in a 486DX for example.
Then came 3dfx, which was a blast.
You can even call the Amiga Blitter as one of the first GPUs, at least a specialized graphics chip. Same goes for coprocessors, like a math unit in a 486DX for example.
Yeah, especially considering Quake 3D was the killer GPU app
Nanite is a software rasterizer running as a compute shader on the GPU though, it still requires the power of a GPU but takes advantage of the increased flexibility of modern GPUs.
Consider: if I'm a bad programmers and I care about slow software but I don't know enough to make it fast, how do I go about learning?
This is something I've considered a bit since listening to some Jonathan Blow talks. When you have loose requirements which change over time, limited time, and you're left figuring stuff out on your own it's difficult to do an outstanding job.
Nobody taught me how to write fast software. At best I learned enough to get stuff working. Usually after I've built something slow but functional I might've learned enough to write a faster version, but there's no time to do another rewrite because the features are changing.
For many developers the choice isn't between fast and slow, it's probably between slow or nothing at all.
Although I do think that big tech companies should be held to slightly higher standards.
This is something I've considered a bit since listening to some Jonathan Blow talks. When you have loose requirements which change over time, limited time, and you're left figuring stuff out on your own it's difficult to do an outstanding job.
Nobody taught me how to write fast software. At best I learned enough to get stuff working. Usually after I've built something slow but functional I might've learned enough to write a faster version, but there's no time to do another rewrite because the features are changing.
For many developers the choice isn't between fast and slow, it's probably between slow or nothing at all.
Although I do think that big tech companies should be held to slightly higher standards.
but I don't know enough to make it fast
I'd say that in the vast majority of cases, you don't actually need to; what you need to instead focus on is how to make it not slow, and to do so is really not that difficult. In a word, it's (total) simplicity. The less code there is in total that the CPU has to execute, the faster it'll be. Thus, don't add unnecessary abstractions/indirections unless necessary, pull in huge libraries and use a tiny fraction of their functionality, etc.
I'm of the opinion that you don't need to know about cycle counting and such to write decently efficient software. You don't need to specifically optimise if you start off with a sane and simple design.
I'd say that in the vast majority of cases, you don't actually need to; what you need to instead focus on is how to make it not slow, and to do so is really not that difficult. In a word, it's (total) simplicity. The less code there is in total that the CPU has to execute, the faster it'll be. Thus, don't add unnecessary abstractions/indirections unless necessary, pull in huge libraries and use a tiny fraction of their functionality, etc.
I'm of the opinion that you don't need to know about cycle counting and such to write decently efficient software. You don't need to specifically optimise if you start off with a sane and simple design.
> focus on is how to make it not slow
That's the same thing, worded differently. A junior programmer might focus on looping for-loops backwards "for speed" and then leave a serial `await` in the loop itself. If you don't see that that's a problem, no "focus" will help.
That's the same thing, worded differently. A junior programmer might focus on looping for-loops backwards "for speed" and then leave a serial `await` in the loop itself. If you don't see that that's a problem, no "focus" will help.
Simplicity is good but often not the solution to make things faster.
Most obvious example. Binary search is faster than linear, at the cost of more complexity (that can often be hidden under an abstraction).
Async code is often perceived as faster than synchronous, at the cost of more complexity. In the case of TFA, loading the file list asynchronously in the background could have been a good candidate for such.
Most obvious example. Binary search is faster than linear, at the cost of more complexity (that can often be hidden under an abstraction).
Async code is often perceived as faster than synchronous, at the cost of more complexity. In the case of TFA, loading the file list asynchronously in the background could have been a good candidate for such.
> The less code there is in total that the CPU has to execute, the faster it'll be.
The more higher-level the language is, the less this is true in my experience
The more higher-level the language is, the less this is true in my experience
The #1 rule of optimization is that it's only 10% or 1% of the code which matters. You only need to optimize the code which runs 1000 times a second, the code which runs 1 time every 10 seconds in a background thread can be inefficient and nobody will notice or care. You don't need to rewrite your entire app to make it run substantially faster, you only need to rewrite the hot-paths.
Another key rule is to limit the "main loop" to as little as possible: don't run big computations which don't affect what the user is focusing on. For example, web browsers will pause and unload tabs when you have several of them: this means you can have over 100 open tabs and your web browser will still run fast, because the unopened tabs are not doing anything, they are just caching the URL and (in some cases) whatever is rendered on the site. Similarly, most games try not to render or update things outside of the player's vision. You don't have to go to extremes like this, but if something doesn't affect the user's flow (e.g. an extra feature only a few people use), its performance impact should be negligible. Specifically: don't put code in your hot-path that doesn't need to be there.
Another key rule is to use well-written libraries for your algorithms. I assure you a vector-math library with 1000 stars implements vector-math operations much faster than you can, and those operations are neatly wrapped in easy-to-use functions.
When you have code in your hot-path which needs to be there, is too slow and you can't replace it with a library, then you bring out the big-O and zero-allocation techniques. And also, caching. A lot of optimization is ultimately caching, as computation is generally much more expensive than memory. Those super pretty render engines all use a ton of caching.
You can have a piece of software with tons of bloat, extra features, inefficient/redundant computations, and an Electron back-end, which is still fast (example: VSCode). Even the Linux kernel has a lot of bloat in the form of various drivers, but it does not affect the main runtime as these drivers are not loaded. Even graphically-intensive games and simulations have redundant computations and excess allocations, they are just not in the hot-path.
And last tip: don't write exponential-time algorithms. The above doesn't apply when you have an exponential-time algorithm, because even when n = 30 it will slow your computer, and when n = 75 it will run past the sun burns out.
Another key rule is to limit the "main loop" to as little as possible: don't run big computations which don't affect what the user is focusing on. For example, web browsers will pause and unload tabs when you have several of them: this means you can have over 100 open tabs and your web browser will still run fast, because the unopened tabs are not doing anything, they are just caching the URL and (in some cases) whatever is rendered on the site. Similarly, most games try not to render or update things outside of the player's vision. You don't have to go to extremes like this, but if something doesn't affect the user's flow (e.g. an extra feature only a few people use), its performance impact should be negligible. Specifically: don't put code in your hot-path that doesn't need to be there.
Another key rule is to use well-written libraries for your algorithms. I assure you a vector-math library with 1000 stars implements vector-math operations much faster than you can, and those operations are neatly wrapped in easy-to-use functions.
When you have code in your hot-path which needs to be there, is too slow and you can't replace it with a library, then you bring out the big-O and zero-allocation techniques. And also, caching. A lot of optimization is ultimately caching, as computation is generally much more expensive than memory. Those super pretty render engines all use a ton of caching.
You can have a piece of software with tons of bloat, extra features, inefficient/redundant computations, and an Electron back-end, which is still fast (example: VSCode). Even the Linux kernel has a lot of bloat in the form of various drivers, but it does not affect the main runtime as these drivers are not loaded. Even graphically-intensive games and simulations have redundant computations and excess allocations, they are just not in the hot-path.
And last tip: don't write exponential-time algorithms. The above doesn't apply when you have an exponential-time algorithm, because even when n = 30 it will slow your computer, and when n = 75 it will run past the sun burns out.
The optimise the hot part strategy, applied repeatedly across different test cases, stamps down the spikes on the profiler until the profile is basically flat.
Sadly at that point rather more thought is needed and it seems to get a bit domain specific what needs to change next.
A corollary of this that bad architecture choices made up front, e.g. because performance was not considered, become both extremely entrenched and difficult to see on the flat flame graph.
Sadly at that point rather more thought is needed and it seems to get a bit domain specific what needs to change next.
A corollary of this that bad architecture choices made up front, e.g. because performance was not considered, become both extremely entrenched and difficult to see on the flat flame graph.
First, learn how to run an execution profile[1], like the author did in this article.
Second, try to find out how fast software is written in the domain. These are often architectural decisions. Is this software better written by loading the entire data into memory? Or should I process it sequentially and keep memory usage bounded? What data needs to be kept "hot"?
Third, try to figure out what parts of the OS or libraries you use have potentially unbounded waiting. You don't want that to be in any thread that has to be responsive. Either make it async or run it in another thread. This is the cause of a lot of slowness in software, where it gets tested mainly on a single machine but it is actually a distributed system, or where there is never contention for a file in testing, nor is a database row ever locked by another application.
1. https://learn.microsoft.com/en-us/visualstudio/profiling/pro...
Second, try to find out how fast software is written in the domain. These are often architectural decisions. Is this software better written by loading the entire data into memory? Or should I process it sequentially and keep memory usage bounded? What data needs to be kept "hot"?
Third, try to figure out what parts of the OS or libraries you use have potentially unbounded waiting. You don't want that to be in any thread that has to be responsive. Either make it async or run it in another thread. This is the cause of a lot of slowness in software, where it gets tested mainly on a single machine but it is actually a distributed system, or where there is never contention for a file in testing, nor is a database row ever locked by another application.
1. https://learn.microsoft.com/en-us/visualstudio/profiling/pro...
Study basic algorithms and data structures. The purpose of this is not to pass a leetcode test, those are obnoxious and those companies that conduct them should be embarrassed. The purpose is to learn how to analyze systems and their performance on two primary metrics: time and space. This isn't all there is to understanding larger systems (simplistic counting the steps of a loop is not sufficient if you're swamped by IO access times, for instance, but it's a start). But you need to be able to examine a system and understand tradeoffs. When deciding between different data structures you're often faced with a situation like: X is faster at A, but slower at B; Y is the opposite. Which do you choose? I can't tell you, you have to know your intended use-cases. Will A dominate in your application, or B? This is the kind of thinking that scales, too. A database that's better at random access reads and writes is better for some applications than one that's optimized for sequential reads but lousy at random writes, so choose the suitable option based on a reasonable expectation of your system's needs.
Study parallel programming (that is, actually having two or more threads/processes/whatevers running simultaneously). This will help you understand the limits of parallelism in speeding up programs, but also the potential for some programs. You'll learn about how to divide work across N processors, the limits of performance gains, and where you will get performance losses (critical ones: synchronization and communication).
Study concurrency so you can see how to make use of asynchronous patterns. I don't mean strictly async/await style, just how to use whatever language/OS facilities to avoid unnecessary blocking. Like, if you're making a network query it can take seconds or longer to resolve (depending on location, size of the data, remote server load, etc.). Your program shouldn't wait if it still has other things it can do (if it doesn't, then let it wait).
Use a profiler, study your real systems and see where they spend their time. What functions get called the most, what functions or syscalls consume the most time. Is there a file/resource that's constantly being used? I once sped up a program by around 60x just by realizing that the previous coder was repeatedly reading from the same file. It wasn't needed, he could have read from it once (optimal), or at least cached it in memory (though memory wasn't so plentiful then, it would've been a tight fit on a mid-00s desktop, but fine today). Apply the ideas above to analyzing it and improving it.
Study parallel programming (that is, actually having two or more threads/processes/whatevers running simultaneously). This will help you understand the limits of parallelism in speeding up programs, but also the potential for some programs. You'll learn about how to divide work across N processors, the limits of performance gains, and where you will get performance losses (critical ones: synchronization and communication).
Study concurrency so you can see how to make use of asynchronous patterns. I don't mean strictly async/await style, just how to use whatever language/OS facilities to avoid unnecessary blocking. Like, if you're making a network query it can take seconds or longer to resolve (depending on location, size of the data, remote server load, etc.). Your program shouldn't wait if it still has other things it can do (if it doesn't, then let it wait).
Use a profiler, study your real systems and see where they spend their time. What functions get called the most, what functions or syscalls consume the most time. Is there a file/resource that's constantly being used? I once sped up a program by around 60x just by realizing that the previous coder was repeatedly reading from the same file. It wasn't needed, he could have read from it once (optimal), or at least cached it in memory (though memory wasn't so plentiful then, it would've been a tight fit on a mid-00s desktop, but fine today). Apply the ideas above to analyzing it and improving it.
On parallelism:
Sufficient attentiin applied to parallelism/concurrency, is indistinguishable from management, without any of the benefits or recognition, or most other practitioners bothering to even try to to grok it.
Does that mean you shouldn't do it? No. However, there comes a point where if you breach that barrier, you will discover that the fundamental problems of today arise out of willingness to inflict pain stemming from the component actors in the Human Virtual Machine.
Sufficient attentiin applied to parallelism/concurrency, is indistinguishable from management, without any of the benefits or recognition, or most other practitioners bothering to even try to to grok it.
Does that mean you shouldn't do it? No. However, there comes a point where if you breach that barrier, you will discover that the fundamental problems of today arise out of willingness to inflict pain stemming from the component actors in the Human Virtual Machine.
Use an old computer. Slow operations are enlarged which makes it more obvious where the problem lies. I have an old HP EliteBook 6930p with a Core 2 Duo P8700 CPU from 2008. I did put an SSD in it because Windows 10 is unusable without it.
Using an SSD hides a lot of IO-related performance issues though. I suspect it's why Windows 10 is super slow on an HDD.
> if I'm a bad programmers and I care about slow software but I don't know enough to make it fast, how do I go about learning?
Learn how to use a sampling profiler.
Learn how to use a sampling profiler.
Under Big O and any time you have an algorithm, see what the complexity is and if you can reduce it.
I just tried opening the Voice Recorder on Windows 11 and was met with this: https://i.imgur.com/K8FViqO.png
I honestly have no words for this. I can't decide if I should laugh or cry.
I honestly have no words for this. I can't decide if I should laugh or cry.
Just wait until it starts nagging you to rate it in the Microsoft Store like the calculator does.
I gathered all the pieces of the Windows7 calculator and transplanted them into Windows 10 because of these stories and after seeing how slow it loaded. Speed to load is a critical feature of a PC calculator IMO.
That's how you get 1 star ratings.
calculator?! jesus fucking christ.
I also don't dig the look of these apps compared to the clean GUI of Windows 2000 for example. I understand everything needs to be big and vectorized because of 4k displays, touchscreens and whatever else, but it always feels like these apps are running on Flash.
And to tell us this you're using imgur that loads JS garbage from 10 domains in order to display a small picture.
What do you recommend instead?
In a pinch, I make a post to a reddit profile and then link direct to the image. Unlike Imgur, that does link directly to the single file and not a page with html/js.
I use ImgBB, they are probably stealing your images and not deleting them if you set it to auto delete but for screenshots like this it probably doesn't matter.
I use ImgBB, they're probably stealing your images but did screenshots like this is doesn't matter.
I’m profoundly disappointed in the direction computing has gone in the last decade. It’s like Idiocracy actually happened and we’re living in that timeline. Desktop software is trash. Apps that worked perfectly well a decade ago on far slower computers run like ass on far faster computers. The new Mac Outlook sucks CPU just sitting there doing nothing. The new Acrobat is so bad I’ve gone back to emacs for reading and annotating PDFs. To be clear, viewing PDFs in Emacs is a total hack. It involves rendering each page to a PNG file in a separate process, and having Emacs display the PNG. And it still causes the fans on my laptop to spin up less than Acrobat. And don’t even get me started on all the pdf.js/Electron stuff.
What on earth is going on in the computing industry? Did all the smart nerds get siphoned off to hedge funds and machine learning, so Microsoft and Google can only hire the dregs to write boring desktop software?
What on earth is going on in the computing industry? Did all the smart nerds get siphoned off to hedge funds and machine learning, so Microsoft and Google can only hire the dregs to write boring desktop software?
I started in the industry in the 90s. Back then most devs were actually interested in computing and motivated to learn. And management was pretty loose. In my first job they basically told "Here is the problem. Come back in a few months.". No standups, no backlog. But I sat in a nice office with two other guys where we could talk about the problems.
Nowadays I feel there are a lot of people who got into computing as a career but don't really care. And management is trying to convert software development into a micromanaged sweatshop with easily replaceable people.
Nowadays I feel there are a lot of people who got into computing as a career but don't really care. And management is trying to convert software development into a micromanaged sweatshop with easily replaceable people.
Software development has become so ubiquitous and lucrative that it attracts folks who are heavily motivated to find some reason to exist in the org structure, regardless of whether or not that's good for the product.
The number of TPMs and co-PMs (both Product and Project varietals) has exploded. There are now "group program managers" and of course all of these no-hard-product-artifact roles have formed hierarchies, so it's now possible to be a senior technical program manager managing other technical program mangers, the real work-product of which is purely additional meetings. Sub-par software engineers now have tons of places in the org structure where they can move laterally to maintain a place in tech - usually by inserting themselves between technical groups.
Thus, the ratio of people-who-can-produce-something-in-the-product to people-who-mostly-use-email-jira-slack-and-${videoconf} has plummeted.
Any non-trivial engineering product requires organizational members, but there's administrative bloat in tech to rival academia.
I suspect that many of the egregiously bad examples here are, rather than the product of a small team of terrible engineers, actually the product of several large teams of engineers being mediated by pseudo-technical people lodged in bureaucratic seams. That's how the worst software I've personally seen has been produced.
The number of TPMs and co-PMs (both Product and Project varietals) has exploded. There are now "group program managers" and of course all of these no-hard-product-artifact roles have formed hierarchies, so it's now possible to be a senior technical program manager managing other technical program mangers, the real work-product of which is purely additional meetings. Sub-par software engineers now have tons of places in the org structure where they can move laterally to maintain a place in tech - usually by inserting themselves between technical groups.
Thus, the ratio of people-who-can-produce-something-in-the-product to people-who-mostly-use-email-jira-slack-and-${videoconf} has plummeted.
Any non-trivial engineering product requires organizational members, but there's administrative bloat in tech to rival academia.
I suspect that many of the egregiously bad examples here are, rather than the product of a small team of terrible engineers, actually the product of several large teams of engineers being mediated by pseudo-technical people lodged in bureaucratic seams. That's how the worst software I've personally seen has been produced.
> What on earth is going on in the computing industry? Did all the smart nerds get siphoned off to hedge funds and machine learning, so Microsoft and Google can only hire the dregs to write boring desktop software?
They’re grinding leetcode and getting a new job every ~1.5 years.
What happens when owning a home and starting a family near your employer is hard and gaming the interview process is easy.
They’re grinding leetcode and getting a new job every ~1.5 years.
What happens when owning a home and starting a family near your employer is hard and gaming the interview process is easy.
I was kinda shocked to learn the turnover at tech companies. Though I hardly think it’s just the Bay Area. From what I learnt the avg tenure is 4+ years nationally and about 1.5 for the likes of Google, Microsoft, Snapchat, Meta, …etc. These companies all pay well. I’ve stayed at companies for longer and for much lesser than that.
While I don’t think company loyalty is warranted, I can’t understand why there aren’t much efforts to retain them. Like matching your current employees with market rather than letting them feel like they have to job hop for a better pay.
While I don’t think company loyalty is warranted, I can’t understand why there aren’t much efforts to retain them. Like matching your current employees with market rather than letting them feel like they have to job hop for a better pay.
The answer to this question is not politically correct, and therefore will not be answered. We will all end up with quantum computers and software that runs slower than any software from the late 90/2000s
[deleted]
It comes down to difficulties of getting a team together that can implement cross-platform software. Shipping native implementations is more than quadruple the work and cost as compared to shipping with Electron, which is reliably cross-platform and doesn't need you to gather programmers who can pull off native work, not to mention the complications in communication and management that arise after headcount of a group gets larger. There are now alternatives to Electron (Tauri/Rust and Wails/Go) and some Electron projects are starting to transfer over (like 1Password -- you can really feel difference) but it'll take time for the ecosystem to mature and become more attractive than Electron for the average project.
But right now, Electron remains a solid choice if you want to ship cross-platform desktop software without overextending on budget and time. Moreover, it's possible to make Electron apps that aren't terrible if they're optimized enough (Discord, VSCode, etc.).
PDF software is a different ball game as PDF is a behemoth of a specification. Did you know you could envelope 3d models within PDF files? Obviously a stripped down implementation (which you should use, if you don't require the fancier parts of PDF) will be snappier.
But right now, Electron remains a solid choice if you want to ship cross-platform desktop software without overextending on budget and time. Moreover, it's possible to make Electron apps that aren't terrible if they're optimized enough (Discord, VSCode, etc.).
PDF software is a different ball game as PDF is a behemoth of a specification. Did you know you could envelope 3d models within PDF files? Obviously a stripped down implementation (which you should use, if you don't require the fancier parts of PDF) will be snappier.
The slowness seems to be from the UI. PDF engines, except pdf.js, are very fast. On my Mac Mini MuPDF can render 60-100 pages per second, which means for a typical PDF you don’t even have to cache more than a couple of pages in memory at a time. Emacs (pdf-tools) uses poppler, which is slower, but still can handle scrolling through 300 mb PDFs no problem. And the Acrobat engine itself is fast. Prior to Acrobat DC it ran very smoothly on old hardware.
PDF has a lot of features, but it’s not complicated. If your PDF doesn’t contain a 3D model, that’s just some code that takes up room in the library but probably doesn’t even get paged in. A PDF is just a command stream, and the commands for drawing text are straightforward: https://www.oreilly.com/library/view/developing-with-pdf/978...
As to cross platform software, what about Qt, or GTK, or WxWidgets? There isn’t a single decent Electron app. Even VSCode uses CPU sitting there doing nothing. It’s a freaking text editor, what’s happening in the background?!
PDF has a lot of features, but it’s not complicated. If your PDF doesn’t contain a 3D model, that’s just some code that takes up room in the library but probably doesn’t even get paged in. A PDF is just a command stream, and the commands for drawing text are straightforward: https://www.oreilly.com/library/view/developing-with-pdf/978...
As to cross platform software, what about Qt, or GTK, or WxWidgets? There isn’t a single decent Electron app. Even VSCode uses CPU sitting there doing nothing. It’s a freaking text editor, what’s happening in the background?!
> PDF has a lot of features, but it’s not complicated.
I'm not sure about, Acrobat reader in particular is very complicated, I'll rehash a famous old reddit comment --
Acrobat doesn't just read PDFs, it's a mail server, document lifecycle management system, DRM client, full-fledged document tracking system, it can have forms, it can gather statistics for your docs when you share them, it has audio and video playback. The manual is a couple thousand pages. Adobe's proposition to businesses is "You want a complete content tracking system? Wanna be able to effortlessly conduct surveys? Get next-level telemetry data from your users? Have we got the thing for you... the best part is all of your clients already have it installed!"
So good on you for ditching Adobe and not enabling this bull, SumatraPDF/emac's pdf-tools is perfect.
Regarding native GUI toolkits, they work but they're more work than Electron to deal with. Secondly, the big thing that us nerds keep failing to recognize is that software needs to be easy to use for grandmas and grandpas, part of that means having consistency in UI for users across the board from one system to the other. See how Zoom is nearly identical when using on iPad or Windows 7 or even Linux? Couldn't pull that off with GTK. I think with Qt you can modify your widgets quite a bit but you still don't have control over the minutiae of UI like you do with Electron.
Don't get me wrong it is still the wrong choice many times; the abomination that is MS Teams and Spotify is a flagrant violation of good taste and should be set on fire and nuked out of the fucking orbit, it is a heinous crime especially for big companies with the means to arrange large teams to build and deploy apps made with non-WebView-based frameworks, but for small shops I don't see any issue with Electron's use. VSCode launches instantaneously on my M2 Macbook Air. Zoom works flawlessly, along with a host of other Electron-based apps. I don't care too much if it's using some CPU here and there if I can't discern any slowness. Anyway, apps made with Tauri are much better in terms of CPU and memory usage, 1Password moved from Electron and Tauri and I'm certain more will follow and the next wave of desktop apps will surely be speedier now that Electron's slowness has become a bit of a meme.
I'm not sure about, Acrobat reader in particular is very complicated, I'll rehash a famous old reddit comment --
Acrobat doesn't just read PDFs, it's a mail server, document lifecycle management system, DRM client, full-fledged document tracking system, it can have forms, it can gather statistics for your docs when you share them, it has audio and video playback. The manual is a couple thousand pages. Adobe's proposition to businesses is "You want a complete content tracking system? Wanna be able to effortlessly conduct surveys? Get next-level telemetry data from your users? Have we got the thing for you... the best part is all of your clients already have it installed!"
So good on you for ditching Adobe and not enabling this bull, SumatraPDF/emac's pdf-tools is perfect.
Regarding native GUI toolkits, they work but they're more work than Electron to deal with. Secondly, the big thing that us nerds keep failing to recognize is that software needs to be easy to use for grandmas and grandpas, part of that means having consistency in UI for users across the board from one system to the other. See how Zoom is nearly identical when using on iPad or Windows 7 or even Linux? Couldn't pull that off with GTK. I think with Qt you can modify your widgets quite a bit but you still don't have control over the minutiae of UI like you do with Electron.
Don't get me wrong it is still the wrong choice many times; the abomination that is MS Teams and Spotify is a flagrant violation of good taste and should be set on fire and nuked out of the fucking orbit, it is a heinous crime especially for big companies with the means to arrange large teams to build and deploy apps made with non-WebView-based frameworks, but for small shops I don't see any issue with Electron's use. VSCode launches instantaneously on my M2 Macbook Air. Zoom works flawlessly, along with a host of other Electron-based apps. I don't care too much if it's using some CPU here and there if I can't discern any slowness. Anyway, apps made with Tauri are much better in terms of CPU and memory usage, 1Password moved from Electron and Tauri and I'm certain more will follow and the next wave of desktop apps will surely be speedier now that Electron's slowness has become a bit of a meme.
Did you just list the advantages of web tech in the first half and then called PDF spec a behemoth? :D
Actually, PDF is not that bad at all. Sure, it has gotten some additions over the years, but even those are not actually too foreign to the original design.
Actually, PDF is not that bad at all. Sure, it has gotten some additions over the years, but even those are not actually too foreign to the original design.
At least webtech has some excuse for being a gargantuan tarantula and having its legs everywhere, PDF software is supposed to be basic document software but overreaches.
End users don't pay for desktop software any more. I wonder if this is part of the answer.
Have you tried using Okular? It's a pretty serviceable PDF reader on Linux, and now has a Windows port as well.
Okular also comes with a per-page cropping mode, though it tends to cause issues when scrolling between pages. I prefer to use pdf-crop-margins (and on encrypted PDFs, qpdf --decrypt so pdf-crop-margins can read them) to generate a pre-cropped PDF that works on any reader, but keep the original PDF around for when I like it better.
Another trick is that on large 32+ inch screens, you can use Firefox's pdf.js in horizontal scrolling view to hold 2-3 pages on-screen simultaneously and cross-reference them easily, then switch to the hand tool for easy continuous scrolling.
Okular also comes with a per-page cropping mode, though it tends to cause issues when scrolling between pages. I prefer to use pdf-crop-margins (and on encrypted PDFs, qpdf --decrypt so pdf-crop-margins can read them) to generate a pre-cropped PDF that works on any reader, but keep the original PDF around for when I like it better.
Another trick is that on large 32+ inch screens, you can use Firefox's pdf.js in horizontal scrolling view to hold 2-3 pages on-screen simultaneously and cross-reference them easily, then switch to the hand tool for easy continuous scrolling.
I was more making a comparison between PDF apps with newfangled UIs (Acrobat, anything based on Electron or PDF.js), and a complete hack in Emacs that somehow still works better. There are, of course, decent PDF apps built on 1980s/1990s UI technology. Sumatra (Win32) and Preview (directly descended from the Preview app in NeXT Step).
Isn't it the natural progression of most technical devices ?
For instance looking at coffee machines, the general direction was:
- no machine, or decent but not so advanced machines - super basic machines, or advanced but finicky machines - crappy capsule machines, or super advanced finicky machines
Depending on how you look at it, there is a path from "advanced but finicky machines" to "crappy capsule machines". Personally I think things still improved, even as the low end is still crappier than the middle/high end from decades ago.
For instance looking at coffee machines, the general direction was:
- no machine, or decent but not so advanced machines - super basic machines, or advanced but finicky machines - crappy capsule machines, or super advanced finicky machines
Depending on how you look at it, there is a path from "advanced but finicky machines" to "crappy capsule machines". Personally I think things still improved, even as the low end is still crappier than the middle/high end from decades ago.
[deleted]
A very slow scan of the Documents folder, by RuntimeBroker.exe which manages the Store app sandbox, reminds me of Android apps waiting up to 10 seconds for the Android system file picker to list the contents of my Downloads folder since apps (kinda) aren't allowed to browse your files directly.
Oh, this is nothing.
Windows force-updated my Notepad.exe to the new modern UI version.
It is very visibly laggy for basic actions such as opening menus or scrolling through text.
You have to be spectacularly bad at programming to make something so basic so slow in 2022 on a very high-end gaming computer.
Not just bad, but utterly oblivious. Casey Muratori excoriated the Windows Terminal dev team for failing to achieve a consistent 60 fps when displaying fixed-width text, which is such a trivial exercise that Casey whipped up a demo in like two weekends. [1]
Now the Notepad team insisted on following in the Windows Terminal team's footprints and slowing down the other fixed-width text app that needs to be lightweight fast for all sorts of reasons. Nobody needs Notepad to be pretty. Everyone needs it to work.[2]
Just amazing to see something like this happen. It's like a slow motion train wreck, ruining one Windows app at a time.
[1] These videos are especially "shocking" because the code he wrote is terse, straightforward, lightning fast, and more correct than whatever madness Microsoft was doing.
First weekend: https://www.youtube.com/watch?v=hxM8QmyZXtg
Second weekend: https://www.youtube.com/watch?v=99dKzubvpKE
[2] For example, opening a 300 MB log file takes a solid minute of time, which is absurdly slow. It also bloats this to 800 MB of memory usage. For comparison, TextPad opens it in 5 seconds and uses 30 MB of memory. VS Code, an electron application(!) opens in under 2 seconds.
Windows force-updated my Notepad.exe to the new modern UI version.
It is very visibly laggy for basic actions such as opening menus or scrolling through text.
You have to be spectacularly bad at programming to make something so basic so slow in 2022 on a very high-end gaming computer.
Not just bad, but utterly oblivious. Casey Muratori excoriated the Windows Terminal dev team for failing to achieve a consistent 60 fps when displaying fixed-width text, which is such a trivial exercise that Casey whipped up a demo in like two weekends. [1]
Now the Notepad team insisted on following in the Windows Terminal team's footprints and slowing down the other fixed-width text app that needs to be lightweight fast for all sorts of reasons. Nobody needs Notepad to be pretty. Everyone needs it to work.[2]
Just amazing to see something like this happen. It's like a slow motion train wreck, ruining one Windows app at a time.
[1] These videos are especially "shocking" because the code he wrote is terse, straightforward, lightning fast, and more correct than whatever madness Microsoft was doing.
First weekend: https://www.youtube.com/watch?v=hxM8QmyZXtg
Second weekend: https://www.youtube.com/watch?v=99dKzubvpKE
[2] For example, opening a 300 MB log file takes a solid minute of time, which is absurdly slow. It also bloats this to 800 MB of memory usage. For comparison, TextPad opens it in 5 seconds and uses 30 MB of memory. VS Code, an electron application(!) opens in under 2 seconds.
> You have to be spectacularly bad at programming to make something so basic so slow in 2022 on a very high-end gaming computer.
I don't understand why desktop app user interfaces aren't blazingly fast and 240fps.
That being said, I think current iOS and macOS can run their UIs at 120fps (and Apple devices still seem to have decent battery life.)
I don't understand why desktop app user interfaces aren't blazingly fast and 240fps.
That being said, I think current iOS and macOS can run their UIs at 120fps (and Apple devices still seem to have decent battery life.)
> I don't understand why desktop app user interfaces aren't blazingly fast and 240fps.
Telemetry back to HQ takes resources.
I began to notice my disk drive light flickering regularly even when I was doing nothing with the computer, and the computer started seeming sluggish. I'd look at the process list, and started randomly forcing them to quit. Suddenly, the drive light stopped flashing. I'd found my man.
Googling the name of the process, it belonged to a program I'd bought that served media files to my Roku box. But I wasn't asking it to stream anything at the time.
I didn't know what the hell it was doing, but couldn't justify it, and removed the program.
Telemetry back to HQ takes resources.
I began to notice my disk drive light flickering regularly even when I was doing nothing with the computer, and the computer started seeming sluggish. I'd look at the process list, and started randomly forcing them to quit. Suddenly, the drive light stopped flashing. I'd found my man.
Googling the name of the process, it belonged to a program I'd bought that served media files to my Roku box. But I wasn't asking it to stream anything at the time.
I didn't know what the hell it was doing, but couldn't justify it, and removed the program.
> Googling the name of the process, it belonged to a program I'd bought that served media files to my Roku box. But I wasn't asking it to stream anything at the time.
Maybe it was indexing.
Maybe it was indexing.
It doesn't take days to index my hard disk. Besides, I configured it with the path the media files. There's no reason to spy on the rest of the drive.
Why would it be indexing if no files are changed?
GPU accelerated vector graphics rendering is an open problem.
For now.
Unfortunately, even if you have very fast drawing, there are plenty of other ways to make software slow. But high performance GUI software is the main thing I'm working on at the moment and I expect to have interesting results in the coming months. If nothing else, it should be a data point of how fast GUI software can be.
Unfortunately, even if you have very fast drawing, there are plenty of other ways to make software slow. But high performance GUI software is the main thing I'm working on at the moment and I expect to have interesting results in the coming months. If nothing else, it should be a data point of how fast GUI software can be.
I’m super excited to hear about this. I’ve been working on a programming language for UI designers, and I ran into some massive issues when I tried to build my own renderer. Gave up and now I’m just transpiling to other languages instead.
Eagerly awaiting this!
Maybe basic productivity applications shouldn't try to tackle open research problems and instead just do the thing which has worked fine for the last 30 years which is CPU rendering? Like, do you think notepad is going to burn your battery because it redraws 20x20 pixels on every mouse event ?
It isn’t an open problem for text editors or terminals, which is the point Casey made in his videos.
text editors and terminals don’t render vector graphics though, do they? they rely on libraries to rasterize glyphs of a font into a bitmap once and use the bitmaps over and over.
> they rely on libraries to rasterize glyphs of a font into a bitmap once
Unfortunately it's not this simple. Unicode and multilingual support means you can't possibly generate a single bitmap to contain all possible characters. Modern UIs (including text editors and terminals) generally support various zoom levels, which requires re-rasterizing glyphs at different zoom levels. This means you typically have to rasterize vector graphics (your font) on the fly as needed and then update your bitmap.
Edit: Also, emojis and certain scripts require composting multiple glyphs together. This means you have to figure out which glyphs are required for your codepoint and use your font metrics to composite the glyphs appropriately. Then throw in mathematical notation and all bets are off.
My point is, rendering text is not a simple done deal, but it's also not an impossible feat and it can be done efficiently.
Unfortunately it's not this simple. Unicode and multilingual support means you can't possibly generate a single bitmap to contain all possible characters. Modern UIs (including text editors and terminals) generally support various zoom levels, which requires re-rasterizing glyphs at different zoom levels. This means you typically have to rasterize vector graphics (your font) on the fly as needed and then update your bitmap.
Edit: Also, emojis and certain scripts require composting multiple glyphs together. This means you have to figure out which glyphs are required for your codepoint and use your font metrics to composite the glyphs appropriately. Then throw in mathematical notation and all bets are off.
My point is, rendering text is not a simple done deal, but it's also not an impossible feat and it can be done efficiently.
that is a very pedantic argument. you completely overlooked the point i was trying to make.
vector graphics are not rendered directly to the screen. fonts are rasterized first, then the rasterization is what the GPU receives.
vector graphics are not rendered directly to the screen. fonts are rasterized first, then the rasterization is what the GPU receives.
> that is a very pedantic argument. you completely overlooked the point i was trying to make.
lmao welcome to HN, if you don't enjoy that experience I don't recommend you try to argue here.
lmao welcome to HN, if you don't enjoy that experience I don't recommend you try to argue here.
> vector graphics are not rendered directly to the screen.
What difference does it make if you're caching the rasterization or not? You're still rendering vector graphics. And you're not just rasterizing once.
Modern font renderers can't get away with rasterizing glyphs once and reusing them anymore than arbitrary vector graphics renderers can. I sure hope my vector graphics editors cache the rasterizations instead of re-rasterizing needlessly.
What difference does it make if you're caching the rasterization or not? You're still rendering vector graphics. And you're not just rasterizing once.
Modern font renderers can't get away with rasterizing glyphs once and reusing them anymore than arbitrary vector graphics renderers can. I sure hope my vector graphics editors cache the rasterizations instead of re-rasterizing needlessly.
once, twice, two hundred times, whatever.
it's not every frame. that's my point. that you missed twice because you can't see past details to the point of the statement.
it's not every frame. that's my point. that you missed twice because you can't see past details to the point of the statement.
a huge amount of emoji are embedded png in font files. newer ones, however, do require more work.
> figure out which glyphs are required for your codepoint and use your font metrics to composite the glyphs appropriately.
this is all done in shaping though, the renderers just see "put this thing there" and (at least in GTK) we just maintain texture atlases that are updated on the fly.
you're correct that it doesn't solve the same glyph at lots of scales though. but glyphy does (and at high FPS too).
> figure out which glyphs are required for your codepoint and use your font metrics to composite the glyphs appropriately.
this is all done in shaping though, the renderers just see "put this thing there" and (at least in GTK) we just maintain texture atlases that are updated on the fly.
you're correct that it doesn't solve the same glyph at lots of scales though. but glyphy does (and at high FPS too).
GTK's implementation of text rendering is is admittedly broken in several subtle ways that nobody seems to know how to properly fix. [1] Also, this is while _not_ supporting fractional scaling, which is something that users definitely want.
To be clear, I'm not bashing on GTK. I'm saying that it definitely isn't as simple as the above comment implies ;)
[1] https://gitlab.gnome.org/GNOME/gtk/-/issues/3787
To be clear, I'm not bashing on GTK. I'm saying that it definitely isn't as simple as the above comment implies ;)
[1] https://gitlab.gnome.org/GNOME/gtk/-/issues/3787
That issue is literally why I have a branch integrating GLyphy. But yes, your casual drive by comment is definitely more knowledgeable than the people who work on the stuff.
harfbuzz creator made https://github.com/behdad/glyphy/ which i integrated into GTK a while back (but we don't ship currently, because we still do the bitmap stuff).
the bitmap stuff still has major drawbacks though, like maintaining grid alignments and pixel boundaries where as this stuff (mostly) goes away using glyphy.
still work to be done around hinting though (hence not merged).
the bitmap stuff still has major drawbacks though, like maintaining grid alignments and pixel boundaries where as this stuff (mostly) goes away using glyphy.
still work to be done around hinting though (hence not merged).
You shouldn't even need a GPU for basic UI like this, especially with "modern" ultra-minimalist flat UIs.
Yeah, but they're not actually completely flat since they're translucent.
Also, on Windows, the "acrylic" thing is a circus show. Every app behaves differently. It also manages to sometimes lag on an RX5600. I know it's an "old" gpu, but still...
Also, on Windows, the "acrylic" thing is a circus show. Every app behaves differently. It also manages to sometimes lag on an RX5600. I know it's an "old" gpu, but still...
But you don’t have to do it every frame. Most apps could just render to bitmaps once and cache the results.
It is being worked on (ahem...)
But in most contexts, it is not necessary to render vector graphics 'live'; caching is acceptable, so this should not be a bottleneck for most desktop software.
But in most contexts, it is not necessary to render vector graphics 'live'; caching is acceptable, so this should not be a bottleneck for most desktop software.
is it? MacOS has rendered the entire UI via PDF since OSX was first released.
It's not really PDF, it just supports similar operations to PDF. There's more SVG UI elements now.
but it’s vector graphics, right?
Only very recently. Like, all that old Aqua stuff wasn't vectors, it was done in Photoshop.
Bitmaps are much faster to draw and can be made pixel-precise up front. With vectors it's easy to have pixel cracks or blurry lines because it's offset .5px.
Bitmaps are much faster to draw and can be made pixel-precise up front. With vectors it's easy to have pixel cracks or blurry lines because it's offset .5px.
> Bitmaps are much faster to draw and can be made pixel-precise up front.
yes, but that alone doesn't require that bitmap graphics be used.
> With vectors it's easy to have pixel cracks or blurry lines because it's offset .5px.
only if your renderer or your artwork isn't done correctly; inkscape used to be notorious for floating point rounding errors showing up with very few manipulations of a shape.
it's also been very easy to very quickly rasterize vector graphics for some time. maybe not fast enough in the past I guess, but computers are certainly fast enough now.
I look forward to a time when UIs are just all vector all the time, and it's the monitor itself which rasterizes, thanks to a program that the attached device uploads as it boots up or as the monitor is connected.
it's going to get more and more difficult to continue increasing display resolutions and refresh rates over long inexpensive cables if we continue to rasterize on a computer and push raw pixels to the display device.
yes, but that alone doesn't require that bitmap graphics be used.
> With vectors it's easy to have pixel cracks or blurry lines because it's offset .5px.
only if your renderer or your artwork isn't done correctly; inkscape used to be notorious for floating point rounding errors showing up with very few manipulations of a shape.
it's also been very easy to very quickly rasterize vector graphics for some time. maybe not fast enough in the past I guess, but computers are certainly fast enough now.
I look forward to a time when UIs are just all vector all the time, and it's the monitor itself which rasterizes, thanks to a program that the attached device uploads as it boots up or as the monitor is connected.
it's going to get more and more difficult to continue increasing display resolutions and refresh rates over long inexpensive cables if we continue to rasterize on a computer and push raw pixels to the display device.
afaik there just hasn’t been a lot of work done towards leveraging the GPU for vector graphics. The one example I can think of is the Slug library.
> afaik there just hasn’t been a lot of work done towards leveraging the GPU for vector graphic
Sure there has been, in the form of the nvidia path rendering extension. https://developer.nvidia.com/nv-path-rendering
It works great, on nvidia cards. Adobe Illustrator on Windows makes good use of it. Calling the extension from C/C++ and Python is no big deal.
Sure there has been, in the form of the nvidia path rendering extension. https://developer.nvidia.com/nv-path-rendering
It works great, on nvidia cards. Adobe Illustrator on Windows makes good use of it. Calling the extension from C/C++ and Python is no big deal.
Like? SVG UIs everywhere? Could you make a UI that's like SVG and onclick handlers do the "magic". We could have a VB6 like SVG UI designer and program "actions" in $LANG?
WPF would probably be that. In any case, modern UI toolkits often are vector-based anyway because of UI scaling. Heck, even ancient ones are, if you look at the Marlett font. That contained scalable UI elements for Windows 95.
> That being said, I think current iOS and macOS can run their UIs at 120fps (and Apple devices still seem to have decent battery life.)
The same critique for Apple applies all the time, when you are completely vertically integrated and make your own chips up to the very software running on them, it's kinda easier.
That aside, there's no display offering above 60Hz IIRC in the Apple line. Correct me if I am wrong.
The same critique for Apple applies all the time, when you are completely vertically integrated and make your own chips up to the very software running on them, it's kinda easier.
That aside, there's no display offering above 60Hz IIRC in the Apple line. Correct me if I am wrong.
The phone and laptop have had them since last year, the iPad Pro has had them for years and years now.
14" and 16" MacBook Pros are equipped with 120 Hz displays
A lot of the speed games get is by having bad load times when starting up.
Keeping startup really fast and all interactions really fast is more challenging than it seems.
This is not the whole story, but one example of the differences in perception.
Keeping startup really fast and all interactions really fast is more challenging than it seems.
This is not the whole story, but one example of the differences in perception.
Streaming in hundreds of assets as you walk around an open world does surely take a different pipeline, but as a show of computing competence it is staggering. You should see how fast some of these truly massive games load nowadays. The fact that the notepad app takes forever to load when all it needs to load is the UI is just a sad state of affairs.
But games are loading enormous textures, meshes, maps, etc. UIs don’t need any of that.
A mesh may have a lot of triangles, but it's still a single object you can memory map. UIs are typically a bunch of heap allocations and pointer jumping all created at runtime. That can be surprisingly inefficient.
then make it efficient. overallocate a block of memory and use it for all your tiny ui elements. its exactly what games do.
Arena allocators only work when everything has the same lifetime.
There are game-like UI libraries (https://github.com/ocornut/imgui) but to use them you have to write your program like you're a gamedev and Conway's Law usually fights that.
There are game-like UI libraries (https://github.com/ocornut/imgui) but to use them you have to write your program like you're a gamedev and Conway's Law usually fights that.
Most bad loading times fall into 2 categories
1. Avoidable: The main one here is shader caching. Shaders could be precompiled for every supported card/driver, but that's more work than anyone is willing to do. Even so, some games (like Rust) have an excessive amount of shaders, which adds a lot of extra waiting time.
2. Unavoidable: The main one here is textures. If you want it to be quickly accessible in VRAM, then it's going to be uncompressed, which means either having some very large files to copy over, or having some semi-large files to decompress. You can save quite a bit of time by micromanaging the texture resolution of each asset to be just high enough. Apart from textures, you have vertex data and other assets; all of which have basically the same considerations.
When it comes to regular software, most UI designs don't have big assets. They aren't loading billions of triangles or thousands of textures. This means there really isn't much need for long startup times.
1. Avoidable: The main one here is shader caching. Shaders could be precompiled for every supported card/driver, but that's more work than anyone is willing to do. Even so, some games (like Rust) have an excessive amount of shaders, which adds a lot of extra waiting time.
2. Unavoidable: The main one here is textures. If you want it to be quickly accessible in VRAM, then it's going to be uncompressed, which means either having some very large files to copy over, or having some semi-large files to decompress. You can save quite a bit of time by micromanaging the texture resolution of each asset to be just high enough. Apart from textures, you have vertex data and other assets; all of which have basically the same considerations.
When it comes to regular software, most UI designs don't have big assets. They aren't loading billions of triangles or thousands of textures. This means there really isn't much need for long startup times.
Fortunately you can copy the binary from an older version of Windows.
At least you could thank Microsoft for being backwards-compatible.
I've done this with calc.exe since ~Vista.
the Notepad team
That's probably part of the problem --- Notepad is something that could be written by a single person in less than a day (it's not more than a wrapper around a Win32 multiline editbox.) Yet give a whole team of multiple people the task of "modernising" it, and work will magically appear to fill the available resources.
At least you could thank Microsoft for being backwards-compatible.
I've done this with calc.exe since ~Vista.
the Notepad team
That's probably part of the problem --- Notepad is something that could be written by a single person in less than a day (it's not more than a wrapper around a Win32 multiline editbox.) Yet give a whole team of multiple people the task of "modernising" it, and work will magically appear to fill the available resources.
As noted in the sibling comment, Notepad does A LOT of stuff, some of it to deal with users who use (abuse?) it to do things it was never intended to do, and MS saw fit to keep it doing those things for compatibility. It did these things prior to the update, and knowing MS I imagine it's had to maintain some compatibility with the older version.
That said, I've not encountered the same slowness issues since updating that are discussed here. I'm not a fan of the menu animations, but it's a minor nitpick. For editing huge texts it is significantly more efficient, in my experience. I remember turning off word-wrap in the old notepad after loading a huge log file, then having to wait minutes for Notepad to become responsive again.
Yeah, I know, anecdata....
Edit: The source code to the older Notepad is rather easy to find online. I won't link to it, but it's an interesting read.
That said, I've not encountered the same slowness issues since updating that are discussed here. I'm not a fan of the menu animations, but it's a minor nitpick. For editing huge texts it is significantly more efficient, in my experience. I remember turning off word-wrap in the old notepad after loading a huge log file, then having to wait minutes for Notepad to become responsive again.
Yeah, I know, anecdata....
Edit: The source code to the older Notepad is rather easy to find online. I won't link to it, but it's an interesting read.
This 'modern' Notepad is actually not 'a wrapper around a Win32 edit box', it's (likely to support additional use cases like macOS TextEdit in the future - most the modernization seems to be to haphazardly 'compete' with arbitrary features from macOS) actually a variant of the Office rich text control, which in some cases actually is more performant.
The WinUI parts and initial launch cost, however, are somewhat hard to forgive.
The WinUI parts and initial launch cost, however, are somewhat hard to forgive.
Obligatory Raymond Chen reference: https://devblogs.microsoft.com/oldnewthing/20180521-00/?p=98...
> For example, did you know that you can type https://www.microsoft.com/ into Notepad’s File Open dialog?
I tried it. Everything in the File Open dialog got disabled except for the cancel button for about 30 seconds until it loaded. For comparison, curl took 2.3s (250ms in later runs so probably waiting for curl to load) and my browser took around 250ms with cache disabled as well.
Why does it take Notepad so long?
I tried it. Everything in the File Open dialog got disabled except for the cancel button for about 30 seconds until it loaded. For comparison, curl took 2.3s (250ms in later runs so probably waiting for curl to load) and my browser took around 250ms with cache disabled as well.
Why does it take Notepad so long?
software is a gas they say.
I have a 2000 line markdown file that I edit with the Mac native Markdown+ application. It is so slow that I can type full sentences before it can update. This on an M1 with 16GB of RAM doing nothing else.
2000 lines is nothing. I routinely edited book length texts in Microsoft Word for DOS on a 4.77 MHz 8088.
2000 lines is nothing. I routinely edited book length texts in Microsoft Word for DOS on a 4.77 MHz 8088.
Not familiar with that specific app, but... I'm guessing it's trying to re-rendering the entire document every keystroke?
That’s my guess. Hard to do it any other way for Markdown. However, it could probably do that on a background thread and still be responsive with large amounts of text.
This can be done without multiple threads. MicroEmacs does it by checking the keyboard input buffer every once in a while while rendering. If there's a key in it, it abandons the rendering to go service the key. When there are no keys in the buffer, it tries rendering again. It works so well you never notice this is happening, you just get instance keyboard response.
As I recall GNU Emacs does this too, by default. https://www.masteringemacs.org/article/improving-performance...
I expect this is normal practice.
Forget background threads. People don’t know basic programming principles. The thought of debouncing user input didn’t even occur to them.
What do you mean by 'debouncing' here? I only know it as ignoring keypresses of the same key/button that happen too fast after each other (because they are likely a mechanical or electrical glitch rather than the user actually meaning to press the button/key multiple times).
If you mean that, I don't know how it relates to the comment/article since I never needed to do debouncing in a desktop context (only for embedded stuff with actual push button circuitry attached).
If you mean that, I don't know how it relates to the comment/article since I never needed to do debouncing in a desktop context (only for embedded stuff with actual push button circuitry attached).
That same concept, but one step removed if you will.
The classic example is a data table with a text field that filters/searches the results, pulled from a server.
Let’s say you want the results to update “live” as the user types, without having to click a “search” button. So if the user types “bacon” then she sees all results that contain the word “bacon” in them.
Problem is, if the act of sending the text field input to the server and waiting for the results is relatively slow and blocks the UI from updating, then each key press causes a delay between the search results updating on the screen, and the next letter showing up in the search field.
1. User types “b” and then the app sends “b” to the server, pauses, gets a response of all results with the letter “b”.
2. User types “a”, and the app sends “ba” to the server… pauses… gets results. She types a “c”, the app sends “bac”, etc.
Debouncing, then, means the app waits a short period of time to make sure the user is finished typing before sending the search term.
Usually this means every time the user enters a letter into the search field, the app sets a timer for, say, 500ms. If the user doesn’t enter another letter for 500ms, great - send the search term, wait for the results.
If the user does enter another letter within 500ms, then she must still be typing, so reset the timer back to 500ms and hold off on sending the search term out to the server.
The classic example is a data table with a text field that filters/searches the results, pulled from a server.
Let’s say you want the results to update “live” as the user types, without having to click a “search” button. So if the user types “bacon” then she sees all results that contain the word “bacon” in them.
Problem is, if the act of sending the text field input to the server and waiting for the results is relatively slow and blocks the UI from updating, then each key press causes a delay between the search results updating on the screen, and the next letter showing up in the search field.
1. User types “b” and then the app sends “b” to the server, pauses, gets a response of all results with the letter “b”.
2. User types “a”, and the app sends “ba” to the server… pauses… gets results. She types a “c”, the app sends “bac”, etc.
Debouncing, then, means the app waits a short period of time to make sure the user is finished typing before sending the search term.
Usually this means every time the user enters a letter into the search field, the app sets a timer for, say, 500ms. If the user doesn’t enter another letter for 500ms, great - send the search term, wait for the results.
If the user does enter another letter within 500ms, then she must still be typing, so reset the timer back to 500ms and hold off on sending the search term out to the server.
Same idea, debouncing replaces a sequence of "re-render Markdown" events (button pushes) with a single "re-render" event (button push). Googling "debouncing" will turn up a bunch of articles about it.
This. Incredible. Debouncing was done in the early 70s and even someone on;y familiar with more modern similar ideas would note that Nagling was invented in 1984 at the latest...
Not a UI / frontend developer. Doesn’t denouncing user input increase the latency before changes are rendered onto the screen?
I would expect the changes to be visible ideally in the next frame, but I guess that’s not really an option?
I would expect the changes to be visible ideally in the next frame, but I guess that’s not really an option?
Debouncing only has to carry a latency penalty for certain kinds of responses. A good UI should be able to start processing an action as soon as the button goes down, and add/amend the actions later once it's clear whether it was a press and hold, press and release, or double-click. If the appropriate responses to those three options are mutually exclusive to the point that the UI can't respond at all until the user input is fully disambiguated, then the keybindings are probably ill-considered and those unrelated or conflicting functions should not be sharing a button.
In the strictest sense, yes. But here clearly rendering is taking longer than 1/60s so it wouldn’t matter.
The compromise would be to always start rendering immediately rather than wait x ms, but if you detect that the previous rendering hasn’t completed when the next input has arrived then you can start debouncing before rendering until you catch up.
The compromise would be to always start rendering immediately rather than wait x ms, but if you detect that the previous rendering hasn’t completed when the next input has arrived then you can start debouncing before rendering until you catch up.
Seems like Markdown could be rendered 1-3 lines at a time, depending on line-height. It isn't like other formats where formatting information is spread all over the place.
Type two backticks, everything below is fine. Type one more… the entire rest of the syntax tree is potentially a new tree. Backspace, do the whole thing again.
Behavior solvable.
When a user inserts to backticks, insert the closing backticks immediately.
When a user inserts to backticks, insert the closing backticks immediately.
Cool, now explain how to do it for backspace.
And now explain how anyone can ever turn existing text into a code block if every user action is "modified" to not turn existing text into a code block.
And now explain how anyone can ever turn existing text into a code block if every user action is "modified" to not turn existing text into a code block.
This may be obvious but if you're trashing the second pair of backticks you obviously want the rest of it to be inside the backticks which required the first re-render so its not a mistake, the user made the request , follow through, render till the next set of bacticks into a block.. no trailing backticks ? Render everything.. thats the choice the user made.
> And now explain how anyone can ever turn existing text into a code block if every user action is "modified" to not turn existing text into a code block.
At no point did I mention that you couldnt wrap code in bacticks, I simply said the default insert would NOT have single pair backticks opening up to a forced re-render of every trailing element.
> And now explain how anyone can ever turn existing text into a code block if every user action is "modified" to not turn existing text into a code block.
At no point did I mention that you couldnt wrap code in bacticks, I simply said the default insert would NOT have single pair backticks opening up to a forced re-render of every trailing element.
That solves your problem but probably doesn’t solve the user’s.
I'm not sure if it's the case with this particular app, but with many apps these days you have to seek out and download the version specifically for apple silicon.
IntelliJ was killing me until I figured this out.
IntelliJ was killing me until I figured this out.
The problem here isn't the intel/arm issue, it's that the application amplifies handling a single keystroke into what is basically a massive re-rendering job instead of applying any of dozens of well-known solutions to the problem.
Applications are simply not well written and most developers are incompetent and unaware of how incompetent they are.
Applications are simply not well written and most developers are incompetent and unaware of how incompetent they are.
Sure, but in MS Word on an 8088 you were editing in DOS text mode.
Markdown+ is parsing and rendering your Markdown in real time. It may be slower than it needs to be, I haven’t used it and don’t know, but it’s doing a lot more work than 1982-era MS Word was.
You can still get blazing fast speed for even gigabytes of Markdown if you use instead use something like BBEdit on a current Mac, which would be the equivalent modern user experience.
Markdown+ is parsing and rendering your Markdown in real time. It may be slower than it needs to be, I haven’t used it and don’t know, but it’s doing a lot more work than 1982-era MS Word was.
You can still get blazing fast speed for even gigabytes of Markdown if you use instead use something like BBEdit on a current Mac, which would be the equivalent modern user experience.
All that parsing and rendering can be done in the background, not grinding the UI to a halt. As soon as I strike a key on my keyboard, the letter should appear. Instantly. (Ok, in 20ms or 40ms or whatever)
If I'm actively editing a markdown file, I do not need for it to fire off a re-render on each keystroke. It can wait 500ms before starting to do it, and cancel if I hit another key in the meantime. If I start typing after it has started rendering, then go ahead and finish that cycle while still showing me what I type in real time.
However these events are scheduled, the appearance of my typed characters on the screen is going to always be the priority. There's no reason to block UI while doing this.
If I'm actively editing a markdown file, I do not need for it to fire off a re-render on each keystroke. It can wait 500ms before starting to do it, and cancel if I hit another key in the meantime. If I start typing after it has started rendering, then go ahead and finish that cycle while still showing me what I type in real time.
However these events are scheduled, the appearance of my typed characters on the screen is going to always be the priority. There's no reason to block UI while doing this.
I use Thunderbird mail. It often freezes for several seconds while typing text. This is ordinary behavior for modern software. I'm not excusing it, it's just that it's hardly exclusive to Microsoft.
TB is a multithreaded program, but it isn't very good at using it effectively. I've learned through bitter experience to not touch a key or the mouse when it is getting messages.
TB is a multithreaded program, but it isn't very good at using it effectively. I've learned through bitter experience to not touch a key or the mouse when it is getting messages.
I've never used Word for DOS. The oldest I've used it "seriously" was word 97 on windows 95 or 98 on a 100 MHz 486. Not book-length documents, but comfortably in the dozens of pages. That thing would fly.
Modern Word lags on a two-page document on an 8 core xeon with 64 GB of RAM. Same PC can run modern games at 4k@60 FPS.
I can agree that maybe modern Word "does more work" than Word 97. But as another commenter said: WTF is that work? And do we even need it?
As a user, all I see is that modern Word's interface has buttons all over the place, taking up half my screen. And when I grab the scroll-bar, it won't scroll until some time after I've released it. Word 97 managed to scroll the contents as I moved the bar just fine.
It also takes forever to start-up on an SSD so fast it can move multiple times the size of that old 486's whole hard drive in one second. WTF is it loading? And once loaded, why is it so slow, since everything can fit in RAM?
Modern Word lags on a two-page document on an 8 core xeon with 64 GB of RAM. Same PC can run modern games at 4k@60 FPS.
I can agree that maybe modern Word "does more work" than Word 97. But as another commenter said: WTF is that work? And do we even need it?
As a user, all I see is that modern Word's interface has buttons all over the place, taking up half my screen. And when I grab the scroll-bar, it won't scroll until some time after I've released it. Word 97 managed to scroll the contents as I moved the bar just fine.
It also takes forever to start-up on an SSD so fast it can move multiple times the size of that old 486's whole hard drive in one second. WTF is it loading? And once loaded, why is it so slow, since everything can fit in RAM?
> it’s doing a lot more work than 1982-era MS Word was.
That's the point of gp's comment. The software is doing a lot of "work" that it does not need to do, thanks to poor programming.
Work is activity that helps to achieve a goal. Whatever Win11's Notepad is doing, it is not that.
That's the point of gp's comment. The software is doing a lot of "work" that it does not need to do, thanks to poor programming.
Work is activity that helps to achieve a goal. Whatever Win11's Notepad is doing, it is not that.
Markdown parsing can be quite fast. I just benchmarked pulldown-cmark parsing all the Markdown in my blog (10k lines, 1MB) and writing HTML, and it does that in 14ms. Admittedly that's on an M1 Max.
If indeed premature optimization is the root of all evil, then I have lots to atone for.
If indeed premature optimization is the root of all evil, then I have lots to atone for.
premature optimization is the root of all evil is the root of all evil.
I think my point was lost on you. Word had code to accommodate the realities of what the application was trying to achieve. This app I mention could trivially be hundreds of times faster just by doing basic things that have been done since before most current SWE were born.
I don't know how it did it, but the old DOS version of MS word actually displayed bold/italic/underline in the DOS terminal. It wasn't WYSIWYG but the display was a bit nicer than Wordstar.
For all the 80s (before the first version for Windows) the terminal was the only thing out there, plus the graphic modes. I think that bold/italic/underline only existed in the graphic mode version (EGA.)
https://en.wikipedia.org/wiki/History_of_Microsoft_Word
https://en.wikipedia.org/wiki/History_of_Microsoft_Word
Ha the DOS terminal, that thing where the fullscreen button wouldn't work and you have to right click -> properties -> layout -> change the window size manually until you have something that work ok.
At least, they have fix it in the most recent version of windows ....
At least, they have fix it in the most recent version of windows ....
I imagine the OP means the dos interface.
First they ruined the perfectly functional calculator software by bloating it up to the point it takes several seconds to even appear on screen. Now you're telling me the next time they update Windows at work, the notepad will also be ruined.
It just keeps getting worse. I mean how hard is it to do nothing and leave these old apps alone?
It just keeps getting worse. I mean how hard is it to do nothing and leave these old apps alone?
There's no excuse for 2 seconds.
Machines are a thousand times as fast as in the 90s, and a text editor didn't take any 2000 seconds, or even 2 seconds, back then.
Now they load up the monstrous libraries they need to negotiate a way to communicate with the desktop, and then negotiate that. A screen buffer is so yesterday, so I guess they have a scene graph now that some library renders whenever a pixel might have changed, triple-buffered to prevent artifacts.
Now they load up the monstrous libraries they need to negotiate a way to communicate with the desktop, and then negotiate that. A screen buffer is so yesterday, so I guess they have a scene graph now that some library renders whenever a pixel might have changed, triple-buffered to prevent artifacts.
It's probably time to start all over. Chuck compatibility for usability — no more POSIX & friends, for instance, along with the rush to compile everything on anything new, with all the cruft those efforts inject into a system. Let the new thing be the new thing for a while. As much as I like what I can do with a modern system, I've found myself longing more than once of late for an 80x24 text display with instant responses from the system.
MacOS X introduced a fully buffered desktop to prevent artifacts 21 years ago. It was a bit pokey on a 500 MHz G4 but still faster than today’s apps.
[deleted]
overpaid ui designers that should probably be contracts maybe. so much pointless 'work'
there is fogies complaining about change, then there is making things objectively worse for zero gain. christ
and they are releasing a new taskmgr on win11 to boot
there is fogies complaining about change, then there is making things objectively worse for zero gain. christ
and they are releasing a new taskmgr on win11 to boot
> and they are releasing a new taskmgr on win11 to boot
Which is borken. In the performance tab, I often see the right-hand side elements cut-off (like RAM size, etc) even though there's still space left in the window. Resizing it brings the text back.
But hey, at least it now supports dark mode!
Which is borken. In the performance tab, I often see the right-hand side elements cut-off (like RAM size, etc) even though there's still space left in the window. Resizing it brings the text back.
But hey, at least it now supports dark mode!
What do you want from a company that cannot fit their progress indicator squares neatly into their own progress bars since 1995? (Anyone who has seen scandisk and defrag from Windows 95 will understand)
nothing is sacred! go figure
And we wonder what 7k employees are doing at Docusign... so many useless positions, people gatta find something to do to pad that promo packet.
to be fair to Microsoft, the terminal developers didn’t have a lot of experience with approaches like the one Casey used. Casey had 25+ years of game development experience to draw from, and the Windows Terminal team didn’t.
game development is an entirely different paradigm than enterprise software development; they aren’t really comparable, other than they both produce software which successfully executes.
in games, any code which does not directly contribute to the final product is not written, or is removed after it is not needed. in enterprise software, tens of thousands of lines of code are produced simply so the project layout is familiar for everyone working on it, because that’s important, somehow?
Casey was right to complain about the performance, and to show them how it should work. licensing it so that they couldn’t even look at it was both 100% intentional and 100% a dick move, and i will not credit him for that. that code could have been used as a wakeup call regarding performance for a lot of people within Microsoft, but he licensed it so they can’t even look at it.
perfomance of microsoft software isn’t great, i’ll agree. poking fun at them then hiding the code that was easy enough to write for him, but completely not obvious to the Windows Terminal team, doesn’t really make Casey the Software Quality Hero you’ve portrayed him as. Casey writes excellent software, don’t get me wrong. this particular example and series of events is not a high point in his career, to me.
game development is an entirely different paradigm than enterprise software development; they aren’t really comparable, other than they both produce software which successfully executes.
in games, any code which does not directly contribute to the final product is not written, or is removed after it is not needed. in enterprise software, tens of thousands of lines of code are produced simply so the project layout is familiar for everyone working on it, because that’s important, somehow?
Casey was right to complain about the performance, and to show them how it should work. licensing it so that they couldn’t even look at it was both 100% intentional and 100% a dick move, and i will not credit him for that. that code could have been used as a wakeup call regarding performance for a lot of people within Microsoft, but he licensed it so they can’t even look at it.
perfomance of microsoft software isn’t great, i’ll agree. poking fun at them then hiding the code that was easy enough to write for him, but completely not obvious to the Windows Terminal team, doesn’t really make Casey the Software Quality Hero you’ve portrayed him as. Casey writes excellent software, don’t get me wrong. this particular example and series of events is not a high point in his career, to me.
Paraphrasing: "To be fair to one of the consistently largest software giants of the last 40 years, with nearly unlimited budget and a reputation for paying its software engineers well, they actually suck ass at writing software".
You seem to roughly agree with this in your 3rd paragraph. Enterprise software is fucking awful despite spending an enormous amount of money on it, and that's because the developers and the development process and the SMEs and the managers and vice presidents are all fucking awful at their jobs. That's the reason. They're fucking shit-ass at making software. Just look at Microsoft Teams. This is not the defense you seem to think it is.
Besides, we're not talking about enterprise software driven by bloat from the demand-side. We're talking about notepad and the console.
Marketing and legal is stellar though.
You seem to roughly agree with this in your 3rd paragraph. Enterprise software is fucking awful despite spending an enormous amount of money on it, and that's because the developers and the development process and the SMEs and the managers and vice presidents are all fucking awful at their jobs. That's the reason. They're fucking shit-ass at making software. Just look at Microsoft Teams. This is not the defense you seem to think it is.
Besides, we're not talking about enterprise software driven by bloat from the demand-side. We're talking about notepad and the console.
Marketing and legal is stellar though.
> Enterprise software is fucking awful despite spending an enormous amount of money on it, and that's because the developers and the development process and the SMEs and the managers and vice presidents are all fucking awful at their jobs
Enterprise software written 20 years ago still runs and works. I can still install it from an Msi package
Half the android apps I bought 5 years ago don't exist any more.
In Google they would have cancelled, closed, relaunched and cancelled again.
Games are hit and miss, admittedly some run for decades.
But above all, they dont care
Enterprise software written 20 years ago still runs and works. I can still install it from an Msi package
Half the android apps I bought 5 years ago don't exist any more.
In Google they would have cancelled, closed, relaunched and cancelled again.
Games are hit and miss, admittedly some run for decades.
But above all, they dont care
Games can avoid a lot of maintenance patterns that look like cruft simply because many games are fire and forget and not truly expected to have a long shelf life.
PS: you wouldn't believe how many tries it took to get my phone to accept the word "cruft" instead of autocorrecting it to craft. Dammit!
PS: you wouldn't believe how many tries it took to get my phone to accept the word "cruft" instead of autocorrecting it to craft. Dammit!
> This is not the defense you seem to think it is.
it’s not a defense at all, it’s my whole point.
enterprise software is absolutely lost, because following enormous sets of rules and enforcing those rules via management is FAR CHEAPER AND EASIER than hiring skilled people who don’t need those rules to produce good software.
if that isn’t the very definition of bad management and business practices, i don’t know what is.
enterprise software being a total shitshow is my point, because a shitshow is far better for shareholders in the short term. filling seats with dopes you hope to train is easy to measure. carefully harvesting the best talent is slow, expensive, and difficult to measure.
microsoft does that. all large publicly traded companies optimize for short-term gain at the expense of long-term loss.
game companies, the good ones, the smaller ones, don’t operate like that. there are shit managers there, too, but game companies attract the exact talent they need, almost by default. they care about what they create because not caring means bad reviews, low sales, no bonuses, and layoffs.
the punishment for writing shit software at any enterprise company is continued employment, and at worst, an involuntary lateral move to another position where software quality does not matter.
microsoft is an enterprise software developer. they are burdened with that management style just like a lot of our employers are.
it’s not a defense at all, it’s my whole point.
enterprise software is absolutely lost, because following enormous sets of rules and enforcing those rules via management is FAR CHEAPER AND EASIER than hiring skilled people who don’t need those rules to produce good software.
if that isn’t the very definition of bad management and business practices, i don’t know what is.
enterprise software being a total shitshow is my point, because a shitshow is far better for shareholders in the short term. filling seats with dopes you hope to train is easy to measure. carefully harvesting the best talent is slow, expensive, and difficult to measure.
microsoft does that. all large publicly traded companies optimize for short-term gain at the expense of long-term loss.
game companies, the good ones, the smaller ones, don’t operate like that. there are shit managers there, too, but game companies attract the exact talent they need, almost by default. they care about what they create because not caring means bad reviews, low sales, no bonuses, and layoffs.
the punishment for writing shit software at any enterprise company is continued employment, and at worst, an involuntary lateral move to another position where software quality does not matter.
microsoft is an enterprise software developer. they are burdened with that management style just like a lot of our employers are.
> microsoft does that. all large publicly traded companies optimize for short-term gain at the expense of long-term loss.
I hear this claim all the time, but nobody is able to explain why they continue to grow profits.
I hear this claim all the time, but nobody is able to explain why they continue to grow profits.
I'm the "main tech guy" of a financial firm and I can answer this question.
It's because people like me don't care about user experience.
We get Teams for free in our M365 subscription. It's integrated with other stuff, it has APIs for automation and I can't be blamed for choosing MS.
I know this is a shitty argument... but seriously, I have many battles to fight, not wasting my energy choosing the best chat software.
Devs think a lot a about software quality and stuff, but that's often irrelevant for business. GP is wrong, MS knows exactly what they need to do to increase profits.
It's because people like me don't care about user experience.
We get Teams for free in our M365 subscription. It's integrated with other stuff, it has APIs for automation and I can't be blamed for choosing MS.
I know this is a shitty argument... but seriously, I have many battles to fight, not wasting my energy choosing the best chat software.
Devs think a lot a about software quality and stuff, but that's often irrelevant for business. GP is wrong, MS knows exactly what they need to do to increase profits.
optimizing for short-term gain is growing profit. what do you think short-term gain is? it's gain!
what it isn't is investing in quality and long-term success; it's betting that you'll be able to find some other short-term optimization in the future, and it relies on the stock market having a memory that lasts no longer than one year
what it isn't is investing in quality and long-term success; it's betting that you'll be able to find some other short-term optimization in the future, and it relies on the stock market having a memory that lasts no longer than one year
Microsoft has had 35 years of long term growth. How can you say optimizing for short term profit at the expense of long term profit is their strategy?
> and it relies on the stock market having a memory that lasts no longer than one year
If that was true, you could easily make $billions shorting those stocks that you know sacrifice the long term. Let us know how it goes!
> and it relies on the stock market having a memory that lasts no longer than one year
If that was true, you could easily make $billions shorting those stocks that you know sacrifice the long term. Let us know how it goes!
Microsoft doesn't do _everything_ with an eye on short-term gain, is why.
Everything Microsoft did in the 1990s was based on returning profit sooner rather than later. That's the whole point of "embrace, extend, extinguish": liquidate this technology and turn it into money as fast as possible.
Every time you make a decision which gives you money or profit today, and that decision is based on money alone, you are optimizing for the short term at the expense of the long term.
It is easier to think about when you think of personal finances. You can have a slightly bigger paycheck every pay period or you can set aside money for retirement. You can maximize profit today if you borrow from your own future.
I should not have said that Microsoft do this exclusively (and at this very second I'm not sure I said that) because they don't do that exclusively. It is a common approach to decisions regarding stock price and "shareholder value" however.
Everything Microsoft did in the 1990s was based on returning profit sooner rather than later. That's the whole point of "embrace, extend, extinguish": liquidate this technology and turn it into money as fast as possible.
Every time you make a decision which gives you money or profit today, and that decision is based on money alone, you are optimizing for the short term at the expense of the long term.
It is easier to think about when you think of personal finances. You can have a slightly bigger paycheck every pay period or you can set aside money for retirement. You can maximize profit today if you borrow from your own future.
I should not have said that Microsoft do this exclusively (and at this very second I'm not sure I said that) because they don't do that exclusively. It is a common approach to decisions regarding stock price and "shareholder value" however.
> Everything Microsoft did in the 1990s was based on returning profit sooner rather than later.
Are you aware of what happened to MSFT stock during the 1990s? The claim is not credible.
> It is easier to think about when you think of personal finances. You can have a slightly bigger paycheck every pay period or you can set aside money for retirement. You can maximize profit today if you borrow from your own future.
This is not what "profit" is.
Are you aware of what happened to MSFT stock during the 1990s? The claim is not credible.
> It is easier to think about when you think of personal finances. You can have a slightly bigger paycheck every pay period or you can set aside money for retirement. You can maximize profit today if you borrow from your own future.
This is not what "profit" is.
how in the hell am I this bad at communicating?
optimizing for short term gain IS OPTIMIZING TO MAXIMIZE PROFIT IN THE SHORT TERM.
optimizing for short term gain IS OPTIMIZING TO MAXIMIZE PROFIT IN THE SHORT TERM.
If your focus is on optimizing for short term profit at the expense of long term, your company won't last long.
If your company does last long, and grows enormously, you are optimizing for the long term.
If your company does last long, and grows enormously, you are optimizing for the long term.
those companies don't last long, except for those that get lucky, and continue to find new (or create new) things to quickly liquidate.
being a public company and having shareholders more or less requires you to increase stock price at the expense of almost anything else.
investors do not care if a company goes under, provided they get a handsome return on their investment beforehand.
being a public company and having shareholders more or less requires you to increase stock price at the expense of almost anything else.
investors do not care if a company goes under, provided they get a handsome return on their investment beforehand.
Games developers have a very different tradeoff between bugginess and speed than most developers. Security bugs are completely fine, inconsistent and unreproducible behaviour is mostly fine, especially if it's funny.
> are completely fine
Can be fine. There are plenty of computer game genres where security bugs are not fine at all and where you need to be super careful to not introduce nondeterminism as that would break the whole game, all while maintaining performance.
Can be fine. There are plenty of computer game genres where security bugs are not fine at all and where you need to be super careful to not introduce nondeterminism as that would break the whole game, all while maintaining performance.
Companies should market their software based on celebrity programmers who wrote it. Similar to Shelby cars or warren Buffett stock pics.
Just look at Teams. It feels more and more like a project run by a few interns that are practicing Agile.
> licensing it so that they couldn’t even look at it
https://github.com/cmuratori/refterm/blob/main/LICENSE
It’s just GPL 2.0, what are you talking about!?
Are Microsoft employees vampires that will burn up instantly if they merely glance at GPL code or something?
This is sour grapes nonsense from Microsoft. “We don’t like your tone so we won’t even dignify your argument by considering it.”
At one point an MS employee said they would love to fix their code as suggested by Casey but he refused to even look at the YouTube video!
“I would love to hear your arguments but I refuse to listen to the sound of your voice.” is next-level dismissive.
https://github.com/cmuratori/refterm/blob/main/LICENSE
It’s just GPL 2.0, what are you talking about!?
Are Microsoft employees vampires that will burn up instantly if they merely glance at GPL code or something?
This is sour grapes nonsense from Microsoft. “We don’t like your tone so we won’t even dignify your argument by considering it.”
At one point an MS employee said they would love to fix their code as suggested by Casey but he refused to even look at the YouTube video!
“I would love to hear your arguments but I refuse to listen to the sound of your voice.” is next-level dismissive.
> just GPL 2.0
perhaps you are not familiar with what the GPL requires of its users, and how wildly incompatible that is with what a company like Microsoft does.
Microsoft employees are forbidden from looking at GPL code (as well as other virally licensed code) because they may unwittingly reproduce some GPL code they saw in a large Microsoft codebase.
If that happens, and they are taken to trial over it, and lose, that entire codebase is now GPL. This is enough of a risk to them that MS employees have computers which simply can not view GPL code.
that is also why they would not watch the video; it isn’t about his voice, it is about the code visible on stream. additionally, watching a video of GPL code being written could be perceived as an attempt to circumvent the restrictions on viewing GPL code, and would likely get the viewer into trouble.
you not already knowing this bothers me a lot, given all the things you’ve said in this discussion. you are making a lot of assumptions that are just not true and basing your opinions on the belief that those assumptions are true.
this is very similar to why software at microsoft is as bad as it is generally; a lot of people assuming they know things they don’t know, and producing results which are more flawed than they needed to be.
perhaps you are not familiar with what the GPL requires of its users, and how wildly incompatible that is with what a company like Microsoft does.
Microsoft employees are forbidden from looking at GPL code (as well as other virally licensed code) because they may unwittingly reproduce some GPL code they saw in a large Microsoft codebase.
If that happens, and they are taken to trial over it, and lose, that entire codebase is now GPL. This is enough of a risk to them that MS employees have computers which simply can not view GPL code.
that is also why they would not watch the video; it isn’t about his voice, it is about the code visible on stream. additionally, watching a video of GPL code being written could be perceived as an attempt to circumvent the restrictions on viewing GPL code, and would likely get the viewer into trouble.
you not already knowing this bothers me a lot, given all the things you’ve said in this discussion. you are making a lot of assumptions that are just not true and basing your opinions on the belief that those assumptions are true.
this is very similar to why software at microsoft is as bad as it is generally; a lot of people assuming they know things they don’t know, and producing results which are more flawed than they needed to be.
There's no reason other than lack of will for something like Windows Terminal to not be GPL compatible. The limits Microsoft places upon themselves are results of their own conscious choices.
Also, Microsoft employees work on GPL2-licensed Linux just fine, so something seems off in your story.
Also, Microsoft employees work on GPL2-licensed Linux just fine, so something seems off in your story.
other Microsoft libraries are used in Windows Terminal, optionally in code form. cmd.exe is in the same source tree, and other proprietary code of Microsoft's relies on that code; it's a non-zero portion of the command-line stuff that ships in Windows.
stuff that is not Windows Terminal but that lives in the Windows Terminal repo on GitHub gets shipped with Windows.
so, yes, there are very good reasons they can't license Windows Terminal via GPL.
WSL kernel folks get that repo whitelisted.
stuff that is not Windows Terminal but that lives in the Windows Terminal repo on GitHub gets shipped with Windows.
so, yes, there are very good reasons they can't license Windows Terminal via GPL.
WSL kernel folks get that repo whitelisted.
None of what you said inherently prevents GPL licensing. I'm not sure you understand the implications of GPL license at all.
Also, it's not just WSL people who work on GPL software inside Microsoft. I guess they are magically expected not to subconsciously reproduce any GPL code in their work ever ;)
Also, it's not just WSL people who work on GPL software inside Microsoft. I guess they are magically expected not to subconsciously reproduce any GPL code in their work ever ;)
you're right, nothing i said inherently prevents GPL licensing, but the existing license terms of the code they write themselves and license from others does inherently prevent GPL licensing. I thought that was clear, but apparently I need to be more deliberate with my comments.
IF THEY COPY IT VERBATIM.
They're free to look at the approach and simply rewrite it.
They'd have to anyway to fit in with their codebase.
They're free to look at the approach and simply rewrite it.
They'd have to anyway to fit in with their codebase.
>They're free to look at the approach and simply rewrite it.
If by "look at the approach" you mean the "look at the code", then no. You're giving a step for your code to be considered a derivative work. If by "look at the approach" you mean someone read the code, made a specification, and then someone else that hasn't looked the code, made a program based on the specification, then yes. But arguably this isn't "simply rewrite".
Those aren't even baseless considerations. ReactOS project had paused development due to concerns that derived (read: rewritten) parts of leaked Windows code may have been used. It took years for ReactOS to show that everything was in fact a clean-room implementation (basically doing what wrote above). If they hadn't, they'll have made themselves open to be sued by Microsoft.
If by "look at the approach" you mean the "look at the code", then no. You're giving a step for your code to be considered a derivative work. If by "look at the approach" you mean someone read the code, made a specification, and then someone else that hasn't looked the code, made a program based on the specification, then yes. But arguably this isn't "simply rewrite".
Those aren't even baseless considerations. ReactOS project had paused development due to concerns that derived (read: rewritten) parts of leaked Windows code may have been used. It took years for ReactOS to show that everything was in fact a clean-room implementation (basically doing what wrote above). If they hadn't, they'll have made themselves open to be sued by Microsoft.
can you, as a lawyer for Microsoft, make a guarantee to the business that if allowed, a software developer at Microsoft who works on the Windows source code will not ever reproduce a method they saw in a GPL codebase in the code they write for their job?
You can not. So what do you do to prepare for a lawsuit which alleges that you've copied GPL code into the Windows codebase? You have technical and legal barriers in place so that those developers can't see GPL code where you can control that, and make them promise that they won't do anything to compromise the license of the Windows code that they work on. Technical and legal (read: contractual) measures to prevent this are the defense that one needs to have in order to defend against a case like this, in court.
So, because you, as a Microsoft attorney, can't guarantee that GPL code won't be reproduced verbatim in any quantity in the Windows codebase, you do what you can to dissuade your developers from looking at GPL code.
please think for 2 seconds on your own and I won't have to ELI5.
You can not. So what do you do to prepare for a lawsuit which alleges that you've copied GPL code into the Windows codebase? You have technical and legal barriers in place so that those developers can't see GPL code where you can control that, and make them promise that they won't do anything to compromise the license of the Windows code that they work on. Technical and legal (read: contractual) measures to prevent this are the defense that one needs to have in order to defend against a case like this, in court.
So, because you, as a Microsoft attorney, can't guarantee that GPL code won't be reproduced verbatim in any quantity in the Windows codebase, you do what you can to dissuade your developers from looking at GPL code.
please think for 2 seconds on your own and I won't have to ELI5.
Do you just not understand copyright? That is totally in violation.
Wait, I'm also confused. I thought completely rewriting the implementation of an algorithm would not violate copyright. It may run afoul of other laws, but not copyright. Am I mistaken?
I don't mean simply renaming variables or changing a couple of lines.
I don't mean simply renaming variables or changing a couple of lines.
It does, but only if you can show its not a derivative work. See also the concept of clean room reimplementation.
"Derivative work" doesn't mean "a work made with knowledge of another work". WP explains, "In copyright law, a derivative work is an expressive creation that includes major copyrightable elements of an original, previously created first work (the underlying work)." Canonical examples include translations into another language, dramatizations of novels, or phonorecords of sheet music.
Algorithms, being purely functional, are not copyrightable in any country that I know of. So a new implementation that uses the same algorithm as the original, but without copying any of its non-functional aspects, is not a derivative work of the original and does not infringe its copyright.
Cleanroom reimplementation is a defense against allegations of copying, because under, for example, US copyright law, access plus substantial similarity is sufficient to show copying. The "He's So Fine" lawsuit is one of the most extreme examples of this doctrine. Cleanroom reimplementations are a defense against copying by eliminating the access element. But copying the functional aspects of a program does not meet the "substantial similarity" bar.
Algorithms, being purely functional, are not copyrightable in any country that I know of. So a new implementation that uses the same algorithm as the original, but without copying any of its non-functional aspects, is not a derivative work of the original and does not infringe its copyright.
Cleanroom reimplementation is a defense against allegations of copying, because under, for example, US copyright law, access plus substantial similarity is sufficient to show copying. The "He's So Fine" lawsuit is one of the most extreme examples of this doctrine. Cleanroom reimplementations are a defense against copying by eliminating the access element. But copying the functional aspects of a program does not meet the "substantial similarity" bar.
> Am I mistaken?
No.
No.
Uh, no. You cannot e.g. listen to a Radiohead song and "look at the approach and simply rewrite it" for instance if you want to be entirely clear of copyright liability - same applies for software.
Isn't copyright about copying verbatim (or with trivial enough changes that it remains recognizably the same) large enough sections of the protected work?
At least not in my country (France). At the very least changing the pitch, and rhythm and interpretation of a melody is not enough to make it an original work there.
To give a very concrete example: french singer Claude François made the song Comme d'habitude: https://youtu.be/avr5nvY4wxg
It was translated and rewritten a few times but he still is recognized as one of the copyright owners on those covers, e.g. https://youtu.be/qQzdAsjWGPg which are definitely not just a mechanical rewrite of the original score
To give a very concrete example: french singer Claude François made the song Comme d'habitude: https://youtu.be/avr5nvY4wxg
It was translated and rewritten a few times but he still is recognized as one of the copyright owners on those covers, e.g. https://youtu.be/qQzdAsjWGPg which are definitely not just a mechanical rewrite of the original score
There's a difference between creating a derivative work and analyzing how something is done in order to create something new based on that knowledge. You can make songs in style of Led Zeppelin that sound like they could be written and performed by Led Zeppelin without violating any copyrights (and to be able to do that you obviously need to know some Led Zeppelin songs beforehand), but you can't take "Dazed and Confused", change the guitar solo and part of the lyrics and call it done...
Well, at least unless you end up as big as Led Zeppelin was before someone catches you, I guess ;)
Well, at least unless you end up as big as Led Zeppelin was before someone catches you, I guess ;)
> Well, at least unless you end up as big as Led Zeppelin was before someone catches you, I guess ;)
Are you talking about the Led Zeppelin who get entangled in judicial pursuits for plagiarism every couple years? If it was as clear cut as you say it is it would never reach any judge.
There is even one case where someone was condemned for not copying any specific song but the general style of another musician : Pharrell Williams had to pay 5M to the Marvin Gaye estate because of this
https://rpjlaw.com/blurred-lines-copying-songs-musical-style...
https://www.jonesday.com/en/insights/2018/07/blurred-lines-b...
Yes, this is extremely shitty but this is the world we live in
Are you talking about the Led Zeppelin who get entangled in judicial pursuits for plagiarism every couple years? If it was as clear cut as you say it is it would never reach any judge.
There is even one case where someone was condemned for not copying any specific song but the general style of another musician : Pharrell Williams had to pay 5M to the Marvin Gaye estate because of this
https://rpjlaw.com/blurred-lines-copying-songs-musical-style...
https://www.jonesday.com/en/insights/2018/07/blurred-lines-b...
Yes, this is extremely shitty but this is the world we live in
> Are you talking about the Led Zeppelin who get entangled in judicial pursuits for plagiarism every couple years?
Yes, I intentionally mentioned a song "by Page – inspired by Jake Holmes": https://en.wikipedia.org/wiki/Dazed_and_Confused_(song)#Led_...
> The Ninth Circuit stated that the appeal was decided “on narrow grounds” and that the “decision does not give license to copyright a musical style or ‘groove.’”
What a pile of bullshit. I wasn't aware of that case.
Yes, I intentionally mentioned a song "by Page – inspired by Jake Holmes": https://en.wikipedia.org/wiki/Dazed_and_Confused_(song)#Led_...
> The Ninth Circuit stated that the appeal was decided “on narrow grounds” and that the “decision does not give license to copyright a musical style or ‘groove.’”
What a pile of bullshit. I wasn't aware of that case.
Of course you can. That's what a huge chunk of art ever created does. You're only (potentially) in trouble when you make it obvious.
No, otherwise authors of books would be suing everyone who read them and learned their ideas.
> Are Microsoft employees vampires that will burn up instantly if they merely glance at GPL code or something?
More like will legal drain life from developers if they hear they looked at GPL code.
We had to switch DB drivers because legal didn't want to deal with GPL. Imagine writing code based on it...
More like will legal drain life from developers if they hear they looked at GPL code.
We had to switch DB drivers because legal didn't want to deal with GPL. Imagine writing code based on it...
It seems perfectly fair to expect Microsoft to throw resources at heavily used apps. "They don't have the experience to make it good" is pretty chickenshit.
> It seems perfectly fair to expect Microsoft to throw resources at heavily used apps.
yes it does seem perfectly fair, doesn’t it? it isn’t. the amount of work that needs to be done always outscales the ability to do the work, forcing prioritization. the Windows Terminal team is not large, and just throwing people at the team will not make it faster.
> "They don't have the experience to make it good" is pretty chickenshit.
you’re very unpleasant already. second sentence, calling me chickenshit. classy.
how many people do you know who are capable of writing what Casey wrote in two weekends who also DON’T work on games full-time?
i don’t know any people like that. so, where do you think those people go to work? on Windows Terminal? like that’s a common career path for someone like Casey? i would be absolutely flabbergasted if there were someone on the Windows Terminal team who knew how to do what Casey did before he did it. “Hey, come work on the command prompt” does not attract people who understand how to dynamically generate a texture atlas of code points and use that atlas to render a texture representing the console window contents 60/120/144/240 frames per second. the people that know how to do that are already spoken for.
yes it does seem perfectly fair, doesn’t it? it isn’t. the amount of work that needs to be done always outscales the ability to do the work, forcing prioritization. the Windows Terminal team is not large, and just throwing people at the team will not make it faster.
> "They don't have the experience to make it good" is pretty chickenshit.
you’re very unpleasant already. second sentence, calling me chickenshit. classy.
how many people do you know who are capable of writing what Casey wrote in two weekends who also DON’T work on games full-time?
i don’t know any people like that. so, where do you think those people go to work? on Windows Terminal? like that’s a common career path for someone like Casey? i would be absolutely flabbergasted if there were someone on the Windows Terminal team who knew how to do what Casey did before he did it. “Hey, come work on the command prompt” does not attract people who understand how to dynamically generate a texture atlas of code points and use that atlas to render a texture representing the console window contents 60/120/144/240 frames per second. the people that know how to do that are already spoken for.
> i would be absolutely flabbergasted if there were someone on the Windows Terminal team who knew how to do what Casey did before he did it.
THAT is precisely the problem!
What Casey did was not some unique and special solution known only to dark wizards working on AAA game engines.
It's an obvious solution that has been implemented many times before by many people for many systems. All green-screen and DOS-era terminals work this way, for example.
Admittedly, I'm a former game engine developer, but I went into that career with essentially zero industry experience of any type. I just muddled my way through a C++ textbook and started writing a game engine.
I needed a text console for my game engine, and I also needed to display Unicode text because we were targetting an international market.
Completely naively I wrote a "text engine". It took about a week, and my solution was virtually identical to Casey's. The difference was that I was slower at implementing it, and Casey's is more feature complete, but the approach was identical.
I didn't go to Hogwarts School for Game Development Wizardry. I just thought about the problem and applied the obvious solution.
The difference between people like Casey and the Microsoft team is not necessarily aptitude, but attitude. The Microsoft team has no mechanical sympathy and no interest in quality.
They made arguments not just along the lines of "this is impossible", but also "nobody needs this".
Through every fibre of their being they believe that computers are slow and that this is fine.
THAT is precisely the problem!
What Casey did was not some unique and special solution known only to dark wizards working on AAA game engines.
It's an obvious solution that has been implemented many times before by many people for many systems. All green-screen and DOS-era terminals work this way, for example.
Admittedly, I'm a former game engine developer, but I went into that career with essentially zero industry experience of any type. I just muddled my way through a C++ textbook and started writing a game engine.
I needed a text console for my game engine, and I also needed to display Unicode text because we were targetting an international market.
Completely naively I wrote a "text engine". It took about a week, and my solution was virtually identical to Casey's. The difference was that I was slower at implementing it, and Casey's is more feature complete, but the approach was identical.
I didn't go to Hogwarts School for Game Development Wizardry. I just thought about the problem and applied the obvious solution.
The difference between people like Casey and the Microsoft team is not necessarily aptitude, but attitude. The Microsoft team has no mechanical sympathy and no interest in quality.
They made arguments not just along the lines of "this is impossible", but also "nobody needs this".
Through every fibre of their being they believe that computers are slow and that this is fine.
> All green-screen and DOS-era terminals work this way, for example
lol no they do not. absolutely not lol
> The Microsoft team has no mechanical sympathy and no interest in quality.
I know people at Microsoft. I have acquaintances on the Windows Terminal team. this is just plain factually incorrect.
lol no they do not. absolutely not lol
> The Microsoft team has no mechanical sympathy and no interest in quality.
I know people at Microsoft. I have acquaintances on the Windows Terminal team. this is just plain factually incorrect.
[deleted]
The proof is in the putting sadly. Anyone with a concern about Quality would not have shipped. As they would be inflicting pain rather than resolving it.
Green-screen and DOS-era terminals don't work by having a custom shader running on a GPU, it's true, but they do have hardware that does the same thing Muratori's custom shader does: generate screen pixels by looking up texture data in a texture atlas indexed by a glyph index taken from a display buffer, which is updated by software. Consequently their display latency averages about 9 milliseconds.
Those character generators don't do rescaling and linear interpolation of the texels, and often they're only one-bit color, but that's extra work refterm is doing that DOS-era terminals weren't doing. It wasn't that Muratori invented a new, more efficient way to do things; he just reproduced the bitmap character-generator approach that was standard from 01975 to 01985, but runs it on faster hardware. So he got about 160 megabytes per second before optimizing, ten times faster than Windows Terminal, and 1600 megabytes per second after he "put in the most basic optimization [he] could think of".
It isn't necessary to do character generation on the GPU to be responsive, either. The Alto running Smalltalk took maybe a second for its ≈0.5 MIPS CPU to redraw a screenful of proportional text, say about 2kB. The original Macintosh took maybe 100 ms using a 1-MIPS CPU and hand-optimized assembly. But the laptop I'm typing this on is about 34000 MIPS so it ought to be able to do the job in ≈3 μs. My word-wrapped proportional-font experiment in http://canonical.org/~kragen/sw/dev3/propfont.c gets 72 MB/s, which works out to ≈30 μs per 2-kB "screenful". Valgrind says it's running about 113 instructions per byte rendered, suggesting my C is closer to Smalltalk performance than QuickDraw. It's still 20× as slow as refterm but it's not using the GPU, but 5× as fast as Windows Terminal. (On the other hand, it doesn't draw to the actual display, just a memory buffer which it outputs to a PNM file at the end.)
My own unoptimized monospace terminal emulator running on the CPU with a full-color font https://gitlab.com/kragen/bubbleos/-/blob/master/yeso/admu-s... takes 6.9 wallclock seconds and 3.1 CPU seconds to display the output of yes {1..90} | head -c 1000000. This is enormously slower, only 145 kilobytes per second or 323 kilobytes per CPU second, a whole 14 milliseconds per 2-kB screenful. (And its screen really is 2 kB.) This is 1000 times slower than refterm and 100 times slower than Windows Terminal, and it still doesn't handle the difficult cases like combining characters, double-width CJK characters, and RTL text. Probably I should optimize it!
Those character generators don't do rescaling and linear interpolation of the texels, and often they're only one-bit color, but that's extra work refterm is doing that DOS-era terminals weren't doing. It wasn't that Muratori invented a new, more efficient way to do things; he just reproduced the bitmap character-generator approach that was standard from 01975 to 01985, but runs it on faster hardware. So he got about 160 megabytes per second before optimizing, ten times faster than Windows Terminal, and 1600 megabytes per second after he "put in the most basic optimization [he] could think of".
It isn't necessary to do character generation on the GPU to be responsive, either. The Alto running Smalltalk took maybe a second for its ≈0.5 MIPS CPU to redraw a screenful of proportional text, say about 2kB. The original Macintosh took maybe 100 ms using a 1-MIPS CPU and hand-optimized assembly. But the laptop I'm typing this on is about 34000 MIPS so it ought to be able to do the job in ≈3 μs. My word-wrapped proportional-font experiment in http://canonical.org/~kragen/sw/dev3/propfont.c gets 72 MB/s, which works out to ≈30 μs per 2-kB "screenful". Valgrind says it's running about 113 instructions per byte rendered, suggesting my C is closer to Smalltalk performance than QuickDraw. It's still 20× as slow as refterm but it's not using the GPU, but 5× as fast as Windows Terminal. (On the other hand, it doesn't draw to the actual display, just a memory buffer which it outputs to a PNM file at the end.)
My own unoptimized monospace terminal emulator running on the CPU with a full-color font https://gitlab.com/kragen/bubbleos/-/blob/master/yeso/admu-s... takes 6.9 wallclock seconds and 3.1 CPU seconds to display the output of yes {1..90} | head -c 1000000. This is enormously slower, only 145 kilobytes per second or 323 kilobytes per CPU second, a whole 14 milliseconds per 2-kB screenful. (And its screen really is 2 kB.) This is 1000 times slower than refterm and 100 times slower than Windows Terminal, and it still doesn't handle the difficult cases like combining characters, double-width CJK characters, and RTL text. Probably I should optimize it!
I didn't call you chickenshit, I called giving them leeway chickenshit (by synthesizing a statement that I thought roughly reflected the leeway).
I have no idea who could match what, but Windows is just an utterly massive product, they have the resources to make the whole thing shine, even if people really don't want to work on it and need big bribes.
I have no idea who could match what, but Windows is just an utterly massive product, they have the resources to make the whole thing shine, even if people really don't want to work on it and need big bribes.
Casey wrote code and made it public. That is helping.
Regardless, this isn't about Casey. This is about MS releasing crap software. Text editors from 30 years ago were faster than WT. Something is horribly broken about the programmer's approach to writing code, or they are being horribly managed, or all of the above.
On the surface, it often feels like there is just an extreme lack of pride in workmanship. In code from lots of vendors I am constantly seeing interfaces with strange glaring issues, with bizarrely convoluted UI paths, with odd render bugs, etc... It feels like there is something unfortunate happening across the industry. Maybe I'm just a cranky older engineer, but seeing so many visual and functional failures being shipped makes me think the issue is far deeper than that.
Regardless, this isn't about Casey. This is about MS releasing crap software. Text editors from 30 years ago were faster than WT. Something is horribly broken about the programmer's approach to writing code, or they are being horribly managed, or all of the above.
On the surface, it often feels like there is just an extreme lack of pride in workmanship. In code from lots of vendors I am constantly seeing interfaces with strange glaring issues, with bizarrely convoluted UI paths, with odd render bugs, etc... It feels like there is something unfortunate happening across the industry. Maybe I'm just a cranky older engineer, but seeing so many visual and functional failures being shipped makes me think the issue is far deeper than that.
Is releasing software for free under the GPL-2.0 a "dick move" ?
> Is releasing software for free under the GPL-2.0 a "dick move" ?
it is when you know that Microsoft employees are forbidden from looking at GPL code because they may unconsciously recall it and use it in a Microsoft codebase.
Casey knew that at the time and admitted to licensing it GPL for that exact reason.
That is a “dick move” to me. maybe i’m wrong. you be the judge, i guess.
it is when you know that Microsoft employees are forbidden from looking at GPL code because they may unconsciously recall it and use it in a Microsoft codebase.
Casey knew that at the time and admitted to licensing it GPL for that exact reason.
That is a “dick move” to me. maybe i’m wrong. you be the judge, i guess.
As you say, Casey is experienced and his code is valuable. MS should pay him if they want to look at his code! He said as much in his tweets… I mean, why should he be doing charity for MS by fixing their products for them?
> Casey is experienced and his code is valuable. MS should pay him if they want to look at his code! He said as much in his tweets… I mean, why should he be doing charity for MS by fixing their products for them?
so difficult that he wrote it so he could demonstrate how easy it is.
he gladly showed the rest of us how to do it, and he intentionally took action which prevented anything he did from being used within Microsoft. Microsoft is so shitty because they don’t know [thing] so i’m going to show everyone but them how to do [thing].
he could have chosen to help. he chose to not help. he chose to complain and ridicule while intentionally avoiding doing anything to change the situation he was unhappy with.
so difficult that he wrote it so he could demonstrate how easy it is.
he gladly showed the rest of us how to do it, and he intentionally took action which prevented anything he did from being used within Microsoft. Microsoft is so shitty because they don’t know [thing] so i’m going to show everyone but them how to do [thing].
he could have chosen to help. he chose to not help. he chose to complain and ridicule while intentionally avoiding doing anything to change the situation he was unhappy with.
Casey did choose to help - explaining the issue and potential solution on github. It was only after his solution was dismissed, then catching the microsoft engineers using his solution and making a blog post about it that he complained.
they didn't use his solution. they began to adopt his approach. began to. they are still not even close to done with it all, and the changes they would need to make to fully line up with Casey's approach can't happen because of other things the code does.
they were not caught using his solution. they published a blog entry describing that they were made aware of performance problems and they didn't mention him. it was rabid fans of Casey that started foaming at the mouth then, even though Casey publicly stated he didn't want or need credit, because the idea wasn't his to begin with.
they were not caught using his solution. they published a blog entry describing that they were made aware of performance problems and they didn't mention him. it was rabid fans of Casey that started foaming at the mouth then, even though Casey publicly stated he didn't want or need credit, because the idea wasn't his to begin with.
This version of events sounds too simplified
For anyone interested about the blog post drama check https://twitter.com/cmuratori/status/1522468481135902725 (hn discussion: https://news.ycombinator.com/item?id=31284419)
Personally I think that he owes them nothing, especially after the way he was treated in https://github.com/microsoft/terminal/issues/10362 so I can't comprehend statements which claim that he was an asshole for using GPL.
For anyone interested about the blog post drama check https://twitter.com/cmuratori/status/1522468481135902725 (hn discussion: https://news.ycombinator.com/item?id=31284419)
Personally I think that he owes them nothing, especially after the way he was treated in https://github.com/microsoft/terminal/issues/10362 so I can't comprehend statements which claim that he was an asshole for using GPL.
He owes them nothing, you're right. I made no claim that he does.
I'm saying he could have helped the situation, and he deliberately chose to not only avoid helping, but to hinder anyone at Microsoft who wanted to look at that code and contribute to Windows Terminal. That was a choice he made intentionally.
He could have taken the high road, there. He chose not to. It's his choice, don't get me wrong, he was well within his rights. I just disagree with that choice. that's why I call that a "dick move". And if you disagree with me, that's fine. "Dick move" means different things to different people at different times.
I'm saying he could have helped the situation, and he deliberately chose to not only avoid helping, but to hinder anyone at Microsoft who wanted to look at that code and contribute to Windows Terminal. That was a choice he made intentionally.
He could have taken the high road, there. He chose not to. It's his choice, don't get me wrong, he was well within his rights. I just disagree with that choice. that's why I call that a "dick move". And if you disagree with me, that's fine. "Dick move" means different things to different people at different times.
Microsoft hires consultants sometimes to better understand Windows. They hire people to come lecture to them about their own software. (I'm not making fun of them, it's a good idea.) If they can't pony up for a few consulting hours to help their developers learn how to write the software they should have written, that's really on them.
The way Casey handled this issue was... not good. He's been a jerk about it and generally dragged this whole dumb argument on way longer than needed (I can't believe we're still talking about it).
He was right, but he does not seem remotely pleasant to work with and has been remarkably adversarial. Hasn't exactly fostered a "let's hire him as a consultant" relationship.
He was right, but he does not seem remotely pleasant to work with and has been remarkably adversarial. Hasn't exactly fostered a "let's hire him as a consultant" relationship.
> Microsoft employees are forbidden from looking at GPL code because they may unconsciously recall it and use it in a Microsoft codebase
Isn't that exactly what Copilot does?
Isn't that exactly what Copilot does?
this is a fucking meme at this point, the amount of assumption around this.
no, that is not what copilot does unless you check the box (or fail to uncheck the box) that allows exact matches of code from public repositories.
https://docs.github.com/en/copilot/configuring-github-copilo...
no, that is not what copilot does unless you check the box (or fail to uncheck the box) that allows exact matches of code from public repositories.
https://docs.github.com/en/copilot/configuring-github-copilo...
If Copilot can look at GPL code and use that to synthesize new code, why can't a Microsoft employee?
Because the lawyers haven't caught up with Copilot yet. Legal has been hounding developers for decades at this point.
How is it a meme? Copilot can throw verbatim licensed code.
https://twitter.com/mitsuhiko/status/1410886329924194309
https://twitter.com/mitsuhiko/status/1410886329924194309
the meme is that you can turn off the verbatim stuff and no one ever remembers that they are told about this, or that it is in the documentation, or that poking around the copilot settings you will see the setting in question.
actual users of copilot seem to know this little bit of information; commenters who like to make comments about things they have never used always overlook details like this, and perpetually restate the things they learned about the topic months or years prior, even when those details were incorrect at the time, and are still incorrect today.
actual users of copilot seem to know this little bit of information; commenters who like to make comments about things they have never used always overlook details like this, and perpetually restate the things they learned about the topic months or years prior, even when those details were incorrect at the time, and are still incorrect today.
>the meme is that you can turn off the verbatim stuff
I see. But the setting (which surprises me that is opt-in) merely checks whether the output is an exact match and forces the system to produce a new output. The system can and does (as seen on the tweet) output verbatim code. So the question that parent commenter asked is legit.
I see. But the setting (which surprises me that is opt-in) merely checks whether the output is an exact match and forces the system to produce a new output. The system can and does (as seen on the tweet) output verbatim code. So the question that parent commenter asked is legit.
It is nice to be a dominant position like Microsoft because if they weren't they would have put aside their "feelings" and hired Casey to help them out. But they don't need to. Feelings of Microsoft seem more important than feelings of all their users who will just have to bear with it.
Come on man, the fact he's able to use GPL2 to annoy you is your problem. Just stop hitting yourself.
I will restate what many others have said here, Microsoft has used and interacted with GPL2 code longer than I have been alive. There is GPL2 licensed software in the Windows Store released by Microsoft.
Microsoft is the one doing the "dick move" by requiring special permission for new GPL2 interaction.
If Microsoft can interact with Linux, the biggest GPL2 project ever, they can interact with whatever Casey releases.
I will restate what many others have said here, Microsoft has used and interacted with GPL2 code longer than I have been alive. There is GPL2 licensed software in the Windows Store released by Microsoft.
Microsoft is the one doing the "dick move" by requiring special permission for new GPL2 interaction.
If Microsoft can interact with Linux, the biggest GPL2 project ever, they can interact with whatever Casey releases.
come on, man, the windows Terminal repo includes code that ships with windows and is not the windows Terminal application itself. cmd.exe lives in that repo, for example, along with some of the structure which supports all command-line applications on windows.
they can't adopt GPL in that repo and Microsoft has a very strong policy about segregation between developers who can read and write GPL code and those who cannot. those who cannot interact with GPL code get to work on stuff that ships with Windows, like the stuff in the Windows Terminal repo.
so, no. they can't look at GPL code, including Casey's refterm code. Casey knew all of this when he chose the GPL for that project.
they can't adopt GPL in that repo and Microsoft has a very strong policy about segregation between developers who can read and write GPL code and those who cannot. those who cannot interact with GPL code get to work on stuff that ships with Windows, like the stuff in the Windows Terminal repo.
so, no. they can't look at GPL code, including Casey's refterm code. Casey knew all of this when he chose the GPL for that project.
What the repo contains should not be very relevant. Proprietary and GPLed software can be in the same repo/disk/etc. We even see this in the official git repo with non-free firmware. Instead the concern is about what code the terminal application ends up using and what it links with. If his terminal did not link to any weird stuff then it is probably the case that they could avoid it as well. Also don't forget that the GPL contains an exception for system libraries specifically.
As for the segregation policy, it sounds like a self-imposed problem.
As for the segregation policy, it sounds like a self-imposed problem.
This argument is pointless. Microsoft doesn't need a YouTube video to teach its engineers how to do their jobs. They have tons of people in house who can code circles around any famous open source engineer, and who'd be happy to explain how to make cmd.exe great again internally
Like others have said, there are organizational issues causing the company's output to be embarrassing trash. Personally, I blame the Ballmer years of "unregretted attrition", but that's just my pet theory as to how it became a dysfunctional bureaucratic treacherous pirate's den
Like others have said, there are organizational issues causing the company's output to be embarrassing trash. Personally, I blame the Ballmer years of "unregretted attrition", but that's just my pet theory as to how it became a dysfunctional bureaucratic treacherous pirate's den
> project layout is familiar for everyone working on it
Surely you understand why something like this may be genuinely beneficial for application software even when it's not needed for games?
Surely you understand why something like this may be genuinely beneficial for application software even when it's not needed for games?
not when it adds tens or hundreds of thousands of lines of code to a project, and makes understanding everything more difficult, because every single action the code takes is indirect and abstracted thrice.
it's stupid.
it's stupid.
But that's not making the project layout more familiar for maintainers.
You are moving the goalposts.
You are moving the goalposts.
have you seen a large project grow ? it's all about organization so no one gets lost. none of it is needed if you have skilled developers, but skilled developers are expensive, so outside of silicon valley, companies hire shit developers and hope that they become semi-productive before their 5th anniversary.
the result is very large projects because no one knows how to write code, and tons of unnecessary abstraction and boilerplate stuck onto EVERYTHING.
just write what you need to solve the problem you need to solve, then stop.
the result is very large projects because no one knows how to write code, and tons of unnecessary abstraction and boilerplate stuck onto EVERYTHING.
just write what you need to solve the problem you need to solve, then stop.
They can totally look at it, but if they copy it into Windows Terminal, they need to relicense Windows Terminal under the GPL. That would be a great outcome for everyone. Microsoft is already shipping Linux under the GPL2 as WSL2 so this shouldn't be unthinkable.
Also, though, it does not require 25+ years of game development experience to figure out that you can write a non-laggy terminal emulator, which should be obvious because people have been building non-laggy terminal emulators since at least 01974 and people routinely watch 1080p videos at 60 fps on modern laptops and even modern hand computers.
We had terminal emulators with acceptable, even snappy, performance on X-Windows when I started using it in 01993. In 01994 I would commonly use an IRIS Indigo, which had a 150 MHz R4400, which was not superscalar, so it was maybe capable of 150 MIPS, or 300 MIPS if you count floating-point too. I preferred using it instead of the X terminals because it was faster. I'm typing this on an obsolete laptop whose CPU averages about 100 times faster than that. It's immediately obvious to the most casual observer that there is no excuse for software on this laptop to flub UI tasks the Indigo handled with ease.
The VT50 https://en.wikipedia.org/wiki/VT50 was released in 01974, and its microprogram emulated a Teletype with added features. (See my comment at https://news.ycombinator.com/item?id=32547666.) A service manual for the 01975 version, the VT52, is at https://news.ycombinator.com/item?id=32547666. It had 60 frames per second (in the US), 240 scan lines per frame, 80 characters per scan line (at 15.36 kHz), and I think 8 pixels per character (in each scan line), one of which was always blank. So its output shift register ("VSR") operated at 9.8 MHz, or actually a little bit faster because of the 10-μs HBI, during which it checked the keyboard. Its microprogram, which had to load the appropriate 7 bits into the VSR 80 times per scan line (as well as processing escape sequences and printable characters during, I think, the VBI), had a 1.3 μs instruction cycle time (770 MHz), divided into 18 72-ns clock cycles (13.824 MHz) during which it did different things.
So I think that in most 72-ns clock cycles about a couple of dozen flip-flops would change state.
This laptop typically executes about 2400 CPU instructions in every 72-ns tick of the VT52's clock, 600 per core. You could compile an RTL-level logic simulation of the VT52 into C and compile it with a C compiler, and it would run faster than the original VT52.
But that would be sort of stupid because you have a framebuffer and a GPU with texture-mapping units; your CPU doesn't actually have to generate pixels one at a time to keep the screen refreshed because you have hardware to do that for you. A 1920×1080 video at 60 frames per second is displaying 124 million new pixels per second, which it has to get by decoding H.264 or H.265 or something. That's a bit over 1.5 million VT52 characters per second. But when you're displaying characters on a terminal that isn't very useful because people can only read about 30 characters per second; even updating a full-page 80×66 display at 60 frames per second is only 0.3 million characters per second.
In most cases the computation required to process a character is to verify that it's a printable plain ASCII character, update the screen image, and update the cursor position. Updating the screen image might involve blitting some texture data or, as in Muratori's refterm, updating a glyph index that gets consulted by a shader to figure out what texture data to draw when the time comes to render a frame. We're talking about an amount of computation that's small compared to a typical C subroutine call and return.
Also, though, it does not require 25+ years of game development experience to figure out that you can write a non-laggy terminal emulator, which should be obvious because people have been building non-laggy terminal emulators since at least 01974 and people routinely watch 1080p videos at 60 fps on modern laptops and even modern hand computers.
We had terminal emulators with acceptable, even snappy, performance on X-Windows when I started using it in 01993. In 01994 I would commonly use an IRIS Indigo, which had a 150 MHz R4400, which was not superscalar, so it was maybe capable of 150 MIPS, or 300 MIPS if you count floating-point too. I preferred using it instead of the X terminals because it was faster. I'm typing this on an obsolete laptop whose CPU averages about 100 times faster than that. It's immediately obvious to the most casual observer that there is no excuse for software on this laptop to flub UI tasks the Indigo handled with ease.
The VT50 https://en.wikipedia.org/wiki/VT50 was released in 01974, and its microprogram emulated a Teletype with added features. (See my comment at https://news.ycombinator.com/item?id=32547666.) A service manual for the 01975 version, the VT52, is at https://news.ycombinator.com/item?id=32547666. It had 60 frames per second (in the US), 240 scan lines per frame, 80 characters per scan line (at 15.36 kHz), and I think 8 pixels per character (in each scan line), one of which was always blank. So its output shift register ("VSR") operated at 9.8 MHz, or actually a little bit faster because of the 10-μs HBI, during which it checked the keyboard. Its microprogram, which had to load the appropriate 7 bits into the VSR 80 times per scan line (as well as processing escape sequences and printable characters during, I think, the VBI), had a 1.3 μs instruction cycle time (770 MHz), divided into 18 72-ns clock cycles (13.824 MHz) during which it did different things.
So I think that in most 72-ns clock cycles about a couple of dozen flip-flops would change state.
This laptop typically executes about 2400 CPU instructions in every 72-ns tick of the VT52's clock, 600 per core. You could compile an RTL-level logic simulation of the VT52 into C and compile it with a C compiler, and it would run faster than the original VT52.
But that would be sort of stupid because you have a framebuffer and a GPU with texture-mapping units; your CPU doesn't actually have to generate pixels one at a time to keep the screen refreshed because you have hardware to do that for you. A 1920×1080 video at 60 frames per second is displaying 124 million new pixels per second, which it has to get by decoding H.264 or H.265 or something. That's a bit over 1.5 million VT52 characters per second. But when you're displaying characters on a terminal that isn't very useful because people can only read about 30 characters per second; even updating a full-page 80×66 display at 60 frames per second is only 0.3 million characters per second.
In most cases the computation required to process a character is to verify that it's a printable plain ASCII character, update the screen image, and update the cursor position. Updating the screen image might involve blitting some texture data or, as in Muratori's refterm, updating a glyph index that gets consulted by a shader to figure out what texture data to draw when the time comes to render a frame. We're talking about an amount of computation that's small compared to a typical C subroutine call and return.
> They can totally look at it, but if they copy it into Windows Terminal, they need to relicense Windows Terminal under the GPL. That would be a great outcome for everyone.
no! it would not! the GPL is a truly awful license. "freedom" my white butt. it's a list of requirements, all of which restrict the freedoms and abilities of those who utilize it.
about rendering:
I'm not trying to say that it's hard. I'm trying to say that they believed they knew how to do it, when in reality they did not.
THAT'S ALL.
Everyone is laser focused on the details of what I am saying and completely missing the point of what I am saying, or trying to say. I am apparently speaking a language that is close to English but not actually English.
no! it would not! the GPL is a truly awful license. "freedom" my white butt. it's a list of requirements, all of which restrict the freedoms and abilities of those who utilize it.
about rendering:
I'm not trying to say that it's hard. I'm trying to say that they believed they knew how to do it, when in reality they did not.
THAT'S ALL.
Everyone is laser focused on the details of what I am saying and completely missing the point of what I am saying, or trying to say. I am apparently speaking a language that is close to English but not actually English.
>no! it would not! the GPL is a truly awful license. "freedom" my white butt. it's a list of requirements, all of which restrict the freedoms and abilities of those who utilize it.
The only freedom the GPL restricts is the freedom to maintain a captive audience in a state of information asymmetry guaranteed helplessness.
You're not making a good case for yourself here.
The only freedom the GPL restricts is the freedom to maintain a captive audience in a state of information asymmetry guaranteed helplessness.
You're not making a good case for yourself here.
GPL forces code others write for your project to also be GPL. that is a loss of freedom for that contributor.
if I license something with the MIT license, anyone can use it for any purpose, and they can take my code with their modifications private if they want, meaning those entities are free to do what they want, and the code I released is still freely available just like it was before. I lose nothing, and no freedoms were taken from anyone who uses my code.
if I license something with the MIT license, anyone can use it for any purpose, and they can take my code with their modifications private if they want, meaning those entities are free to do what they want, and the code I released is still freely available just like it was before. I lose nothing, and no freedoms were taken from anyone who uses my code.
Windows Terminal isn't licensed under any of the various MIT licenses, though; it's licensed under the Windows EULA. The Windows EULA forces code others write for Windows Terminal to never exist in the first place, and it forces the people on the Windows Terminal team to lose access to their own code if they leave Microsoft. So moving Windows Terminal to the GPL2 would be a gain of freedom for all those people. Only Satya Nadella would suffer a loss of freedom.
That sort of simple numerical counting isn't enough for an ethical argument, of course. If my house is converted to a public park, then everyone gains the freedom to drink beer in it, or at least everyone in Buenos Aires does, and only I lose any freedom, like the freedom to sleep in a dry bed when it's raining without raucous beer-drinkers guffawing all around me. But this is still a net loss because the benefits to everyone else are not as important as the loss to me.
In the Windows Terminal case, the situation is the other way around: Nadella would suffer a loss of freedom that affects him only in a de minimis way, while everyone else would gain substantially.
That sort of simple numerical counting isn't enough for an ethical argument, of course. If my house is converted to a public park, then everyone gains the freedom to drink beer in it, or at least everyone in Buenos Aires does, and only I lose any freedom, like the freedom to sleep in a dry bed when it's raining without raucous beer-drinkers guffawing all around me. But this is still a net loss because the benefits to everyone else are not as important as the loss to me.
In the Windows Terminal case, the situation is the other way around: Nadella would suffer a loss of freedom that affects him only in a de minimis way, while everyone else would gain substantially.
> Everyone is laser focused on the details of what I am saying and completely missing the point of what I am saying, or trying to say. I am apparently speaking a language that is close to English but not actually English.
Have you considered that people are disagreeing with you because you are wrong rather than because they don't understand you?
Have you considered that people are disagreeing with you because you are wrong rather than because they don't understand you?
Well, it's true that I spent a lot of words attacking the claim that making a terminal emulator snappy (in the usual case, maybe not Zalgo cases) would be difficult, and it is also true that naikrovek didn't actually make that claim. Rather, they claimed that the Windows Terminal team didn't know how to do it. Which, I mean, maybe that's true, but it isn't that hard to figure out. I think it's more accurate that they didn't bother and made up rationalizations for their shitty software after the fact.
Oh, well, I agree that the Windows Terminal team overestimated their own competence. I just think they had to argue some pretty obviously implausible positions in order to do so. You have to dumb pretty hard to start believing that a 3 GHz 2-core superscalar i7 with AVX256, 24 MB of L3 cache, and a SIMT GPU can't handle the load of emulating a VT100.
And having the Windows Terminal licensed under the GPL instead of a Microsoft EULA would strictly increase the freedoms and abilities of its users: they could still legally do all the things with Windows Terminal they do now, plus lots of new things as well. Opt out of undesired updates, for example. Even the Windows Terminal team would have more freedom that way: they could go work at a different employer and keep hacking on the code they wrote as part of Windows Terminal.
And having the Windows Terminal licensed under the GPL instead of a Microsoft EULA would strictly increase the freedoms and abilities of its users: they could still legally do all the things with Windows Terminal they do now, plus lots of new things as well. Opt out of undesired updates, for example. Even the Windows Terminal team would have more freedom that way: they could go work at a different employer and keep hacking on the code they wrote as part of Windows Terminal.
[deleted]
I've had the misfortune of watching Windows 11's Notepad crash on opening a specific text file I was taking notes in.
in fairness the old XP/Server 2003 notepad would frequently freak out on me and suddenly remove all <CR> (or <LF>, I don't really know) from a file I was editing (not a unix or mac generated file, so it was not a <CR><LF> issue.) That would force me to quit and reload the file and start my edits over again. This was almost always on servers, so I never bothered to find a fix or workaround, but it made me never trust notepad. So old notepad sucked, and apparently new notepad sucks.
Windows 10 notepad was good for a brief glorious period, they fixed mixing status bar with line wrap, they made files with LF only render as multiple lines (don't know what happens when you save them). I have no plans to move off Ameliorated 10 to Windows 11 with their botched Notepad, slow store apps, and reduced ability to install explorer and taskbar customizations (though 7 already required patching out signature checks, and 8 through 10 progressively broke Aero Glass and widget controls patching).
That was just the malware owning your system
To be fair, this HP consumer laptops did come with a lot of preinstalled bloatware malware, some which even lacked uninstallers. But IIRC the Notepad crash persisted after a so-called clean Windows reinstall from Settings, which removed most bloatware apps but not advertising shortcuts in the Start menu and etc.
i had the same experience after they revamped the Windows Calculator. it went from instantaneous to ~2s startup with UI jank.
(highly recommend Notepad3)
i don't miss the ceremony of trying to debloat Windows after every unstoppable auto-update; what a goddamn nightmare that was.
now permanently on EndeavourOS KDE/Plasma and everything "just works, very fast". i do wish that Affinity products where available for Linux, i dont think you can run them with Proton/Wine :(
(highly recommend Notepad3)
i don't miss the ceremony of trying to debloat Windows after every unstoppable auto-update; what a goddamn nightmare that was.
now permanently on EndeavourOS KDE/Plasma and everything "just works, very fast". i do wish that Affinity products where available for Linux, i dont think you can run them with Proton/Wine :(
I wrote a rant about the Windows Calculator change as well. They're loading things like Windows 10 Hello for Business login recovery assistant modules, NVIDIA telemetry, creating threadpools, writing multiple log files, and on and on.
If you asked me what a calculator app "should" do, none of those would ever enter into my brain.
If you asked me what a calculator app "should" do, none of those would ever enter into my brain.
I didn't get a chance to use the new calculator yet (I usually use Linux. Windows 11 changes make it unlikely I'll 'upgrade' a VM). Is it this one?
https://github.com/Microsoft/calculator
https://github.com/Microsoft/calculator
Yes, that's the horrible telemetry-infested (yes, really!) UWP monstrosity that MS was proud to open-source instead of the good old classic one (whose source got leaked a long time ago...)
https://news.ycombinator.com/item?id=19321217
https://news.ycombinator.com/item?id=26579481
https://news.ycombinator.com/item?id=19321217
https://news.ycombinator.com/item?id=26579481
Someone once explained that MS hires a lot of short-term contractors who then don't own the consequences of the code they write. I wonder if this is still true and responsible for this.
Same thing in other faang companies, even with employees, who got the promotion and got hunting for another one. You won't get promoted for supporting an existing project.
Probably not. Contractors tend to be hired for a lot for non engineering tasks (admin, recruiting, facilities) and the sometimes to fill out a dev team with specific skill sets or to cover some grunt work. In my experience on dev teams (in the US at least) FTEs outnumber contractors by at least 8:1, so I’d be surprised if much was done on Windows Terminal by contractors. (I work for Microsoft but not on anything even adjacent to Windows Terminal).
Nothing annoys me more than having to use notepad when remoted into a server. It is the biggest pile of crap and having something even halfway competent included in a very expensive operating system would be a baseline I could accept.
> It is very visibly laggy for basic actions such as opening menus or scrolling through text.
Some (not all) of the lag in Windows is on purpose by the UI engineers. Google "disable windows animation" for a starting point on how to fix it.
Some (not all) of the lag in Windows is on purpose by the UI engineers. Google "disable windows animation" for a starting point on how to fix it.
>Casey Muratori excoriated the Windows Terminal dev team for failing to achieve a consistent 60 fps
Oh I remember that!
He even got lectured by some expert nitwits at MS which were making dozens of arguments about how it was allegedly impossible to create a fast terminal (on modern machines, most of them with GPUs in the TFLOPS range, LOL).
I have no idea why those "engineers" weren't fired on the spot.
Oh I remember that!
He even got lectured by some expert nitwits at MS which were making dozens of arguments about how it was allegedly impossible to create a fast terminal (on modern machines, most of them with GPUs in the TFLOPS range, LOL).
I have no idea why those "engineers" weren't fired on the spot.
My text editor (a version of MicroEmacs) is 55Kb in size, and opens instantly. Of course, it was designed to be performant under DOS.
It's not just the slowness, but the loss of basic functionality too.
Keyboard shortcuts, for example.
On my machine, Notepad still seems to have the traditional keyboard shortcuts. But in Paint they are completely missing.
Alt-F-A? No Save As dialog for you.
Close the app with unsaved changes? You get something like the traditional "do you want to save your work?" dialog, but the left and right arrow keys do not work in the dialog. And you can't type N or Alt+N for "Don't save".
These do work in the latest Notepad on my machine.
These keyboard shortcuts used to work in every Windows app, and especially in all the standard Microsoft apps. Now it seems to depend on whether the individual developer knows about them.
Keyboard shortcuts, for example.
On my machine, Notepad still seems to have the traditional keyboard shortcuts. But in Paint they are completely missing.
Alt-F-A? No Save As dialog for you.
Close the app with unsaved changes? You get something like the traditional "do you want to save your work?" dialog, but the left and right arrow keys do not work in the dialog. And you can't type N or Alt+N for "Don't save".
These do work in the latest Notepad on my machine.
These keyboard shortcuts used to work in every Windows app, and especially in all the standard Microsoft apps. Now it seems to depend on whether the individual developer knows about them.
Yeah, I'm using Windows Home at the house of the person I'm working for and It's remarkable how much worse it is than either my Linux laptop or Windows of ten years ago.
Media Player has apparently been replaced by "DVD Player(free)" which is not only intentionally crippled (you have to use the keyboard to access the DVD menu and if the Menu begin with no item having focus, you're SOL) but also crashes 75% of the time upon attempting to play the disk (I spent five starting the thing until I installed VLC).
The entire scheme is apparently to try to leverage the monopoly MS has on low-end systems.
Media Player has apparently been replaced by "DVD Player(free)" which is not only intentionally crippled (you have to use the keyboard to access the DVD menu and if the Menu begin with no item having focus, you're SOL) but also crashes 75% of the time upon attempting to play the disk (I spent five starting the thing until I installed VLC).
The entire scheme is apparently to try to leverage the monopoly MS has on low-end systems.
> force-updated my Notepad.exe to the new modern UI version.
> opening a 300 MB log file takes a solid minute of time
But notepad.exe is always slow when opening large files. If anything I think it's better now than it was in XP/7.
> opening a 300 MB log file takes a solid minute of time
But notepad.exe is always slow when opening large files. If anything I think it's better now than it was in XP/7.
If anything I think it's better now than it was in XP/7.
Unless you're using the same hardware, that's probably all hardware improvement.
Unless you're using the same hardware, that's probably all hardware improvement.
> You have to be spectacularly bad at programming to make something so basic so slow in 2022 on a very high-end gaming computer.
With all the telemetry going on in the background, Windows is worse every day. I tried Windows 11 and was amazed at how it completely nullified having a top-tier mobile i7 paired with 32GB RAM.
Take a look at how much a modern install of Windows is thrashing your drive. It's scanning everything, all the time.
With all the telemetry going on in the background, Windows is worse every day. I tried Windows 11 and was amazed at how it completely nullified having a top-tier mobile i7 paired with 32GB RAM.
Take a look at how much a modern install of Windows is thrashing your drive. It's scanning everything, all the time.
Casey Muratori is an extremely proficient programmer. Windows probably employs a couple of new grads to work on the terminal. (Even if they aren't, it'd be hard to come close to the breadth of experience that Casey has.) I never understood why this is a surprise at all - "Casey Muratori excoriated the Windows Terminal" basically translates to "L8 smarter than L2".
Everyone has somehow internalised the "fact" that all performance optimisation is a black art that only "hugely experienced" devs can possibly know.
I'll give you an example.
I've seen developers and DBAs working on databases simply create new indexes on the fly to fix some performance issue. It takes ten seconds, it's fairly safe, and it is relatively easy to gauge the effect. "Before minutes, after milliseconds". Done.
I've seen other places where merely suggesting the use of an index -- no matter how beneficial, trivial, or safe -- resulted in everyone foaming at the mouth in abject terror. Wailing and gnashing of teeth. "How dare you suggest that we touch the mystical black box!? Begone with you, foul wizard!"
This is what I saw with the Casey Muratori vs Windows Terminal team arguments.
Everyone acts like Casey pulled out some dark tome only the most grey haired wizards are allowed to see and spoke secrets that Microsoft isn't even allowed to legally hear.
Do you actually know what he suggested?
Make a grid, just a 2D array. Put the characters in them.
Use the characters to look up small images of individual characters in another grid.
If the other grid is missing a characters, find an empty spot an replace it with the missing character image. (If the second grid is bigger than the first one, this is always possible.)
That's... it. Just array lookups, 2D images, and a hashtable.
This is child's play not wizardry.
I'll give you an example.
I've seen developers and DBAs working on databases simply create new indexes on the fly to fix some performance issue. It takes ten seconds, it's fairly safe, and it is relatively easy to gauge the effect. "Before minutes, after milliseconds". Done.
I've seen other places where merely suggesting the use of an index -- no matter how beneficial, trivial, or safe -- resulted in everyone foaming at the mouth in abject terror. Wailing and gnashing of teeth. "How dare you suggest that we touch the mystical black box!? Begone with you, foul wizard!"
This is what I saw with the Casey Muratori vs Windows Terminal team arguments.
Everyone acts like Casey pulled out some dark tome only the most grey haired wizards are allowed to see and spoke secrets that Microsoft isn't even allowed to legally hear.
Do you actually know what he suggested?
Make a grid, just a 2D array. Put the characters in them.
Use the characters to look up small images of individual characters in another grid.
If the other grid is missing a characters, find an empty spot an replace it with the missing character image. (If the second grid is bigger than the first one, this is always possible.)
That's... it. Just array lookups, 2D images, and a hashtable.
This is child's play not wizardry.
> That's... it. Just array lookups, 2D images, and a hashtable.
No, that's not all that Casey did. I mean, perhaps from a very reductionist point of view it is, yes, but there's a thousand ways to implement "array lookups, 2D images and a hashtable" that are less efficient than the way that Casey did it (and probably most would be less efficient than the MS team as well...) Do you know the ins and outs of Direct2D in order to write all that out efficiently? I certainly don't.
No, that's not all that Casey did. I mean, perhaps from a very reductionist point of view it is, yes, but there's a thousand ways to implement "array lookups, 2D images and a hashtable" that are less efficient than the way that Casey did it (and probably most would be less efficient than the MS team as well...) Do you know the ins and outs of Direct2D in order to write all that out efficiently? I certainly don't.
Sigh...
Casey demonstrated... and it ought to be obvious... that the "ins-and-outs" of Direct2D don't matter.
If you actually bother to cache the glyphs into sprites, then Direct2D can be absurdly slow (it isn't) and still get something like 1000 fps.
Your comment is almost a parody of what went down.
How can you not see that generating a 16x20 pixel image can't possibly be the bottleneck on a modern PC unless you're doing something spectacularly wrong!?
Casey demonstrated... and it ought to be obvious... that the "ins-and-outs" of Direct2D don't matter.
If you actually bother to cache the glyphs into sprites, then Direct2D can be absurdly slow (it isn't) and still get something like 1000 fps.
Your comment is almost a parody of what went down.
How can you not see that generating a 16x20 pixel image can't possibly be the bottleneck on a modern PC unless you're doing something spectacularly wrong!?
> Casey demonstrated... and it ought to be obvious... that the "ins-and-outs" of Direct2D don't matter.
I don't know what this means. Casey clearly uses Direct2D in his repo. If he were to demonstrate that Direct2D doesn't matter, he shouldn't have used it.
> Your comment is almost a parody of what went down.
There are thousands of programmers out there who haven't built a terminal as fast as RefTerm, and then there's Casey Muratori. If a hundred more Refterms show up tomorrow, I'll happily change my mind. Heck, you can even prove me wrong by coding one up yourself, if you'd like!
I don't know what this means. Casey clearly uses Direct2D in his repo. If he were to demonstrate that Direct2D doesn't matter, he shouldn't have used it.
> Your comment is almost a parody of what went down.
There are thousands of programmers out there who haven't built a terminal as fast as RefTerm, and then there's Casey Muratori. If a hundred more Refterms show up tomorrow, I'll happily change my mind. Heck, you can even prove me wrong by coding one up yourself, if you'd like!
He artificially constrained refterm to use DirectText specifically, because that's what the Windows Terminal team was using, and Casey wanted to prove that this could be made performant. The specific argument the Microsoft guys were making was that DirectText/Direct2D had inherent limitations that fundamentally prevented certain scenarios running fast, such as a mix of many different colours. Casey proved them wrong, but to do that he had to use the same underlying libraries so as not to be seen as somehow "cheating".
Well said. I also like your assertion that performance optimization is believed to be a black art. As you say, it is not. In fact, there are much better tools available for performance monitoring today than there ever have been. How many programmers actually open those tools and look for performance bottlenecks? I would guess not too many, at least based on what we are seeing in shipping products.
Story time!
When I was something like 23 years old, my boss told me to go help optimise IIS performance at a large dev shop.
Think about how absurd it sounds: A junior tech with essentially no knowledge of IIS performance tuning doing consulting to an entire company of 30-40 year old experienced developers on the finer points of web serving.
I turned up, and it turned out that these guys were writing what seemed like a trivial web app, but if two people submitted a form simultaneously then the page would take a minute to submit.
A minute you ask? I timed it: 60 seconds exactly, plus or minus a few hundred milliseconds.
That's a lock timeout.
I asked to talk to the database team. They told me the scope of works was to look at IIS and "tune it". To put things in perspective, they were using the largest server I had ever seen (8 sockets!) and it couldn't handle two users.
"Maybe we need a 16-socket server, what do you thinkg?"
I told them... no-no-no-no, I need to talk to the DBA team, just trust me.
After some negotiation, I finally sat down next to the "senior" DBA and asked him what database engine he was using.
"Sybase."
Shit. I had heard of Sybase, but had never used it. I had a smidge of experience with SQL Server, and I had heard that Microsoft based it off the Sybase code that they had licensed.
In a mild panic I asked him if Sybase had any performance analyzer tool or somesuch?
"No."
Err... that's not good. Now in a cold sweat, I asked to borrow his mouse for a second so I could click through his Start menu, where I found the Sybase folder, with a "Profiler" tool that was a carbon copy of the "SQL Server Profiler", except for the icon and some color schemes.
It took about 10 minutes to repro the scenario and demonstrate that as expected, the codebase was Deadlock City and the mysterious 1 minute "performance issues" were in fact caused by 60.0 second timeouts as I had guessed.
When I walked away from that two storey building full of "professional" developers, I had a wide-eyed look on my face wondering why literally nobody in the entire database team was even aware of the existence of the database profiler tool... which was in their Start menu the whole time.
When I was something like 23 years old, my boss told me to go help optimise IIS performance at a large dev shop.
Think about how absurd it sounds: A junior tech with essentially no knowledge of IIS performance tuning doing consulting to an entire company of 30-40 year old experienced developers on the finer points of web serving.
I turned up, and it turned out that these guys were writing what seemed like a trivial web app, but if two people submitted a form simultaneously then the page would take a minute to submit.
A minute you ask? I timed it: 60 seconds exactly, plus or minus a few hundred milliseconds.
That's a lock timeout.
I asked to talk to the database team. They told me the scope of works was to look at IIS and "tune it". To put things in perspective, they were using the largest server I had ever seen (8 sockets!) and it couldn't handle two users.
"Maybe we need a 16-socket server, what do you thinkg?"
I told them... no-no-no-no, I need to talk to the DBA team, just trust me.
After some negotiation, I finally sat down next to the "senior" DBA and asked him what database engine he was using.
"Sybase."
Shit. I had heard of Sybase, but had never used it. I had a smidge of experience with SQL Server, and I had heard that Microsoft based it off the Sybase code that they had licensed.
In a mild panic I asked him if Sybase had any performance analyzer tool or somesuch?
"No."
Err... that's not good. Now in a cold sweat, I asked to borrow his mouse for a second so I could click through his Start menu, where I found the Sybase folder, with a "Profiler" tool that was a carbon copy of the "SQL Server Profiler", except for the icon and some color schemes.
It took about 10 minutes to repro the scenario and demonstrate that as expected, the codebase was Deadlock City and the mysterious 1 minute "performance issues" were in fact caused by 60.0 second timeouts as I had guessed.
When I walked away from that two storey building full of "professional" developers, I had a wide-eyed look on my face wondering why literally nobody in the entire database team was even aware of the existence of the database profiler tool... which was in their Start menu the whole time.
Really?
Many of us discovered around the age or 13 or 14 that POKE #B8000 plus a little arithmetic was significantly faster than using OS calls to write text to the screen. We were wrapping those in functions, inlining ASM, and building text-mode UIs as teenagers - not as long-time professionals, not college grads, and without all the information available on the modern internet.
Over the next few years, as graphical stuff became more common, we were trying different algorithms to optimize polygon fills, mouse drivers, custom fonts, etc. That was a part-time hobby for teenagers. Did not require greybeard gurus with lifetime achievement awards and college degrees and the entire internet of information always available.
It was just the basics, the fundamentals, a few decades ago.
Many of us discovered around the age or 13 or 14 that POKE #B8000 plus a little arithmetic was significantly faster than using OS calls to write text to the screen. We were wrapping those in functions, inlining ASM, and building text-mode UIs as teenagers - not as long-time professionals, not college grads, and without all the information available on the modern internet.
Over the next few years, as graphical stuff became more common, we were trying different algorithms to optimize polygon fills, mouse drivers, custom fonts, etc. That was a part-time hobby for teenagers. Did not require greybeard gurus with lifetime achievement awards and college degrees and the entire internet of information always available.
It was just the basics, the fundamentals, a few decades ago.
I'm of the opinion that all the telemetry is used to identify the remaining parts of Windows users can still rely upon and so they can be 'improved' by a PM looking to redirect an existing user base onto some new-crap.
In this way, users have nowhere to hide. Any refuge from the new UI changes will be tracked, identified and targeted for termination.
In this way, users have nowhere to hide. Any refuge from the new UI changes will be tracked, identified and targeted for termination.
> excoriated
look at that- with so many wonderful words to chose from, why do people fall back on 'decimated' ?
look at that- with so many wonderful words to chose from, why do people fall back on 'decimated' ?
To excoriate means to condemn. to decimate means to heavily damage. They are not synonyms.
The Roman punishment of “decimation” was to make each ten-man team in an army murder one of their number (the whole team being killed if they refused).
Practically “Saw”, in real life.
Practically “Saw”, in real life.
crap i thought it meant 'skin alive'
It apparently does have continued use as an approximate synonym to chafe or flay when used in the appropriate context:
https://www.merriam-webster.com/dictionary/excoriate
https://www.collinsdictionary.com/dictionary/english/excoria...
https://www.merriam-webster.com/dictionary/excoriate
https://www.collinsdictionary.com/dictionary/english/excoria...
That would be flaying. But in a medical context I think excoriating is skin removal... it's just not being used literally here.
Just install notepad++ and associate it with the files you want, it will open instantly and has 100x the capability of notepad, even if you don't need it.
I liked the modern Notepad UI when I tried it, but to be fair I didn't use it for long, having discovered a regedit fix to replace it with N++.
“electron application”, so many smart people working on optimizing javascript, simplicity wins again
Clever way to push users from Notepad into OneNote, ruin their previous experience.
why the fn heck does "notepad" even need an 'update'? An update to do what.
>c:\copy con FILE.txt
F6
>c:\copy con FILE.txt
F6
[deleted]
notepad dot electron
The new Notepad uses XAML, probably using Win UI 2 or 3.
VS Code, which is an Electron application, opens text files something like 20 times faster than the new Notepad.
Whoever wrote and shipped the new Notepad should feel ashamed at their failure as a professional software engineer.
This isn't some Github toy project. It's being forced onto billions of users, without the slightest bit of quality assurance or adherence to the basic practices of the art.
VS Code, which is an Electron application, opens text files something like 20 times faster than the new Notepad.
Whoever wrote and shipped the new Notepad should feel ashamed at their failure as a professional software engineer.
This isn't some Github toy project. It's being forced onto billions of users, without the slightest bit of quality assurance or adherence to the basic practices of the art.
> Whoever wrote and shipped the new Notepad should feel ashamed at their failure as a professional software engineer.
Steady on old chap. Who's to say the hapless developer isn't embarrassed about things themselves, but is prevented from doing the right thing by some management edict, ancient process etc.
My money's on that rather than one person's incompetence.
Steady on old chap. Who's to say the hapless developer isn't embarrassed about things themselves, but is prevented from doing the right thing by some management edict, ancient process etc.
My money's on that rather than one person's incompetence.
I would absolutely refuse to ship anything of this piss-poor quality. I would refuse to even participate if I wasn't allowed to achieve at least the existing tool's quality level.
In fact, I have done similar things.
For example, I was asked to develop a protocol-shim for an authentication system used by tens of thousands of people accessing one of the largest SAP systems in the country. I wrote the code, tested it, and said to the PM that the only thing missing for go-live is some HTTPS certificates to make it secure.
He asked me to go to production anyway to meet his deadline, and that the certificates will be fixed up later.
I straight up told him: "No."
The look on his face was hilarious. He called my boss, my boss' boss, and up.
I told all of them: "No."
We waited for the certificates to be issued.
In fact, I have done similar things.
For example, I was asked to develop a protocol-shim for an authentication system used by tens of thousands of people accessing one of the largest SAP systems in the country. I wrote the code, tested it, and said to the PM that the only thing missing for go-live is some HTTPS certificates to make it secure.
He asked me to go to production anyway to meet his deadline, and that the certificates will be fixed up later.
I straight up told him: "No."
The look on his face was hilarious. He called my boss, my boss' boss, and up.
I told all of them: "No."
We waited for the certificates to be issued.
That's a difference between developer and engineer. It's part of an engineer's job to not put out the garbage required by bad management, and instead signal to regulatory agents and media that management is attempting to say, put changes into a bridge that will cause it to call down, killing the first people to step on it
I’m ready to agree with you as long as you understand software engineering isn’t a thing.
Your fantasy of how engineering is supposed to work does not correspond to the reality of the term “software engineer”, at least in the US.
we're talking about the default text editor on an OS that supports endless alternatives. yes, it's an embarrassment, but (approximately) no one is dying because notepad is janky.
There can be late night disaster recovery scenarios where logging into an under-resourced virtual machine and using Notepad to view a huge log file is the difference between success and failure. Failure can mean huge companies going under, life & death, suicides, end of careers, you name it.
The new Notepad is not fit-for-purpose. You cannot use it to open a file several gigabytes in size. It'll just die, take minutes, fill memory, and even if it opens successfully, it'll be unusably slow.
It's the Windows equivalent of 'vi'. It has to work, all of the time.
The new Notepad is not fit-for-purpose. You cannot use it to open a file several gigabytes in size. It'll just die, take minutes, fill memory, and even if it opens successfully, it'll be unusably slow.
It's the Windows equivalent of 'vi'. It has to work, all of the time.
> No one is dying
I hate this argument so much
It's like being perfectly fine with a universe with as much suffering as long as no one is dying. I am exaggerating of course, but why inflict this suffering to a BILLION people? I feel bad when I annoy someone for a few seconds (like struggling to open a door) and yet people are perfectly nonchalant to create ecosystem that billions of people use everyday for the next decade at least.
I hate this argument so much
It's like being perfectly fine with a universe with as much suffering as long as no one is dying. I am exaggerating of course, but why inflict this suffering to a BILLION people? I feel bad when I annoy someone for a few seconds (like struggling to open a door) and yet people are perfectly nonchalant to create ecosystem that billions of people use everyday for the next decade at least.
>no one is dying
Are you sure?
---- "Larry started to explain about some of the places where he thought that he could improve things, but Steve wasn't interested. He continued, "You know, I've been thinking about it. How many people are going to be using the Macintosh? A million? No, more than that. In a few years, I bet five million people will be booting up their Macintoshes at least once a day."
"Well, let's say you can shave 10 seconds off of the boot time. Multiply that by five million users and thats 50 million seconds, every single day. Over a year, that's probably dozens of lifetimes. So if you make it boot ten seconds faster, you've saved a dozen lives. That's really worth it, don't you think?" --- https://www.folklore.org/StoryView.py?project=Macintosh&stor...
Are you sure?
---- "Larry started to explain about some of the places where he thought that he could improve things, but Steve wasn't interested. He continued, "You know, I've been thinking about it. How many people are going to be using the Macintosh? A million? No, more than that. In a few years, I bet five million people will be booting up their Macintoshes at least once a day."
"Well, let's say you can shave 10 seconds off of the boot time. Multiply that by five million users and thats 50 million seconds, every single day. Over a year, that's probably dozens of lifetimes. So if you make it boot ten seconds faster, you've saved a dozen lives. That's really worth it, don't you think?" --- https://www.folklore.org/StoryView.py?project=Macintosh&stor...
You put up the stink of your life or you quit. Anything less and you are a coward and shouldn't work in the industry.
In this hypothetical, the individual is choosing between possible homelessness and making Microsoft look bad by following Microsoft’s orders and releasing bad software.
And they should choose possible homelessness?
And they should choose possible homelessness?
"possible homelessness"? For someone who at least met the bar for getting hired by Microsoft? On the contrary, I think that individual would easily find a job somewhere else.
Who are these software engineer's working at Microsoft in the year 2022 at risk of homlessness?
In this hypothetical, the same person who is a “failure as a software engineer”, according to a parent commenter.
You mixed up two scenarios.
The original claim was that they were a failure.
The hypothetical is that they are not a failure, but are "prevented from doing the right thing". In the hypothetical, they should quit and get a new job.
The original claim was that they were a failure.
The hypothetical is that they are not a failure, but are "prevented from doing the right thing". In the hypothetical, they should quit and get a new job.
Cowards ship. Shippers get raises and promotions. Software quality declines.
> without the slightest bit of quality assurance or adherence to the basic practices of the art.
You really don't do your argument any favours with exaggerations like this.
You really don't do your argument any favours with exaggerations like this.
> It's being forced onto billions of users, without the slightest bit of quality assurance or adherence to the basic practices of the art.
whoa there, you don’t know that. you think you do because you’re getting a ton of upvotes and you’re leaning into the feeling you had when you wrote that comment — the anti-Microsoft comment on a site known for its anti-microsoft comments.
if you don’t work at Microsoft, you have very little hope of understanding the motivations of a team, a product manager, or even a single developer within Microsoft. there’s a lot more there than meets the eye, just like there is at any huge company.
whoa there, you don’t know that. you think you do because you’re getting a ton of upvotes and you’re leaning into the feeling you had when you wrote that comment — the anti-Microsoft comment on a site known for its anti-microsoft comments.
if you don’t work at Microsoft, you have very little hope of understanding the motivations of a team, a product manager, or even a single developer within Microsoft. there’s a lot more there than meets the eye, just like there is at any huge company.
Famously, Microsoft fired their entire QA team and now relies only on telemetry. They similarly fired their technical writing team, and now beg the public to write their documentation through pull-requests.
I do know what Microsoft is or isn't doing.
The evidence is in the pudding: opening a large text file in the new Notepad is orders of magnitude slower than any other editor on my system. This is a common failure mode that is trivially caught by testing. It's a well understood problem space, much like how databases "get slower" as the data volume increases, but the well understood solution is to use indexing to make this not occur.
The fact that the Notepad team spent months (years?) working on a trivial text editor and never tested it with text files speaks volumes.
If they had ever opened a large text file, they would have immediately noticed that a mistake had been made. Much like how if a database developer runs a query over large amounts of data, the missing indexes are immediately obvious.
It takes a minute for Notepad to open a file that isn't even "huge".
A minute isn't slow. It's broken.
They shipped broken software to the public, at the scale of hundreds of millions or even billions of users.
> understanding the motivations of a team, a product manager, or even a single developer within Microsoft.
Clearly, performance and quality are never motivations at Microsoft.
I do know what Microsoft is or isn't doing.
The evidence is in the pudding: opening a large text file in the new Notepad is orders of magnitude slower than any other editor on my system. This is a common failure mode that is trivially caught by testing. It's a well understood problem space, much like how databases "get slower" as the data volume increases, but the well understood solution is to use indexing to make this not occur.
The fact that the Notepad team spent months (years?) working on a trivial text editor and never tested it with text files speaks volumes.
If they had ever opened a large text file, they would have immediately noticed that a mistake had been made. Much like how if a database developer runs a query over large amounts of data, the missing indexes are immediately obvious.
It takes a minute for Notepad to open a file that isn't even "huge".
A minute isn't slow. It's broken.
They shipped broken software to the public, at the scale of hundreds of millions or even billions of users.
> understanding the motivations of a team, a product manager, or even a single developer within Microsoft.
Clearly, performance and quality are never motivations at Microsoft.
>if you don’t work at Microsoft, you have very little hope of understanding the motivations of a team, a product manager, or even a single developer within Microsoft
Their motivations are irrelevant for an end-user. Their results are.
Their motivations are irrelevant for an end-user. Their results are.
It's important to understand that Microsoft are staffed by people just like us, trying to make things better. Except Microsoft get shit on daily simply because they are Microsoft. That stuff adds up and gets to people who work on things.
If you don't think it should, I remind you that you can't take the humanity out of a human.
If you don't think it should, I remind you that you can't take the humanity out of a human.
That doesn't justify horrifically slow applications. It just seems like Microsoft isn't fit for purpose.
God, I'm glad I use linux.
> Windows force-updated my
If you don't like how your OS works, pick a different one.
If you don't like how your OS works, pick a different one.
There are also bash scripts to stop Windows from force-updating. Its a lot less work than switching to an entirely new OS and doesn't require giving up video games. But if someone can afford to learn Linux, or any OS that doesn't force updates, then its probably better to do so in the long run.
With Proton you can run a hell of a lot of your Steam library. It's far enough along that you don't need to be terminal guru any more.
I've heard of Proton a few times but last time I checked only 600 out of Steam's 50,000+ games are compatible with it. Not bad by any means but using an older Windows version still seems to offer more at the moment.
I have 400+ titles in my Steam library and play most of them on a Linux machine. Nowadays it's very rare to find one that does not work out of the box, and in those rare cases, there's always glorious eggroll to try.
Check by yourself on protondb.
Using an older Windows version to run Steam is a really bad idea. Steam requires an internet connection, old versions of Windows, are by definition, not up to date.
If the first 19,000 updates weren't secure, I don't have much confidence the next one will be. Is Linux any different in this regard?
Programmers too abstracted from the impact of what's going on to realize where real slow spots might exist.
I would be interested what percentage have ever opened it/used it?
I extremely rarely use the software I write as intended. I am just running it against tests.
I extremely rarely use the software I write as intended. I am just running it against tests.
So most of them. This sort of thing happens all the time everywhere. Unfortunately.
There are no incentives for bottom of the barrel programmers to write efficient and effective code, particularly when hardware in general have gotten so powerful and their own dev machines are probably overpowered for purpose.
The market doesn’t care. Making the program run in 1 second instead of 2 seconds takes the same effort as adding a new feature. The new feature empowers users more than saving a second, where they have the option of just buying a faster computer if they care.
Market doesn't have options. There either isn't alternatives or you can't refund anything with reason of it being slow...
There are always options. And always those looking to exploit untapped opportunities. And in the subscription age, you don’t refund, you just stop renewing.
There is no market for a photoshop competitor with half the features but runs faster for example. Because every tool is already tuned to be just fast enough that customers are satisfied.
There is no market for a photoshop competitor with half the features but runs faster for example. Because every tool is already tuned to be just fast enough that customers are satisfied.
Except this isn't a 1 second is good performance situation. Given the error the writer would need to buy a computer 11000x faster to make up for it.
Assuming scanning 44,000 files instead of 4 takes ~11000x longer.
Assuming scanning 44,000 files instead of 4 takes ~11000x longer.
What incentive exists for any programmers to care about speed unless it is a real constraint?
I’m not paid for a good user experience. My manager is happiest when features ship quickly.
I’m not paid for a good user experience. My manager is happiest when features ship quickly.
Can any devs at Microsoft explain/speculate how this happened? Are these bundled apps simply unmaintained?
Because it seems like this is a major and obvious flaw that is relatively easy to fix in a lot of different ways… yet it shipped.
It’s easy to just say that Microsft’s developers are incompetent/dumb/etc but this is low hanging fruit, and makes me think the problem is something else.
Because it seems like this is a major and obvious flaw that is relatively easy to fix in a lot of different ways… yet it shipped.
It’s easy to just say that Microsft’s developers are incompetent/dumb/etc but this is low hanging fruit, and makes me think the problem is something else.
I don't work at MS, but there's a few potential things going on:
Voice Recorder is pretty old, it got a big revamp/renaming ("Sound Recorder") in Windows 11. Problems might be fixed now?
The native development story on Windows is in shambles. UWP and WinRT have a ton of baggage but there isn't a great replacement for them yet. Many bundled apps were written in UWP when it seemed like it was the future, and now they're in an awkward not-quite-unmaintained place.
I see the trace is spending a lot of time in the Windows.Storage.* namespace and... there be dragons. Filesystem access from UWP apps is awful, it has several usability and performance shortcomings. Would not surprise me if that's a platform issue and not a Voice Recorder issue.
Voice Recorder is pretty old, it got a big revamp/renaming ("Sound Recorder") in Windows 11. Problems might be fixed now?
The native development story on Windows is in shambles. UWP and WinRT have a ton of baggage but there isn't a great replacement for them yet. Many bundled apps were written in UWP when it seemed like it was the future, and now they're in an awkward not-quite-unmaintained place.
I see the trace is spending a lot of time in the Windows.Storage.* namespace and... there be dragons. Filesystem access from UWP apps is awful, it has several usability and performance shortcomings. Would not surprise me if that's a platform issue and not a Voice Recorder issue.
There could be infinitely many reasons for this kind of slow software, but most cases many programmers don't care what happens under the hood as long as their code over abstraction layers appears to be working as intended in their own test scenario. I saw many websites developed by someone who doesn't really understand how query plans are generated and executed, how data is indexed, etc... and in the result they accidentally write the query that simply scans over the entire table. And thanks to modern hardware and database engine this scales to hundreds or even thousands of users!
Not a Microsoftie but it's probably going through various emulation layers.
What emulation layers are you speculating there are?
Windows-on-Windows/WoW64, depending on the age of the application in question. Has the voice recorder application been updated in recent versions of Windows (or whatever version the author was using)?
Unless we're talking running x86/x64 binaries on ARM or something, WOW64 is not an emulator, since x86_64 chips support IA32 instructions natively. There is some small overhead from filesystem and registry path translation, as well as switching the processor into/out of 32-bit mode.
There is, in fact, a new sound recorder app on W11: https://www.pcmag.com/news/first-look-windows-11s-new-sound-....
There is, in fact, a new sound recorder app on W11: https://www.pcmag.com/news/first-look-windows-11s-new-sound-....
I think this is an obligatory read for anyone who caused performance problems like this, especially with commonly used utilities (like the discussion of Notepad in these comments): https://www.folklore.org/StoryView.py?story=Saving_Lives.txt
Everyone talking about the algorithm being slow instead of wondering why an app needs to do this in the first place.
At the very least, this isn’t hugely useful, and if it was, why can’t it rely on a OS level file search cache to help it?
At the very least, this isn’t hugely useful, and if it was, why can’t it rely on a OS level file search cache to help it?
> why an app needs to do this in the first place
This is – besides all those programming related oddities – a question of holistic UX: if collecting the list takes that long, make the list on-demand, hide it behind a button or documents icon – and already your app is responsive for the majority of use cases and users.
This is – besides all those programming related oddities – a question of holistic UX: if collecting the list takes that long, make the list on-demand, hide it behind a button or documents icon – and already your app is responsive for the majority of use cases and users.
More importantly, why can’t you start recording before the file history has been populated? Those seem like two completely separate functions.
I'll take it back a bit further, as someone who used voice recorder back in windows 98:
Why does a voice recorder program need to list files?
That's what a file browser is for.
Why does a voice recorder program need to list files?
That's what a file browser is for.
Thought experiment: if the feature was implemented correctly, would you still prefer it not be in app?
It says in the article that it finds, and lists, previous recordings. I think it makes complete sense.
I guess I'm just getting old. I don't need a voice recorder to find and list previous recordings. I need it to record voices. Quickly. Inside one process.
I guess you are too file-oriented. If you were more task-oriented, it would be completely fine to use the voice recorder to find and listen the old voice recordings. Just like you would do if you had a physical device which can record and playback older records.
>Too file oriented
>Not task oriented enough?
Jesus, is this why modern devs are completely ignorant as to how to use filesystems?
Jesus, is this why modern devs are completely ignorant as to how to use filesystems?
[deleted]
What do you mean? We are talking about user oriented features, not devs
>Devs are all Users
>Not all Users are Devs.
>There is no intrinsic barrier standing between a User and becoming a Dev other than the patience to navigate the hell that is other people's code.
All users should understand files, in terms of how to locate them, what it means to open them, what it means to close them. They need to know how to navigate filesystems.
That's basic OS use. Kernel independent. OS independent. Hell, that's basic information retrieval system use.
Being "Task oriented" warranting blurring lines to the point you have "Sound Recorders" doing directory operations is a signal something has gone horribly, horribly wrong.
All users should understand files, in terms of how to locate them, what it means to open them, what it means to close them. They need to know how to navigate filesystems.
That's basic OS use. Kernel independent. OS independent. Hell, that's basic information retrieval system use.
Being "Task oriented" warranting blurring lines to the point you have "Sound Recorders" doing directory operations is a signal something has gone horribly, horribly wrong.
It is no doubt caused by the "app" model infecting everything, and justified as "security" to silo all data only within the app that created it.
"Task oriented" is an excuse for making closed proprietary solutions that don't interoperate.
"Task oriented" is an excuse for making closed proprietary solutions that don't interoperate.
Linux on an SSD is fast. Android is fast. I don't see much slowness. Windows has a bit here and there but in general things are pretty snappy, it's not like the tail end of the HDD era where stuff really was slow.
My Linux distro has never slowed down for me, except where I've personally done some weird config muckup. And I know exactly who is to blame... insert "of course I know him, he's me" meme here...
In other words, developer has no idea what the hell he’s making. He’s just implementing features, and doesn't think about anything or gives any damn.
It's the reason for Steve Job's tyranny back in the day imo.
It's the reason for Steve Job's tyranny back in the day imo.
For all the praise MS gets lately, i do think they have bloated windows and made them worse, which in turn drags everything down with it. Win7 was a great standard, quick, consistent, no bullshit. I have pretty much downloaded all the Win7 apps that i could find and use them instead of the new ones that come with win10. Photos? Better. Windows Movie maker is so much better than the new 'hidden' videos app it's not even funny. I use skype, and i wish they had stayed with their native version. Pretty much anything that uses the new UIs is consistently slow , it's like my computer is ignoring me. Sad
As much a schadenfreude opportunity as I want this to be, I must admit my Linux desktop takes so long to launch a snap that I have time to ps and see whether it crashed or is just slow. (It's always just slow.)
Reading this made me miss BeOS. Blazingly fast UI. Mandatory to use separate threads for app rendering and processing user input to stay responsive under load. File system was implemented as queryable database with arbitrary metadata, so scanning for media was almost instant. Built in components for text rendering, media playback, etc, all had extremely performant implementation.
I remember easily playing half a dozen video files simultaneously, butter smooth, with the system still being instantly responsive as if it was doing nothing. On a 200mhz 603e. Those were the good old days.
I remember easily playing half a dozen video files simultaneously, butter smooth, with the system still being instantly responsive as if it was doing nothing. On a 200mhz 603e. Those were the good old days.
I find it depressing that developers are so willing to blame themselves over a basic fact of life.
Let's categorize performance into 4 poorly-defined groups: perfectly optimized, highly optimized, reasonably optimized, poorly optimized
Everyone expects you to do the reasonable thing. I mean no disrespect, but finding examples of poor optimization does not justify elitist attitudes when most developers are just being reasonable otherwise. We all agree poorly optimized code is bad and never justified. I hope we all agree that anyone is capable of at least optimizing beyond what is reasonable. Anyone who has slaved over it knows this is not a skill to be proud of. We should all wish this aspect be automated (much of it has been). If a business case is ever made to optimize beyond what is considered reasonable, then you'll get to perform your hobby for pay. Does anyone seriously believe any products are perfect?
Bonus words: To address the complaints about web dev, get familiar with the waterfall diagram in the network tab of your browser's dev tools and behold where almost all the time goes. Stop embarrassing yourself complaining about javascript. What you will typically find is slow backend responses, poorly organized backend APIs that demand a long series chain of requests to get what the page needs, huge unoptimized images, and yes ads. Blaming slow web pages on anything but the organization as a whole that produced it is naive.
Let's categorize performance into 4 poorly-defined groups: perfectly optimized, highly optimized, reasonably optimized, poorly optimized
Everyone expects you to do the reasonable thing. I mean no disrespect, but finding examples of poor optimization does not justify elitist attitudes when most developers are just being reasonable otherwise. We all agree poorly optimized code is bad and never justified. I hope we all agree that anyone is capable of at least optimizing beyond what is reasonable. Anyone who has slaved over it knows this is not a skill to be proud of. We should all wish this aspect be automated (much of it has been). If a business case is ever made to optimize beyond what is considered reasonable, then you'll get to perform your hobby for pay. Does anyone seriously believe any products are perfect?
Bonus words: To address the complaints about web dev, get familiar with the waterfall diagram in the network tab of your browser's dev tools and behold where almost all the time goes. Stop embarrassing yourself complaining about javascript. What you will typically find is slow backend responses, poorly organized backend APIs that demand a long series chain of requests to get what the page needs, huge unoptimized images, and yes ads. Blaming slow web pages on anything but the organization as a whole that produced it is naive.
As a user of software, I don't care about how reasonable the optimization is, I care about how responsive the software is. Code that feels fast but is poorly optimized is fine. The problem is often that developers don't know how the software actually performs because they don't have proper analytics and don't do proper QA.
I almost agree.
Poor optimization is trivial to avoid even without "proper analytics" or "proper QA". Code won't merely feel fast if it's reasonably optimized. Taking shortcuts around slow code is harder work than just never letting it be slow in the first place. Your team's level of experience may vary, hence what "reasonable" means.
I'd also replace "developers don't know" with "project managers don't know", but if you work for a small business that distinction may not exist.
Poor optimization is trivial to avoid even without "proper analytics" or "proper QA". Code won't merely feel fast if it's reasonably optimized. Taking shortcuts around slow code is harder work than just never letting it be slow in the first place. Your team's level of experience may vary, hence what "reasonable" means.
I'd also replace "developers don't know" with "project managers don't know", but if you work for a small business that distinction may not exist.
Developers are often very aware of how the software performs but are refused time to work on it.
[deleted]
Developers are using over-powered machines to build programs. Slower and less capacity machines lead to better programming optimisation. Ask anyone that knows about game console programming.
Programmers can't tell when they're making slow programs because their computers make up for it.
Programmers can't tell when they're making slow programs because their computers make up for it.
Somewhat relevant: https://tonsky.me/blog/disenchantment/
Recording audio clips on the iPhone 11 I have is similar. Why the heck is this thing laggier from when I press "record" than a 386 running Windows 3.11?
I know some will take this badly as its questioning one of the core holy tennants of software engineering, but I do believe its in part down to an overly strong interpretation of the phrase "premature optimisation is the root of all evil".
About a year ago, I was (sidenote, still am) looking to get a new computer. In my free time, I do game dev, so I wanted something powerful enough to give me some flexibility in that, but also weak enough that Im not making my game on a machine 10x more powerful than most people will play it on. Of course you test on other machines as well, but I believe poor optimisation should be seen as a bug, not just something to improve on. I've explained it badly, but hopefully you get what I mean. Anyway, I opened a reddit thread about this on r/gamedev, and got swamped by the old "premature optimisation..." rule.
Its not the first, or last, time I've seen this happen. And of course, premature optimisation can ruin or stall a project, but refusing any and all possible optimisation steps prior to it being a problem seems really odd to me.
About a year ago, I was (sidenote, still am) looking to get a new computer. In my free time, I do game dev, so I wanted something powerful enough to give me some flexibility in that, but also weak enough that Im not making my game on a machine 10x more powerful than most people will play it on. Of course you test on other machines as well, but I believe poor optimisation should be seen as a bug, not just something to improve on. I've explained it badly, but hopefully you get what I mean. Anyway, I opened a reddit thread about this on r/gamedev, and got swamped by the old "premature optimisation..." rule.
Its not the first, or last, time I've seen this happen. And of course, premature optimisation can ruin or stall a project, but refusing any and all possible optimisation steps prior to it being a problem seems really odd to me.
Well that's just it - if you're running into slowness that's going to effect the core of what you're building optimizing it isn't "premature".
If you're fiddling with getting a rarely used subsystem to load in 50ms instead of 60ms that's premature.
If you're fiddling with getting a rarely used subsystem to load in 50ms instead of 60ms that's premature.
Would it make more sense for you to build a high-end PC, and artificially limit the performance when testing the game? Or keep some of your old components, to build a low-end testing machine?
Both of those things are absolutely possible, the problem is that it separates the dev and testing steps. Its difficult to explain why exactly this is a problem, so I'll give an example.
Imagine your developing a game on your high end PC, you run it and it runs at like 10 fps. Clearly something is wrong, thats a bug that needs to be fixed before you do anything else.
Now imagine it runs fine on your high end PC, but in testing, it runs at 10 fps.
The issues are the same, and should be treated the same, but in reality they rarely are. The testing issue would certainly be addressed, but not quite with the same urgency.
Imagine your developing a game on your high end PC, you run it and it runs at like 10 fps. Clearly something is wrong, thats a bug that needs to be fixed before you do anything else.
Now imagine it runs fine on your high end PC, but in testing, it runs at 10 fps.
The issues are the same, and should be treated the same, but in reality they rarely are. The testing issue would certainly be addressed, but not quite with the same urgency.
This is interesting, I have talked about this on my blog[1] where I have linked to Jonathan Blow's famous talk.
I guess that we stopped caring about performance in software a while ago...
[1]: https://apogliaghi.com/2022/09/software-midlife-crisis/
I guess that we stopped caring about performance in software a while ago...
[1]: https://apogliaghi.com/2022/09/software-midlife-crisis/
I wonder if the file scanning is hitting a variant of the N+1 problem, and if there's even a way to avoid that using the file system APIs. There's an old paper titled "New Need Not Be Slow"[1] that goes into the right and wrong ways to scan a filesystem for lots of files.
1 ftp://ftp.cs.toronto.edu/doc/programming/cnews.{ps,pdf}
1 ftp://ftp.cs.toronto.edu/doc/programming/cnews.{ps,pdf}
Minor correction: should be ftp://ftp.cs.toronto.edu/doc/programming/c-news.ps
There are loads of technical reasons, especially around how software is assembled today instead of created from "purpose-built programmning". But, ultimately the root cause is that programmers no longer create programs for users. They create programs (oftentimes just subsets of programs) for their managers. And, managers many times work towards MVP products instead of actually usable products for their end-users .
The incentive for creating a user's program has become an incentive to complete quarterly assessments.
The incentive for creating a user's program has become an incentive to complete quarterly assessments.
I remember waiting almost two minutes for Netscape Navigator to start.
I bet Netscape 2.0 would startup more or less instantly on modern hardware. Modern computers are so bonkers fast that everything should be instant on them. I mean, new CPUs can do 5 billion cycles per second, per core. To have an operation take 1 second, someone wrote software so slow that the computer took billions of steps to do something. How wild is that!
Whenever you have to wait for things on modern computers, you’ve essentially been robbed.
Whenever you have to wait for things on modern computers, you’ve essentially been robbed.
> 5 billion cycles per second
IF everything is in the cache, there is no dependency on I/O, and cycles aren't wasted with context switching between usermode->kernel.
Unless your program is purely computational it's going to burn cycles on system calls and I/O.
IF everything is in the cache, there is no dependency on I/O, and cycles aren't wasted with context switching between usermode->kernel.
Unless your program is purely computational it's going to burn cycles on system calls and I/O.
Your computer can do an awful lot of syscalls and IO in 1 second.
And almost all of the network traffic and syscalls can be mitigated almost all of the time. Eg by caching, buffered IO, cutting out bloat, minifying javascript, static rendering, using lower res assets, and so on.
There’s exceptions - AI, video games, rendering and simulations work can take real time.
But other than that, modern computers should boot instantly, and apps should open instantly. Webpages should open within about 3x the server round trip time. There is very little software I’ve ever worked with that, with some effort, couldn’t be massively optimized.
And almost all of the network traffic and syscalls can be mitigated almost all of the time. Eg by caching, buffered IO, cutting out bloat, minifying javascript, static rendering, using lower res assets, and so on.
There’s exceptions - AI, video games, rendering and simulations work can take real time.
But other than that, modern computers should boot instantly, and apps should open instantly. Webpages should open within about 3x the server round trip time. There is very little software I’ve ever worked with that, with some effort, couldn’t be massively optimized.
I was in middle school. We called it Nutscrape. It was abysmally slow, agreed.
But it wasn't a simple piece of software, either. This sound recorder is.
And I'll let an expert chime in here, but I think we're many orders of magnitude faster now, on the hardware side.
But it wasn't a simple piece of software, either. This sound recorder is.
And I'll let an expert chime in here, but I think we're many orders of magnitude faster now, on the hardware side.
Nothing to waiting for contemporary versions of Word to collect the fonts list…
Whether you're trying to avoid premature optimisation, or just don't care, software is always as slow as it can be. You only tune it when you notice a problem. The only way to make software reliably run 'fast' is to run it on old hardware, and even then that won't catch problems like this that don't appear until a special condition is met.
In this case I can believe the machines this program was tested on at MS had a lot less than 44'000 files. Should they have known to test with more files? I wouldn't have.
To ensure quality, Microsoft also need to make bug reporting easy and effective. But unfortunately that's a herculean task with so many users. I'm guessing most actual bug fixes are a result of big companies with support contracts complaining. And how many of those are going to know or care about the microphone app?
In this case I can believe the machines this program was tested on at MS had a lot less than 44'000 files. Should they have known to test with more files? I wouldn't have.
To ensure quality, Microsoft also need to make bug reporting easy and effective. But unfortunately that's a herculean task with so many users. I'm guessing most actual bug fixes are a result of big companies with support contracts complaining. And how many of those are going to know or care about the microphone app?
For big companies (like microsoft) software is ultimately controlled by the product teams. They prioritise the work that is to be done not the dev teams. Think of SCRUM for example the priority of the backlog is PO's responsibility not the devs.
Software is slow because it's not devs who make it all by themselves it's now a team and often team members are ambivalent or ignorant of performance. Requirements come first then perf later when often requirements and perf are the same.
Example you can't have a page that shows 100 million pieces of data (a requirement) and be fast. You have to pagination, lazy load ...etc. Those are requirements just as much as the fact you are showing 100 million bits of data.
Good teams have non-tech people that understand this. Bad teams (most teams tbh) don't and so insane slow shit gets put out to customers.
Software is slow because it's not devs who make it all by themselves it's now a team and often team members are ambivalent or ignorant of performance. Requirements come first then perf later when often requirements and perf are the same.
Example you can't have a page that shows 100 million pieces of data (a requirement) and be fast. You have to pagination, lazy load ...etc. Those are requirements just as much as the fact you are showing 100 million bits of data.
Good teams have non-tech people that understand this. Bad teams (most teams tbh) don't and so insane slow shit gets put out to customers.
The beauty of irony: slow software rant published on Wordpress :)
I find many apps on both Linux and Windows are slowed down by disk operations. You may say "get an SSD then". Well, not all your users will have an SSD, and if they already have one, then what? You could maybe offload some stuff into RAM, but many users have just 8GB, and you're not the only thing using it.
If you write a voice recorder for example, you _already_ know that the first thing you will do is scan the directories. Do this during start-up, let the user know that the system is initializing and then for the most part you only need to check your assertions are still true just before recording.
In general I think we need to make people care about performance more. The only thing I can think of is to make running on low-end hardware part of the validation process.
If you write a voice recorder for example, you _already_ know that the first thing you will do is scan the directories. Do this during start-up, let the user know that the system is initializing and then for the most part you only need to check your assertions are still true just before recording.
In general I think we need to make people care about performance more. The only thing I can think of is to make running on low-end hardware part of the validation process.
There is no money in making software fast, but there is a lot of money/promos in shipping garbage quickly. If it was the opposite, software shops would run their half baked garbage on slow VMs, and block releases until it runs fast enough, nobody at the shop would benefit from this.
Why is modern software slow? Because Joe developer's solution to a given problem is to Google up a package that solves the problem, check the license, and then run:
> { packageMgr } install { randomPkg }
> npm install node-plus
import { readFile } from 'node:fs/promise'
import { dirname, join } from 'node:path'
import additionChecker from 'add-check'
// redundant useless comments
return function(n1, n2) {
// redundant useless comments
n1 + n2
}In many cases this is the opposite of true. GTA online had a bug in their home-rolled json parser that caused it to read files in O(n*2), which went unfixed for years and added minutes to the loading time. That bug is much less likely to exist in a popular open-source package, because there are more eyes to catch it and fix it.
Yep. Game devs handroll lots and release back to public much less than other tech companies.
Not unheard of to even forego c++ stdlib and rolling their own structures.
In these cases it's not a matter of not understanding the low level internals.
In these cases it's not a matter of not understanding the low level internals.
That kind of thing won't make a program significantly slower other than a second or two to load, and won't make compiled programs slower at all.
No no you're so right. Let's have 500 packages installed to do multiplication. Zero performance impact, I'm sure.
You seem sarcastic, but you're saying something very true. The number of packages installed should have basically zero impact on performance. What matters is how much code the program runs, and the only extra code you added was import statements. That won't do much in many situations, and in compiled code it will have zero performance impact. The compile will take longer but you're not even the one that has to compile it.
Dylan is correct. The impact of the packages will be disk space and perhaps startup time, but that's it. The add function gets inlined in basically any runtime environment or compiler, even at O0 most likely.
It all adds up.
You can sum as many zeroes as you want, the result is still zero.
This is necessary in JS because the language itself is broken. You actually can't JUST add 2 "number"s because they might not actually be numbers. Writing raw JS is like writing raw assembly.
Forgive me for having chosen pseudo js for the example - and npm. That conflates the issue. I meant to convey an attitude - let some random package do the work, who cares whether it is deep and complex and handling cases I won't ever need, offering API surface that will go unused, and perhaps doing simple things inefficiently. The language doesn't much matter if "ship ship ship rn" is the ethic that drives decision making.
You guys check the license?
It's a long-time design flaw at Microsoft.
Long time ago, in Windows for Workgroups 3.11 times, when I opened Explorer, it was getting slower and slower. Finally, changing a directory took 5 secs. The hard disk was working hard (there were LED indicators!).
I've figured out that the root cause was my C:\Documents directory, which contained a bunch of files, maybe some hundreds of mp3s, don't remember. Moved _all_ files from C:\Documents made Explorer lightning fast.
I think, Explorer wanted to show a summary of different file types, which was completely unnecessary and couldn't turnded off.
Long time ago, in Windows for Workgroups 3.11 times, when I opened Explorer, it was getting slower and slower. Finally, changing a directory took 5 secs. The hard disk was working hard (there were LED indicators!).
I've figured out that the root cause was my C:\Documents directory, which contained a bunch of files, maybe some hundreds of mp3s, don't remember. Moved _all_ files from C:\Documents made Explorer lightning fast.
I think, Explorer wanted to show a summary of different file types, which was completely unnecessary and couldn't turnded off.
There was no Explorer in Windows 3.11, nor a Documents directory.
I think one problem is that these days you use so many frameworks and libraries that is hard to get a good mental model of how software performs and where bottlenecks are.
Another problem is that performance often given lower priority in enterprisey software development. For example, i'm currently working on a project where two componenens share data through a message bus. The architects decided that all messages on this bus have to use json with a bunch of headers. For my project, it could literally be 50x faster with very little effort if we could use a more efficient format, but it is not allowed.
Another problem is that performance often given lower priority in enterprisey software development. For example, i'm currently working on a project where two componenens share data through a message bus. The architects decided that all messages on this bus have to use json with a bunch of headers. For my project, it could literally be 50x faster with very little effort if we could use a more efficient format, but it is not allowed.
So, nobody in the team questioned - "why are we loading the list of recordings on the program startup instead of giving a button to the user so that they can decide when to see the list of recordings".
Or just do it asynchronously so it doesn't lock stuff up.
Too many computers and not enough scientists
Software is like water, it will always fill up the hardware that contains it.
> A twenty-second delay for recording a forty-second clip is a pretty bad efficiency ratio.
What a weird way of thinking, I wouldn't hire anyone who says something like this. 20 seconds to load up or do just about anything on a computer is bad, regardless of some round about reasoning of ratios. I find these days there is something wrong with most developers' brains. Like we need to start a new parallel software industry and make sure no unreasonable people can be part of it.
What a weird way of thinking, I wouldn't hire anyone who says something like this. 20 seconds to load up or do just about anything on a computer is bad, regardless of some round about reasoning of ratios. I find these days there is something wrong with most developers' brains. Like we need to start a new parallel software industry and make sure no unreasonable people can be part of it.
That's just bad dev, for a native app.
But anyone complaining that modern Electron/Web apps have bad performance is naive. Of _course_ they do, and yes we should try to improve performance, but much of the industry as a whole has decided that features > performance for the most part.
But anyone complaining that modern Electron/Web apps have bad performance is naive. Of _course_ they do, and yes we should try to improve performance, but much of the industry as a whole has decided that features > performance for the most part.
[deleted]
For those familiar with UWP, what does RuntimeBroker.exe do exactly?
It's part of the UWP sandboxing system ("AppContainer"). UWP apps chat with the Runtime Broker for potentially insecure operations like opening the common file open dialog or printing.
There's a bit more info about it on page 709 of Windows Internals 7th Edition (Part 1).
There's a bit more info about it on page 709 of Windows Internals 7th Edition (Part 1).
Modern software is so slow because objects, by definition, are post-paradigmatic. Every program is, in essence, an exercise in metaprogramming that can't be reliably optimized, built upon, or chained together with other objects.
And then you have hundreds of failed attempts to solve that problem which don't actually strike at the root problem, further making a mess of the computing environment so that it requires multiple gigabytes of resident memory to write and execute "hello world."
And then you have hundreds of failed attempts to solve that problem which don't actually strike at the root problem, further making a mess of the computing environment so that it requires multiple gigabytes of resident memory to write and execute "hello world."
In Spring you can just use the @Paradigmatic annotation to solve this.
the author used to ship latency sensitive software with -fPIC... great that they're advocating for this now, but major stalls were very common with their implementations.
This is not hard.
This particular app is slow because nobody cared. Not many people record voice on PCs anymore, and the ones they do, they use professional software. People just use phones for simple tasks.
Over time, someone had the idea to add search for files, they implemented the shortest hack that worked, and they left it in, because nobody prioritized this app.
Notes recording on phone is snappy and working, because people record on phones.
This particular app is slow because nobody cared. Not many people record voice on PCs anymore, and the ones they do, they use professional software. People just use phones for simple tasks.
Over time, someone had the idea to add search for files, they implemented the shortest hack that worked, and they left it in, because nobody prioritized this app.
Notes recording on phone is snappy and working, because people record on phones.
Modern software is slow because the tools which developers user are slow! I don't have time to hand-optimize my code, do you know how long it takes to run the full test suite?
It’s slow because nobody cares enough to make it fast.
Scanning for id3 tags at the end of files in case they’re mp3? Except doing it by scanning from the start rather than seeking from the end?
I’ve probably fallen into this trap. Several years ago I had to have offline voice recognition and I used some library I found online for the iPhone. It worked perfectly from a functionality perspective. But I have no idea how it worked except it was based on some CMU code. I simply didnt have the energy nor desire to audit its performance. It performed well enough.
Mobile apps must be available for use within a second or two at most every time they are run. If not, we just won’t use the app. On the other hand, mobile websites can commonly take 10 seconds to become readable.
So, why are apps slow? Because they don’t need to be fast unless users will not use them otherwise.
So, why are apps slow? Because they don’t need to be fast unless users will not use them otherwise.
Because it can is probably the reason (thanks to hardware).
Making better/faster/more reliable software is possible and I think this is an opportunity that will only reach its full potential at some point in the future, when hardware improvements will be slow enough for this effort to be competitive.
Making better/faster/more reliable software is possible and I think this is an opportunity that will only reach its full potential at some point in the future, when hardware improvements will be slow enough for this effort to be competitive.
It is slow because programming is like playing the guitar: anyone can do it, but not everyone SHOULD do it.
The modern software I write is very fast. It has to be. It is used to optimise pairing/rostering of very large airlines and rail companies. A single percentage of speed improvement has big impacts on competitiveness and customer perceived value.
The operation is likely tested on common systems (a few files) and anyone with thousands of files is in a scenario that wasn’t tested.
Second, the “antivirus law”: IO is considered free if it’s on a background thread. Never mind that it bogs down the computer.
Second, the “antivirus law”: IO is considered free if it’s on a background thread. Never mind that it bogs down the computer.
Sometimes opening a folder with Windows Explorer can take 15-30 seconds. Even when there are just a handful of files. It is very clearly scanning the contents of the files (so that it can sort them by date descending).
Windows allows a lot of middleware in Explorer. The shell extensions can eat performance like crazy if they get out of hand.
Linux file managers manage to be nice and snappy. On my machine, KDE's Dolphin, on an NVME, takes, maybe, 4 seconds to open and scan a folder that has more than 9000 folders and files.
What's Windows' problem? It has no excuse. Maybe, just maybe, the Windows Explorer, plus the stack it runs on, is utter garbage.
What's Windows' problem? It has no excuse. Maybe, just maybe, the Windows Explorer, plus the stack it runs on, is utter garbage.
My experience was opposite on directory with 30k files via SMB. Windows Explorer took within a second while Dolphin took 30 seconds. Nautilus took over a minutes iirc.
That’s not usually how file systems are organized; modified date and such are part of the metadata along with the filename so that you don’t have to scan the contents of the file just to list it. If it’s really that slow for a dozen files, using some kind of performance tool may be in order to figure out where the perf is going.
Actually it doesn't matter what the sort order is. This behavior exists.
> Blindly applying one explanation without a bit of investigation is the software equivalent of a cargo cult.
How is this a "cargo-cult"?
How is this a "cargo-cult"?
On an open sourced application, I could submit this as a bug in a bug tracker. Does Microsoft Connect still exist for everyone?
Is this not just a Microsoft problem?
Microsoft and bloatware is a synonym in the annuals of computer commentary.
Audacity on my computer also starts 20s
I read this and failed to find an answer to the rhetorical title. I got a lot of why "a particular operation on a particular operating system compounded to create a very suboptimal scenario." But there was no redux tl;dr that basically gave me a unified field theory about why modern software is slow. I agree that it often is, but this didn't further the fundamental praxis about why it often happens. Just ranted about a symptom. Did I miss a gem buried in there?
Instantly starts recording for me on Windows 11 so maybe this was reported and fixed long ago.
because its optimized to serve ads not to be useful
So sick of my laggy, jittery, POS gaming laptop.
The hardware is great. The software has officially shit itself.
Stuff like "guitar pri" lags it.
Just constant memory spikes and slow downs. PATHETIC.
The hardware is great. The software has officially shit itself.
Stuff like "guitar pri" lags it.
Just constant memory spikes and slow downs. PATHETIC.
[deleted]
This blog post seems like a Windows problem specifically...
That being said, in general, modern software is more bloated than ancient software because it does more...
That being said, in general, modern software is more bloated than ancient software because it does more...
I feel like "it does more" is often a cop out that's used without justifying whether the performance is actually commensurate to the additional features. If slowness if 20% extra features and 80% waste, "it does more" is just a deflection from the real issue.
Well, this blog post specifically seems like MS is doing 95% idiotic things.
But you look at something like Linux where obviously the Linux of 2022 is going to lag on 2002 hardware the answer is definitely that it's prettier and does more in general.
But you look at something like Linux where obviously the Linux of 2022 is going to lag on 2002 hardware the answer is definitely that it's prettier and does more in general.
I run linux on 2010 hardware, and it's fine; perfectly snappy.
Aka "software is slow because they didn't anticipate my use-case". This person has 44k files in a scanned directory. Someone at MS was careless in configuring the scan. This would have been caught if their test harness had more files on it.
FWIW I think software is slow these days because everything seems to want/need the network to function. Software is bloated for a different reason, namely the existence of a healthy, free, high quality sea of dependencies, from which we freely use.
FWIW I think software is slow these days because everything seems to want/need the network to function. Software is bloated for a different reason, namely the existence of a healthy, free, high quality sea of dependencies, from which we freely use.
For commercial airplanes it may be safety first, ticket price second (passenger capacity, fuel efficiency) and speed third. For most software, functionality, signing up for a subscription, platform availability etc are usually prioritized higher than response times and keyboard shortcuts.
Game devs worry a lot about latency and frame rates and professional software care a lot about keyboard shortcuts. This proves (anecdotally) that performance isn't unacheivable at all, but rather deprioritized. Nobody wants slow apps but it's just that developer velocity, metrics, ads etc etc are higher priorities, and that comes with a cpu and memory cost that the vendor doesn't care about.