Yeah, when you have tall servers this can be a really surprising factor. In some sense you could view this as an extension of processor caching behaviors, which also causes some memory accesses to be lower - just due to cache behaviors, not physical location. But in many cases, the same tools can be used to fight both "far" memory accesses and cache trashing, by using a thread-isolated architecture.
I have been dealing with the topic for a few years now and it was surprisingly hard to track down the bottlenecks to actual numbers. Some time ago I managed to find a good example to demonstrate the effect in a tangible way and wrote up an article about it. If the topic sounds interesting, you might enjoy https://sander.saares.eu/2025/03/31/structural-changes-for-4... (Structural changes for +48-89% throughput in a Rust web service).
The Revelation Space book series by Alastair Reynolds [spoilers ahead] ends with such a paperclip maximizer scenario. Someone accidentally turns on advanced sentient self-replicating terraforming robots without programming their target area. They promptly begin to terraform the galaxy and don't take kindly to interference.
This is a tiny shade of background in the story, not the main story, but read the books if you like this kind of tale.
I have been dealing with the topic for a few years now and it was surprisingly hard to track down the bottlenecks to actual numbers. Some time ago I managed to find a good example to demonstrate the effect in a tangible way and wrote up an article about it. If the topic sounds interesting, you might enjoy https://sander.saares.eu/2025/03/31/structural-changes-for-4... (Structural changes for +48-89% throughput in a Rust web service).