So what’s wrong with 1975 programming? (2006)(varnish-cache.org)
varnish-cache.org
So what’s wrong with 1975 programming? (2006)
http://varnish-cache.org/docs/trunk/phk/notes.html#so-what-s-wrong-with-1975-programming
111 comments
But the whole point of the article is that given the way modern hardware and kernels work, you don't actually have the ability to program against lower layers of abstraction, and attempting to do so can degrade performance.
I agree completely that you shouldn’t be digging into that stuff unless you have a reason to and possess the appropriate competence. But the reasons for doing that is the same everywhere: (1) complex systems are difficult and subtle, and (2) “premature optimization is the root of all evil”.
In fact, Knuth’s aphorism is a much better description of why one should avoid the complexity of this issue than a clearly incorrect argument proposing that the distinction between storage classes doesn’t exist, that 1975 programmers were ignorant of storage abstraction, and that this issue has become less complex over time. (Edit: corrected typo. Programmers in 1075 were definitely unfamiliar with storage abstractions.)
In fact, Knuth’s aphorism is a much better description of why one should avoid the complexity of this issue than a clearly incorrect argument proposing that the distinction between storage classes doesn’t exist, that 1975 programmers were ignorant of storage abstraction, and that this issue has become less complex over time. (Edit: corrected typo. Programmers in 1075 were definitely unfamiliar with storage abstractions.)
> I'd argue that it's actually more complex now. I programmed in the early 1980s, and did have to worry about where my data physically resided inside the machine, but I never had to worry about whether it was in a different room on a different machine, or sitting on a hard drive in some Amazon Data Center a continent away.
Oof, I hear that. That’s just the tip of the iceberg. We also design for those computers being deleted or disconnected spontaneously. Even more complex than designing distributed systems (in my opinion) is designing the continuous deployment systems which get our code from git to running in production (without running incompatible versions of services even for a moment). It certainly makes the idea of writing single-host software seem simple and romantic.
Oof, I hear that. That’s just the tip of the iceberg. We also design for those computers being deleted or disconnected spontaneously. Even more complex than designing distributed systems (in my opinion) is designing the continuous deployment systems which get our code from git to running in production (without running incompatible versions of services even for a moment). It certainly makes the idea of writing single-host software seem simple and romantic.
Some past threads:
What's wrong with 1975 programming? - https://news.ycombinator.com/item?id=13435988 - Jan 2017 (3 comments)
So what's wrong with 1975 programming? (2006) - https://news.ycombinator.com/item?id=9260169 - March 2015 (20 comments)
So what's wrong with 1975 programming? (2008) - https://news.ycombinator.com/item?id=4874304 - Dec 2012 (128 comments)
What's wrong with 1975 programming? - https://news.ycombinator.com/item?id=1760811 - Oct 2010 (12 comments)
What's wrong with 1975 programming - https://news.ycombinator.com/item?id=1554656 - July 2010 (115 comments)
What's wrong with 1975 programming? - https://news.ycombinator.com/item?id=13435988 - Jan 2017 (3 comments)
So what's wrong with 1975 programming? (2006) - https://news.ycombinator.com/item?id=9260169 - March 2015 (20 comments)
So what's wrong with 1975 programming? (2008) - https://news.ycombinator.com/item?id=4874304 - Dec 2012 (128 comments)
What's wrong with 1975 programming? - https://news.ycombinator.com/item?id=1760811 - Oct 2010 (12 comments)
What's wrong with 1975 programming - https://news.ycombinator.com/item?id=1554656 - July 2010 (115 comments)
The post should be marked as [2006]. A lot has changed since then. For example it doesn't talk about NUMA but it should, if it were written today.
I would love to hear some opinions from the authors of more contemporary web servers like one of H2O or Envoy. (The latter may have slightly different use cases though.)
I would love to hear some opinions from the authors of more contemporary web servers like one of H2O or Envoy. (The latter may have slightly different use cases though.)
Forgive my ignorance, but doesn't http://varnish-cache.org/docs/trunk/phk/notes.html#more-cach... talk about that, with an example?
It's about cache vs memory. NUMA is about local memory vs remote memory. So similar but different IMO.
What's different about NUMA today vs 2006?
I would say nothing, except maybe for how many mentions it gets in HW-review articles.
Unless you are in numerical HPC or kernel programming, you do not need to think about NUMA, but it is not black/white, it is a slope.
Varnish tries to do certain things "NUMA-friendly", without actually being "NUMA-aware" as such, and the kernels seem to do a really good job with that.
Unless you are in numerical HPC or kernel programming, you do not need to think about NUMA, but it is not black/white, it is a slope.
Varnish tries to do certain things "NUMA-friendly", without actually being "NUMA-aware" as such, and the kernels seem to do a really good job with that.
NUMA was not widely available / understood in 2006. I don’t recall seeing my first NUMA servers until probably 2008ish.
That makes sense to me for the entire computing landscape, like if we're including windows, various 32-bit x86 unix like OS's, etc. But varnish was used on Solaris, IRIX, and HP/UX. All of which had cc-NUMA support fairly well established by 2006. I suppose, though, it was still new-ish.
First Opterons were released in 2003
Even for 2006, what would make this argument for me is actual numbers. Is it plausible that some programs worked well like that then? Sure. But I can promise not everything did. The kernel's approach to caching is optimized for the general case, but each program is specific. It's plausible to me that a caching service like Varnish is a very close match. But that's an empirical question, not a theoretical one.
Well if by contemporary you mean written by Google builds with Bazel and runs in Docker, stuff like Envoy is pretty good and it uses Joyent's HTTP parser which is more readable than the pigeon C parsing Varnish likes to do. But Varnish understands UNIX much better than Envoy does, as evidenced by its portability and the things you've read in the blog post. Compared to Squid then hands down Varnish is the best. I'm not sure if it's better than open source GFE but it's pretty good.
None of them go as fast as redbean of course. The blog post talks about how they only need 18 system calls to serve a local asset. redbean only needs one system call: writev(). That's because redbean cheats by having a novel design. The zip central directory means we don't need file system system calls. The invariant RDTSC instruction means we can also have timestamps without context switching.
None of them go as fast as redbean of course. The blog post talks about how they only need 18 system calls to serve a local asset. redbean only needs one system call: writev(). That's because redbean cheats by having a novel design. The zip central directory means we don't need file system system calls. The invariant RDTSC instruction means we can also have timestamps without context switching.
Yes, 2006 would be a more proper tag.
NUMA is interesting, I've had a lot of fun with it on a system to control the adaptive mirrors in ESO's ELT telescope, and it is /amazing/ what you can do with a modern server, even on a vanilla kernel.
However, it is a lot harder to exploit NUMA in an application like Varnish, because the requests arrive as a randomized stream, and the effort to figure out which NUMA pool is best situated to handle the request, is not significantly different from just handling the request to begin with.
We used to serve a cache hit in seven system calls, an accept(2), a read(2), a write(2) and four timestamps, and as a result, Varnish servers ran with CPU's >70% idle.
If we had added NUMA-awareness on top of that, we would on average Nnuma-1/Nnuma of the time add at two system-calls to migrate the request to another NUMA domain, and that made little sense.
Since then NICs have become NUMA aware and since we use scatter/gather I/O, that matters a lot more than which CPU core did the overhead processing.
Where NUMA may become mandatory is 100G and 400G networking, but very few people seem interested in running that much traffic through a single server, for reasons of reliability, but I'm keeping an eye on it.
NUMA is interesting, I've had a lot of fun with it on a system to control the adaptive mirrors in ESO's ELT telescope, and it is /amazing/ what you can do with a modern server, even on a vanilla kernel.
However, it is a lot harder to exploit NUMA in an application like Varnish, because the requests arrive as a randomized stream, and the effort to figure out which NUMA pool is best situated to handle the request, is not significantly different from just handling the request to begin with.
We used to serve a cache hit in seven system calls, an accept(2), a read(2), a write(2) and four timestamps, and as a result, Varnish servers ran with CPU's >70% idle.
If we had added NUMA-awareness on top of that, we would on average Nnuma-1/Nnuma of the time add at two system-calls to migrate the request to another NUMA domain, and that made little sense.
Since then NICs have become NUMA aware and since we use scatter/gather I/O, that matters a lot more than which CPU core did the overhead processing.
Where NUMA may become mandatory is 100G and 400G networking, but very few people seem interested in running that much traffic through a single server, for reasons of reliability, but I'm keeping an eye on it.
Checkout TCP_REUSPORT_LB_NUMA on FreeBSD.
It will filter incoming TCP connections to listen sockets owned by threads bound to the same NUMA domain as the NIC that received the packets. This is part of the Netflix FreeBSD NUMA work described here https://papers.freebsd.org/2019/eurobsdcon/gallatin-numa_opt...
The idea is that you can keep connections local to a NUMA domain. We (Netflix) have a local patch to nginx to support this (basically just setting the socket ioctl on the listen socket after the nginx worker is bound)
It will filter incoming TCP connections to listen sockets owned by threads bound to the same NUMA domain as the NIC that received the packets. This is part of the Netflix FreeBSD NUMA work described here https://papers.freebsd.org/2019/eurobsdcon/gallatin-numa_opt...
The idea is that you can keep connections local to a NUMA domain. We (Netflix) have a local patch to nginx to support this (basically just setting the socket ioctl on the listen socket after the nginx worker is bound)
Yeah, so there are a lot of things we could optimize for on different kernels, but we prefer to not do so, until we have a really good reason.
As I said above, it is not obvious to me that we would gain much over what we already do, but I am keeping an eye on it.
As I said above, it is not obvious to me that we would gain much over what we already do, but I am keeping an eye on it.
Why did the timestamps require syscalls / do they still do now? (I know CPU counters were a mess back then, but was there any other factor about that?)
Back then they did.
But it is one of those things were I just chose to trust the OS to DTRT, and automatically reaped the benefits of advances "under the hood".
But it is one of those things were I just chose to trust the OS to DTRT, and automatically reaped the benefits of advances "under the hood".
Envoy is not a web server.
Why not? And what’s the definition of a „web server“ anyway?
In my opinion it’s about whether a software can process http requests on a configurable port - and envoy can for sure do that.
In my opinion it’s about whether a software can process http requests on a configurable port - and envoy can for sure do that.
It's a function from request to response.
A webserver, as bare minimum, should be able to serve static files from the disk - but you are correct, there is no such definition, so this is an opinion.
Regardless, calling envoy a webserver is misleading: it's both more and less, than a "traditional" webserver. It's a proxy; call it a proxy, please.
Regardless, calling envoy a webserver is misleading: it's both more and less, than a "traditional" webserver. It's a proxy; call it a proxy, please.
> A webserver, as bare minimum, should be able to serve static files from the disk.
You can configure routes with a `direct_response` handler from envoy.yaml that is read from disk ;)
You can configure routes with a `direct_response` handler from envoy.yaml that is read from disk ;)
As a historical note, I'll point out that 1975 computers weren't as primitive as the article implies. Virtual memory was introduced commercially in 1961 with the Burroughs B5000. Cache memory was introduced on the IBM System/360 Model 85 in 1969.
Absolutely, but did you ever try to use any of that ?
Back then /everything/ was a special case and everything needed to be programmed as such.
This is literally why JCL is a nightmare.
The genius of Ken & Dennis was to boil all the special cases down to one simple abstraction.
(Look at the socket API to if you want an example what happens when people dont understand the importance of sticking with the dominant abstraction.)
Back then /everything/ was a special case and everything needed to be programmed as such.
This is literally why JCL is a nightmare.
The genius of Ken & Dennis was to boil all the special cases down to one simple abstraction.
(Look at the socket API to if you want an example what happens when people dont understand the importance of sticking with the dominant abstraction.)
Any good examples of how the socket API could have been better if they did stick to the dominant abstraction?
On what design features came to market when, two examples:
(1) In 1972, I was at Georgetown U., and IBM was proposing that we buy a 370/135 with virtual memory.
(2) In 1973 there was the IBM 360/67 with 31 bit addressing, virtual memory, and virtual machine.The virtual machine software was CP/67 (Control Program 67), and the user interface was from CMS (Conversational Monitor System). I programmed it in PL/I to schedule the fleet at FedEx.
(1) In 1972, I was at Georgetown U., and IBM was proposing that we buy a 370/135 with virtual memory.
(2) In 1973 there was the IBM 360/67 with 31 bit addressing, virtual memory, and virtual machine.The virtual machine software was CP/67 (Control Program 67), and the user interface was from CMS (Conversational Monitor System). I programmed it in PL/I to schedule the fleet at FedEx.
RCA/Sperry Univac had reasonably large scale time sharing systems running basic, FORTRAN, apl, algol and 360-compatible assembly language. They had both physical and logical I/O APIs and supported vm to both disks and drums. Fill in the rest of the old time computer room as you’d like. Did a chess program and various other things in assembly, wrote the required Star Trek game in basic etc. As they provided the source code to the OS, even messed around with changing things like adding 2-step logins and other hacks.
Basically, that decade saw the continuation and growth of the hacker culture that started in the early 60’s on trivially small machines (google spacewar). As there were no small machines, things were more social / collaborative than now out of necessity. And yeah, the hardware is bazillion times faster but the software less so as code bloat is real.
Basically, that decade saw the continuation and growth of the hacker culture that started in the early 60’s on trivially small machines (google spacewar). As there were no small machines, things were more social / collaborative than now out of necessity. And yeah, the hardware is bazillion times faster but the software less so as code bloat is real.
Just from the title, I would have expected the opposite approach: not treating RAM as a disk cache, but using RAM only and ignoring the disk. One of the biggest differences between now and then is that we've gone from RAM deficit to RAM surplus. (The VMs that contain so much of today's software can be seen as a way of cutting too-big physical RAM back down to slices that match problems.)
This means that batch orientation makes sense for a much smaller slice of problems. Compilers, for example, come from a world of scarce RAM. When my dad started coding, one iteration of change-the-code-and-watch-it-run took days. Now that loop can be seconds: if I pause typing, my IDE starts running my unit tests automatically. It seems to me that things like compilers and runtimes should by default keep everything hot, incrementally updating as fast as possible.
This means that batch orientation makes sense for a much smaller slice of problems. Compilers, for example, come from a world of scarce RAM. When my dad started coding, one iteration of change-the-code-and-watch-it-run took days. Now that loop can be seconds: if I pause typing, my IDE starts running my unit tests automatically. It seems to me that things like compilers and runtimes should by default keep everything hot, incrementally updating as fast as possible.
> Now that loop can be seconds: if I pause typing, my IDE starts running my unit tests automatically. It seems to me that things like compilers and runtimes should by default keep everything hot, incrementally updating as fast as possible.
Is it only me that finds this incredibly distracting?
When I write code I pause often. It might be in the middle of a sentence, or a function definition, the most recent example I can think of is because I realised I needed another function, and this function's state is dependent on it, or I needed to look something up. Getting my mindstate trashed every time I pause because it's found a dozen irrelevant errors sounds horrifying.
Is it only me that finds this incredibly distracting?
When I write code I pause often. It might be in the middle of a sentence, or a function definition, the most recent example I can think of is because I realised I needed another function, and this function's state is dependent on it, or I needed to look something up. Getting my mindstate trashed every time I pause because it's found a dozen irrelevant errors sounds horrifying.
Agreed, I hate that. I don't want to hear about test failures, syntax errors, compile errors, or anything like that until I ask for it.
Because there's latency. It wouldn't be distracting if there's no latency at all because it feels natural even if the code is in stale shapes.
I use Wallaby[0] to run JavaScript tests, and it shows the result on the fly just near every line of test code. The latency is very small so it's hard to notice it.
After I was used to this approach the idea of press the test button, and waiting for 1s to compile, another 3s to run tests is simply tormenting. It breaks the flow because I have to repeatedly wait for the computer to catch my thought.
[0]: https://wallabyjs.com/
I use Wallaby[0] to run JavaScript tests, and it shows the result on the fly just near every line of test code. The latency is very small so it's hard to notice it.
After I was used to this approach the idea of press the test button, and waiting for 1s to compile, another 3s to run tests is simply tormenting. It breaks the flow because I have to repeatedly wait for the computer to catch my thought.
[0]: https://wallabyjs.com/
I think one could make the same argument for a zillion things a modern IDE does, from syntax highlighting forward. It's more of a problem in theory and on early use than it is for sustained use.
For unit tests in specific, if the interface is designed well and one is used to TDD anyhow, I think the distraction goes away pretty quickly. When I'm used to TDD, I always have a bit of my mind on what the test impact is, so if the errors showing are the ones I expected anyhow, then it's not disruptive at all. Indeed, it's welcome confirmation that I'm not off track.
That's how syntax highlighting and feedback feels for me too. I may pause at a point with unparseable code, so some things will be the wrong colors and may be marked with issues. It's fine, because that matches my mental syntax parser. And when it doesn't, I'm glad to learn as early as possible that I have an issue somewhere.
For unit tests in specific, if the interface is designed well and one is used to TDD anyhow, I think the distraction goes away pretty quickly. When I'm used to TDD, I always have a bit of my mind on what the test impact is, so if the errors showing are the ones I expected anyhow, then it's not disruptive at all. Indeed, it's welcome confirmation that I'm not off track.
That's how syntax highlighting and feedback feels for me too. I may pause at a point with unparseable code, so some things will be the wrong colors and may be marked with issues. It's fine, because that matches my mental syntax parser. And when it doesn't, I'm glad to learn as early as possible that I have an issue somewhere.
It's a matter of UX. If the errors are displayed in a subtle way it's not an issue.
We have come a long way since 2006 in terms of both storage and RAM. There are a lot of Varnish servers out there running straight out of RAM, and some of them have a /lot/ of RAM.
But the most important question is your data set and your traffic pattern, and whatever that is, Varnish can deal with it, given suitable hardware.
But the most important question is your data set and your traffic pattern, and whatever that is, Varnish can deal with it, given suitable hardware.
The viewpoint of phk, the squid disk cache is the only thing that matters and ram is simply there to accelerate the hottest data.
Multipass compilers were originally engineered that way because there simply wasn't available memory to keep all the intermediate data in memory.
Now the nature of compilers has gone from batch to incremental to handle the feedback the IDE needs to give to the programmer.
Multipass compilers were originally engineered that way because there simply wasn't available memory to keep all the intermediate data in memory.
Now the nature of compilers has gone from batch to incremental to handle the feedback the IDE needs to give to the programmer.
As a general rule, I would suggest people refrain from trying to express what my viewpoint is, because, quite frankly, N-1 person on this planet suck at it :-)
There still is a cost differential between 1TB RAM and 1TB disk, and that matters in the real world.
Varnish lets you decide where you want to spend your money, if you want to run straight out of RAM, it can do that, if you want to have a bigger cheaper and slightly slower cache on disk or ssd, you can do that too.
There still is a cost differential between 1TB RAM and 1TB disk, and that matters in the real world.
Varnish lets you decide where you want to spend your money, if you want to run straight out of RAM, it can do that, if you want to have a bigger cheaper and slightly slower cache on disk or ssd, you can do that too.
I didn't put enough caveats, I should have said, "the viewpoint that I think phk is trying to express _in this article_ ..." :)
You have written enough publicly that we could probably cook up a GPT2 level simulacrum with your posts. Useful for code reviews, discussions, etc.
Aren't we saying the same thing? The RAM is there to support the disk and the disk holds the data (the asset), so your article is disk centric. If disk was fast enough, there would be no ram. The ram is just a tool to support a latency target.
You have written enough publicly that we could probably cook up a GPT2 level simulacrum with your posts. Useful for code reviews, discussions, etc.
Aren't we saying the same thing? The RAM is there to support the disk and the disk holds the data (the asset), so your article is disk centric. If disk was fast enough, there would be no ram. The ram is just a tool to support a latency target.
Remember 2006 ? You know, back when Twitter was a new thing :-)
Back then you couldn't buy machines with 1TB of RAM, most systems were 32bit and limited to about 3GB usable RAM.
Varnish was written with 64bit in mind from the start, but it also had to work on 32 bit systems, so we had to be able to use disk-backed VM to get past a couple of GB.
Back then you couldn't buy machines with 1TB of RAM, most systems were 32bit and limited to about 3GB usable RAM.
Varnish was written with 64bit in mind from the start, but it also had to work on 32 bit systems, so we had to be able to use disk-backed VM to get past a couple of GB.
This post elicited a response from antirez back in the day. Then he and "the varnish guy" /g had a meeting of geek minds. You'll learn quite a bit from both.
http://oldblog.antirez.com/post/what-is-wrong-with-2006-prog...
http://oldblog.antirez.com/post/what-is-wrong-with-2006-prog...
This article is a bit strange and conflates disk storage with swap and RAM. Modern systems don't necessarily even have swap (looking at you, k8s), so the whole article falls on its face.
> And then there were the secondary store, paper tape, magnetic tape, disk drives the size of houses, then the size of washing machines and these days so small that girls get disappointed if think they got hold of something else than the MP3 player you had in your pocket.
This is also written terribly.
> And then there were the secondary store, paper tape, magnetic tape, disk drives the size of houses, then the size of washing machines and these days so small that girls get disappointed if think they got hold of something else than the MP3 player you had in your pocket.
This is also written terribly.
> This article is a bit strange and conflates disk storage with swap and RAM.
The author is pointing out that many programs duplicate OS virtual memory functionality by paging temporary data to persistent storage and loading it when needed. This duplicates the operating system's built in virtual memory capability and has negative effects on the system. The whole idea of virtual memory is to allow a system to handle loads where memory allocated exceeds the size of RAM.
> Modern systems don't necessarily even have swap (looking at you, k8s)
Modern linux does have swap, and it is quite useful. Proper support for swap is coming in k8s (looks like 1.23). Quite a few workloads need swap to run safely, so adding this to k8s will be an improvement.
> This is also written terribly.
Bad joke was bad.
The author is pointing out that many programs duplicate OS virtual memory functionality by paging temporary data to persistent storage and loading it when needed. This duplicates the operating system's built in virtual memory capability and has negative effects on the system. The whole idea of virtual memory is to allow a system to handle loads where memory allocated exceeds the size of RAM.
> Modern systems don't necessarily even have swap (looking at you, k8s)
Modern linux does have swap, and it is quite useful. Proper support for swap is coming in k8s (looks like 1.23). Quite a few workloads need swap to run safely, so adding this to k8s will be an improvement.
> This is also written terribly.
Bad joke was bad.
>> Modern systems don't necessarily even have swap (looking at you, k8s)
> Modern linux does have swap, and it is quite useful. Proper support for swap is coming in k8s (looks like 1.23). Quite a few workloads need swap to run safely, so adding this to k8s will be an improvement.
Ah, I see the HN pedants have arrived. I was not claiming that Modern Linux does not have swap. Note the words "don't necessarily". I was claiming not all machines have swap.
Most of mine do not have it enabled, and I take effort to ensure on machines with 16GB or more RAM that it is disabled. I do run a K8S cluster as well, and those machines do not have swap enabled, not because K8S requires it to be off, but because having it on would be adverse to the health of the flash storage they use.
TBH, this is the kind of vapid pedantic response I've come to expect of the average of HN comments these days, and why I'm considering just not using it anymore at this point.
> Modern linux does have swap, and it is quite useful. Proper support for swap is coming in k8s (looks like 1.23). Quite a few workloads need swap to run safely, so adding this to k8s will be an improvement.
Ah, I see the HN pedants have arrived. I was not claiming that Modern Linux does not have swap. Note the words "don't necessarily". I was claiming not all machines have swap.
Most of mine do not have it enabled, and I take effort to ensure on machines with 16GB or more RAM that it is disabled. I do run a K8S cluster as well, and those machines do not have swap enabled, not because K8S requires it to be off, but because having it on would be adverse to the health of the flash storage they use.
TBH, this is the kind of vapid pedantic response I've come to expect of the average of HN comments these days, and why I'm considering just not using it anymore at this point.
It would have been funny if I was still 15. But then again, this article is from 2006; The blogosphere was a different place in those days, and I've matured since then as well - hopefully the author has too. Still: Bad joke was bad.
slver(3)
This is probably the thing that annoys me the most about golang: It abstracts everything as if you were on a 1975 computer, so there's allocations, allocations everywhere! And trying to keep them under control requires intimate knowledge about go's implementation details. Failure to do so dooms your program to run 3-4x slower since allocations are horribly slow to begin with, and now your data is spread all over the place. And even then, there are still some things (like []byte <-> string) that you simply can't get around (unless you want to delve into unsafe land and risk your code blowing up once the runtime implementation changes).
To be fair to Go, it provides an escape analysis log and a memory profiler in the standard distribution, so while fixing the heap allocations can require good knowledge of implementation details, finding out that you have heap allocations is really easy.
Go is also not that clever, so you can often intuit fairly easily where the allocations are taking place and how to avoid them. As far as optimizing goes, it’s a lot easier to optimize away allocations in Go than in many other languages. I also think this is the right tradeoff—most code isn’t that sensitive to allocations so why make the programmer think about the 100% of the time, rather than opting into thinking about them in order to optimize the hot path?
I think this comes down to the right tool for the job.
If you have a lot of programmers churning out code, languages like golang makes it harder for them to make mistakes, sometimes at the cost of performance, but 99.9% of the time, performance does not matter.
If performance matters, there is no better language than C to bend the hardware to your precise will, but that comes at a cost of programming difficulty.
If you have a lot of programmers churning out code, languages like golang makes it harder for them to make mistakes, sometimes at the cost of performance, but 99.9% of the time, performance does not matter.
If performance matters, there is no better language than C to bend the hardware to your precise will, but that comes at a cost of programming difficulty.
>It abstracts everything as if you were on a 1975 computer, so there's allocations, allocations everywhere!
Care to elaborate? Generally you don't care about memory management in Go at all, so I wouldn't call it 1975 programming where you certainly would. Sure, taking the address of a variable heap-allocates but that's an implementation detail and I don't see how it's related to the article at all. It's automatic and you don't even think about it. The article talks about userland software reinventing what's already implemented in a modern operating system. How is it the case with heap allocations in Go, escape analysis etc.? Operating systems don't have means to do escape analysis for userland variables. Or did you get the conclusion from the article that slow=1975 programming? I don't think that's what the author meant.
Care to elaborate? Generally you don't care about memory management in Go at all, so I wouldn't call it 1975 programming where you certainly would. Sure, taking the address of a variable heap-allocates but that's an implementation detail and I don't see how it's related to the article at all. It's automatic and you don't even think about it. The article talks about userland software reinventing what's already implemented in a modern operating system. How is it the case with heap allocations in Go, escape analysis etc.? Operating systems don't have means to do escape analysis for userland variables. Or did you get the conclusion from the article that slow=1975 programming? I don't think that's what the author meant.
> Generally you don't care about memory management in Go at all
I think this is true for the majority of devs, just as it is for any managed language (C#, Java...).
But sometimes you are working on something performance sensitive, and there minimising allocations can be a real problem.
I think this is true for the majority of devs, just as it is for any managed language (C#, Java...).
But sometimes you are working on something performance sensitive, and there minimising allocations can be a real problem.
Abstractions do have costs, no doubt. I just don't see the connection between "Go heavily uses interfaces which force heap allocation in the current implementation" and "1975 programming". How would one go from 1975 to current year? By stopping using abstractions? Isn't that exactly what 1975 programming was to begin with?
Well, yes, I have to agree that I don't see the link between allocations, abstractions and 1975 ¯\_(ツ)_/¯
Varnish seemed really popular a decade ago, but I wonder how it fits the modern web and who's using it today and for what purpose.
The lack of built-in HTTPS seems killer to me. On the client-facing side it needs a separate daemon for TLS termination and on the upstream side there's no TLS support at all unless you're using the paid version.
The lack of built-in HTTPS seems killer to me. On the client-facing side it needs a separate daemon for TLS termination and on the upstream side there's no TLS support at all unless you're using the paid version.
Hi, Varnish Author here...
The reason why Varnish does not have built in TLS, is a matter of security design and flexibility.
First, as to flexibility:
If we added TLS support to Varnish, we would have to pick a TLS library, which means our users would automatically be locked into that library.
This would violate one of our core principles, which is "Tools, not policies".
And given the state of TLS implementations, I /really/ dont want to make that choice, I want to leave that to the person responsible for the security at the site running Varnish.
Second, I think it is sound security engineering to confine the server certificate in a process which does that one thing and does it well.
I know several sites that have two different TLS frontends in front of Varnish, using different TLS implementations, so that whenever a CVE hits one of them, they can just shut that half down until the issue is fixed, and still remain in production. That would not be possible if TLS was bolted into Varnish.
The reason why Varnish does not have built in TLS, is a matter of security design and flexibility.
First, as to flexibility:
If we added TLS support to Varnish, we would have to pick a TLS library, which means our users would automatically be locked into that library.
This would violate one of our core principles, which is "Tools, not policies".
And given the state of TLS implementations, I /really/ dont want to make that choice, I want to leave that to the person responsible for the security at the site running Varnish.
Second, I think it is sound security engineering to confine the server certificate in a process which does that one thing and does it well.
I know several sites that have two different TLS frontends in front of Varnish, using different TLS implementations, so that whenever a CVE hits one of them, they can just shut that half down until the issue is fixed, and still remain in production. That would not be possible if TLS was bolted into Varnish.
I guess Hitch was one of those TLS frontends, what was the other one?
https://www.hitch-tls.org/
https://www.hitch-tls.org/
Wow! That's a neat trick
"The lack of built-in HTTPS seems killer to me."
I agree. I think varnish remained popular for 2 reasons:
- It was, at one time, higher performing than an nginx proxy cache. That doesn't seem to be the case anymore.
- VCL is very rich and flexible with primitives for reading/writing cookie contents, cache metadata, cache flushing, client and server state, and so on.
So probably the only reason left if that you're doing something really complicated with your cache in VCL.
Otherwise, as you say, the built in HTTPS of nginx, haproxy, nuster, or something else is a big advantage.
I agree. I think varnish remained popular for 2 reasons:
- It was, at one time, higher performing than an nginx proxy cache. That doesn't seem to be the case anymore.
- VCL is very rich and flexible with primitives for reading/writing cookie contents, cache metadata, cache flushing, client and server state, and so on.
So probably the only reason left if that you're doing something really complicated with your cache in VCL.
Otherwise, as you say, the built in HTTPS of nginx, haproxy, nuster, or something else is a big advantage.
As for who uses Varnish, the best public data I know is:
https://trends.builtwith.com/Web-Server/Varnish
My personal estimate is that around 20% of all HTTP traffic pass through a Varnish instance somewhere, many of which are not public-facing.
As to why people use Vanish, I think the main reason is that VCL gives you full and instant control over your HTTP traffic, and people really like that, because it lets unify heterogenous sites and pamper legacy systems as necessary.
https://trends.builtwith.com/Web-Server/Varnish
My personal estimate is that around 20% of all HTTP traffic pass through a Varnish instance somewhere, many of which are not public-facing.
As to why people use Vanish, I think the main reason is that VCL gives you full and instant control over your HTTP traffic, and people really like that, because it lets unify heterogenous sites and pamper legacy systems as necessary.
There are advantages of using HTTP cache on application side even if you use CDN:
- Varnish can guarantee that a given resource will be fetched at most once every TTL expiration since you can set up one instance to do so (+ some HA solution like hot standby). This complements a large CDN that is highly scalable but at a cost that there is no hard "synchronization" between servers. You will see many requests for the same resource, even when the CDN uses cache hierarchy because they can't risk such bottleneck not knowing in advance your traffic pattern. You can intentionally do that with Varnish and benefit from that.
- Complex caching rules, even including what to do if application is not available/slow. That includes serving stale content, how long to serve stale after TTL expires, guarantee that fetching a new version is performed in background. Example: https://varnish-cache.org/docs/trunk/users-guide/vcl-grace.h...
We've used varnish + some other magic in addition to Cloudflare to withstand Black Friday when marketing insisted the promo has to start at specified time, anuonced much earlier to all customers. The landing page had up-to date info on available products updated almost instantly, thanks to serving stale (in practice < 1s) content and guaranteed prefresh in background. Cache key was properly set up to exclude tracking elements from URLs (from FB, adwords etc) and only include what is necessary.
- Varnish can guarantee that a given resource will be fetched at most once every TTL expiration since you can set up one instance to do so (+ some HA solution like hot standby). This complements a large CDN that is highly scalable but at a cost that there is no hard "synchronization" between servers. You will see many requests for the same resource, even when the CDN uses cache hierarchy because they can't risk such bottleneck not knowing in advance your traffic pattern. You can intentionally do that with Varnish and benefit from that.
- Complex caching rules, even including what to do if application is not available/slow. That includes serving stale content, how long to serve stale after TTL expires, guarantee that fetching a new version is performed in background. Example: https://varnish-cache.org/docs/trunk/users-guide/vcl-grace.h...
We've used varnish + some other magic in addition to Cloudflare to withstand Black Friday when marketing insisted the promo has to start at specified time, anuonced much earlier to all customers. The landing page had up-to date info on available products updated almost instantly, thanks to serving stale (in practice < 1s) content and guaranteed prefresh in background. Cache key was properly set up to exclude tracking elements from URLs (from FB, adwords etc) and only include what is necessary.
One thing I've heard a lot is "We know we can always fix it in Varnish".
By this people mean things like printing the wrong URL in a full-page sunday newspaper add or splitting traffic based on first digit of customer-number and stuff like that.
Some of the screwups Varnish-ops have fixed are truly the stuff of legends, but telling them belongs to their heros :-)
By this people mean things like printing the wrong URL in a full-page sunday newspaper add or splitting traffic based on first digit of customer-number and stuff like that.
Some of the screwups Varnish-ops have fixed are truly the stuff of legends, but telling them belongs to their heros :-)
Also CDNs became cheaper and easily accessible. They don’t eat anywhere near 100% of your traffic for most sites but it’s likely enough to get a big chunk of the cacheable traffic at lower latency than your own Varnish server can unless you’re running them all over like Fastly. If a significant fraction of the traffic coming back isn’t cacheable the benefits of running Varnish might sink below the level where it’s worth dealing with another layer of production infrastructure.
The Fastly CDN is built on Varnish VCL IIRC.
<< girls get disappointed if think they got hold of something else than the MP3 player you had in your pocket>>
Sexist B.S.
Sexist B.S.
Not really
[deleted]
anoncake(1)
Nah.
How is that sexist?
Honest answer: because it treats "you," the potential reader of the article, as separate from the class of "girls."
Adding my voice to this. I know that the article was written in 2006, but still. If one wants to be taken seriously as a writer, one shouldn't insert puerile jokes for 12 year olds into their prose. It made me wince when when I read it, and I was ready to comment here about it, but GP got here first.
Tell that to Shakespeare and Chaucer.
Shakespeare and Chaucer were not writing technical documentation.
I was responding in jest to the narrow claim about not making puerile jokes if one wishes to be taken as a serious writer.
You shouldn’t get downvoted for that.
Welcome to Hacker News, please enjoy your stay. /s
Where people are either socialists or nazis, without a ton of middle ground so you’re likely to get downvoted no matter what you say.
Any article that assumes something about the reader is in some way discriminatory or is it just when it assumes that you're a man?
Generally, the accusation of "sexist" is used when someone is making unfair or irrelevant assumptions on the basis of sex. It is not generally leveled at technical documentation that assumes the reader has the technical background to read the documentation, for instance.
Technical documentation sometimes does contain humor, it's not that uncommon for developers to include jokes and easter eggs. Furthermore, this is listed under category "Poul-Hennings random outbursts", described as "You may or may not want to know what Poul-Henning thinks.". So while technically included in it, it's not really a documentation of anything. It's basically a blog post. And just like I don't find anything wrong about a blog about cosmetics or fashion assuming that the reader is a woman, I don't find anything wrong in a blog post about technology assuming that the reader is a man.
I propose the article branches at this point as a set of selectable tabs, each labeled over a suitable set of properties describing the reader, and then offering a matching joke when you click the tab.
Doesn't that still have the problem of categorizing the reader, being wildly off-topic from the technical content of the article and distracting the reader by making them wonder, at least a bit, if the technical content of the article is going to be different for them depending on who they are?
The joke barely makes sense in context. Is the physical size of the memory relevant to the point about memory-mapped views of disk - and is it a problem that the physical size of memory is smaller? Should one program an MP3 player the way described in this article, or not? What about USB flash drives (which one is likely to have in one's pocket, and which are roughly the same form factor as the other object alluded to here) - they're much slower than main disk and more likely to fail. Is this an intended reference to such memory devices, and in either case, how does the advice from the article apply to such devices?
The article would be better (more effective at communicating its point) without it.
The joke barely makes sense in context. Is the physical size of the memory relevant to the point about memory-mapped views of disk - and is it a problem that the physical size of memory is smaller? Should one program an MP3 player the way described in this article, or not? What about USB flash drives (which one is likely to have in one's pocket, and which are roughly the same form factor as the other object alluded to here) - they're much slower than main disk and more likely to fail. Is this an intended reference to such memory devices, and in either case, how does the advice from the article apply to such devices?
The article would be better (more effective at communicating its point) without it.
OK, I propose then we replace the tabs with just a single link labeled "Please click here if you'd like a dick joke".
Then have more detailed tabs on a dedicated page.
Then have more detailed tabs on a dedicated page.
You can make a dick joke without assuming the listener is the owner of a dick.
The conflict here is that a joke where you include the listener is automatically funnier, and a dick joke is also automatically funnier.
Source: https://www.youtube.com/watch?v=xwGIHzR5r0Y
So the challenge lies in combining both, resulting in either the listener, or someone the listener is interacting with in the narrative having a dick.
It's one of the big unsolved problems of telling jokes online.
Source: https://www.youtube.com/watch?v=xwGIHzR5r0Y
So the challenge lies in combining both, resulting in either the listener, or someone the listener is interacting with in the narrative having a dick.
It's one of the big unsolved problems of telling jokes online.
[deleted]
Arenas are great for latency sensitive programming. But some of the advice in this piece is a bit limited in that it assumes the presence of swap (often disabled on server hardware) and programs whose important "hot" datasets are likely to fit in memory (i.e. the memory use is nice and predictable--an enviable but not always achievable property).
When working with bursty memory demands and code/data you don't always fully control (think a big thick runtime with its own allocation decisions, some of them decidedly "1975") on a system whose swapping behavior you can't predict, hand managing on-disk versus in-memory data residency in userland (via manual memory mapping or something like sqlite/lmdb to take care of it for you) becomes necessary.
When working with bursty memory demands and code/data you don't always fully control (think a big thick runtime with its own allocation decisions, some of them decidedly "1975") on a system whose swapping behavior you can't predict, hand managing on-disk versus in-memory data residency in userland (via manual memory mapping or something like sqlite/lmdb to take care of it for you) becomes necessary.
TLDR; Varnish is very proud they use memory-mapped files for cache, instead of managing RAM and disk cache separately.
That said, don't overestimate the OS ability to understand your usage of RAM. The entity with highest insight on that usage is the application.
Maybe there's some fortunate overlap in the case of a caching service (Varnish) using a caching service (OS disk cache / virtual memory). Varnish itself has little clue how their entries will be used beyond what the OS sees.
But in more complex applications that's decidedly not the case. Photoshop for example to this day uses a "scratch disk" separate from virtual memory. In fact you better hope you never use swap file with Photoshop, because it becomes unusable.
That said, don't overestimate the OS ability to understand your usage of RAM. The entity with highest insight on that usage is the application.
Maybe there's some fortunate overlap in the case of a caching service (Varnish) using a caching service (OS disk cache / virtual memory). Varnish itself has little clue how their entries will be used beyond what the OS sees.
But in more complex applications that's decidedly not the case. Photoshop for example to this day uses a "scratch disk" separate from virtual memory. In fact you better hope you never use swap file with Photoshop, because it becomes unusable.
Thanks for making this point. If you access your memory sequentially, as Varnish does, great. But if you need to stride across a giant matrix in transposed order, you'll bring your spinning disk backed virtual memory to a halt, as it does a million seeks. To use virtual memory successfully you have to write in-memory algorithms that recognize the non-random-access nature of the memory. If you had simply acknowledged there's a storage system, you wouldn't put out much more effort, if any, to get the same performance. In fact, by pretending it's all RAM, you give up the opportunity to overlap your I/O and do something useful with the time the system would have to spend paging in your data.
Edit: btw, happy and satisfied Varnish user for many years!
Edit: btw, happy and satisfied Varnish user for many years!
Amazing until you need persistence.
Sure, but the point remains: If you fight the operating system you WILL lose. Varnish doesn’t solve the same problem as Squid does, the author have said as much in talks when Varnish was first released. Still that doesn’t excuse an outdated programming model in Squid.
We actually do have a persistent `stevedore` in Varnish Cache, but we have marked it "deprecated" because it did not live up to expectations, but we keep it around to keep the internal API's honest.
Redesigning that code has not bubbled up to the top of my TODO list yet, and shows now signs of doing so.
But if you need persistence, there is a commercial version of Varnish which has it, but I have not been involved in that.
Redesigning that code has not bubbled up to the top of my TODO list yet, and shows now signs of doing so.
But if you need persistence, there is a commercial version of Varnish which has it, but I have not been involved in that.
[deleted]
Since the moderators seem to have collapsed the subthread about it (but not flagged it, so I can't vouch for it), I'd just like to say that the random puerile joke in this article detracts from its technical merit.
I'd argue that it's actually more complex now. I programmed in the early 1980s, and did have to worry about where my data physically resided inside the machine, but I never had to worry about whether it was in a different room on a different machine, or sitting on a hard drive in some Amazon Data Center a continent away.
As others here note, the idea of using the same abstraction for all storage is at least as old as MMUs and Multics (mid 1960s). What is different is that programmers in the 60s and (early) 70s were usually coding pretty close to the bare metal. Nowadays, Moore's law has allowed us sufficient power to permit programming on top of a big pile of abstractions that try very hard to hide the actual hardware from us.
That's a luxury afforded by the sheer power of what we're working with, but the people writing those abstraction layers still have to pay attention to the layer beneath them, and if you go down far enough, you'll find some code that needs to pay attention to what class of memory something is stored in and how to optimally move it around. It's just that work was probably done for you by someone else who wrote your operating system.
Just because your programming language doesn't require you to use pointers doesn't meant that indirection isn't being used. You just don't have to deal with it (until it rears up and reminds you it's still there). Joel Spolsky's Law of Leaky abstractions (https://www.joelonsoftware.com/2002/11/11/the-law-of-leaky-a...) is relevant here.