I couldn't agree with too much tooling can abstract what is usually simple - just serve some HTML. I think that the whole API economy and saas/paas stuff really has to be evaluated carefully. You have business considerations around lockin, time to integrate vs time to build your own, etc. I think that they work really well when you're building something simple, but there is a range of the size of your site where it is more of a hinderance. The decision to use a service should be about what it gives you, not because it is cool.
Aside: I have totally been that engineer that has made something "clever". I am sure there are other engineers that curse me for what I thought was a great tool b/c I looked at the site for 0.1 seconds (sorry!).
I really wanted to address the talk about static.
Let's take the instance of a blog (like any of the heroku/rails tutorials out there). Yes, you must have a canonical place for the copy to live. Be it in a db or flat files on disk or in your git repo. But you don't need to have the actual request go to the origin for that info and then jam it through some jinja/unicorn/etc template. Just to render a silly article to the end user. When you write that article, you know what that page is going to look like, _why dynamically generate it_? This is the way that static can work, generate all the versions of the content and rely on JS to do magic on the frontend (https://www.destroyallsoftware.com/talks/the-birth-and-death...). Removing the whole call back to the origin db for what is essentially static content. This obviously is going to be faster than a DB query + template render + network traffic, as well as more secure. It is an http GET ~ hard exploit vector.
Now does this extend into the arena of apps (react, angular, newest fanciest JS framework). The actual assets are also static, no? They should be served exactly the same as the HTML we have. Then it is up to the JS to query whatever service/API you want and automagically generate some HTML.
The big thing is that services like wordpress/drupal/rails have made it very easy for people to build sites in a classic LAMP stack, but that is kinda flawed in a lot of ways. Wordpress's plugin system essentially lets you remotely run your code on their servers. That is a dangerous game to play. All to do something that doesn't even need a server in the first place. Why risk it when you don't need to? And you'd get some nice improvements if you don't. People shouldn't even know what a LAMP stack is to make there business site.
Now is this approach right for every site? Nopezzzz. I don't believe in silver bullets, but there are a lot of sites that fit this mold. And it is a different approach to building your site out.
Either way - sorry to hear about Capistrano. Shell scripts ftw (though I have some that are terrible out there too).
Absolutely you can solve this all with your own code. It is totally reasonable.
But it is your own code. Which means dev cost & maintenance. Hence there are services for it. Could you set up your own DB? rack and stack your own servers? Absolutely, but AWS is way easier. It is about letting the developer work on the part that matters; not rote problems that have been solved.
To specifically address your cache invalidation. (1) you have to manually invalidate _each_ resource. And you have to do that atomically or you run into issues surrounding different versions globally. (2) you release each of your resources to a versioned destination (e.g. <img src="v2/stupidimage.jpg"> -> <img src="v3/stupidimage.jpg">). Again, yes it is straight forward (not to say easy) but it it tedious, error prone and really just annoying to do. Because you actually want to use something like the SHA-1 of the content (better cache hits). Again, you have an issue around the atomic behaviour of updating the site.
And then you need to make sure that you set your headers right (which I always screw up). This is your E-Tag and cache control headers. This is to force the browser to do a conditional get request.
And continuous delivery is about removing the person from pipeline of push -> deploy. That means that even if you have a script that does your FTP drop, you have to configure that. Easy enough in something like circleCI, travis, or (for the daring) jenkins. But you still have an issue surrounding the atomic actions. FTP uploads _take time_ if you're constantly serving traffic - this could lead to odd problems around "what does the customer see". Those are real issues for sites, usually unreported by the user (They just refresh - but it colors the "feeling" of your site).
The other part about this is that it raises the barrier of entry into website development. Should you need a neckbeard and CS degree just to make a site for your mom? Hosting services like gitlab, github pages, netlify, s3 and the likes try to make it easier.
Yes - you can solve this all yourself; there are a finite amount of checkboxes you need to...well..check. But like you don't really want to rack and stack all your own servers, do you really want to spend your time thinking about that?
So static sites really have come a long way from the 1994 geocities versions. There are people that statically build sites with 1000's of pages. The idea is that you don't need to have the whole backend that we've all built before is growing (look at things like the serverless movement).
As soon as you have to do more than a junk website that you check occasionally and care about the actual user experience you need to start considering things like CDNs, cache invalidation (global consistency matters) and C/D integrations. This is why you have a growth of static site hosting services (and hence the article).
I couldn't agree with too much tooling can abstract what is usually simple - just serve some HTML. I think that the whole API economy and saas/paas stuff really has to be evaluated carefully. You have business considerations around lockin, time to integrate vs time to build your own, etc. I think that they work really well when you're building something simple, but there is a range of the size of your site where it is more of a hinderance. The decision to use a service should be about what it gives you, not because it is cool.
Aside: I have totally been that engineer that has made something "clever". I am sure there are other engineers that curse me for what I thought was a great tool b/c I looked at the site for 0.1 seconds (sorry!).
I really wanted to address the talk about static.
Let's take the instance of a blog (like any of the heroku/rails tutorials out there). Yes, you must have a canonical place for the copy to live. Be it in a db or flat files on disk or in your git repo. But you don't need to have the actual request go to the origin for that info and then jam it through some jinja/unicorn/etc template. Just to render a silly article to the end user. When you write that article, you know what that page is going to look like, _why dynamically generate it_? This is the way that static can work, generate all the versions of the content and rely on JS to do magic on the frontend (https://www.destroyallsoftware.com/talks/the-birth-and-death...). Removing the whole call back to the origin db for what is essentially static content. This obviously is going to be faster than a DB query + template render + network traffic, as well as more secure. It is an http GET ~ hard exploit vector.
Now does this extend into the arena of apps (react, angular, newest fanciest JS framework). The actual assets are also static, no? They should be served exactly the same as the HTML we have. Then it is up to the JS to query whatever service/API you want and automagically generate some HTML.
The big thing is that services like wordpress/drupal/rails have made it very easy for people to build sites in a classic LAMP stack, but that is kinda flawed in a lot of ways. Wordpress's plugin system essentially lets you remotely run your code on their servers. That is a dangerous game to play. All to do something that doesn't even need a server in the first place. Why risk it when you don't need to? And you'd get some nice improvements if you don't. People shouldn't even know what a LAMP stack is to make there business site.
Now is this approach right for every site? Nopezzzz. I don't believe in silver bullets, but there are a lot of sites that fit this mold. And it is a different approach to building your site out.
Either way - sorry to hear about Capistrano. Shell scripts ftw (though I have some that are terrible out there too).