This is a really good point - I've said it before and I'll say it again - containers neither add to (nor subtract) from whatever you're doing today. If you have a single VM and no shared storage, you're exactly as vulnerable as if you were doing things in a container. And, in the majority of cases, the exact same techniques you'd use in a VM work in a container or Kubernetes too.
I should mention Kubernetes is 100% open source, runs on AWS, Google Cloud, Azure, Digital Ocean, Vagrant, bare metal, VMWare, Rackspace and lots more I'm probably forgetting. Then you can pick the cloud you like, and lock-in be gone.
I put that disclaimer at the top of all my posts, but this one truly is HIGHLY biased.
The absolute easiest way to do what you describe is to use GKE (Google's hosted Kubernetes). For $0.15/hr, we'll manage everything for you, and you can build out teeny tiny clusters that do everything you need. It's even free for clusters <5 nodes. Start it up, use a sample app from this directory (https://github.com/kubernetes/kubernetes/tree/master/example...) and you're done.
Yep - and the ability to run just one per node is coming "soon" (ideally by next milestone). It's something we wanted to get in, but just didn't have the time by 1.1.
My #1 recommendation is to do whatever you do today. If you mount an NFS mount into your VM to store your MySQL data, that's what you should do again. The speed at restarting a pod is very low - but you do need to do the (small) hack today if you want to be absolutely sure you don't have multiple pods with on the same node.
To be clear, the ugly-ness is one line in your config file:
Portability, as you mentioned, is huge - but beyond that there is also binpacking benefits. You can basically cut your serving costs by 50% or more by using lots of single process containers instead of poorly utilized VMs.
The number one feature we see people get out of containers (as we did at Google) is flexibility. Bin packing and (some) isolation is important and valuable, but the vast majority is just being able to throw 25k containers at a system (Borg for internal work, now Kubernetes externally) and just say "Go run this somewhere." The system figures it out for you - and you never have to deal with server config. Then if you need to move to a new data center - presto, happens in seconds.
I vibe-coded^W^Wwrote this - please tell me what we can do to help make it more useful!