> Is terraform generally used to deploy workloads to nomad instead of writing tasks directly?
It can be, although it has some weird shortcomings. For example, if the job is already present in Nomad but not running ("dead"), I don't think you can use the terraform provider to start it again.
HashiCorp themselves suggest using terraform to provision the base nomad system (ACL, quotas, base system jobs), but perhaps not your actual applications:
> This can be used to initialize your cluster with system jobs, common services, and more. In day to day Nomad use it is common for developers to submit jobs to Nomad directly, such as for general app deployment. In addition to these apps, a Nomad cluster often runs core system services that are ideally setup during infrastructure creation. This resource is ideal for the latter type of job, but can be used to manage any job within Nomad.
* A DIY templating thing (e.g. python and jinja2 templates)
* A webapp that submits jobs as JSON directly to the nomad API, perhaps modifying it to match certain policies (kind of like k8s validating / mutating admission webhooks)
I understand what you're saying. Fewer runtimes = fewer problems.
It's not going to force your hand. You can disable all the task drivers except the docker driver if you want a container-only cluster. The drivers themselves are lightweight.
In an ideal world, every company is two years old and only produces software in perfect docker containers, but in reality there's always some service that doesn't work in a container but could benefit from job scheduling.
I think it's great that we can add scheduling to different runtimes. Some folks want or need to use those different runtimes, and I like that Nomad lets you do that.
This is Nomad's most underrated feature, IMHO. You don't have to use containers for everything if you don't want to. For example, if you're a golang shop you can run everything as native binaries and cut out docker completely.
Nomad has much simpler networking, i.e. no web of iptables rules to figure out. You can add Consul connect as a service mesh if you need it, but if you don't, you can keep things very simple. Simple = easy to understand, run, and debug.
The main downside for me is a lack of plug and play pieces, e.g. a helm chart for Grafana or Prometheus. You'll have to write your own job for those, though it's very easy to learn. I'd love to see a series of Nomad recipes that people could use.
I think it's the ideal choice for on-prem, bare-metal, or 'weird' deployments where you need a bit more control. You can build the exact stack you need with the different HashiCorp projects with minimal overhead.
I can't recommend it enough! I help people move to Nomad, my email is in my profile if you want to chat :)
I only have praise for the HashiStack (Nomad, Consul, Vault). Folks often reach for Kubernetes as part of a larger initiative to containerize their apps, add ephemeral test environments, etc, but I really think Nomad is better suited to this.
Choosing k8s is just the first step - you have to deal with what 'distribution' of k8s to use, upgrades, pages of yaml, secrets stored in plaintext by default...
Once you've got Nomad running, it just works.
I help people move to Nomad, my email is in my profile if you want to chat :)
That's terrible. There's nothing worse than losing data.
I'm working on this exact problem at the moment: a dashboard to keep track of your backups to check they never fail. https://backupshq.com
* Get notified when backups fail or take too long via email, slack, webhook, etc.
* Run your backup scripts and tools with an open source agent, which takes cares of reporting backup results to the dashboard.
* Alternatively, write your own integration with our API to report backup results and send logs yourself.
I'm hoping to launch privately soon, I'd love to hear what you think. We haven't made any pricing decisions yet, but definitely want to give free credit to homelabbers!
Oh! I'm working on this exact problem at the moment: a dashboard to keep track of your different backups.
* Get notified when backups fail or take too long via email, slack, webhook, etc.
* Run your backup scripts and tools with our open source agent, which takes cares of reporting backup results to the dashboard.
* Alternatively, write your own integration with our API to report backup results and send logs yourself.
We'll launch privately soon at https://backupshq.com, I'd love to hear what you think of the idea. We haven't decided on pricing yet, but would like to offer free usage for personal use. Either way, I'll hook you up with some free credit.
Firstly, sorry to hear about your backup woes. There's nothing worse than losing data.
Three tricks that have worked for me:
1) Reduce the resistance in your brain to making backups
Easy wins can be made here without changing anything. Keep the external drive ready to go on your desk at any time, have it start backing up as soon as you plug it in, etc. Reduce the dissonance you feel in your brain to making backups regularly.
2) Automate it as much as possible
Take the resistance entirely out of the equation by automating it. For example, in your case you might want to try plugging the external drive into an always-on raspberry PI, and use Syncthing to constantly send your files there.
3) Have a process that keeps you accountable
You need something to make you feel guilty for not having backups, that overpowers the resistance in your brain with shame! Even a recurring calendar entry can work well.
I've taken it a step further by writing software to solve this very problem. I've seen and heard about way too many backup failures in my short career, and determined to help bring good backup practices to more people. Check out the idea at https://backupshq.com - we'll be launching privately soon and always interested in feedback on the idea.
https://github.com/hashicorp/nomad/issues/3854
I've used an nginx-based S3 proxy in the past to get around this. Not ideal but it works.