> Is terraform generally used to deploy workloads to nomad instead of writing tasks directly?
It can be, although it has some weird shortcomings. For example, if the job is already present in Nomad but not running ("dead"), I don't think you can use the terraform provider to start it again.
HashiCorp themselves suggest using terraform to provision the base nomad system (ACL, quotas, base system jobs), but perhaps not your actual applications:
> This can be used to initialize your cluster with system jobs, common services, and more. In day to day Nomad use it is common for developers to submit jobs to Nomad directly, such as for general app deployment. In addition to these apps, a Nomad cluster often runs core system services that are ideally setup during infrastructure creation. This resource is ideal for the latter type of job, but can be used to manage any job within Nomad.
* A DIY templating thing (e.g. python and jinja2 templates)
* A webapp that submits jobs as JSON directly to the nomad API, perhaps modifying it to match certain policies (kind of like k8s validating / mutating admission webhooks)
I understand what you're saying. Fewer runtimes = fewer problems.
It's not going to force your hand. You can disable all the task drivers except the docker driver if you want a container-only cluster. The drivers themselves are lightweight.
In an ideal world, every company is two years old and only produces software in perfect docker containers, but in reality there's always some service that doesn't work in a container but could benefit from job scheduling.
I think it's great that we can add scheduling to different runtimes. Some folks want or need to use those different runtimes, and I like that Nomad lets you do that.
This is Nomad's most underrated feature, IMHO. You don't have to use containers for everything if you don't want to. For example, if you're a golang shop you can run everything as native binaries and cut out docker completely.
Nomad has much simpler networking, i.e. no web of iptables rules to figure out. You can add Consul connect as a service mesh if you need it, but if you don't, you can keep things very simple. Simple = easy to understand, run, and debug.
The main downside for me is a lack of plug and play pieces, e.g. a helm chart for Grafana or Prometheus. You'll have to write your own job for those, though it's very easy to learn. I'd love to see a series of Nomad recipes that people could use.
I think it's the ideal choice for on-prem, bare-metal, or 'weird' deployments where you need a bit more control. You can build the exact stack you need with the different HashiCorp projects with minimal overhead.
I can't recommend it enough! I help people move to Nomad, my email is in my profile if you want to chat :)
I only have praise for the HashiStack (Nomad, Consul, Vault). Folks often reach for Kubernetes as part of a larger initiative to containerize their apps, add ephemeral test environments, etc, but I really think Nomad is better suited to this.
Choosing k8s is just the first step - you have to deal with what 'distribution' of k8s to use, upgrades, pages of yaml, secrets stored in plaintext by default...
Once you've got Nomad running, it just works.
I help people move to Nomad, my email is in my profile if you want to chat :)
https://github.com/hashicorp/nomad/issues/3854
I've used an nginx-based S3 proxy in the past to get around this. Not ideal but it works.