Deploying Mastodon on Digital Ocean(startuplab.io)
startuplab.io
Deploying Mastodon on Digital Ocean
https://startuplab.io/post/deploying-mastodon-on-digital-ocean
51 comments
Why do we still need to manage domain names and DNS settings through web interfaces? Isn't there a command-line tool for this? I'm asking because every provider uses a different web interface, which is very inconvenient.
You may be interested by DNSControl, a tool developed by StackExchange:
https://stackexchange.github.io/dnscontrol/getting-started
I wish they had use the standard bind format to describe the zones, instead of JavaScript, but the tool looks great.
https://stackexchange.github.io/dnscontrol/getting-started
I wish they had use the standard bind format to describe the zones, instead of JavaScript, but the tool looks great.
On a kubernetes cluster you can run external-dns, which always runs in the cluster and watches for new Ingress records routing traffic on a certain domain. And then external-dns will go manage Route53 DNS records (also works with the goog).
https://github.com/kubernetes-incubator/external-dns
A similar thing happens for ssl certs from LetsEncrypt https://github.com/jetstack/cert-manager
So I just `kubecctl create ingress` with a new domain and after a minute or two, the cluster has set it all the certs/dns set up for me (and in the case of CI, the pipeline will continue and run e2e tests against it).
It obv took investment to learn and set it all up, but on a team I could see how it could save people time who don't want to wrange dns records and such.
https://github.com/kubernetes-incubator/external-dns
A similar thing happens for ssl certs from LetsEncrypt https://github.com/jetstack/cert-manager
So I just `kubecctl create ingress` with a new domain and after a minute or two, the cluster has set it all the certs/dns set up for me (and in the case of CI, the pipeline will continue and run e2e tests against it).
It obv took investment to learn and set it all up, but on a team I could see how it could save people time who don't want to wrange dns records and such.
Choose provider that offers API. It doesn't have to be the VPS provider itself. And probably shouldn't be.
Ok, is the API standardized?
No, but choosing a tool that implements support for multiple providers will give you a common interface.
There's RFC2136 and RFC3645 but they're not very widely implemented.
DNS provided by your hosting service is really just there for convenience IMO. Using services like DNS Made Easy or the like is worth the small annual fee to not have to worry about it anymore.
There are free options with an API, such as Cloudflare (no, one doesn't have to use the CDN/proxy the traffic through them to use the DNS).
At the risk of hijacking this discussion, why is installing server-side web apps so ludicrously complicated? It's madness.
I'd love to see more developers succeed with their self-hosted web apps. But it simply will not happen when the barrier to installation is so high. Too many developers vastly underestimate just how complicated installation is.
Or maybe they prefer the complications because it makes SaaS a much more attractive option to customers when there isn't a simple one-click install option?
I asked this question previously: https://news.ycombinator.com/item?id=18385497
I just find the situation exasperating. Would love to hear why people think this is a non-issue. And let me repeat: a simple, easy-to-install option for server-side software would unlock countless opportunities for developers.
I'd love to see more developers succeed with their self-hosted web apps. But it simply will not happen when the barrier to installation is so high. Too many developers vastly underestimate just how complicated installation is.
Or maybe they prefer the complications because it makes SaaS a much more attractive option to customers when there isn't a simple one-click install option?
I asked this question previously: https://news.ycombinator.com/item?id=18385497
I just find the situation exasperating. Would love to hear why people think this is a non-issue. And let me repeat: a simple, easy-to-install option for server-side software would unlock countless opportunities for developers.
>Would love to hear why people think this is a non-issue.
Most developers have never seen anything better. They start working within an artificially complicated ecosystem, get used to it and perpetuate all of the assumptions the ecosystem makes when they create their own products.
And when you point this out, most people just get angry, because for them it suggests they have been doing something wrong for as long as they've done engineering. This is why you always hear things like "buy my website is an application" or "but it has to be complicated to use, because it had complicated functionality".
Simplicity is not easy or intuitive.
I've only started to learn the principles of designing simple systems that do complex things around 10-12 years in my career, after accidentally discovering certain people's talks online (Alan Kay, Rich Hickey) and learning a bit about old-school AI systems. Most people aren't familiar with any of that stuff and don't really care.
One thing I personally cannot comprehend is why web applications are almost universally installed and managed through command line, rather than web interfaces. The only advantage of a command-line interface is pipe chaining, and it's never uses installation/configuration. There is nothing fundamental about command line. It's just a software simulation of a teletype terminal.
Most developers have never seen anything better. They start working within an artificially complicated ecosystem, get used to it and perpetuate all of the assumptions the ecosystem makes when they create their own products.
And when you point this out, most people just get angry, because for them it suggests they have been doing something wrong for as long as they've done engineering. This is why you always hear things like "buy my website is an application" or "but it has to be complicated to use, because it had complicated functionality".
Simplicity is not easy or intuitive.
I've only started to learn the principles of designing simple systems that do complex things around 10-12 years in my career, after accidentally discovering certain people's talks online (Alan Kay, Rich Hickey) and learning a bit about old-school AI systems. Most people aren't familiar with any of that stuff and don't really care.
One thing I personally cannot comprehend is why web applications are almost universally installed and managed through command line, rather than web interfaces. The only advantage of a command-line interface is pipe chaining, and it's never uses installation/configuration. There is nothing fundamental about command line. It's just a software simulation of a teletype terminal.
The command line has the advantage of presenting portable, terse and editable instructions that can be cobbled together into a recipe which can be abstracted and automated.
It's a very powerful interface.
It's a very powerful interface.
Web interfaces are used by half the population of the planet. Any command you run in a shell can be represented by a URL, an HTML tag or a form. URLs and HTML have editable text representations, and they also have semantics, standard composition rules and generic UI.
Any "recipe" you're getting is likely something you get from a web page to begin with. That can be a tool for composition and automation on it own.
Any "recipe" you're getting is likely something you get from a web page to begin with. That can be a tool for composition and automation on it own.
So very few of those web interfaces used by half the population of the planet actually utilize semantics, standard composition rules, and generic UIs - even though they're available.
> Any "recipe" you're getting
Not necessarily. Distro package maintainers use and construct recipes that are not web pages, in order to allow folks to install complex software with, in many cases, one click.
Not necessarily. Distro package maintainers use and construct recipes that are not web pages, in order to allow folks to install complex software with, in many cases, one click.
[deleted]
> cannot comprehend is why web applications are almost universally installed and managed through command line, rather than web interfaces. The only advantage of a command-line interface is pipe chaining
Repeatability, scriptability, debugability ... ?
Repeatability, scriptability, debugability ... ?
Are you talking about these instructions in particular? What exactly is overcomplicated if you don't mind me asking?
This is not some trivial web application, it's a federated social network, and it's all laid out for you, basically up to and including steps for ensuring security and production readiness.
I am exasperated too by this stuff, and I am a professional that works with it every day (but these instructions are especially well laid out.)
I'm counting on Helm to save us. I've been admining a Jenkins server on Helm and Kubernetes and I just haven't seen the "miserable experience" I've heard of other Jenkins admins describing.
I've run several other projects using Helm and I'm convinced it's a pattern of admin-friendliness that is common to stable and well-maintained charts. They are just generally easier to work with, don't feel cobbled together, and you basically never get Configuration Drift.
But it still won't be universally easy until Kubernetes is ubiquitous. It seems easy to me now, because I've already grown accustomed to basic Kubernetes maintenance and it's mostly optional, thanks to the proliferation of managed Kubernetes providers on-the-cheap.
It's still a lot to reason about, and at my place of business the status quo is simply worse than that because of struggling with adoption.
This is not some trivial web application, it's a federated social network, and it's all laid out for you, basically up to and including steps for ensuring security and production readiness.
I am exasperated too by this stuff, and I am a professional that works with it every day (but these instructions are especially well laid out.)
I'm counting on Helm to save us. I've been admining a Jenkins server on Helm and Kubernetes and I just haven't seen the "miserable experience" I've heard of other Jenkins admins describing.
I've run several other projects using Helm and I'm convinced it's a pattern of admin-friendliness that is common to stable and well-maintained charts. They are just generally easier to work with, don't feel cobbled together, and you basically never get Configuration Drift.
But it still won't be universally easy until Kubernetes is ubiquitous. It seems easy to me now, because I've already grown accustomed to basic Kubernetes maintenance and it's mostly optional, thanks to the proliferation of managed Kubernetes providers on-the-cheap.
It's still a lot to reason about, and at my place of business the status quo is simply worse than that because of struggling with adoption.
> This is not some trivial web application, it's a federated social network
To be fair, its cousin Pleroma is comparatively easy to install
To be fair, its cousin Pleroma is comparatively easy to install
Is it though? I had to dig for a while reading "what is/about pleroma" documents to find the link to the install doc, I think this is it:
https://i2p.rocks/blog/the-magical-world-of-pleroma-setting-...
Is this easier? It looks at least a bit more complicated at a glance to me. I haven't honestly tried either of them, but neither of these are simple to install. Add a config file to nginx, install erlang and use mix to grab the dependencies... these things are not totally foreign to me as a Ruby programmer, but I could see how a newbie would trip over them.
I still don't know which part people are tripping over, I'd like to hear from GP... The Pleroma doc might actually be simpler because it skips over all of the "Hardening your instance" stuff, but I'm only guessing about what people see as the hard part, since none of this looks particularly hard to me.
To be equally fair, it's really never as simple as "docker run someone/image:latest" – that's only the first step, so you can see it running. In Kubernetes terms, that's the equivalent of firing off a "run-once" pod. In other words, if that's all you did then it's not deployed yet.
https://i2p.rocks/blog/the-magical-world-of-pleroma-setting-...
Is this easier? It looks at least a bit more complicated at a glance to me. I haven't honestly tried either of them, but neither of these are simple to install. Add a config file to nginx, install erlang and use mix to grab the dependencies... these things are not totally foreign to me as a Ruby programmer, but I could see how a newbie would trip over them.
I still don't know which part people are tripping over, I'd like to hear from GP... The Pleroma doc might actually be simpler because it skips over all of the "Hardening your instance" stuff, but I'm only guessing about what people see as the hard part, since none of this looks particularly hard to me.
To be equally fair, it's really never as simple as "docker run someone/image:latest" – that's only the first step, so you can see it running. In Kubernetes terms, that's the equivalent of firing off a "run-once" pod. In other words, if that's all you did then it's not deployed yet.
The install docs are on the wiki: https://git.pleroma.social/pleroma/pleroma/wikis/Installing%...
Because most server-side web apps are complicated. The ones that aren't (too some extend) can just get [apt-get|dnf|yum|pacman]ed into your server and you're ready to go. WordPress and Roundcube are one command away from running on your server. If running apt-get, picking a database password and setting your domain name are too much to ask, it's likely that someone like that might not have the expertise to securely run a web-facing server-side application securely.
There's several systems set up to make self-hosting projects easy. It usually involves some behind-the-scenes docker containers with custom scripts and a custom web interface. If you're interested in something like this, try Sandstorm: https://sandstorm.io/ It has a free demo that shows exactly what it can do. It works quite nice, actually.
As you might have seen, there is only a limited amount of software packages available on Sandstorm. The trouble comes with dependencies and overhead. Not every project can talk to Postgres (WordPress mostly relies on MySQL), leading to multiple databases running at once unless you go and manually configure addons and config files to make software talk to each other. Software is designed to run well on a specific base system, be it Ubuntu Server with postgres, CentOS with MySQL or Arch with sqlite.
To make this easier, large software packages are usually packed into Docker containers. Running multiple Docker containers still requires some effort (you need to specify what persistent data needs to go where) but that's nothing compared to manual installation. It's also a requirement one cannot easily get around, as most people/companies have their own preferred storage paths and data storage requirements for important data (think encrypted databases, separate encrypted disk mounts, etc.). This is tiresome and requires reading up on the relevant documentation, but not difficult.
With most of the complexity removed (just ports and paths), you quickly end up running multiple versions of Nginx and Apache at once because every docker container comes with its own web server and its own specific, non-compatible add-ons. This wastes RAM and CPU which could be used to have your services run more smoothly. Most big applications therefore are designed to run best on their own separate hosts to take full advantage of the (virtual) hardware.
The article linked is a good example. It is not a manual about writing config stanzas and downloading packages; it's the bare minimum one needs to handle to configure something like Mastodon.
There's several systems set up to make self-hosting projects easy. It usually involves some behind-the-scenes docker containers with custom scripts and a custom web interface. If you're interested in something like this, try Sandstorm: https://sandstorm.io/ It has a free demo that shows exactly what it can do. It works quite nice, actually.
As you might have seen, there is only a limited amount of software packages available on Sandstorm. The trouble comes with dependencies and overhead. Not every project can talk to Postgres (WordPress mostly relies on MySQL), leading to multiple databases running at once unless you go and manually configure addons and config files to make software talk to each other. Software is designed to run well on a specific base system, be it Ubuntu Server with postgres, CentOS with MySQL or Arch with sqlite.
To make this easier, large software packages are usually packed into Docker containers. Running multiple Docker containers still requires some effort (you need to specify what persistent data needs to go where) but that's nothing compared to manual installation. It's also a requirement one cannot easily get around, as most people/companies have their own preferred storage paths and data storage requirements for important data (think encrypted databases, separate encrypted disk mounts, etc.). This is tiresome and requires reading up on the relevant documentation, but not difficult.
With most of the complexity removed (just ports and paths), you quickly end up running multiple versions of Nginx and Apache at once because every docker container comes with its own web server and its own specific, non-compatible add-ons. This wastes RAM and CPU which could be used to have your services run more smoothly. Most big applications therefore are designed to run best on their own separate hosts to take full advantage of the (virtual) hardware.
The article linked is a good example. It is not a manual about writing config stanzas and downloading packages; it's the bare minimum one needs to handle to configure something like Mastodon.
I think cloudron provides exactly this. Have you tried it? Mastadon package is also under way.
Cloudron.io installation experience is the best I've ever seen. Even easier than Wordpress:
wget https://cloudron.io/cloudron-setup chmod +x ./cloudron-setup ./cloudron-setup --provider [digitalocean,ec2,generic,ovh,...]
https://cloudron.io/get.html And that's it, after that it's only a web UI.
Does anybody know other examples of software which provides command line install personnalized to the provider you use?
wget https://cloudron.io/cloudron-setup chmod +x ./cloudron-setup ./cloudron-setup --provider [digitalocean,ec2,generic,ovh,...]
https://cloudron.io/get.html And that's it, after that it's only a web UI.
Does anybody know other examples of software which provides command line install personnalized to the provider you use?
Shameless plug: https://git.cloudron.io/synchrone/mastodon-app
I am currently working on an end-to-end test suite, as required by cloudron.io platform, to add it to their appstore,
I am currently working on an end-to-end test suite, as required by cloudron.io platform, to add it to their appstore,
> a simple, easy-to-install option for server-side software > would unlock countless opportunities for developers.
Docker. Its how any software can be packaged with its dependencies. Best for linux serverside/webapp software.
Install and run a docker with one console command, just provide a network-port and folder mapping.
Or use a webinterface, i like how Unraid does this. If you have a old server/PC to spare, definitely try Unraid.net by just booting it from USB stick.
Docker. Its how any software can be packaged with its dependencies. Best for linux serverside/webapp software.
Install and run a docker with one console command, just provide a network-port and folder mapping.
Or use a webinterface, i like how Unraid does this. If you have a old server/PC to spare, definitely try Unraid.net by just booting it from USB stick.
Docker only solves the dependency problem for static file dependencies. For things like databases or other services, you still have to stand those up yourself or use another orchestration tool. Beyond that, Docker also puts the onus on the user to solve for networks and volumes and environment variables and secrets management and configuration management.
"Docker" is perhaps part of the solution, but only a very small part.
"Docker" is perhaps part of the solution, but only a very small part.
Yep. That’s where docker compose comes in. It takes care of the whole process of linking together different docker containers as well as configuration of each of them (such as env vars).
Compose is just a declarative syntax for your stack. Ironically it's not composable. In other words, it's not a package definition--you can't ship a compose file that will "just run" anywhere. For example, most web apps require a database, but you don't want to run one database per web app. How do you write a compose file that says, "use the system postgres if available, otherwise spin up a new one". It also totally punts on secret and config management.
Helm is the closest thing to a package manager for Docker, but it targets Kubernetes, not vanilla Docker (and k8s is overkill for a single server, at least considering the time/expertise required to stand up a productionized k8s installation).
Helm is the closest thing to a package manager for Docker, but it targets Kubernetes, not vanilla Docker (and k8s is overkill for a single server, at least considering the time/expertise required to stand up a productionized k8s installation).
Ah I see what you mean. Yeah you’re right
> Would love to hear why people think this is a non-issue
It is an issue, but like, you have to get over it.
Most of the complications come from dependencies (db, caching, etc). How can I, as a web service creator, make it easier to install and configure a postgres DB? I can't! Software is complicated, and sometimes the requirements for software are complicated.
Sure, I could release a version that uses sqlite but then people will whine that it's too slow and doesn't scale. If I release a version that supports both, now I've introduced an entire layer of complexity that I have to support (and tends to break endlessly in my experience).
Yeah, it sucks. but if you're really that worried about it, like you mentioned, use a SaaS service for the app itself or for its dependencies.
It is an issue, but like, you have to get over it.
Most of the complications come from dependencies (db, caching, etc). How can I, as a web service creator, make it easier to install and configure a postgres DB? I can't! Software is complicated, and sometimes the requirements for software are complicated.
Sure, I could release a version that uses sqlite but then people will whine that it's too slow and doesn't scale. If I release a version that supports both, now I've introduced an entire layer of complexity that I have to support (and tends to break endlessly in my experience).
Yeah, it sucks. but if you're really that worried about it, like you mentioned, use a SaaS service for the app itself or for its dependencies.
Have you tried Sandstorm? That's exactly what it offers - simple installation and management of server-side apps, sandboxed and using a common authentication and authentication infrastructure.
As a php developer I don't share this pain point.
It's okay, you have plenty of pain points of your own.
I can honestly say developing in php feels great. Php removes the pain points. That's why it has a bad rap because it enables the novice to create and they make novice mistakes.
What languages have a lower pain point level in your opinion?
What languages have a lower pain point level in your opinion?
Well, yes you do, because the thing that makes this complicated is all the various out of process dependencies like the database.
The database is the easiest part in a lamp setup provided you are not using postgres.
If you are using something like digitalocean they have a one click lamp install. It couldn't be easier.
If you are using something like digitalocean they have a one click lamp install. It couldn't be easier.
You mentioned previously that sandstorm and cloudron are not options. Why not? You’re only talking about hosting on your local machine?
Deploying my go web application is as simple as copying a small binary to a server and running ./server
So it doesn't have a database (or it has an embedded database)?
With me, the gripes of installing these apps are not just the app itself but the dependencies, which is a necessary evil and impossible to avoid once things get sufficiently complicated.
With me, the gripes of installing these apps are not just the app itself but the dependencies, which is a necessary evil and impossible to avoid once things get sufficiently complicated.
How about stop buying back so many freaking shares all the time. So tired of companies spending so much money on this.
FYI there's a more recent, and much more in-depth official installation manual: https://docs.joinmastodon.org/administration/installation/
The documentation also includes various guides on horizontal scaling, using pgBouncer etc.
The documentation also includes various guides on horizontal scaling, using pgBouncer etc.
I don't see any reference to running Mastodon using Docker on the official installation manual. If there is, could you point me to the right direction?
See here: https://github.com/tootsuite/documentation/blob/master/Runni...
For some reason Eugen didn't port everything to the new doc website.
For some reason Eugen didn't port everything to the new doc website.
Right, but is that up-to-date? The parent to my reply states that "more recent, and much more in-depth official installation manual" exists, but without Docker. Does it means Docker will no longer be officially supported?
The new docs site is definitely a WIP, every upgrade guide still has notes for how to upgrade both with and without Docker.
Nice tutorial!
> Now let's add docker to the sudo group, that will allow us to run all the docker commands without sudo
This command is actually adding the current user to the docker group, not adding docker to the sudo group. Same end result though.
> Now let's add docker to the sudo group, that will allow us to run all the docker commands without sudo
This command is actually adding the current user to the docker group, not adding docker to the sudo group. Same end result though.
I see what you snuck in there...(last item)
from TFA: `sudo apt-get install nginx emacs`
from TFA: `sudo apt-get install nginx emacs`