Launch Your App: Why We Don’t Use Docker (We Don’t Need It)(launchyourapp.meezeeworkouts.com)
launchyourapp.meezeeworkouts.com
Launch Your App: Why We Don’t Use Docker (We Don’t Need It)
https://launchyourapp.meezeeworkouts.com/2021/03/why-we-dont-use-docker-we-dont-need-it.html
94 comments
Previous discussion: https://news.ycombinator.com/item?id=26472452
> But we have software we need to ship in order to get users in order to drive subscriptions. Anything that doesn’t directly serve that goal is a complication.
I think you're only seeing Docker as a packaging format and missing the distribution aspect. There's a lot of value in telling users "My app is accessible with a single `docker run` command", that doesn't involve them downloading a standalone binary manually, placing it in their PATH, etc., and also works across platforms and operating systems. Or worse, having to setup the Go toolchain and compile it themselves, no matter how easy it is for a developer (and ignoring what can go wrong with Go modules).
Of course, then you open yourself up to all sorts of support questions related to Docker, but I still think it's a missed opportunity to not use it to make your software as widely and most easily accessible as possible.
I think you're only seeing Docker as a packaging format and missing the distribution aspect. There's a lot of value in telling users "My app is accessible with a single `docker run` command", that doesn't involve them downloading a standalone binary manually, placing it in their PATH, etc., and also works across platforms and operating systems. Or worse, having to setup the Go toolchain and compile it themselves, no matter how easy it is for a developer (and ignoring what can go wrong with Go modules).
Of course, then you open yourself up to all sorts of support questions related to Docker, but I still think it's a missed opportunity to not use it to make your software as widely and most easily accessible as possible.
> and also works across platforms and operating systems
As a mac user, I'm not sure I agree with this. Docker does technically work cross platform of course. But it's slow everywhere except for linux. I'd much rather have binary (or even better a `brew install`) than a docker image.
As a mac user, I'm not sure I agree with this. Docker does technically work cross platform of course. But it's slow everywhere except for linux. I'd much rather have binary (or even better a `brew install`) than a docker image.
It's a lot faster if you don't rely on the default volume mounting: https://nodewood.com/blog/how-to-speed-up-docker-on-macos/
Still not as fast as on Linux, but within 10% of running it natively for me.
Still not as fast as on Linux, but within 10% of running it natively for me.
SO the argument here is that it's the storage layer that slows things down? I deduce this since the linked article is about installing a new storage driver to sync to external volumes.
What about server workloads where the storage is completely inside the container / on a different server?
What about server workloads where the storage is completely inside the container / on a different server?
It's less the storage layer and more the volume mount on OSX - the default is very reliable, but at the cost of a LOT of speed. Mutagen offers varying levels of reliability, and occasionally I have to restart the sync (like maybe once a week), but it's _much_ faster.
The circumstances you're describing sound like it wouldn't be suffering this specific issue in the first place, so I'm not sure how much it'd help.
The circumstances you're describing sound like it wouldn't be suffering this specific issue in the first place, so I'm not sure how much it'd help.
A lot of people don't realize docker on macos runs in a vm.
It would be much nicer if apple coexisted instead of apple-language, apple-frameworks, apple-laserlike-navel-focus. Do community interaction, not just over-the-wall to opensource.apple.com
a native apple docker would be great. Imagine:
It would be much nicer if apple coexisted instead of apple-language, apple-frameworks, apple-laserlike-navel-focus. Do community interaction, not just over-the-wall to opensource.apple.com
a native apple docker would be great. Imagine:
FROM macos:10.13.2
RUN xcode-build ....Apple will never commoditize its own stuff.
They're perfectly willing to ruin the lives of countless devs / CI/CD devs, who have to support their crappy platforms, because you can't virtualize MacOS (only AWS can, and even then, it can only be rented in 24h increments at super high prices).
Apple controls the end users, their end users have money so devs just have to come crawling to be able to make and sell their apps.
They're perfectly willing to ruin the lives of countless devs / CI/CD devs, who have to support their crappy platforms, because you can't virtualize MacOS (only AWS can, and even then, it can only be rented in 24h increments at super high prices).
Apple controls the end users, their end users have money so devs just have to come crawling to be able to make and sell their apps.
I’ve used Anka for running macos VMs with good success. So much so that we replaced all our bare metal workloads with portable and repeatable environments. The trade offs are performance related and bugs with the VM tech but those are less costly than managing the bare metal.
That's true, unfortunately, volumes are especially problematic. Though I think it's gotten better over the years, and performance should be acceptable for most workflows.
I'm not saying it should be the _only_ distribution option, and standalone binaries and other packages should be provided as well, but there's no reason for Docker to not be one of them.
I'm not saying it should be the _only_ distribution option, and standalone binaries and other packages should be provided as well, but there's no reason for Docker to not be one of them.
Also, I guess, it's needless to say, most of people don't have Docker already installed (unless your target audience is a dev-ops community). So it's not just "docker run".
> and also works across platforms and operating systems
That's the greatest lie that Docker managed to convince people of.
Docker is not portable, its images are not portable. It runs on Linux and that's it. It doesn't even run on other *NIXen.
Docker on Mac runs inside a VM. Docker on Windows runs inside a VM. Technically Docker for Windows supports Windows containers. But they only run on... Windows.
Docker is as portable as any software I can get my hands on and that I can run inside a VM. So almost all the desktop software I know of.
Docker just managed to package its software in a nicer way so that many people don't even realize that there's a VM behind the scenes (most usually notice because Docker is slower/much slower on non-Linux platforms).
That's the greatest lie that Docker managed to convince people of.
Docker is not portable, its images are not portable. It runs on Linux and that's it. It doesn't even run on other *NIXen.
Docker on Mac runs inside a VM. Docker on Windows runs inside a VM. Technically Docker for Windows supports Windows containers. But they only run on... Windows.
Docker is as portable as any software I can get my hands on and that I can run inside a VM. So almost all the desktop software I know of.
Docker just managed to package its software in a nicer way so that many people don't even realize that there's a VM behind the scenes (most usually notice because Docker is slower/much slower on non-Linux platforms).
I don't care if it runs in a VM or not. That's just an implementation detail.
What matters is whether my app and the myriad of dependencies it relies on can be run easily and consistently.
What matters is whether my app and the myriad of dependencies it relies on can be run easily and consistently.
Docker for Windows is a thing, and it does run Windows binaries natively.
So you can publish for two platforms.
Java is still a good way to have bulletproof cross-platform experience, but JRE is sort of large. JDK 10 or 11, IIRC, allows to package your code, only the needed parts of stdlib, and the VM into a reasonably lean self-contained executable.
So you can publish for two platforms.
Java is still a good way to have bulletproof cross-platform experience, but JRE is sort of large. JDK 10 or 11, IIRC, allows to package your code, only the needed parts of stdlib, and the VM into a reasonably lean self-contained executable.
Rust/Go are much better at than Java. You do need a separate binary for each platform which makes it slightly more of a pain for the maintainer to setup CI, but it's much more convenient for the end user.
Good point for services being installed. Especially when I'm just testing something I don't want to see a full page on how to get it up and running, or even care about the underlying tech. Just give me a docker oneliner. Doesn't have to be the standard way of distributing the app, but at least make a simple image for local testing / development.
Tested RabbitMQ this weekend, and the getting started was soo easy.
Tested RabbitMQ this weekend, and the getting started was soo easy.
Some context to my comment, last time I was doing app distribution I used the contemporary best practice which was SVN. I’ve read a dozen articles on the theory of Docker vs VM but still don’t get it. Apparently it saves resources by not replicating the whole OS. Do everyone’s apps really not rely on the specific kernel at all? Even in i.e. PHP apps I usually have binaries tied in that are compiled for the kernel I’ve developed on.
Containers are peripheral to my work so I don’t understand but I’m curious.
Containers are peripheral to my work so I don’t understand but I’m curious.
Docker takes advantage of various linux sandboxing features. On a linux bare metal host it's just a combination of namespaces and resource constraints (cgroups) whereas on other platforms it will have to be the above within a bonafide VM running a linux kernel. The general rule is that linux maintains backward compatability even in ABI so as long as the host kernel is the same or newer than the image requirements it should just work. Things wont work for example if your image requires iouring (a brand new syscall interface) but your host OS is running 4.x (doesn't map iouring syscalls to anything).
what kind of php app are you running that requires to be linked against a specific linux kernel
Let's say I integrate with a CLI utility, which comes up. Docker wouldn't work? If the answer is no I think I'd say I understand the theory better. Unless docker is providing some kind of universal kernel API or something?
In that case you would bundle a copy of the CLI utility into your docker container, with the correct version. This also applies to any system libraries your code might be compiled against. Things like shared libraries or CLI utilities are part of the "operating system", but not part of the "kernel".
This does mean that you end up with multiple copies of the same utilities and libraries on the system - although docker does have a mechanism to avoid duplication between different docker containers. It also means that if you take your docker container and move it over to a different system with a newer operating system (or a different distribution) all the parts of the operating system it cares about come with it.
The Linux kernel - the part that gets shared between containers - is mostly just responsible for process and memory management, networking and interacting with the computer hardware.
This does mean that you end up with multiple copies of the same utilities and libraries on the system - although docker does have a mechanism to avoid duplication between different docker containers. It also means that if you take your docker container and move it over to a different system with a newer operating system (or a different distribution) all the parts of the operating system it cares about come with it.
The Linux kernel - the part that gets shared between containers - is mostly just responsible for process and memory management, networking and interacting with the computer hardware.
What makes you assume that all people have or even want docker runtime in their machines?
Not to mention all the security headaches it brings with it.
The OCI image format is an established standard and registries are supported across tools. Publishing an image on Docker Hub also makes it available to tools like Podman that don't require a runtime and can run rootless containers. You're also free to download, extract the image and run the binaries however you wish, even outside a container.
Docker is a packaging format with a distribution platform and a common execution interface. I see offering it as an option to users only as a net positive. Not to mention that it opens the orchestration door making the application easy to integrate into CI pipelines and complex workflows.
Docker is a packaging format with a distribution platform and a common execution interface. I see offering it as an option to users only as a net positive. Not to mention that it opens the orchestration door making the application easy to integrate into CI pipelines and complex workflows.
[deleted]
> Building is as easy as:
> go build
> Testing is as easy as
> go test
> Deploying the app is as easy as:
> scp app user@host:
> ssh user@host “nohup ./app”
I suppose we'll see in 1-3 years whether the decision to skip any/all build and deploy tools in favor of staking a business on someone scp'ing to prod servers was a good idea. I have a very strong assumption, but who knows.
> go build
> Testing is as easy as
> go test
> Deploying the app is as easy as:
> scp app user@host:
> ssh user@host “nohup ./app”
I suppose we'll see in 1-3 years whether the decision to skip any/all build and deploy tools in favor of staking a business on someone scp'ing to prod servers was a good idea. I have a very strong assumption, but who knows.
Twitch made it to acquisition with amazon with basically this system for both a rails app and about 20 other go apps. I'd know, I helped transition us to ecs later.
For go apps, docker isn't really doing much over golang binaries. The bigger transition was more just moving us away from any unneeded dependence on system level apps. This happened at the same time we were dockerizing.
I think the ECS abstraction is better than the simple daemon scripts we were using and scp for deploys, but we made it work up to around 30k servers before we hit scale limits. We could have pretty easily taken it even further forward but ecs is simpler and better integrated with the rest of AWS and our build tools. It only matters for a few special teams.
The real issue with SCP at some point was the parallelism of deploys: opening 30k ssh connections single threaded or with bash starts bogging down at some point, and we were at the point of writing a massively parallel version when we switched to ECS for much of this (also around the time ECS became stable).
For go apps, docker isn't really doing much over golang binaries. The bigger transition was more just moving us away from any unneeded dependence on system level apps. This happened at the same time we were dockerizing.
I think the ECS abstraction is better than the simple daemon scripts we were using and scp for deploys, but we made it work up to around 30k servers before we hit scale limits. We could have pretty easily taken it even further forward but ecs is simpler and better integrated with the rest of AWS and our build tools. It only matters for a few special teams.
The real issue with SCP at some point was the parallelism of deploys: opening 30k ssh connections single threaded or with bash starts bogging down at some point, and we were at the point of writing a massively parallel version when we switched to ECS for much of this (also around the time ECS became stable).
I posted below, but to me this post isn't really about Docker at all, as are my issues with it. You don't need to use Docker to see the need for sane build and deploy tooling.
Who is running this scp? Where do they get the build? Do they have to know which hosts to scp to? Questions/problems like this are not solved at all, which generally need to be solved one way or another to scale any production system.
Who is running this scp? Where do they get the build? Do they have to know which hosts to scp to? Questions/problems like this are not solved at all, which generally need to be solved one way or another to scale any production system.
This was years ago. it was bare metal. The answer was that it got the exe from the build machine which it was running from. It knew where to deploy as we were on bare metal and used consul to list the hosts.
Down the road, pulling the binary in a container for deployment is trivial, so I don’t see a downside here. Of course nobody will advocate that scping a binary on each production server for deployment is viable in the long run, but that should not be your primary concern when going through the early phases of a product.
Not focusing on docker from the start forces you to have a clean build and runtime software chain, and reduces the amount of magic required (ironically, going to containers was supposed to be the opposite, but many server-side packages are now docker-only because the build and/or runtime requirements are so messed up, e.g. discourse).
Not focusing on docker from the start forces you to have a clean build and runtime software chain, and reduces the amount of magic required (ironically, going to containers was supposed to be the opposite, but many server-side packages are now docker-only because the build and/or runtime requirements are so messed up, e.g. discourse).
I just don't get it. Docker is just not that hard to set up. It takes a day or two and you get a lot of flexibility you can use down the line.
Sure you could convert later, but now you have to move your whole workflow over during a time you're in dire need of the new features instead of casually from the start. Now instead of your artifact history being consistent, its truncated at the point you switched over.
I guess I don't understand whats so complicated about docker?
Sure you could convert later, but now you have to move your whole workflow over during a time you're in dire need of the new features instead of casually from the start. Now instead of your artifact history being consistent, its truncated at the point you switched over.
I guess I don't understand whats so complicated about docker?
Docker in itself, apart from everything else, is not that hard to setup. But you have to look at the bigger picture here.
Maybe the deployment is on "pet servers" and not on-demand, which means if you are a responsible admin and have a nftables config you need to analyze how docker will mess up your rules (or why it is not working).
Maybe you don’t want to lose time setting up an artifactory, and a docker registry with proper access controls.
Maybe you don’t want to have to focus on the ops side of things until you actually need it. Yes that may mean losing time later when you need it fast, but at the same time this can be done iteratively when things are ramping up.
From experience at a previous workplace, using docker as part of your local development process can be a huge PITA, and getting developers to adopt it without grumbling and complaining about it every day is also a hurdle. Given enough computers, you will get weird network errors, some borked modules due to an upgrade, some containers refusing to die, etc…
I am not saying it cannot be done, and in the general case docker mostly works without (too many) glitches, but it is always another layer that hinders your productive work.
(also, "docker" in production nowadays is mostly a shortcut for k8s, and that is a whole new amount of investment required)
Maybe the deployment is on "pet servers" and not on-demand, which means if you are a responsible admin and have a nftables config you need to analyze how docker will mess up your rules (or why it is not working).
Maybe you don’t want to lose time setting up an artifactory, and a docker registry with proper access controls.
Maybe you don’t want to have to focus on the ops side of things until you actually need it. Yes that may mean losing time later when you need it fast, but at the same time this can be done iteratively when things are ramping up.
From experience at a previous workplace, using docker as part of your local development process can be a huge PITA, and getting developers to adopt it without grumbling and complaining about it every day is also a hurdle. Given enough computers, you will get weird network errors, some borked modules due to an upgrade, some containers refusing to die, etc…
I am not saying it cannot be done, and in the general case docker mostly works without (too many) glitches, but it is always another layer that hinders your productive work.
(also, "docker" in production nowadays is mostly a shortcut for k8s, and that is a whole new amount of investment required)
> (also, "docker" in production nowadays is mostly a shortcut for k8s, and that is a whole new amount of investment required)
It didn't help that Docker tied docker compose to swarm and then swarm kinda died. So I'm really scratching my head at the people recommending docker in this thread. I'm assuming they mean K8S but at the same time surely they can't be serious.
Today if you need something at the startup/mid range level you're probably looking at Dokku, Caprover, or maybe even nomad. This is still more complexity than scp, of course.
It didn't help that Docker tied docker compose to swarm and then swarm kinda died. So I'm really scratching my head at the people recommending docker in this thread. I'm assuming they mean K8S but at the same time surely they can't be serious.
Today if you need something at the startup/mid range level you're probably looking at Dokku, Caprover, or maybe even nomad. This is still more complexity than scp, of course.
Its really not hard at all to set up CI/CD into managed k8s on one of the clouds. You can still trivially use docker compose to set up a local dev environment with all the application appliances like the DB or redis or what have you as docker containers.
You get turn key environments with a ton of flexibility for what is honestly much less effort than even maintaining the setup documentation for installing the apps directly.
You get turn key environments with a ton of flexibility for what is honestly much less effort than even maintaining the setup documentation for installing the apps directly.
Bridge networking, probably.
Overlay filesystem performance across distros?
Attaching a debugger to a process?
Idk. Just some ideas off the top of my head. I don’t think it’s unimaginable to consider how docker is an increase in complexity and understanding.
Overlay filesystem performance across distros?
Attaching a debugger to a process?
Idk. Just some ideas off the top of my head. I don’t think it’s unimaginable to consider how docker is an increase in complexity and understanding.
Getting VSCode debugger to hook into a process running in a docker container running inside WSL was insanely frustrating. In the end, I stopped using Docker during development and only use it for deploying to staging/production.
I agree. Debugging docker is a total nightmare across various stacks. I don't know how people bother with it. Like you use it for deployments but not for development. I'd rather have slightly more time setting up a project (which in all honesty is usually a few commands) vs hours/days of flakey debugging.
Did you ever find a satisfying solution? I am trying to make more effective use of docker for development and anticipate needing GDB at some point.
I imagine one could install GDB in the container and connect to it remotely? Or from the container's shell?
I imagine one could install GDB in the container and connect to it remotely? Or from the container's shell?
Ignoring VSCode's insanely fragile config for this, any debugger with a remote protocol tends to work. Personally I have better luck manually setting up a headless GDB session and connecting with intellij, but it's also difficult sometimes.
Since the remote stuff is sometimes an incredible pain to set up and use, I usually install gdb and just use its CLI in the container. That's usually trivial in comparison, and it's a decent skill to learn regardless since so many debuggers use a similar CLI UX. It's extremely cross-platform and cross-language, so you can just jump on a new thing and be a wizard from day 1.
Since the remote stuff is sometimes an incredible pain to set up and use, I usually install gdb and just use its CLI in the container. That's usually trivial in comparison, and it's a decent skill to learn regardless since so many debuggers use a similar CLI UX. It's extremely cross-platform and cross-language, so you can just jump on a new thing and be a wizard from day 1.
For node / php and python I just expose the debugger ports in docker, and if I'm running docker in a VM I just forward them over SSH. VSCode connects to the forwarded port easily, but sometimes you have to struggle with workspace / source mapping and other issues (like connecting to child processes).
I was eventually able to get it to work by digging through a lot of old GitHub issue discussions. Sadly it was so flaky and slow that I just threw out Docker for development after awhile.
IIRC, you just need to install gdb when you build your image and use `docker run -it bash` to run any shell commands in your container
IIRC, you just need to install gdb when you build your image and use `docker run -it bash` to run any shell commands in your container
> Bridge networking, probably.
You can just use --net=host if you don't like it.
> Overlay filesystem performance across distros?
I've used docker on several distros and never noticed an issue with this.
> Attaching a debugger to a process?
This is also not an issue if you don't use bridge networking.
You can just use --net=host if you don't like it.
> Overlay filesystem performance across distros?
I've used docker on several distros and never noticed an issue with this.
> Attaching a debugger to a process?
This is also not an issue if you don't use bridge networking.
The above post isn't _really_ about Docker, I don't think. Relying on scp to deploy to prod also has nothing to do with Docker inherently. This post is mostly about rejecting any "modern" build and deploy tools in favor of perceived-simplicity. The problem with perceived-simplicity is it's usually not actually that simple, the developer that thought up the idea just inherently understands it. My post above was mostly to highlight that that pereived-simplicity falls apart when you add more devs, or increased code complexity, etc.
Using legitimate tools to handle building and deploying has nothing to do with Docker, but the author of this blog post has combined them.
Using legitimate tools to handle building and deploying has nothing to do with Docker, but the author of this blog post has combined them.
'Day or two' isn't actually a trivial amount of time.
This exactly. Docker provides a way to deploy but also to self-document and version control the deployment process, and having the Dockerfile from the starts brings a lot of value.
Without this, deployment starts being spread over various bash files with often poor documentation (they’re already mentioning such a script in their article). Then when it becomes too messy, time to write that Dockerfile I guess, except now it’s more complicated than if it had been done from the start.
Without this, deployment starts being spread over various bash files with often poor documentation (they’re already mentioning such a script in their article). Then when it becomes too messy, time to write that Dockerfile I guess, except now it’s more complicated than if it had been done from the start.
I'm pretty sure we've already answered that question over the last 20-30 years, and it turns out that in most cases: yes, it's absolutely fine.
In the "our business teeters on the edge of solvency every month" stage of a company lifecycle, does it really matter that much?
Plus the article never mentions "we'll never need to add more ci/cd", they're simply saying they don't need it right now.
Plus the article never mentions "we'll never need to add more ci/cd", they're simply saying they don't need it right now.
Just to re-enforce this, this article is all about context. It's making it very clear that in this organisations context this method of deployment and production is absolutely fine. And that's the key thing, you can keep things this simple for as long as your risk assessment says so.
The key for me is making sure you are taking the time to reassess those early life decisions so when the time comes you can course-adjust
The key for me is making sure you are taking the time to reassess those early life decisions so when the time comes you can course-adjust
Yeah, we should use deploy tools instead.
$ cat deploy.sh
#!/bin/sh
scp app user@host;
ssh user@host "nohup ./app"
The reason to use build and deployment tools has nothing to do with _how_ the tools actually work. Deployment issues are mostly issues of consistency and access. Who runs the build/deploy, from where, with what arguments, when? Using a CI/CD pipeline isn't just wrapping an scp call in a bash script, they provide tools for managing the when, the where, the why.
Do you think using only scp will prevent you from deploying the wrong artifact? Or deploying to all-but-one production node? Or scping but forgetting to run the nohup?
Honestly setting up a github action to do this takes very little time, doesn't depend on containers, doesn't really add complexity to your stack. CircleCI exists and is equally easy. To suggest that your above-posted script is just as good is, to me, mostly just NIH + developer bikeshedding. YAGNI is great for being honest about the needs of an org and ignoring tools or frameworks that aren't appropriate, but there really are tools that exist that add consistency in ways that don't really increase complexity all that much.
Do you think using only scp will prevent you from deploying the wrong artifact? Or deploying to all-but-one production node? Or scping but forgetting to run the nohup?
Honestly setting up a github action to do this takes very little time, doesn't depend on containers, doesn't really add complexity to your stack. CircleCI exists and is equally easy. To suggest that your above-posted script is just as good is, to me, mostly just NIH + developer bikeshedding. YAGNI is great for being honest about the needs of an org and ignoring tools or frameworks that aren't appropriate, but there really are tools that exist that add consistency in ways that don't really increase complexity all that much.
Because push instead of pull deployment means reboots and now servers have to be handled manually.
Because this way doesn't leave explicit audit/deployment logs.
Because this way doesn't leave explicit audit/deployment logs.
[deleted]
I'd almost argue that the main benefit to not using docker at a scale where scp works is you won't be tempted to use kubernetes.
[deleted]
The main reason I push for Docker containers is that it's a universally-understood encapsulation of an app or service amongst technical and non-technical people alike. "I'll have John write a Docker service that handles the integration" is an easy goal for both management and John to understand. Behind the scenes, we can use some fancy deployment or just run it on a single machine like the author describes.
I would definitely not be okay with an engineer in my organization telling me their plan is to ignore CI/CD, especially in the present age where GitHub Actions is trivially easy to set up, and rather set up some custom procedure over SSH. Technical choices aside, it fails the "hit by a bus" test - if you disappear for some reason, or even just your laptop gets stolen, does the company grind to a halt?
I would definitely not be okay with an engineer in my organization telling me their plan is to ignore CI/CD, especially in the present age where GitHub Actions is trivially easy to set up, and rather set up some custom procedure over SSH. Technical choices aside, it fails the "hit by a bus" test - if you disappear for some reason, or even just your laptop gets stolen, does the company grind to a halt?
> if you disappear for some reason, or even just your laptop gets stolen, does the company grind to a halt
As in, no one in the whole company knows how to scp a binary?
As in, no one in the whole company knows how to scp a binary?
With CI
Boss: "Alice, can you release Super Mega Money Maker"
Alice: clicks deploy on a UI It'll be out in 5 minutes.
Boss: "Thanks!"
Without CI
Boss: "Bob is out sick, can you deploy Super Mega Money Maker?"
Alice: "Ok. hmm... I pulled the code and I'm trying to run the build but I'm getting an error. It wants a super-secret-token.pem file. Do you know where that is?"
Boss: "Let me call Bob" "Ok, he said it's a file on his laptop but he left his laptop in the office. He's on his way in."
Boss: "Alice, can you release Super Mega Money Maker"
Alice: clicks deploy on a UI It'll be out in 5 minutes.
Boss: "Thanks!"
Without CI
Boss: "Bob is out sick, can you deploy Super Mega Money Maker?"
Alice: "Ok. hmm... I pulled the code and I'm trying to run the build but I'm getting an error. It wants a super-secret-token.pem file. Do you know where that is?"
Boss: "Let me call Bob" "Ok, he said it's a file on his laptop but he left his laptop in the office. He's on his way in."
To be fair, scp was officially deprecated: it is effectively "legacy knowledge" now, and so soon it will be difficult to find anyone who can use it effectively. https://lwn.net/Articles/835962/ In contrast, we all know that knowledge of the Docker container format is both universal and likely to "stand the test of time". That anyone would believe that merely downloading a binary and then running it directly could ever be easier than first downloading and installing Docker so that you can then run docker on yet another file clearly has never worked as a systems administrator.
[deleted]
if the app in question can be installed by dropping some files on some servers, how could another full layer of abstraction be easier? an encapsulation is by definition adding complexity.
I mean that it promotes a style of operating where a sudden change, for whatever reason, can cause all kinds of problems.
> Docker containers is [...] a universally-understood encapsulation of an app or service amongst technical and non-technical people alike
On what planet does a non-technical person even know what a Docker container is?
On what planet does a non-technical person even know what a Docker container is?
I mean someone at the CEO/product manager level. They don't need to understand the details, and rarely do they. The anecdote that came to mind was a conference where a non-technical CEO gave a high-level explanation of "how Docker transformed my company" (not much substantive content) through all kinds of shipping container analogies.
Well it's unlikely that person was the only one on your team that could run a two-line scp script. Choose any two engineers and their expertise in, and understanding of Docker is going to differ, but they should equally and completely understand basic file copy tools.
I would guess you are more likely to be in the situation where only one developer really knows your complicated build tools and how exactly they are put together. That's your actual "hit by a bus" vulnerability.
I would guess you are more likely to be in the situation where only one developer really knows your complicated build tools and how exactly they are put together. That's your actual "hit by a bus" vulnerability.
You're missing my bigger point, which is that I want to use off-the-shelf tools because I can explain it more easily to other developers and management.
GitHub Actions is not complicated.
GitHub Actions is not complicated.
You can commit the deployment script in the app repo. I'm not recommending not using ci/cd, but not using it doesn't mean you run every step manually from memory.
I feel like this is one of those articles meant to convince the author most of all.
At my startup we do the same but with Rust. All the reasons stand. Insanely fast, self contained (web service / embedded UI with static js/html). Could not be happier with the architecture.
I wrote it about it here if anyone interested https://logicai.io/blog/recommendation-systems-in-rust/.
I wrote it about it here if anyone interested https://logicai.io/blog/recommendation-systems-in-rust/.
A lot of modern tooling has come from people using microservices instead of making libraries. If you actually just write libraries it turns out you don't need to solve distributed systems problems, orchestration and configuration management as part of your normal development cycle.
> we did get a little more complicated and create a SystemD script to launch it at startup
If only there was a tool to:
- bundle, distribute and deploy applications
- ...and configuration files
- ...and systemd unit files
- ...even directories containing a Python virtualenv
- keep track of what is installed and in what version
- roll forward or roll back
I would call it Advanced Packaging Tool.
If only there was a tool to:
- bundle, distribute and deploy applications
- ...and configuration files
- ...and systemd unit files
- ...even directories containing a Python virtualenv
- keep track of what is installed and in what version
- roll forward or roll back
I would call it Advanced Packaging Tool.
This is the normal progression. A new technology comes along, and folks are so happy to use it everywhere that they teach new people to use it, without any understanding of why it's useful or what it was made to fix. And they have little to no understanding of the ecosystem that preceded it.
I watched it happen with virtualization. Everyone was so hyped about virtualization that everything was run in VMs. Later there was a counterrevolution of folks going "hey, we discovered that running things on bare metal servers is a thing you can do, and it's way faster and cheaper", a push that has mostly died out now that virtualization has hit near bare metal performance anyway and cloud hosting is the norm. But that counterrevolution happened because everyone embraced VMs and cloud services without dwelling on its actual use case, or in what cases it was an improvement over bare metal hosting. I watched similar happen with both the NoSQL and Serverless revolutions, and now we're watching it with containers.
So yes, this would be better setup as a deb package, or deployed as part of a cloudinit startup script or similar. But if you came in when everyone was just shoving docker down each other's throats, you don't know what stages there were before that, and which one is the most appropriate one for your use.
I watched it happen with virtualization. Everyone was so hyped about virtualization that everything was run in VMs. Later there was a counterrevolution of folks going "hey, we discovered that running things on bare metal servers is a thing you can do, and it's way faster and cheaper", a push that has mostly died out now that virtualization has hit near bare metal performance anyway and cloud hosting is the norm. But that counterrevolution happened because everyone embraced VMs and cloud services without dwelling on its actual use case, or in what cases it was an improvement over bare metal hosting. I watched similar happen with both the NoSQL and Serverless revolutions, and now we're watching it with containers.
So yes, this would be better setup as a deb package, or deployed as part of a cloudinit startup script or similar. But if you came in when everyone was just shoving docker down each other's throats, you don't know what stages there were before that, and which one is the most appropriate one for your use.
I am all for keeping complexities at a minimum and it's great that it's working for them.
However, Docker is so simple to work with and so valuable in what it delivers that a headline like this reads to me a bit like "We don't need git". Sure, you might not need it. But it would probably be nice.
However, Docker is so simple to work with and so valuable in what it delivers that a headline like this reads to me a bit like "We don't need git". Sure, you might not need it. But it would probably be nice.
> No, the reason we went with go was because golang’s packaging is so much better than node. Or java. Or C#. We get a single binary.
Same can be achieved with node/java/C#. There are 3rd party tools to achieve this for many languages, but at least dotnet has this option built-in: https://docs.microsoft.com/en-us/dotnet/core/whats-new/dotne...
Same can be achieved with node/java/C#. There are 3rd party tools to achieve this for many languages, but at least dotnet has this option built-in: https://docs.microsoft.com/en-us/dotnet/core/whats-new/dotne...
Yeah, fat jars in Java has been a thing for years and, IMO, docker for Java is unnecessary overhead: if you need _containers_, use something like Google’s jib to turn your fat jar into an OCI container.
If you can make your app fit in a single binary and you do not _need_ Docker's features, then yes, I don't see why you'd bother using it. Docker shouldn't be your immediate go-to. It's just another tool that can help with some problems.
Things do become more complicated when your app is a single logical unit that depends on a number of resources / services. Maybe you want an nginx reverse proxy in front of it, a database engine behind it; maybe Go isn't the only language you're going to use. Maybe you want the ability to effortlessly run the exact same setup as your production environment somewhere else without having to run a bunch of Ansible playbooks on the target host, and without worrying about the Linux distribution on it. Docker and docker-compose can help with these use-cases and get you up and running quickly.
I'm not saying Docker is ever a _necessity_. But it can be a great convenience. In the end, use the simplest solution that fits your requirements, and make sure you can extend it later on if those requirements change.
Things do become more complicated when your app is a single logical unit that depends on a number of resources / services. Maybe you want an nginx reverse proxy in front of it, a database engine behind it; maybe Go isn't the only language you're going to use. Maybe you want the ability to effortlessly run the exact same setup as your production environment somewhere else without having to run a bunch of Ansible playbooks on the target host, and without worrying about the Linux distribution on it. Docker and docker-compose can help with these use-cases and get you up and running quickly.
I'm not saying Docker is ever a _necessity_. But it can be a great convenience. In the end, use the simplest solution that fits your requirements, and make sure you can extend it later on if those requirements change.
I don't know, learning Docker probably took me a while, but now that I know how to do it I can spit out a Dockerfile in like five minutes (especially because most of the time its "FROM language COPY app RUN ./build.sh COMMAND ./app" unless you want to get fancy with multi container builds), and a docker-compose for single-server or workstation usage (which is all my hobbies amount to) in 15. The longest time I've spent writing a Dockerfile (because I was trying to build a gigantic Rails app where their build instructions were wrong) was probably a couple hours.
I can understand not wanting to take the time to learn it because growing a business, but if you know how to write a bash script you'll probably pick it up in like a few hours, once. It's not like we're talking about k8s or something, which I can understand not wanting to immerse yourself in for weeks only to wonder why you didn't just pay for Heroku and hire an ops guy in a year to get off of it.
I can understand not wanting to take the time to learn it because growing a business, but if you know how to write a bash script you'll probably pick it up in like a few hours, once. It's not like we're talking about k8s or something, which I can understand not wanting to immerse yourself in for weeks only to wonder why you didn't just pay for Heroku and hire an ops guy in a year to get off of it.
From the comments, it looks like they aren't even using a database and storing data client side ("to be stateless" they say). I don't know what kind of application they are serving but that has a whole can of complications. They are trying to make money yet I assume they don't have accounts etc. That's a bit overkill. So, no database, a single web service, yeah you might not need docker or alike. Depending on their use case they might need high availability so a backup server with a failover would be nice of course, ideally kicking in automatically and doing the necessary notifications.
I've used pretty much this approach at a couple of places, mostly working with Java. A production server is a vanilla Red Hat machine with an aftermarket JDK at a known path. The deployment artifact is a zip file containing jar files (and anything else) and a shell script called run.sh. Deployment is copying, unzipping, and overwriting a symlink. A cron job or a systemd etc service starts it. CI is a cron job which pulls, runs build.sh, and copies build/distribution to an NFS server. It works.
You can also build a fat jar, which is basically a zip file containing everything but that the JVM knows how to load. All the static assets can go in there, too. It saves you the build.sh.
The build.sh is how CI builds the project from source. I suppose you meant that it saves the run.sh - but it doesn't, because you still have to give the path to the JDK, and any JVM command line arguments you want.
Honestly, i've never really got the point of fat jars. Fat jars come with all sorts of annoyances (what do you do with the manifests from each jar?). Unpacking a zip is not hard.
Honestly, i've never really got the point of fat jars. Fat jars come with all sorts of annoyances (what do you do with the manifests from each jar?). Unpacking a zip is not hard.
I thought recent JDKs had the tools to build executable jars with the JVM built in. Can't find a reference now, though.
Perhaps you're talking about jpackage - https://openjdk.java.net/jeps/392
That sounds close, but I thought you got a plain executable out rather than an installer.
> This might end up getting a lot of hate.
no hate from me. it's perfectly fine to run one's stuff without docker.
no hate from me. it's perfectly fine to run one's stuff without docker.
This works for their particular use case because their server doesn't even have a database.
It doesn't address runtime configuration like remote hosts, passwords, etc. It doesn't even address the difference between provisioning and deploying a host.
It doesn't address runtime configuration like remote hosts, passwords, etc. It doesn't even address the difference between provisioning and deploying a host.
Interesting. I wrote an Internal Go Application (not very mission critical) and i literally deployed the binary using supervisor and nginx as a proxy in front. A quick scp command as needed for changes down the line.
Similar to what I do, only I use C++ instead of Go and CMake / make instead of Go Build. Vertical scalability is stellar and exceeds any realistic expectation my own business or those of my clients.
X rant about technology Z because they don't understand it. Of course Docker is not simpler, because you can do so much more with it than copying a single stateless binary to a single machine...
More people need to do this. Focus on growing your business not optimizing your pipeline.
This was posted previously a couple weeks/months ago.