For a more full fledged use case FRRouting may be the way to go. We are using Quagga here mainly to maintain internal routes with iBGP and route reflectors. This is a fairly simple use case.
Quagga is available in the default package managers of most distros so its a good place to start.
We have been working on a simplified container build system which does away with layers altogether. [1]
The use of layers at the build stage adds a lot of needless complexity with very little benefits and users
really need to step back and question the value they are getting from the use of layers. [2]
Words like 'immutability', 'declarative' and 'reproduciblity' are often used in ways that can lead to user misunderstanding and can be accomplished with simpler workflows. For instance immutability, reuse, composition do not require layers. There needs to be a lot more technical scrutiny to avoid confusion.
We need to spin up a lot of VMs of various Linux distributions for testing Flockport and we found the existing tooling too heavy and involved.
In the end we rolled our own solution that is much more lightweight, uses QEMU directly and provides better visibility, ease of use and even some potentially useful networking features. Here are a few screencasts [1].
This is mainly designed as a module for Flockport and we will preview this soon.
What is wrong with Docker? This does not address any technical or other shortcoming and only seeks to replace one set of over engineered tools with another with the exact same problems. [1]
The is yet more of the ecosystem continuing to push over engineered tooling and 'winners' breathlessly without basic technical scrutiny that leaves end users dealing with needless complexity and debt.
Containers can be useful as a lightweight, efficient alternative to VMs and those who want containers untouched by questionable ideas should try the LXC project on which all this was based.
Any additional layer on top of this be it non standard OS environment or layers should meet technical scrutiny for end user benefits and most users will be surprised by the results.
This is where containers can help. Ubuntu ships with LXC and its relatively simple to download an Alpine Linux container and then install Nginx and you are set. Alternatively Flockport [self plug] provides an open source app store [1] for LXC and you can download a prebuilt Nginx114 container all set to go.
But containers are some work with both upsides and downsides and upsides compared to having it out of the box in the distribution.
Layers are integral in Docker, layers were pushed as enabling reuse, composibility, 'immutability', every 'run' statement is a layer. They are not an implementation detail. Managing all these layers adds complexity and impact users from choosing storage drivers, to using layers to building images, and if you are going to use layers you need to be a privileged user to mount all these layers. [1]
Here is an example. I build an Nginx container with no layers from a base alpine image. Users can download and run it in a layer so the original Nginx container remains untouched or run a copy of it. If there is a security alert or update I rerun the build. No layers used. You build the same Nginx container will multiple layers, and have all the overhead of tracking and managing these layers. And users then download the Nginx container made up of all these layers and run it. If there is a security update in any lower layers it needs to rebuild. You have used a workflow and designed an entire system based on stacking layers to compose and build your image. That is a lot of overhead and complexity in basic image management. What is the benefit?
Docker was based on LXC and there are some important differences that should be more well known.
The main difference is LXC boots an init in the namespace as the first process so you get a standard OS environment with multi-process support. Docker runs the application directly so you get a single process environment, unless you decide to run a process manager. IMHO this creates far more problems than it solves as anything nonstandard would, as you now need workarounds for basics like logging, networking, daemons and anything that expects a standard OS environment
Docker also uses layers to build containers and ephemeral data. None of these 3 critical decisions have seen much technical scrutiny resulting in persistent confusion in the ecosystem and with scrutiny many users will find they can add far more management overhead and complexity to containers. We have written about some of these issues here. [1]
For instance OP is referring to unprivileged containers and user namespaces that let you run containers as a user process. LXC has supported these since 2013. Docker added user namespaces support in recent versions. However taking just one issue - because of the use of layers to build containers you can't use unprivileged containers for app builds as only root users can mount filesystems. So a lot of these decisions have all kinds of tradeoffs that should be better known and discussed. [2]
Disclaimer: Founder at Flockport and trying to build a simpler alternative with LXC with support for app builds, provisioning, orchestration, service discovery and an app store. [3]
Layers are interesting but the benefits of layers are vastly overstated, you can achieve the same without layers and the overhead and complexity in image management. Piling layers upon layers to build a container is a really bad idea and adds a lot of complexity in basic image management. [1]
The big problem is technical decisions like the use of layers, single process containers and ephemeral storage that add a lot of complexity and overhead have not met technical scrutiny and thus not well understood. But they should be as they can add a lot of management overhead and debt. [2]
Many users who use LXC will realize a lot of benefits and flexibility actually flow from containers and not these 'modifications'.
Disclaimer: Founder at Flockport and trying to build a simpler alternative with LXC containers.
Thanks for the feedback. This is far from negative. There is definitely value in comparative docs that lets users get a quick overview, and we will put one together.
However we also want to start new conversations in the community about containers and orchestration and reach users who find existing platforms too complex. And we have put together a lot of content to kindle these discussions.
More technical scrutiny could lead to simplification and lower barriers for both users and developers.
Glad to hear you are aware of Flockport. Flockport was initially conceived as an app store for LXC containers. This was still in the early days of containers when the awareness was just building.
We always wanted to do more and our early vision was for a container management platform that is more accessible and easier to use and now of course we have built out a whole platform that delivers a lot more functionality and capabilities, and hopefully delivers on some of that early vision.
We are pleased to preview Flockport with the HN community. Flockport is a new container management platform designed to deliver a lot of functionality and ease of use. We designed this to be simple, it was the core driving force and we hope we have succeeded.
Flockport lets you quickly provision servers and deploy and manage containers across servers, supports overlay networks, distributed storage, service discovery, load balancers and high availability.
Flockport also provides a container build system and an app store. Apps can be deployed cluster wide. We also built in support for rolling out things like databases across clusters. Users can add and remove both database instances and also individual databases cluster wide. There is also similar support for web servers and load balancers builtin.
We are eager to have the community give it a go and get their early feedback and support. For those who do not want to install it there are preinstalled VMs on the downloads page and also screencasts on the resources page.
You can try LXC containers by Ubuntu, this is what Docker was based on initially. The main difference is LXC runs an init in the container so you get a standard multi process OS environment while Docker containers are single process environments.
We have been working on Flockport [1] that supports LXC containers and provides orchestration, an app store, service discovery and repeatable builds. It's still in early preview and we have not started proper outreach but it may be worth looking at.
Ubuntu also provides the LXD project that provides some orchestration across servers.
We have a guide [1] up on using Wireguard and go into some detail on the configuration and how it works. But we mainly use it to build container networks.
An app store, provisioning servers, overlay networks with vxlan, bgp and wireguard, distributed storage, service discovery and a build and packaging system.
We have tried to provide a lot of documentation so do visit if you want to learn more.
LXD is excellent and is by the authors of LXC. A lot of users may not need a lot of the functionality Flockport provides.
Flockport uses a standard networking bridge served by a Dnsmasq instance that all containers connect to. They all get their IP by DHCP unless you set static IPs and can be discovered by their name on local systems.
The DB command basically rolls out a fresh Mysql or Postgresql container instance and sets up the databases. The discovery happens over dns.
Flockport supports LXC so both aufs and overlayfs are available for use but not enforced. Flockport let's you launch containers in a layer so its used at run time if required but not to build containers.
Layers are interesting but they are still maturing and have hard to detect bugs and incompatibilities [1]. The more layers the worse it becomes so they can add management overhead. Containers and layers are separate technologies so its useful to leave it as a choice.
The benefits have also often been oversold. For instance reuse, all container platforms provide a library of base OS images. These can simply to used as required instead of trying to use layers. How many upper layers are there going to be on top the base OS that can be reused? This sounds good as an idea but often does not pan out, usually its just the base OS or base OS plus dev environment being reused so why use layers?
And If there are updates to any of the lower layers for instance security or version updates usually the container needs to be rebuilt so again you are not benefiting from using layers to build containers.
Using it at run time like you would run a copy of a container to keep the original intact still makes sense, but using it to build containers adds a lot of complexity and management overhead.
Unfortunately Selinux can interfere with processes in weird ways without clear messages to end users. When a container starts networking devices are created, if using layers or btrfs/zfs overlays or snapshots may be created, bind mounts activated. There is a lot of potential for permission issues.
Similarly when creating overlay networks ports across systems need to be open. The idea behind this is users can ensure the functionality is working as desired before enabling firewalls and other security features so they can debug issues effectively.
We have tried to provide a lot of documentation so new users can get started and get comfortable with containers and networking. Often users get discouraged if even after following the docs they run into issues.
Hi, the build system is quite flexible. It's used to build all the open source apps currently available in the app store.
That DB keyword basically allows you to roll out a linked database container for your app if required. Only Mysql and Postgresql is currently supported. These are Mysql and Postgresql instances that can be used for this.
A lot of apps require databases and instead of configuring it manually this allows some degree of automation so a linked database container can be easily deployed if required.
Quagga is available in the default package managers of most distros so its a good place to start.