> We build packages in package groups, to take into account separate packages that need to be upgraded together. Once the whole group has been built, we run a virtualized test suite to make sure none of our core components and developer workflows are broken. Each group is tested separately with a full system installation, boot and local test suite run on that version of the operating system.
> We then proceed to carefully guide this release to the fleet utilizing SRE principles like incremental canarying and monitoring the fleet health.