Thanks! I totally get it — I think a lot of time with acquisitions in general there's also a lot of good faith intentions that never manifest for whatever practical reason(s), so I can totally understand why the cynicism arises. I appreciate the positive wishes and I will do my best to convert those into positive outcomes!
I'd point out that most of the foundational components of Docker's technology stack are FOSS (e.g. [0] [1]). Mutagen also has (and had, prior to acquisition) closed-source components and components licensed under non-OSI licenses (e.g. SSPL), so evaluate my response in that context (and also please note that these are my responses, and not representative of Docker's opinions/positions). Also, as mentioned in the acquisition FAQ [2], we don't have any plans to change the open-source licensing structure, and we're still open to contributions.
It's a delicate balance to strike, and almost more delicate to discuss. At the end of the day, an open source business is still a business, and you have to make money to eat, shelter, and continue writing code. I can certainly appreciate that there are different approaches to balancing that with open-source (e.g. consulting rather than close-sourcing), but so long as you're setting natural (non-contrived) boundaries between open-source and proprietary, then I don't think you're doing anything wrong. In fact, several companies were embedding and making money from Mutagen before I was, but that never really bothered me — that was the freedom I was affording them as users.
I have nothing but the utmost respect for user freedom and I've always strived to offer that with Mutagen (e.g. allowing people to disable components that might not fit the OSI definition of FOSS). I also have nothing but the utmost respect for the authors of open-source software; Mutagen stands on the shoulders of many different dependencies and I've always strived to ensure that we're acknowledging those in a manner that is compliant with their licensing requirements (or going above and beyond that).
In the end, it's a tough but fair question. I don't think my views have changed and I don't think there's any real incompatibility. It's a balance I'll necessarily continue to assess on a daily basis, both at Docker and in any other FOSS I write.
I'm really happy with this outcome! I've been a Docker Captain for almost 3 years now, so being an employee at Docker was a very easy switch to flip (at least mentally). As I said in another comment, this really felt like the logical step given that almost all of Mutagen's users are Docker users, and I think this is the best possible outcome for the project too!
Mutagen's core users are primarily Docker users, and I honestly think they'll be much better served by the tighter integrations that we'll be able to offer now. If I didn't think that were the case, and I didn't think there were a net win (for Docker, Mutagen, and their users), then I wouldn't have joined.
Well, that's one of those things I won't answer, but Docker is actually really great about listing nominal salary ranges for their positions on their Docker Careers page [1] - I'd encourage anyone to check out the openings there!
If anyone has any questions about Mutagen (or integrating it into their Docker-based workflows), I'm happy to help.
Just one clarification on the article: Mutagen offers Docker Compose integration, not Composer integration (Composer is a PHP package manager). However, as mentioned, DDEV is a great option if you're looking to do containerized PHP development while using Mutagen for performance improvements.
I can't predict ultra far into the future (who can these days... :|), but Mutagen has been under active development for about 6 years now[0]. At the moment I have enough funding to work on it full-time until at least the middle of next year, though I also do Mutagen-related contracting and consulting work to support the project. Mutagen's Docker Desktop extension is going to be a freemium product designed to support the project more directly, which will hopefully allow development to continue indefinitely.
There's no conflict resolution UI at the moment (either graphical or command-line based). Mutagen's conflict resolution is primarily performed via the specification of its synchronization mode (which can automate the resolution of most conflicts) and (in the case where conflicts can't be resolved automatically) by manually deleting the file that should "lose" the conflict (because Mutagen will synchronize a modification over a deletion).
Mutagen provides fairly detailed reporting of synchronization status, file staging progress, and change application problems via its "mutagen sync monitor" and "mutagen sync list" commands. These also support JSON output (and Go-template-style formatting), so you can pipe this information into other tooling. If you need REALLY detailed information, you can look at the debug or trace-level logs from the Mutagen daemon, but that's typically only for debugging during development of Mutagen itself.
Mutagen is somewhere in between Unison and Syncthing on the topology front, but closer to Unison. It still only supports two endpoints per synchronization session, but unlike Unison it doesn't require that one is local (i.e. you can do remote-to-remote sync using your local system as a proxy). But with both Mutagen (and Unison), you can set up a hub-and-spoke topology if you want to sync with multiple nodes.
One option I often recommend is setting up multiple synchronization sessions targeting different parts of a particular codebase with different synchronization configurations (kept mutually exclusive via ignore specifications). Quite often people want unidirectional replicas for a certain folder (e.g. a build directory) with more standard bidirectional synchronization for the rest of the code. It can be a little more complex to orchestrate, but you can use Mutagen's project functionality or a shell script to automate the setup a bit.
In many ways Mutagen and VSCode's remote extensions are the same idea, with trade-offs in terms of flexibility vs. integration.
Shared systems with multiple non-admin users was one of the original motivating use cases for tighter default permissions.
I don't think there's any scenario where one can perform truly secure development work on an untrusted system. You could certainly store encrypted code in an untrusted location, but there's not much you could do with it on that system (without a hypothetical compiler or tool that maybe supported some sort of homomorphic-encryption compilation operations?). Even decryption on-the-fly for processing by regular tools wouldn't be secure on an untrusted system. And running any code there would be equally insecure.
I'd imagine that for any seriously sensitive work, one would only want to work in highly controlled, trusted, and firewalled environments. If there's a scenario I'm missing though, definitely let me know.
- Mutagen performs bidirectional synchronization (though it can also operate unidirectionally); rsync is unidirectional
- Mutagen uses recursive filesystem watching to avoid full filesystem rescans (whereas rsync always does a full filesystem rescan). This allows Mutagen to provide a more "real time" sync.
- Mutagen has an active synchronization loop that doesn't require manual invocation.
- Mutagen has more idiomatic Windows support.
- Mutagen doesn't require that it be pre-installed on both endpoints.
Both use differential transfers (i.e. the "rsync algorithm") for transferring individual files.
There are other differences, of course, as well as similarities. Mutagen's design is tuned for development work, rsync's design is tuned for replication. I still use rsync for archival operations on a daily basis - it's great!
Conceptually speaking, Mutagen and Unison are very similar (and actually I mentioned Benjamin Pierce's work in another comment here asking about the sync algorithm - fantastic stuff!). I tend to avoid direct comparisons because they always come across one-sided, but some cursory differences:
- Mutagen tries to integrate recursive filesystem watching very tightly into its synchronization loop to drive synchronization and allow for near-instant filesystem rescans
- Mutagen automatically copies an "agent" binary to remote systems to support synchronization, so no remote install is required
- Mutagen uses Protocol Buffers for its data storage, so synchronization sessions created with older versions continue to work with newer versions
- Mutagen written in Go, Unison in OCaml (which allows Mutagen broader platform support "for free")
- Mutagen tries to treat Windows as a first-class citizen
Obviously the internal implementations are different, but both use differential (rsync-style) file transfers, both use the same reconciliation concepts, etc.
Mutagen has the advantage of Go, recursive filesystem watching, and modern POSIX/Windows APIs that didn't exist when Unison was originally written, though some of that functionality has been brought into Unison.
For a comparison with Syncthing (and to some extent Unison), check out this comment[0].
It should work fine. Many users use Mutagen on multi-GB codebases. If we're talking something larger (say 10s of GBs or TB-sized monorepos), then there are some tweaks you can do to make life with Mutagen a little easier. Feel free to reach out to jacob[-at-]mutagen.io if you have a specific use case, or pop over to the Mutagen Community Slack Workspace[0] to chat.
The general philosophy with Mutagen is to (a) delegate encryption to other tools and (b) use secure defaults (especially for permissions).
So, for example, Mutagen doesn't implement any encryption, instead relying on transports like OpenSSH to provide the underlying transport encryption. In the Docker case, Mutagen does rely on the user securing the Docker transport if using TCP, but works to make this clear in the docs, and Mutagen is generally using the Docker Unix Domain Socket transport anyway. When communicating with itself, Mutagen also only uses secure Unix Domain Sockets and Windows Named Pipes.
When it comes to permissions, Mutagen doesn't do a blanket transfer of file ownership and permissions. Ownership defaults to the user under which the mutagen-agent binary is operating and permissions default to 0700/0600. The only permission bits that Mutagen transfers are executability bits, and only to entities with a corresponding read bit set. The idea is that synchronizing files to a remote, multi-user system shouldn't automatically expose your files to everyone on that system. These settings can be tweaked, of course, and in certain cases (specifically the Docker Desktop extension), broader permissions are used by default to emulate the behavior of the existing virtual filesystems that Mutagen is replacing.
Also, while Mutagen's exact implementation is novel in a number of ways, I would be remiss to not point out that huge amount of academic work in this field was done by Benjamin Pierce[0] and later implemented in Unison[1].
The synchronization uses a repeated three-way merge algorithm, very similar to Git's merge when merging branches. It is triggered by recursive filesystem watching, which is also used to accelerate filesystem rescans. It maintains a virtual most-recent-ancestor and uses the two synchronization endpoints as the "branches" being merged. Much like Git has "-X ours" and "-X theirs" options, Mutagen also has automated conflict resolution[0] modes that can be specified. You can find the reconciliation algorithm here[1] (and there are an exhaustive set of test cases in the corresponding _test.go file).
To avoid a large class of race conditions (at least to the extent possible allowed by POSIX and Windows), Mutagen will use `*at` style system calls for all filesystem traversal on POSIX systems, with a similar strategy on Windows.
Also, to avoid race conditions due to filesystem changes between scan time and change-application time, Mutagen will perform just-in-time checks that filesystem contents haven't changed from what was fed into the reconciliation algorithm.
The benchmarks will likely be highly dependent on your use case, but SSHFS-style virtual filesystems (specifically those backed by FUSE) typically have significantly lower performance than something like an APFS/ext4/NTFS filesystem that Mutagen could target with synchronization.
All of your readdir()/stat()/open()/read()-style calls will suffer significantly on virtual filesystems, and unfortunately these get hit a lot by things like IDEs (e.g. when indexing code), compilers, and dynamic language runtimes (especially PHP).
No tool is at fault in this chain, of course, it's a hard problem. Mutagen is able to offer better performance by being a little less dynamic and creating "real" copies of all the files on a more persistent filesystem.
On the synchronization front, Mutagen's only goal is to facilitate the synchronization of files (albeit with a focus on development-related settings and low-latency for a "real time" feel). It doesn't attempt to integrate with any higher-level tooling (except in the cases of Docker Desktop and Compose, which is facilitated via external projects). That sort of tooling, language, and framework-specific integration is a bit outside the project's target scope (and something that becomes very domain-specific).
Mutagen will, however, happily operate between different operating systems and architectures, so things like working with a remote amd64-based Docker engine from your local arm64-based laptop are totally possible.
Also, several external projects (such as DDEV[0] and Garden[1]) do use Mutagen as a low-level component in their stack to provide synchronization that does "know" a bit more about the framework that you're using.