Ask HN: Developer Workstation Backups
51 comments
I just don't store anything locally that I don't mind losing. If we are talking about software development then everything is tracked in git. If I'm still part way through some changes at the end of the day I will just `git commit -am "wip"` and push to a remote branch for safe keeping. Then you can rework the commit[s] locally before opening / flagging the PR for review.
Documents and stuff are all in google docs/drive or something like that, all my dotfiles / config is in a git repo. I even work like this on my personal machine which is also used for gaming. In case of failure I can just wipe the OS drive and start fresh without any worries.
Documents and stuff are all in google docs/drive or something like that, all my dotfiles / config is in a git repo. I even work like this on my personal machine which is also used for gaming. In case of failure I can just wipe the OS drive and start fresh without any worries.
>Documents and stuff are all in google docs/drive or something like that
There have been horror stories of people randomly loosing access to their Google account, probably because some automatic check decided that they violated some term of use. I see how convenient it is to work "in the cloud", but if those are important documents you should keep a local copy as well.
There have been horror stories of people randomly loosing access to their Google account, probably because some automatic check decided that they violated some term of use. I see how convenient it is to work "in the cloud", but if those are important documents you should keep a local copy as well.
[deleted]
Treat your workstation much like one would a Docker container: everything should be ephemerally configured, driven by automation, and reproducible on the fly. Always commit frequently, backup large files to cloud storage, and wipe your device clean on a regular basis.
nice
My workstation backup is a word document that explains how to quickly reproduce a configured workstation, given the usual sort of Windows fresh install we get from IT. This is useful because, far more often than needing a backup restored, we have new people starting, and established folks needing fresh clean Windows images because theirs are trash from 12 months of development. It's just a side benefit that if somebody's machine were to fall into a bathtub, we could deal with that, too.
All documentation and non-code files are in OneDrive, SharePoint, Teams, or good old fashioned shared network drives. IT handles backing that up for us. $Deity bless them, we've never lost anything from a database or file storage.
All documentation and non-code files are in OneDrive, SharePoint, Teams, or good old fashioned shared network drives. IT handles backing that up for us. $Deity bless them, we've never lost anything from a database or file storage.
Arq backup + GSuite/Workspace as a target. Needs fewer resources than Backblaze, and is (was?) cheaper than it.
I had to exclude Dropbox etc. folders from Arq because of the warnings about these online-only placeholder files, and I would prefer it if we could silently ignore those.
Additionally, I recommend a solution that can restore entire volume images in case of boot drive failures. Time Machine or Carbon Copy Cloner on Mac and Macrium on Windows are nice choices in my experience. Acronis was also very reliable in the past, but since then, they have shifted to yearly nonsense upgrades with questionable value. It reminds me of the transition of Nero Burning ROM into the Nero Media-whatever-center-thingy, where the core functionality remained unchanged while more and more auxiliary features were added.
Additionally, I recommend a solution that can restore entire volume images in case of boot drive failures. Time Machine or Carbon Copy Cloner on Mac and Macrium on Windows are nice choices in my experience. Acronis was also very reliable in the past, but since then, they have shifted to yearly nonsense upgrades with questionable value. It reminds me of the transition of Nero Burning ROM into the Nero Media-whatever-center-thingy, where the core functionality remained unchanged while more and more auxiliary features were added.
* Dropbox for documents and important files
* SyncThing (to local Synology and cloud server) for my root git folder (I use the following "style" ~/git/<company>/<project>)
* CarbonCopyCloner (to T7 SSD) running daily for a full clone
* Backblaze for my cloud copy
* Time Machine to my Synology just to be safe
Not sure if that meets "without a lot of fuss" but it's all on auto-pilot for me and I never think about it.
* SyncThing (to local Synology and cloud server) for my root git folder (I use the following "style" ~/git/<company>/<project>)
* CarbonCopyCloner (to T7 SSD) running daily for a full clone
* Backblaze for my cloud copy
* Time Machine to my Synology just to be safe
Not sure if that meets "without a lot of fuss" but it's all on auto-pilot for me and I never think about it.
What is T7 SSD?
A line of small and fast portable SSD disks from Samsung. Random example:
https://www.samsung.com/it/memory-storage/portable-ssd/porta...
https://www.samsung.com/it/memory-storage/portable-ssd/porta...
https://www.amazon.com/SAMSUNG-Portable-SSD-1TB-MU-PC1T0T/dp...
(Not an affiliate link, AFAIK)
(Not an affiliate link, AFAIK)
Portable SSD by Samsung
I too used to use Crashplan! It steadily went downhill over the years though, so I was looking for an alternative.
I ultimately ended up evaluating various options and deciding on Kopia: https://github.com/kopia/kopia -- been using it for the better part of a year and loving it!
It's basically everything I wanted out of the box: content-addressed storage with built-in deduplication, built-in encryption, compression, many supported backends (filesystem, Backblaze B2, other cloud providers, etc...).
I currently take snapshots on a cron every 30 minutes to a triple-replicated GlusterFS cluster, as well as off-site syncs once daily (B2 currently). Because all of the data is encrypted at rest, I don't mind it sitting in a cloud bucket.
I ultimately ended up evaluating various options and deciding on Kopia: https://github.com/kopia/kopia -- been using it for the better part of a year and loving it!
It's basically everything I wanted out of the box: content-addressed storage with built-in deduplication, built-in encryption, compression, many supported backends (filesystem, Backblaze B2, other cloud providers, etc...).
I currently take snapshots on a cron every 30 minutes to a triple-replicated GlusterFS cluster, as well as off-site syncs once daily (B2 currently). Because all of the data is encrypted at rest, I don't mind it sitting in a cloud bucket.
my dev workstation is so simple i don’t even really need to back it up. All my projects are stored on git. Even assets are stored in the cloud somewhere. I can copy sensitive data like recovery keys onto USB sticks, and occasionally back up everything to a hard drive.
But the actual workstation is IntelliJ, git, language tooling, neovim, fish shell, and firefox. The rest are constantly changing based on the project, e.g. language tooling and libraries, and most projects either install these automatically or make them clear to install.
i’ve set up my dev workstation so often it takes me like 30 minutes to go from fresh OS to working on a project. maybe one day ill make a shell script or something but the software i need changes so often it doesn’t really matter.
But the actual workstation is IntelliJ, git, language tooling, neovim, fish shell, and firefox. The rest are constantly changing based on the project, e.g. language tooling and libraries, and most projects either install these automatically or make them clear to install.
i’ve set up my dev workstation so often it takes me like 30 minutes to go from fresh OS to working on a project. maybe one day ill make a shell script or something but the software i need changes so often it doesn’t really matter.
I have around the same - any setup of systems I am working on if will be gone I will just learn to set it up again.
Besides that code is in GIT and passwords are in password manager.
Besides that code is in GIT and passwords are in password manager.
I like https://www.borgbackup.org/ and you can get a relatively cheap storage plan on rsync.net just for Borg (https://www.rsync.net/products/borg.html)
Retrieving backups is a little manual. Perhaps someone has created a nice GUI for it.
Retrieving backups is a little manual. Perhaps someone has created a nice GUI for it.
There are at least two GUI's for borgbackup:
* Vorta (https://github.com/borgbase/vorta)
* Pika Backup for Gnome (https://apps.gnome.org/app/org.gnome.World.PikaBackup/)
Vorta is multiplatform and more reliable by my experience.
* Vorta (https://github.com/borgbase/vorta)
* Pika Backup for Gnome (https://apps.gnome.org/app/org.gnome.World.PikaBackup/)
Vorta is multiplatform and more reliable by my experience.
I just mount the backup repository and then copy what is needed.
I went to restore an old Crashplan backup of my desktop PC before I started traveling. The restore hung for at least a week without downloading anything. I'm guessing it was put on something Glacier like due to age, but there was no indication of how long it was going to take to get the files, and I was left wondering if they even still existed.
Run, don't walk, imo. I swapped to backblaze.
Run, don't walk, imo. I swapped to backblaze.
I use ansible to configure the OS and to maintain my home. From there I am able to spawn my environment within minutes on a fresh installed system.
Local history of data (not backup) is done by scheduled file system snapshots (zfsnap). I use this whenever I accidentally delete files or directories. It saved my a55 at least once within all the years I have it running.
For instant sync of data between laptops and remote systems, I use syncthing (direct connection through ssh tunnels)
And to the question of backup: borgbackup. But only selected directories of my home. The rest can be reinstalled via ansible.
Local history of data (not backup) is done by scheduled file system snapshots (zfsnap). I use this whenever I accidentally delete files or directories. It saved my a55 at least once within all the years I have it running.
For instant sync of data between laptops and remote systems, I use syncthing (direct connection through ssh tunnels)
And to the question of backup: borgbackup. But only selected directories of my home. The rest can be reinstalled via ansible.
Im a dev that moved to devops and use a VM for dev work (or spin up a machine in Cloud). All code is in repos stored on Github, Gitlab etc. I also have a repo that contains scripts that will build the VM using Packer & SaltStack, and a text file (that is stored on usb's) to say how to add packer and any other base tools needed for the VM (saltstack scripts can be used a the cloud machine)
I'm using Kopia + {Wasabi, B2}
Second! I'm also using Kopia + B2 (as well as LAN-based filesystems), it's incredible!
Would like to add a second cloud destination as a redundancy (especially because B2 is so cheap that it's basically free), but I haven't gotten around to it yet. Will check out Wasabi!
Would like to add a second cloud destination as a redundancy (especially because B2 is so cheap that it's basically free), but I haven't gotten around to it yet. Will check out Wasabi!
Any dev related stuff is checked in, so dockerfiles and code, so that’s easy to spin up again.
Then I have a folder in my home called Sync that has a library and resources folder inside. Documents and Pictures etc are linked to library, and resources contains docs or things that I want to keep.
The sync folder is setup in syncthing to my laptop and nas, then the nas also backups up that to Backblaze.
Dotfiles and ansible books are in GitHub, via chezmoi.
Otherwise I don’t mind if I lose anything else.
Then I have a folder in my home called Sync that has a library and resources folder inside. Documents and Pictures etc are linked to library, and resources contains docs or things that I want to keep.
The sync folder is setup in syncthing to my laptop and nas, then the nas also backups up that to Backblaze.
Dotfiles and ansible books are in GitHub, via chezmoi.
Otherwise I don’t mind if I lose anything else.
I have a local NAS (running TrueNAS) and use Time Machine on it for recovery style backups, any important media, documents, etc are synced separately via Syncthing to a separate volume in TrueNAS that is synced to Backblaze on a schedule.
Bonus is that Syncthing is really good for syncing development working directories between machines when you don't want to use Git because unlike dropbox it lets you exclude things like node_modudules.
Bonus is that Syncthing is really good for syncing development working directories between machines when you don't want to use Git because unlike dropbox it lets you exclude things like node_modudules.
I do software development for living and do not have any special backups for dev related files other than git. For general office documents I use Seafile (selfhosted Dropbox equivalent) and have redundant copies of important files on multiple machines and the central Seafile server. If I need real backups with the ability to restore a set of files in a specific point in time I use restic.
Using borg here:
- configured through Borgmatic
- encrypted
- long retention (daily, weekly, monthly)
- stored on a relatively fast and large external backup disk (right now a Sandisk Extreme Pro SSD)
The backup plan is automatically triggered when I connect my notebook to the docking station. Never failed me and already saved me from the embarressment of accidently modifying or deleting important files that are not part of a git repo.
- configured through Borgmatic
- encrypted
- long retention (daily, weekly, monthly)
- stored on a relatively fast and large external backup disk (right now a Sandisk Extreme Pro SSD)
The backup plan is automatically triggered when I connect my notebook to the docking station. Never failed me and already saved me from the embarressment of accidently modifying or deleting important files that are not part of a git repo.
I push my code changes to origin. I copy my media files to a local NAS. Other than that, nothing. In 30 years I've never lost a file. My .vimrc was around 8k at one point, but I pretty much use vanilla vim now and defaults for most things.
I'm not saying this is the correct approach but it has worked for me.
I'm not saying this is the correct approach but it has worked for me.
Since the only thing I care about is my homedir, I just use restic + s3. Add a bunch of CACHEDIR.TAG files to big dirs I don't want to back up, then just run it regularly, or could set via some type of cron.
https://restic.net/
https://restic.net/
I run Fedora and use Deja Dup. It's basically just a frontend for duplicity, which makes it much easier to set up more complicated scheduled backups. The backups are then synced across various devices via Nextcloud, though I imagine any sort of dropbox-esque service would work.
https://rclone.org/ + backblaze b2.
It Works on every desktop and server and takes minimal cpu and memory. Backblaze b2 buckets can be configured to retain overwritten copies, or you can take periodic snapshots within b2.
It Works on every desktop and server and takes minimal cpu and memory. Backblaze b2 buckets can be configured to retain overwritten copies, or you can take periodic snapshots within b2.
The most important stuff is backed up by version control, obviously. For everything else I use Syncthing.
"Is Syncthing my ideal backup application?
No. Syncthing is not a great backup application because all changes to your files (modifications, deletions, etc.) will be propagated to all your devices. You can enable versioning, but we encourage you to use other tools to keep your data safe from your (or our) mistakes."
https://docs.syncthing.net/users/faq.html#is-syncthing-my-id...
No. Syncthing is not a great backup application because all changes to your files (modifications, deletions, etc.) will be propagated to all your devices. You can enable versioning, but we encourage you to use other tools to keep your data safe from your (or our) mistakes."
https://docs.syncthing.net/users/faq.html#is-syncthing-my-id...
Github and my organization's OneDrive account.
The actual computer/OS that I happen to be using is entirely not a concern for me anymore. Even accounting for installing VS2022, office, etc., I can go from zero to productive in about 30-45 minutes.
Fast storage and networks changed everything for me.
The actual computer/OS that I happen to be using is entirely not a concern for me anymore. Even accounting for installing VS2022, office, etc., I can go from zero to productive in about 30-45 minutes.
Fast storage and networks changed everything for me.
Arq Backup, with both a Linux VPS and a local drive as the target. Plus attached Time Machine backup.
I run macOS on an iMac on an external SSD and then have another SSD connected to the machine that is a Time Machine clone of the first drive. The internal drive on the iMac acts as an emergency boot device.
My development "machine" is a virtual one and I just snapshot it and rsync it somewhere. Redundantly somewhat, I also commit my code at the end of the day if possible and push it up.
git, docker vim
nothing else, if my machine crashes, I just install linux, git and docker and jump back into it.
when the source has been checked out by git, I run the build script which fetches the build environment container for me, which contains the interpreters, compilers, libraries and other tools I need.
nothing else, if my machine crashes, I just install linux, git and docker and jump back into it.
when the source has been checked out by git, I run the build script which fetches the build environment container for me, which contains the interpreters, compilers, libraries and other tools I need.
tarsnap, if you want a decent all-in-one solution. It's pricey though.
codeberg.org for code and notes, rsync to local HDDs, SyncThing between workstation, laptop and mobile, Sync.com for things that I need to store privately in the cloud
restic . Use whatever you want for the back end. If there's important stuff on the machine that's not anywhere else, do NOT rely on any one cloud service.
SyncThing, BackBlaze, TimeMachine, and in the future Tarsnap
External HD (automated weekly mirror) + Backblaze (remote)
Datto...I frickin love that company.
Not easily accessible though :(
Not easily accessible though :(
Restic frequently to cloud and external drives.
An external hard drive with Windows file history.
for OSX, network time machine.
for linux, ZFS, and a couple of scripts to do snapshotting and copying the backups to a target host.
for linux, ZFS, and a couple of scripts to do snapshotting and copying the backups to a target host.
borgbackup + local unraid server + cron
So long story short, what are people using to reliably backup their machines without a lot of fuss? Most services don't support Macs or they don't support Linux, or they don't do infinite revisions.