I recently implemented a backup workflow for me. I heavily use restic for desktop backup and for a full system backup of my local server. It works amazingly good. I always have a versioned backup without a lot of redundant data. It is fast, encrypted and compressed.
But I wondered, how do you guys do your backups? What software do you use? How often do you do them and what workflow do you use for it?
I’ve found that the easiest and most effective way to backup is with an rsync cron job. It’s super easy to setup (I had no prior experience with either rsync or cron and it took me 10 minutes) and to configure. The only drawback is that it doesn’t create differential backups, but the full task takes less than a minute every day so I don’t consider that a problem. But do note that I only backup my home folder, not the full system.
For reference, this is the full line I use: sync -rau --delete --exclude-from=‘/home/<myusername>/.rsync-exclude’ /home/<myusername> /mnt/Data/Safety/rsync-myhome
“.rsync-exclude” is a file that lists all files and directories I don’t want to backup, such as temp or cache folders.
(Edit: two stupid errors.)
Rsync can do incremental backups with a command-line switch and some symlink jugglery. I’m using it to back up my self-hosted stuff.
You might be interested in “rsnapshot” which uses rsync and manages daily, monthly, etc. snapshots.
only drawback is that it doesn’t create differential backups
This is a big drawback because even if you don’t need to keep old versions of files, you could be replicating silent disk corruption to your backup.
It’s not a drawback because rsync has supported incremental versioned backups for over a decade, you just have to use the --link-dest flag and add a couple lines of code around it for management.
Sure, but that’s not in their answer.
They didn’t provide an rsync example until later in the post, the comment about not supporting differential backups is in reference to using rsync itself, which is incorrect, because rsync does support differential backups.
I agree with you that not doing differential backups is a problem, I’m simply commenting that this is not a drawback of using rsync, it’s an implementation problem on the user’s part. It would be like somebody saying “I like my Rav4, it’s just problematic because I don’t go to the grocery store with it” and someone else saying “that’s a big drawback, the grocery store has a lot of important items and you need to be able to go to it”. While true, it’s based on a faulty premise, because of course a Rav4 can go to the grocery store like any other car, it’s a non-issue to begin with. OP just needs to fix their backup script to start doing differential backups.
My one and only purpose was to warn them that their “drawback” is more of a gator pit. It’s noble that you’re here defending rsync’s honor, but maybe let them know instead? My preferred backup tool has “don’t eat my data” mode on by default.
Borg to a NAS.
500GB of that NAS is “special” so I then rsync that to a 500GB old laptop hdd, of which is is duplicated again to another 500GB old laptop hdd.
Same 500GB rsync’d to Cloud Server.
-
daily important stuff (job stuff, Documents folder, Renoise mods) is kept synced between laptop, desktop and home server via Syncthing. A vimwiki additionally also syncs with the phone. Sync happens only when on home network.
-
the rest of the laptop and desktop I’ll roll into a tar backup every now and then with a quick bash alias. The tar files also get synced onto home server’s big file system (2 TB ssd) via Syncthing. Home server backs itself up on it’s own once a week.
-
clever thing is that the 2 TB ssd replaced an old 2 TB spinning disk. I kept the old disk and set up a systemd thing that keeps it spun down, but starts and mounts it once a week and rsyncs the changes to the ssd over, then unmounts it so that it sleeps again for a week. That old drive is likely to serve for years still with this frugal use.
How do you make sure the disk spins down? Is unmounting enough?
Unmounting is enough if the disk has spindown configured. I’ve got this in /etc/udev/rules.d/ :
ACTION=="add", SUBSYSTEM=="block", KERNEL=="sd[a-z]", ENV{ID_SERIAL_SHORT}=="S2H7J9FZB02854", RUN+="/usr/bin/hdparm -S 70 /dev/%k"
-
I use borg the same way you describe. Part of my nixos config builds a systemd unit that starts a backup on various directories on my machine at midnight every day. I have 2 repos: one to store locally and on a cloud backup provider (borgbase) and another thats just stored locally. That is, another computer in my house. That local only is for all my home media. I havent yet put the large dataset of photos and videos on the cloud or offsite.
My work flow is pretty similar to yours:
For my desktop and laptops: systemd timer and service that backups every 15 minutes using restic to my NAS.
For my NAS : daily backup using restic + ZFS snapshots.
All restic backups are then uploaded daily to Backblaze B2.
I recently bought a storagebox from Hatzner and set up my server to run borgmatic every day to backup to it.
I’ve also discovered that Pika Backup works really well as a “read only” graphical browser for borg repos.
Do you use some kind of encryption on the VPS?
Yep, borgmatic encrypts it before it sends data to the server.
Backup? What?
Your car.
i do backups of my home folder with Vorta, tha uses borg in the backend. I never tried restic, but borg is the first incremental backup utility i tried that doesnt increase the backup size when i move or rename a file. I was using backintime before to backup 500gb on a 750gb drive and if I moved 300gb to a different folder, it would try to copy those 300gb again onto the backup drive and fail for lack of storage, while borg handles it beautifully.
as an offsite solution, i use syncthing to mirror my files to a pc at my fathers house that is turned on just once in a while to save power and disc longevity.
My systems are all on btrfs, so I make use of subvolumes and use
brkbkto backup snapshots to other locations.This looks a bit like borgbackup. It is also versioned and stores everything deduplicated, supports encryption and can be mounted using fuse.
Thanks for your hint towards borgbackup.
After reading the Quick Start of Borg Backup they look very similar. But as far as I can tell, borg can be encrypted and compressed while restic is always. You can mounting your backups in restic to. It also seems that restic supports more repository locations such as several cloud storages and via a special http server.
I also noticed that borg is mainly written in python while restic is written in go. That said I assume that restic is a bit faster based on the language (I have not tested that).
It was a while ago that I compared them so this may have changed, but one of the main differences that I saw was that borg had to backup over ssh, while restic had a storage backend for many different storage methods and APIs.
I haven’t used either, much less benchmarked them, but the performance differences should be negligible due to the IO-bound nature of the work. Even with compression & encryption, it’s likely that either the language is fast enough or that it’s implemented in a fast language.
Still, I wouldn’t call the choice of language insignificant. IIRC, Go is strongly typed while Python isn’t. Even if type errors are rare, I would rather trust software written to be immune to them. (Same with memory safety, but both languages use garbage collection, so it’s not really relevant in this case.)
Of course, I could be wrong. Maybe one of the tools cannot fully utilize the network or disk. Perhaps one of them uses multithreaded compression while the other doesn’t. Architectual decisions made early on could also cause performance problems. I’d just rather not assume any noticeable performance differences caused by the programming language used in this case.
Sorry for the rant, this ended up being a little longer than I expected.
Also, Rust rewrite when? :P
My conclusion after researching this a while ago is that the good options are Borg and Restic. Both give you incremental backups with cheap timewise snapshots. They are quite similar to each other, and I don’t know of a compelling reason to pick one over the other.
As far as I know, by definition, at least restic is not incremental. It is a mix of full backup and incremental backup.
Since most of the machines I need to backup are VMs, I do it by the means of hypervisor. I’d use borg scheduled in crontab for physical ones.
Borg Backup, whenever I feel like it - usually monthly.
I keep all of my documents on a local server so all that is on any of my computers is software. So if I need to reinstall Linux I cab just do it without wording about losing anything.
- Offline Backup on 2 separate HDD/SSD
- Backup on HDD within my desktop pc
- Backup offsite with restic to Hetzner Storage Box
