Skip to main content
 
I use #Syncthing on a #Pi that sits in some place (not home) behind a firewall, I installed #Tor on it and can SSH to it via a hidden service, very convenient, so now that #RPi got a 2TB external HDD that has to be mounted manually (due to encryption). The #RPi is used for #backup and it works like charm, it is up without a problem for over a month now and keeps some tens of GB now.
Really very nice thing. I guess you can also use #Rsync via the hidden service, but that wouldn't be fast. The thing is, this way you can take the Pi to any place, hook it to the LAN there, wait 5 minutes (max) and SSH into it from all over the world. Off-site backup made easy, just drop the Pi at your grandmothers, hook it up to the router and go.

You can also do this without the external HDD and use a 512 GB micro SD.
#syncthing #pi #tor #rpi #backup #rsync
Ɽł₵Ⱨ₳ⱤĐ
Ɽł₵Ⱨ₳ⱤĐ 3 months ago
I also know #syncthing and I love it!
#syncthing
DanTheFan
DanTheFan 3 months ago
@Utzer
im thinking about replacing my pi. it regularly dies out of no reason. im tired...
@utzer
utzer
utzer 3 months ago
I had (maybe still have) a problem like that with an other Pi, I bought a new one and that one is running just fine.
The older one now got a bigger micro SD, but I was to lazy to get is configured, but I will do that next week I think.
I am not sure why it died that often, but I think there is some Pi that have problems with the internal power supply.
@danthefan
Stephen Judge
Stephen Judge 3 months ago
I don't suppose you documented your process of setting this up, did you?
mezzodrinker
mezzodrinker 3 months ago
@Stephen Judge Setting up Syncthing is a very straightforward process which is also documented. Adding a Tor hidden service to connect via SSH is a different story, though adding that capability probably wouldn't be necessary if you keep the backup drive mounted.
@stephen judge
MacLemon
MacLemon 3 months ago
@utzer I built something rougly related only that the remote machine would NAT punch upon the first SSH connection and I'd automatically tunnel through that one after the initial SSH over Tor trigger. (Full speed, less latency, still discoverable from everywhere. Needs a spot for connecting to on demand though.)

Also, given your latency very quite high over Tor, you may want to skip rsync.
You could be interested in restic though.

restic/restic

restic - Fast, secure, efficient backup program
github.com
@utzer
setThemFree
setThemFree 3 months ago
Are you sure that Syncthing is enough for a backup? My main concern is that it merely mirrors your live data somewhere else. But what if you accidentally delete something and only notice it later, say a week later? The deletion by that time would have been propagated by Syncthing to the other drive and the data would have been deleted there as well. I think a backup solution should also keep historical backups for a configurable period of time.
mezzodrinker
mezzodrinker 3 months ago
@setThemFree I don't have access to a Syncthing client at the moment, but IIRC, there was an option to have a trash bin of sorts if any files are deleted.
@setthemfree
setThemFree
setThemFree 3 months ago
Maybe, but what about accidentally overwritten files?

In general, I think Syncthing and/or rsync themselves are not enough for a reliable backup. The ability to travel back in time is crucial. I have had a situation when I learned that a file was deleted only a year after the fact. Luckily, I have had obnam snapshot of the data. Everything deduplicated and stored efficiently.
setThemFree
setThemFree 3 months ago
(Since obnam was retired I have moved to borg-backup).
utzer
utzer 3 months ago
Syncthing can be configured to keep revisions of each file, I use staggered file revisions, for different durations (90 days, 180 days), these will thin out the older revisions, have a look in the documentation linked in the comments. My initial idea was to combine this with ZFS/BTRFS snapshots, but I did not implement that yet, I could at any time.

Syncthing can be reached through the firewall, there is relays out there that it will connect to, so it is indirect connection then, but it works fine. Data transferred is encrypted, but you can also disable the relay function.

I use the tor hidden service because it is just nice, also in the LAN, to just plugin the Pi, then wait 2 minutes and it is available by SSH over the .onion address, this spares me from trying to find the local IP. Yeah I know how to do that by a network scan.

Also it is more fault prove, but I learned that Tor updates do not work fine when done in a screen and being connected via the hidden service. 🙁
Solution for this: sudo apt-get update && sudo apt-get install -o ... Show more...
setThemFree
setThemFree 3 months ago
Syncthing can be configured to keep revisions of each file, I use staggered file revisions, for different durations (90 days, 180 days), these will thin out the older revisions, have a look in the documentation linked in the comments.
Thanks, good to know.
Solution for this: sudo apt-get update && sudo apt-get install -o Dpkg::Options::="--force-confold" -y tor
Thanks for the tip.
I was always looking for a backup script that allows me to do a backup via some SSH connection to some server, it should be run daily and retain revision of each run up to a defined maximum age while using using hard links (I think that would save space in the destination), but then I would need a more performance computer that a RPi, because we are talking about a few hundred GB.
I use borg-backup to make local backups of ~300 GB amount of data. It deduplicates everything while in the same time giving me a complete snapshot of each backup. I haven't tested it over a network, but backing up loc... Show more...
utzer
utzer 3 months ago
Yes I guess borg it will need CPU resources on the destination, I assume it does compare the files (checksum) before transfer. I will have a look in it, because I need to set up some backup for home folder and some system folders and bord sounds interesting. Thanks.
setThemFree
setThemFree 3 months ago
Please ping back about your findings.
⇧