Hello selfhosted! Sometimes I have to transfer big files or a large amounts of small files in my homelab. I used rsync but specifying the IP address and the folders and everything is bit fiddly. I thought about writing a bash script but before I do that I wanted to ask you about your favourite way to achieve this. Maybe I am missing out on an awesome tool I wasn’t even thinking about.
Rsync and NFS for me.
And me.
Magic wormhole is pretty dead simple https://magic-wormhole.readthedocs.io/en/latest/welcome.html#installation
I use this a lot at work for moving stuff between different test vms, as you don’t need to check IPs/hostnames
As I understand it, the establishing of the connection is reliant on a relay server. So this would not work on a local network without a relay server and would, by default, try to reach a server on the internet to make connections.
Syncthing and/or ftp.
I work from home, however my two systems (home and work) are on the same LAN, they don’t see each other for file sharing. I get paid via direct deposit like everyone else which means my pay stubs are all electronic. I print those out and then use WinSCP to copy those over to my desktop. No other files are ever sent.
At home, depending on the amount of files, I either use SFTP via Filezilla, or if the mood strikes me and for a single file, I will just use SCP if I’m already on the cli which is most of the time it seems anymore doing work on my personal servers. I’ve found that SFTP is faster at transferring than doing a copy/paste to the NFS share to the same drive.
smb share if its desktop to desktop. If its from phone to PC, I throw it on nextcloud on the phone, then grab it from the web ui on pc.
Smb is the way to go if you have identity set up, since your PC auth will carry over for the connection to the smb share. Nextcloud will be less typing if not since you can just have persistent auth on the app / web.
By “homelab”, do you mean your local network? I tend to use shared folders, kdeconnect, or WebDAV.
I like WebDAV, which i can activate on Android with DavX5 and Material Files, and i use it for Joplin.
Nice thing about this setup is that i also have a certificate secured OpenVPN, so in a pinch i can access it all remotely when necessary by activating that vpn, then disconnecting.
I have a shared syncthing folder on all my devices
Ye old samba share.
But I do like using Nextcloud. I use it for syncing my video projects so I can pick up where I left off on another computer.
Samba Bamba!!
Resilion Sync
WinSCP for editing server config
Rsync for manual transfers over slow connections
ZFS send/receive for what it was meant for
Samba for everything else that involves mounting on clients or other servers.
As a lazy person, I just prefer
sftpon thunar.Not gonna lie, I just map a network share and copy and paste through the gui.
Sounds very straight forward. Do you have a samba docker container running on your server or how do you do that?
I dont have a docker container, I just have Samba running on the server itself.
I do have an owncloud container running, which is mapped to a directory. And I have that shared out through samba so I can access it through my file manager. But that’s unnecessary because owncloud is kind of trash.
I have two servers, one Mac and one Windows. For the Mac I just map directly to the smb share, for the Windows it’s a standard network share. My desktop runs Linux and connects to both with ease.
I just type
sftp://[ip, domain or SSH alias]into my file manager and browse it as a regular folderYOU CAN DO THAT???
Dolphin?
Any file manager on Linux supports this
Set up smb on my file share VM.
My dedicated docker host accesses it through an NFS mount.
Same lol, somebody please enlighten me on a faster way!
SFTP! 😃
What’s wrong with rsync? If you don’t like IP addresses, use a domain name. If you use certificate authentication, you can tab complete the folders. It’s a really nice UX IMO.
If you’ll do this a lot, just mount the target directory with sshfs or NFS. Then use rsync or a GUI file manager.
I never even set up DNS for things that aren’t public facing. I just keep /etc/hosts updated everywhere and ssh/scp/rsync things around using their non-fqdn hostnames.
You could also use mDNS to the same effect.
rclone. I have a few helper functions;
fn mount { rclone mount http: X: --network-mode } fn kdrama {|x| rclone --multi-thread-streams=8 --checkers=2 --transfers=2 --ignore-existing --progress copy http:$x nas:Media/KDrama/$x --filter-from ~/.config/filter.txt } fn tv {|x| rclone --multi-thread-streams=8 --checkers=2 --transfers=2 --ignore-existing --progress copy http:$x nas:Media/TV/$x --filter-from ~/.config/filter.txt } fn downloads {|x| rclone --multi-thread-streams=8 --checkers=2 --transfers=2 --ignore-existing --progress copy http:$x nas:Media/Downloads/$x --filter-from ~/.config/filter.txt }So I download something to my seedbox, then use
rclone lsd http:to get the exact name of the folder/files, and runtv "filename"and it runs my function. Pulls all the files (based on filter.txt) using multiple threads to the correct folder on my NAS. Works great, and maxes out my connection.
