ag@linux:~$

history 27 of 36

Rsync over SSH and the Trailing Slash Question

Copying folders between servers with rsync over SSH, using a custom port and key, and how a trailing slash on the source changes where files end up.

Written

From my Linux study notes, 2016. Some commands and versions here are old, so please reuse them carefully. Anything marked "added later", and the notes at the end, explain what has changed since.

rsync copies files and folders, and on later runs it only sends what has changed. Run over SSH, the transfer is encrypted. These are my notes on how I use it.

Before you start

  • rsync must be installed on both machines, not just the one you run it from.
  • You need an SSH login to the remote machine.
  • Test with a folder you can afford to lose until you are sure of the command.

Using the rsync command

Terminal window
rsync -Pzarve ssh -p (port) -i (ssh key location) /local/dir/ user@hostname:/remote/dir/

What the options mean:

  • -P shows progress (and keeps partly transferred files)
  • -z compresses data during the transfer
  • -a archive mode: keeps permissions, times, symbolic links and so on
  • -r recursive: copies folders and everything in them
  • -v verbose: lists what is being copied
  • -e the remote shell method to use, here ssh

Correction (added later): in the line above, -p and -i would be read as rsync options, not SSH options. When you pass options to SSH, put the whole SSH command in quotes after -e, as in the next example. Also, -a already includes -r, so -r is harmless but not needed.

If you are using an RSA key, do this:

Terminal window
rsync -Pzarve "ssh -i /location/rsa_key -p 1234" /source /destination

Here 1234 is the SSH port and /location/rsa_key is the path to your private key. A full example, with placeholders for your own values:

Terminal window
rsync -Pzav -e "ssh -i ~/.ssh/id_ed25519 -p 1234" /local/dir/ user@hostname:/remote/dir/

If your server uses the normal SSH port (22) and your default key, plain -e ssh is enough, and recent rsync versions use SSH by default anyway.

Done this way, you have to type a password every time you run rsync. If you do not want to enter a password every time, set up passwordless SSH login. See my other post: Passwordless SSH login.

To be / or not to be: that is the big question

Terminal window
#1: rsync -Pzarve /local/dir/path/ /remote/other-dir/
#2: rsync -Pzarve /local/dir/path /remote/other-dir/

Answer:

  1. #1 (with a trailing / on the source) will make /remote/other-dir mirror the contents of /local/dir/path/.
  2. #2 (no trailing /) will create a directory /remote/other-dir/path/ whose contents mirror /local/dir/path/.

Only the slash on the source matters. A slash on the destination makes no difference.

Note (added later): these two examples use -e with no value straight after it, so rsync would take the next word as the shell command. For local copies like these, drop the e: rsync -Pzav /local/dir/path/ /remote/other-dir/.

Be warned! If you are not careful and use #1 when you meant #2, you can lose files in /remote/other-dir.

Note (added later): plain rsync does not delete files on the destination; it can overwrite files that have the same name. Files are only deleted if you add --delete, so take extra care with the slash when you use that option.

Another method

Terminal window
rsync -Pzarv /local/dir/path/. /remote/other-dir/

Ending the source with /. has the same effect as the trailing slash in #1: the contents of the folder are copied, not the folder itself.

Check before you copy

Add -n (--dry-run) to see what rsync would do without copying anything:

Terminal window
rsync -avn /local/dir/path/ user@hostname:/remote/dir/

When the list looks right, run the same command without -n.

Notes for today

  • Modern SSH keys are usually ed25519 keys, created with ssh-keygen -t ed25519. They work with rsync exactly like RSA keys.
  • If you connect to the same server often, you can put the port and key in ~/.ssh/config so you do not need them on every command. See man ssh_config.
  • Documentation: https://rsync.samba.org/