The rsync (remote synchronization) command is a file copy tool that can synchronize files across local storage disks as well as over a network.
It’s prevalent because it’s very good. It is commonly used for backing up files, keeping file servers up-to-date with each other, and for deploying code and assets for web apps to servers.
rsync can and will copy just about every file it can see and will synchronize based on file size and modification date to see what’s changed and what needs to be updated.
The rsync command is BIG, and we’ll get you started with some practical applications that will cover the most common usage scenarios in this guide.
Syntax
rsync OPTIONS SOURCE DESTINATION
Note that:
- OPTIONS are a list of options from the below list, separated by a space
- SOURCE is the path to the source of the file or folder to synchronize and can include the details for a remote host
- DESTINATION is the path to the destination you wish to synchronize the file or folder to and can include the details for a remote host
- SOURCE and DESTINATION must be in order!
Options
Here are the commonly used options for rsync, straight from the manual:
What to copy:
-r, --recursive Recurse into directories
-R, --relative Use relative path names
--exclude=PATTERN Exclude files matching PATTERN
--exclude-from=FILE Read exclude patterns from FILE
-I, --ignore-times Don't exclude files that match length and time
--size-only Only use file size when determining if a file should be transferred
-@ --modify-window=NUM Timestamp window (seconds) for file match (default=0)
--include=PATTERN Don't exclude files matching PATTERN
--include-from=FILE Read include patterns from FILE
How to copy:
-n, --dry-run Perform a trial run with no changes made
-l, --links Copy symlinks as symlinks
-L, --copy-links Transform symlink into referent file/dir
--copy-unsafe-links Only "unsafe" symlinks are transformed
--safe-links Ignore links outside the destination tree
--munge-links Munge symlinks to make them safer
-H, --hard-links Preserve hard links
--devices Preserve device files (super-user only)
--specials Preserve special files
-D, --devices --specials Preserve devices (super-user only) +files
-g, --group Preserve group
-o, --owner Preserve owner (super-user only)
-p, --perms Preserve permissions
--remove-source-files Sender removes synchronized files (non-dir)
-t, --times Preserve times
-S, --sparse Handle sparse files efficiently
-x, --one-file-system Don't cross filesystem boundaries
-B, --block-size=SIZE Force a fixed checksum block-size (default 700)
-e, --rsh=COMMAND Specify rsh replacement
--rsync-path=PATH Specify path to rsync on the remote machine
--numeric-ids Don't map uid/gid values by user/group name
--timeout=SECONDS Set IO timeout in seconds
-W, --whole-file Copy whole files, no incremental checks
Destination options:
-a, --archive Archive mode equals -rlptgoD (no -H,-A,-X)
-b, --backup Make backups (see --suffix & --backup-dir)
--backup-dir=DIR Make backups into this directory
-z, --compress Compress file data during the transfer
-c, --checksum Skip based on checksum, not mod-time & size
-C, --cvs-exclude Auto ignore files in the same way CVS does
--existing Only update files that already exist
--delete Delete files that don't exist on the sending side
--delete-excluded Also delete excluded files on the receiving side
--delete-after Receiver deletes after transfer, not during
--force Force deletion of directories even if not empty
--ignore-errors Delete even if there are IO errors
--max-delete=NUM Don't delete more than NUM files
--log-file-format=FMT Log file transfers using specified format
--partial Keep partially transferred files
--progress Show progress during transfer
-P Equivalent to --partial --progress
--stats Give some file transfer stats
-T --temp-dir=DIR Create temporary files in directory DIR
--compare-dest=DIR Also compare destination files relative to DIR
-u, --update Update only (don't overwrite newer files)
Misc options:
--address=ADDRESS Bind to the specified address
--blocking-io Use blocking IO for the remote shell
--bwlimit=KBPS Limit I/O bandwidth, KBytes per second
--config=FILE Specify alternate rsyncd.conf file (daemon)
--daemon Run as a rsync daemon
--no-detach Do not detach from the parent (daemon)
--password-file=FILE Get daemon-access password from FILE
--port=PORT Specify alternate rsyncd port number
-f, --read-batch=FILE Read batch file
-F, --write-batch=FILE Write batch file
--version Print version number
-v, --verbose Increase verbosity
-q, --quiet Decrease verbosity
-4, --ipv4 Prefer IPv4
-6, --ipv6 Prefer IPv6
-h, --help Show this help screen
For the full list of options, consult the rsync user manual by running:
man rsync
Examples
I won’t repeat what each option means below – make sure you refer back to the list above if you aren’t sure what an option means – you don’t want to run the wrong command and lose any data!
Local to Local (Single File)
Here’s how to synchronize a single file from one directory to another on the local machine using rsync:
rsync -zvh /path/to/my-file.tar /path/to/destination/
From here, all examples will focus on syncing directories, as that is the most common usage scenario for rsync
Local to Local (Directory)
Synchronize all files from one directory to another directory on the same machine:
rsync -avzh /path/to/source /path/to/destination
What’s the -avzh thing? It’s a bunch of options, merged. It’s just a quicker way of typing -a -v -z -h
Local to Remote (Directory)
Synchronize all files from a local source directory to a destination on a remote computer over a network:
rsync -avz /path/to/source/ [email protected]:/path/at/destination/
Note that:
- username is the user on the remote system
- /path/to/source/ is the path to the source directory on the local computer
- 192.168.1.11 is the IP address of the remote system – you can use an IP address or hostname
- /path/at/destination/ is the path on the remote file system
- The synchronization will occur using the rsync daemon
Local to Remote using SSH (Directory)
The -e option lets you specify the protocol used by rsync, allowing us to specify we want to transfer files using SSH securely:
rsync -avzh -e ssh /path/to/source/ [email protected]:/path/at/destination/
Remote to Local (Directory)
Synchronize all files from a source directory on a remote computer to a destination on the local computer over a network:
rsync -avzh [email protected]:/path/to/source /path/to/destination
Note that:
- username is the user on the remote system
- 192.168.1.11 is the IP address of the remote system – you can use an IP address or hostname
- /path/to/source is the path on the remote file system
- /path/to/destination is the destination on the local computer
- The synchronization will occur using the rsync daemon
Remote to Local using SSH (Directory)
rsync -avzh -e ssh [email protected]:/path/to/source /path/to/destination
You can also sync single files from local to remote, remote to local, and using SSH, just as we’ve shown with directories
Other Things to Consider When Synchronising
Consider Bandwidth
If you are synchronizing over the Internet, you may want to limit the bandwidth used so that you don’t slow down other services on your network:
rsync -avz --bwlimit 512 /path/to/source/ [email protected]:/path/at/destination/
Consider Deletions
If a file or folder has been removed at the source but exists at the destination, you may want to delete the file or folder at the destination so that it is an exact mirror of the source. Always make sure you’re certain you want things being deleted before using this option:
rsync -avz --delete /path/to/source/ [email protected]:/path/at/destination/
Including or Excluding Files
Include and exclude files by matching file patterns – if there are files you don’t want to sync, they can be left out:
rsync -avz -e ssh --exclude '*' --include 'keep*' /path/to/source/ [email protected]:/path/at/destination/
This rsync command will sync files and directories only, which starts with ‘keep-’ and exclude all other files and directories (*).
Consider a Dry Run
A dry run will print out the files that will be changed, but it won’t actually do anything – no files will be synchronized or altered, so you can experiment with different options and see the effects:
rsync -avz --dry-run /path/to/source/ [email protected]:/path/at/destination/
Conclusion
When moving a large number of files around on your systems, make sure you check your input before hitting the enter key – you don’t want to synchronize in the wrong direction and remove important files!
