The Power of RSYNC

I first encountered Rsync several years ago during my early days as a system administrator while deploying servers on Linux systems. Before that, my knowledge of file transfers was limited to conventional methods like SCP, FTP, and SFTP, often aided by client applications such as FileZilla and Cyberduck.

However, as my responsibilities grew, I encountered a unique challenge. I needed to move a massive 80GB file from one Linux server to another. SCP, which had served me well for smaller files, was no longer a practical option with internet speeds ranging between 2-7MB/s. This led me to explore alternative solutions, and my quest for a more efficient file transfer method began.

I embarked on a series of Google searches, aiming to find a way to copy a large number of files quickly between two servers, copy files from a local computer to a remote server, and transfer files from one remote server to another. My search didn’t take long; I stumbled upon Rsync.

Rsync proved to be a game-changer. After meticulously following the instructions on how to use it, I successfully moved that 80GB file within a remarkably short period. Below is a sample of how to use Rsync:

Use Rsync to move files from a local computer to a remote server with SSH Key Authentication

rsync -avz -e "ssh -i /path/to/your/key.pem" /path/to/local/source user@server:/path/to/remote/destination

Use Rsync to move files from a local computer to a remote server with Password Authentication

rsync -avz /path/to/local/source user@server:/path/to/remote/destination

Use Rsync to move files from a remote server to another remote server with Password Authentication

sshpass -p 'your_password' rsync -avz -e "ssh -o StrictHostKeyChecking=no" user1@server1:/path/to/source user2@server2:/path/to/destination

Use Rsync to move files from a remote server to another remote server with SSH Authentication

rsync -avz -e "ssh -i /path/to/your/key.pem" user1@server1:/path/to/source user2@server2:/path/to/destination

Notables

• Rsync: The primary command for synchronizing files and directories.
• -avz: These options and flags are used with Rsync:
•    – -a: Abbreviation for “archive” mode, which preserves file attributes, including permissions, timestamps, and more, making it ideal for replicating source directories.
•    – -v: Stands for “verbose,” providing detailed information about the files being copied, aiding in tracking the transfer progress.
•    – -z: Enables compression during the transfer, reducing data transfer over the network, especially for large files.
• -e “ssh -i /path/to/your/key.pem”: Specifies the remote shell for the transfer, using SSH, and the `-i` option to indicate the path to your private key (`key.pem`) for secure authentication.
• /path/to/local/source: This represents the path to the local source directory or file you intend to copy. Replace it with the actual path of the file or directory you wish to synchronize.
• user@server:/path/to/remote/destination: Specifies the remote server and destination directory:
•    – user: The username used for logging into the remote server.
•    – server: The hostname or IP address of the remote server.
•    – :/path/to/remote/destination: The path to the destination directory on the remote server where your files will be copied.