[HowTo] Mirror Your Website With Rsync

For this couple weeks there are lot of changes on my blog. Start from being suspended because of high traffic, move to another hosting server and then optimize the blogs with caching to improve performance, and change themes to optimize for Internet Explorer. Since that i did a lot of backup before doing anything.

I made a script to backup all files and database and upload to HotFile, you can read that post here. And now i rent another hosting server to mirror my website. So if something happen with this host, then i just repoint my nameserver and it would be up again. Yes since i got suspended from the old host, i am become a backup freaks.

So to do mirror your website content, you can use rsync. This useful programs can copy all your files over the ssh between linux or unix server.

Okay now let’s prepare. I give the name CurrentUser, CurrentPassword and CurrentHost for the Current server. And RemoteUser, RemotePassword and RemoteHost for Remote server.

Configure Current Server

1. Open your terminal and login to your ssh server.
2. Generate public/private pairs key to allow ssh connection without asking for a password.

   1 2 3 4 5 6 7 8

   $ ssh-keygen -t dsa -b 2048 -f /home/CurrentUser/cron/CurrentHost-rsync-key Generating public/private dsa key pair. Enter passphrase (empty for no passphrase): [press enter here] Enter same passphrase again: [press enter here] Your identification has been saved in /home/CurrentUser/cron/CurrentHost-rsync-key. Your public key has been saved in /home/CurrentUser/cron/CurrentHost-rsync-key.pub. The key fingerprint is: 2e:28:d9:ec:85:21:e7:ff:73:df:2e:07:78:f0:d0:a0 thisuser@thishost

3. Copy the .pub file to the RemoteHost.

   1	$ scp /home/CurrentUser/cron/CurrentHost-rsync-key.pub RemoteUser@RemoteHost :/home/RemoteUser/

Now we are done with the configuration on the current host. Now we move to remote host.

Configure RemoteServer

1. Connect to the remote host with ssh:

   1 2	$ ssh RemoteUser@RemoteHost RemoteUser@RemoteHost's password: [RemotePassword]

2. Move the CurrentHost-rsync-key.pub to .ssh folder:

   1 2	$ mv CurrentHost-rsync-key.pub .ssh/ cd .ssh/

3. Append CurrentHost-rsync-key.pub to authorized_keys in the .ssh folder

   1	cat CurrentHost-rsync-key.pub >> authorized_keys

4. Edit the authorized_keys with vim:

   1	vim authorized_keys

5. Change from:

   01 02 03 04 05 06 07 08 09 10 11 12 13 14

   ssh-dss AAAAB3NzaC1kc3MAAAEBAKYJenaYvMG3nHwWxKwlWLjHb77CT2hXwmC8Ap+fG8wjlaY/9t4u A+2qx9JNorgdrWKhHSKHokFFlWRj+qk3q+lGHS+hsXuvta44W0yD0y0sW62wrEVegz+JVmntxeYc0nDz 5tVGfZe6ydlgomzj1bhfdpYe+BAwop8L+EMqKLS4iSacNjoPlHsmqHMnbibn3tBqJEq2QJjEPaiYj1iP 5IaCuYBhuTKQGa+oyH3mXEif5CKdsIKBj46B0tCy0/GC7oWcUN92QdLrUyTeRJZsTWsxKpRbMliD2pBh 4oyX/aXEf8+HZBrO5vQjDBCfTFQA+35Xrd3eTVEjkGkncI0SAeUAAAAVAMZSASmQ9Pi38mdm6oiVXD55 Kk2rAAABAE/bA402VuCsOLg9YS0NKxugT+o4UuIjyl6b2/cMmBVWO39lWAjcsKK/zEdJbrOdt/sKsxIK 1/ZIvtl92DLlMhci5c4tBjCODey4yjLhApjWgvX9D5OPp89qhah4zu509uNX7uH58Zw/+m6ZOLHN28mV 5KLUl7FTL2KZ583KrcWkUA0Id4ptUa9CAkcqn/gWkHMptgVwaZKlqZ+QtEa0V2IwUDWS097p3SlLvozw 46+ucWxwTJttCHLzUmNN7w1cIv0w/OHh5IGh+wWjV9pbO0VT3/r2jxkzqksKOYAb5CYzSNRyEwp+NIKr Y+aJz7myu4Unn9de4cYsuXoAB6FQ5I8AAAEBAJSmDndXJCm7G66qdu3ElsLT0Jlz/es9F27r+xrg5pZ5 GjfBCRvHNo2DF4YW9MKdUQiv+ILMY8OISduTeu32nyA7dwx7z5M8b+DtasRAa1U03EfpvRQps6ovu79m bt1OE8LS9ql8trx8qyIpYmJxmzIdBQ+kzkY+9ZlaXsaU0Ssuda7xPrX4405CbnKcpvM6q6okMP86Ejjn 75Cfzhv65hJkCjbiF7FZxosCRIuYbhEEKu2Z9Dgh+ZbsZ+9FETZVzKBs4fySA6dIw6zmGINd+KY6umMW yJNej2Sia70fu3XLHj2yBgN5cy8arlZ80q1Mcy763RjYGkR/FkLJ611HWIA= CurrentUser@CurrentHost

   To

   01 02 03 04 05 06 07 08 09 10 11 12 13 14 15

   from="10.1.1.1",command="/home/RemoteUser/cron/validate-rsync" ssh-dss AAAAB3Nza C1kc3MAAAEBAKYJenaYvMG3nHwWxKwlWLjHb77CT2hXwmC8Ap+fG8wjlaY/9t4uA+2qx9JNorgdrWKhH SKHokFFlWRj+qk3q+lGHS+hsXuvta44W0yD0y0sW62wrEVegz+JVmntxeYc0nDz5tVGfZe6ydlgomzj1 bhfdpYe+BAwop8L+EMqKLS4iSacNjoPlHsmqHMnbibn3tBqJEq2QJjEPaiYj1iP5IaCuYBhuTKQGa+oy H3mXEif5CKdsIKBj46B0tCy0/GC7oWcUN92QdLrUyTeRJZsTWsxKpRbMliD2pBh4oyX/aXEf8+HZBrO5 vQjDBCfTFQA+35Xrd3eTVEjkGkncI0SAeUAAAAVAMZSASmQ9Pi38mdm6oiVXD55Kk2rAAABAE/bA402V uCsOLg9YS0NKxugT+o4UuIjyl6b2/cMmBVWO39lWAjcsKK/zEdJbrOdt/sKsxIK1/ZIvtl92DLlMhci5 c4tBjCODey4yjLhApjWgvX9D5OPp89qhah4zu509uNX7uH58Zw/+m6ZOLHN28mV5KLUl7FTL2KZ583Kr cWkUA0Id4ptUa9CAkcqn/gWkHMptgVwaZKlqZ+QtEa0V2IwUDWS097p3SlLvozw46+ucWxwTJttCHLzU mNN7w1cIv0w/OHh5IGh+wWjV9pbO0VT3/r2jxkzqksKOYAb5CYzSNRyEwp+NIKrY+aJz7myu4Unn9de4 cYsuXoAB6FQ5I8AAAEBAJSmDndXJCm7G66qdu3ElsLT0Jlz/es9F27r+xrg5pZ5GjfBCRvHNo2DF4YW9 MKdUQiv+ILMY8OISduTeu32nyA7dwx7z5M8b+DtasRAa1U03EfpvRQps6ovu79mbt1OE8LS9ql8trx8q yIpYmJxmzIdBQ+kzkY+9ZlaXsaU0Ssuda7xPrX4405CbnKcpvM6q6okMP86Ejjn75Cfzhv65hJkCjbiF 7FZxosCRIuYbhEEKu2Z9Dgh+ZbsZ+9FETZVzKBs4fySA6dIw6zmGINd+KY6umMWyJNej2Sia70fu3XLH j2yBgN5cy8arlZ80q1Mcy763RjYGkR/FkLJ611HWIA= CurrentUser@CurrentHost

Setup on Remote Server done. Now you can test it from your Current server, try run this command:

Setup Cron

1. Create a bash file called it backup-rsync.sh
2. Copy this script:

   01 02 03 04 05 06 07 08 09 10 11

   #!/bin/sh   RSYNC=/usr/bin/rsync SSH=/usr/bin/ssh KEY=/home/CurrentUser/cron/thishost-rsync-key RUSER=RemoteUser RHOST=RemoteHost RPATH=/remote/dir LPATH=/this/dir/   $RSYNC -az -e "$SSH -i $KEY" $RUSER@$RHOST:$RPATH $LPATH

3. Register to cron (i run in once a week):

   1 2	crontab -e 0 0 * * 0 bash /path/to/backup-rsync.sh

   To run in another schedule see the cron manual by using man crontab for more details.

Troubleshooting:

1. Check the permission. On current host, the private key file should be chmod 600. And On Remote host the authorized_keys should be chmod 600.
2. Check the log files in the /var/log directory.
3. Verbose the rsync command see where the command goes wrong:

   1	$ rsync -avvvz -e "ssh -i /home/CurrentUser/cron/CurrentHost-rsync-key" RemoteUser@RemoteHost:/remote/dir/backup /this/dir/backup