[HowTo] Mirror Your Website With Rsync

For this couple weeks there are lot of changes on my blog. Start from being suspended because of high traffic, move to another hosting server and then optimize the blogs with caching to improve performance, and change themes to optimize for Internet Explorer. Since that i did a lot of backup before doing anything.

I made a script to backup all files and database and upload to HotFile, you can read that post here. And now i rent another hosting server to mirror my website. So if something happen with this host, then i just repoint my nameserver and it would be up again. Yes since i got suspended from the old host, i am become a backup freaks.

So to do mirror your website content, you can use rsync. This useful programs can copy all your files over the ssh between linux or unix server.

Okay now let’s prepare. I give the name CurrentUser, CurrentPassword and CurrentHost for the Current server. And RemoteUser, RemotePassword and RemoteHost for Remote server.

Configure Current Server

  1. Open your terminal and login to your ssh server.
  2. Generate public/private pairs key to allow ssh connection without asking for a password.
    $ ssh-keygen -t dsa -b 2048 -f /home/CurrentUser/cron/CurrentHost-rsync-key
    Generating public/private dsa key pair.
    Enter passphrase (empty for no passphrase): [press enter here]
    Enter same passphrase again: [press enter here]
    Your identification has been saved in /home/CurrentUser/cron/CurrentHost-rsync-key.
    Your public key has been saved in /home/CurrentUser/cron/CurrentHost-rsync-key.pub.
    The key fingerprint is:
    2e:28:d9:ec:85:21:e7:ff:73:df:2e:07:78:f0:d0:a0 [email protected]
    
  3. Copy the .pub file to the RemoteHost.
    $ scp /home/CurrentUser/cron/CurrentHost-rsync-key.pub [email protected] :/home/RemoteUser/ 

Now we are done with the configuration on the current host. Now we move to remote host.

Configure RemoteServer

  1. Connect to the remote host with ssh:
    $ ssh [email protected]
    [email protected]'s password: [RemotePassword] 
  2. Move the CurrentHost-rsync-key.pub to .ssh folder:
    $ mv CurrentHost-rsync-key.pub .ssh/
    cd .ssh/ 
  3. Append CurrentHost-rsync-key.pub to authorized_keys in the .ssh folder
    cat CurrentHost-rsync-key.pub >> authorized_keys 
  4. Edit the authorized_keys with vim:
    vim authorized_keys 
  5. Change from:
    ssh-dss AAAAB3NzaC1kc3MAAAEBAKYJenaYvMG3nHwWxKwlWLjHb77CT2hXwmC8Ap+fG8wjlaY/9t4u
    A+2qx9JNorgdrWKhHSKHokFFlWRj+qk3q+lGHS+hsXuvta44W0yD0y0sW62wrEVegz+JVmntxeYc0nDz
    5tVGfZe6ydlgomzj1bhfdpYe+BAwop8L+EMqKLS4iSacNjoPlHsmqHMnbibn3tBqJEq2QJjEPaiYj1iP
    5IaCuYBhuTKQGa+oyH3mXEif5CKdsIKBj46B0tCy0/GC7oWcUN92QdLrUyTeRJZsTWsxKpRbMliD2pBh
    4oyX/aXEf8+HZBrO5vQjDBCfTFQA+35Xrd3eTVEjkGkncI0SAeUAAAAVAMZSASmQ9Pi38mdm6oiVXD55
    Kk2rAAABAE/bA402VuCsOLg9YS0NKxugT+o4UuIjyl6b2/cMmBVWO39lWAjcsKK/zEdJbrOdt/sKsxIK
    1/ZIvtl92DLlMhci5c4tBjCODey4yjLhApjWgvX9D5OPp89qhah4zu509uNX7uH58Zw/+m6ZOLHN28mV
    5KLUl7FTL2KZ583KrcWkUA0Id4ptUa9CAkcqn/gWkHMptgVwaZKlqZ+QtEa0V2IwUDWS097p3SlLvozw
    46+ucWxwTJttCHLzUmNN7w1cIv0w/OHh5IGh+wWjV9pbO0VT3/r2jxkzqksKOYAb5CYzSNRyEwp+NIKr
    Y+aJz7myu4Unn9de4cYsuXoAB6FQ5I8AAAEBAJSmDndXJCm7G66qdu3ElsLT0Jlz/es9F27r+xrg5pZ5
    GjfBCRvHNo2DF4YW9MKdUQiv+ILMY8OISduTeu32nyA7dwx7z5M8b+DtasRAa1U03EfpvRQps6ovu79m
    bt1OE8LS9ql8trx8qyIpYmJxmzIdBQ+kzkY+9ZlaXsaU0Ssuda7xPrX4405CbnKcpvM6q6okMP86Ejjn
    75Cfzhv65hJkCjbiF7FZxosCRIuYbhEEKu2Z9Dgh+ZbsZ+9FETZVzKBs4fySA6dIw6zmGINd+KY6umMW
    yJNej2Sia70fu3XLHj2yBgN5cy8arlZ80q1Mcy763RjYGkR/FkLJ611HWIA= [email protected]

    To

    from="10.1.1.1",command="/home/RemoteUser/cron/validate-rsync" ssh-dss AAAAB3Nza
    C1kc3MAAAEBAKYJenaYvMG3nHwWxKwlWLjHb77CT2hXwmC8Ap+fG8wjlaY/9t4uA+2qx9JNorgdrWKhH
    SKHokFFlWRj+qk3q+lGHS+hsXuvta44W0yD0y0sW62wrEVegz+JVmntxeYc0nDz5tVGfZe6ydlgomzj1
    bhfdpYe+BAwop8L+EMqKLS4iSacNjoPlHsmqHMnbibn3tBqJEq2QJjEPaiYj1iP5IaCuYBhuTKQGa+oy
    H3mXEif5CKdsIKBj46B0tCy0/GC7oWcUN92QdLrUyTeRJZsTWsxKpRbMliD2pBh4oyX/aXEf8+HZBrO5
    vQjDBCfTFQA+35Xrd3eTVEjkGkncI0SAeUAAAAVAMZSASmQ9Pi38mdm6oiVXD55Kk2rAAABAE/bA402V
    uCsOLg9YS0NKxugT+o4UuIjyl6b2/cMmBVWO39lWAjcsKK/zEdJbrOdt/sKsxIK1/ZIvtl92DLlMhci5
    c4tBjCODey4yjLhApjWgvX9D5OPp89qhah4zu509uNX7uH58Zw/+m6ZOLHN28mV5KLUl7FTL2KZ583Kr
    cWkUA0Id4ptUa9CAkcqn/gWkHMptgVwaZKlqZ+QtEa0V2IwUDWS097p3SlLvozw46+ucWxwTJttCHLzU
    mNN7w1cIv0w/OHh5IGh+wWjV9pbO0VT3/r2jxkzqksKOYAb5CYzSNRyEwp+NIKrY+aJz7myu4Unn9de4
    cYsuXoAB6FQ5I8AAAEBAJSmDndXJCm7G66qdu3ElsLT0Jlz/es9F27r+xrg5pZ5GjfBCRvHNo2DF4YW9
    MKdUQiv+ILMY8OISduTeu32nyA7dwx7z5M8b+DtasRAa1U03EfpvRQps6ovu79mbt1OE8LS9ql8trx8q
    yIpYmJxmzIdBQ+kzkY+9ZlaXsaU0Ssuda7xPrX4405CbnKcpvM6q6okMP86Ejjn75Cfzhv65hJkCjbiF
    7FZxosCRIuYbhEEKu2Z9Dgh+ZbsZ+9FETZVzKBs4fySA6dIw6zmGINd+KY6umMWyJNej2Sia70fu3XLH
    j2yBgN5cy8arlZ80q1Mcy763RjYGkR/FkLJ611HWIA= [email protected]

Setup on Remote Server done. Now you can test it from your Current server, try run this command:

$ rsync -avz -e "ssh -i /home/CurrentUser/cron/CurrentHost-rsync-key" [email protected]:/remote/dir/backup /this/dir/backup 

Setup Cron

  1. Create a bash file called it backup-rsync.sh
  2. Copy this script:
    #!/bin/sh
    
    RSYNC=/usr/bin/rsync
    SSH=/usr/bin/ssh
    KEY=/home/CurrentUser/cron/thishost-rsync-key
    RUSER=RemoteUser
    RHOST=RemoteHost
    RPATH=/remote/dir
    LPATH=/this/dir/
    
    $RSYNC -az -e "$SSH -i $KEY" [email protected]$RHOST:$RPATH $LPATH
    
  3. Register to cron (i run in once a week):
    crontab -e
    0 0 * * 0 bash /path/to/backup-rsync.sh
    

    To run in another schedule see the cron manual by using man crontab for more details.

Troubleshooting:

  1. Check the permission. On current host, the private key file should be chmod 600. And On Remote host the authorized_keys should be chmod 600.
  2. Check the log files in the /var/log directory.
  3. Verbose the rsync command see where the command goes wrong:
    $ rsync -avvvz -e "ssh -i /home/CurrentUser/cron/CurrentHost-rsync-key" [email protected]:/remote/dir/backup /this/dir/backup 

Comments

  1. voku says:

    don't use scp, mv, cat … 😉

    ssh-copy-id -i ~/.ssh/id_rsa.pub [email protected]

Give me your feedback

This site uses Akismet to reduce spam. Learn how your comment data is processed.