Copy Latest Backup Preserving Folder Structure Over SSH with rsync

Table of Contents

1. Prerequisites
2. Usage
3. Command
4. Explanation
5. Permissions
6. Example
7. Note
8. Troubleshooting

This script finds the most recently modified file in a specified directory and uses rsync to copy it to a remote server while preserving the folder structure.

Prerequisites

• Ensure you have rsync installed on both your local machine and the remote server.
• SSH access to the remote server should be configured and working.
• The specified directory (e.g., .mongo/) should exist and contain files.

Usage

Command

find .mongo/ -type f -printf '%T@ %p\n' | sort -n | tail -1 | cut -d' ' -f2- | xargs --no-run-if-empty -I {} rsync --mkpath -avvzhRP {} remote-user@remote-ip:~/$(pwd | sed "s#$HOME/##")

Explanation

• find .mongo/ -type f -printf '%T@ %p\n': This command searches the .mongo/ directory for files and prints their last modification time followed by their path.

  • -type f: Finds files only.
  • -printf '%T@ %p\n': Prints the last modification time (epoch time) and the file path.

• | sort -n: Sorts the output numerically by modification time.

• | tail -1: Selects the last file in the sorted list, which is the most recently modified file.

• | cut -d' ' -f2-: Extracts the file path from the sorted output.

• | xargs --no-run-if-empty -I {} rsync --mkpath -avvzhRP {} remote-user@remote-ip:~/$(pwd | sed "s#$HOME/##"):

  • xargs --no-run-if-empty -I {}: Runs the rsync command for each file found, if any.
  • rsync --mkpath -avvzhRP {}: Copies the file using rsync with the following options:
    • --mkpath: Creates the destination directory if it doesn't exist.
    • -a: Archive mode, which preserves permissions, timestamps, symbolic links, and other attributes.
    • -v: Verbose output, providing detailed information about the transfer.
    • -v: Increases verbosity.
    • -z: Compresses file data during the transfer.
    • -h: Outputs numbers in a human-readable format.
    • -R: Uses relative file paths.
    • -P: Shows progress during transfer and keeps partially transferred files.
  • remote-user@remote-ip:~/$(pwd | sed "s#$HOME/##"): Specifies the remote destination:
    • remote-user: Replace with your remote username.
    • remote-ip: Replace with your remote server's IP address.
    • ~/$(pwd | sed "s#$HOME/##"): Constructs the remote path by appending the current directory structure, relative to the home directory, to the remote user's home directory.

Permissions

If the you encounter issues related to permission on the source file, you should add the sudo before find and rsync.

Example

If your current directory is /home/user/project and the most recently modified file in .mongo/ is backup.bson, running the command will copy the file to ~/project/.mongo/backup.bson on the remote server.

Note

• Make sure to replace remote-user and remote-ip with your actual remote username and server IP address.
• The $(pwd | sed "s#$HOME/##") part ensures that the folder structure is preserved relative to your home directory on the remote server. If you prefer a different structure, adjust this part accordingly.

Troubleshooting

• If you encounter issues with SSH authentication, ensure that your SSH keys are set up correctly and that you can manually SSH into the remote server.
• If the rsync command fails, check the availability of rsync on both the local and remote machines and ensure that the destination path is correct.

By following these instructions, you can automate the process of copying the latest backup file while preserving the directory structure using rsync over SSH.