How to Create a Bash File Sync Tool

I manage a handful of machines — a home server, a laptop, a couple of VPS instances — and keeping certain folders in sync between them used to be a manual, error-prone chore. Eventually I got fed up and built a Bash file sync tool on top of rsync that handles local syncing, remote syncing over SSH, dry runs, and logging. In this article I’ll take you through the whole build, explaining exactly what each piece does so you can adapt it to your own setup.

Why rsync Is the Right Foundation

Bash itself doesn’t need to reimplement file syncing — rsync already does the hard part brilliantly: it only transfers the differences between files (using a clever delta algorithm), preserves permissions and timestamps, and supports both local and remote (SSH) destinations. Our job is to wrap it in a script that’s easy to invoke, safe by default, and repeatable.

Prerequisites

sudo apt install rsync openssh-client

For remote syncing, make sure SSH key-based authentication is set up so the script can run non-interactively:

ssh-keygen -t ed25519 -C "sync-tool"
ssh-copy-id user@remote-host

Step 1: A Basic Local Sync Script

#!/usr/bin/env bash
set -euo pipefail

SOURCE="${1:-}"
DEST="${2:-}"

if [[ -z "$SOURCE" || -z "$DEST" ]]; then
    echo "Usage: $0 <source-dir> <destination-dir>"
    exit 1
fi

rsync -av --progress "$SOURCE"/ "$DEST"/

Understanding the Core Command

• rsync -av — -a is “archive mode,” a shorthand that preserves permissions, timestamps, symbolic links, ownership, and recurses into subdirectories all at once. -v makes the output verbose so you can see what’s being copied.
• --progress shows a live progress indicator for each file being transferred, which is reassuring on large syncs.
• The trailing slashes on "$SOURCE"/ and "$DEST"/ matter a lot in rsync — a trailing slash on the source means “copy the contents of this directory,” whereas omitting it would copy the directory itself into the destination, creating an extra nested folder. This is one of the most common rsync mistakes, so I always add it explicitly in scripts.

Step 2: Adding a Dry-Run Mode

Before syncing anything for real, I like to preview exactly what will change:

DRY_RUN="${3:-}"

if [[ "$DRY_RUN" == "--dry-run" ]]; then
    rsync -av --dry-run "$SOURCE"/ "$DEST"/
    echo "Dry run complete. No files were changed."
else
    rsync -av --progress "$SOURCE"/ "$DEST"/
fi

--dry-run tells rsync to calculate and display everything it would do without actually touching any files — it’s one of the most valuable safety features available for any sync or deletion workflow.

Step 3: Supporting Remote Sync Over SSH

Now let’s extend the script to support remote destinations, detected automatically by the presence of a colon (:) in the destination argument, which is the standard rsync/scp convention for remote paths:

#!/usr/bin/env bash
set -euo pipefail

SOURCE="${1:-}"
DEST="${2:-}"
DRY_RUN="${3:-}"

if [[ -z "$SOURCE" || -z "$DEST" ]]; then
    echo "Usage: $0 <source-dir> <destination-dir-or-user@host:path> [--dry-run]"
    exit 1
fi

RSYNC_OPTS=(-av --progress --human-readable)

if [[ "$DRY_RUN" == "--dry-run" ]]; then
    RSYNC_OPTS+=(--dry-run)
fi

if [[ "$DEST" == *:* ]]; then
    echo "Detected remote destination. Syncing over SSH..."
    rsync "${RSYNC_OPTS[@]}" -e ssh "$SOURCE"/ "$DEST"/
else
    rsync "${RSYNC_OPTS[@]}" "$SOURCE"/ "$DEST"/
fi

Breaking This Down

• RSYNC_OPTS=(-av --progress --human-readable) builds our flags as a Bash array, which makes it easy to conditionally append more flags (like --dry-run) without messy string concatenation.
• [[ "$DEST" == *:* ]] uses Bash’s pattern-matching inside [[ ]] to check whether the destination string contains a colon anywhere — a simple but effective way to detect user@host:/path syntax.
• -e ssh explicitly tells rsync to use SSH as the transport for remote operations. This is technically the default when a remote path is detected, but specifying it explicitly makes the script’s intent clear and lets you customize SSH options later (like specifying a non-standard port with -e "ssh -p 2222").

Step 4: Adding Exclusions and Deletion Sync

Real syncing setups usually need to ignore certain files and, sometimes, mirror deletions too:

EXCLUDES=(".git/" "node_modules/" "*.tmp" ".DS_Store")
EXCLUDE_ARGS=()
for pattern in "${EXCLUDES[@]}"; do
    EXCLUDE_ARGS+=(--exclude="$pattern")
done

MIRROR="${4:-}"
if [[ "$MIRROR" == "--mirror" ]]; then
    RSYNC_OPTS+=(--delete)
fi

rsync "${RSYNC_OPTS[@]}" "${EXCLUDE_ARGS[@]}" "$SOURCE"/ "$DEST"/

--delete makes rsync remove files from the destination that no longer exist in the source, turning a one-way sync into a true mirror. This is powerful but also the single most dangerous flag in rsync — always test with --dry-run --delete together first.

Step 5: Logging Every Sync

For anything running unattended (like a cron job), you want a record of what happened:

LOG_DIR="$HOME/.local/share/sync-tool/logs"
mkdir -p "$LOG_DIR"
LOG_FILE="$LOG_DIR/sync-$(date +%Y%m%d-%H%M%S).log"

rsync "${RSYNC_OPTS[@]}" "${EXCLUDE_ARGS[@]}" "$SOURCE"/ "$DEST"/ | tee "$LOG_FILE"

echo "Log saved to $LOG_FILE"

tee "$LOG_FILE" is a genuinely useful Bash idiom — it takes standard input and writes it to both the terminal and a file simultaneously, so you get live feedback while also keeping a permanent record.

Full Combined Script

#!/usr/bin/env bash
set -euo pipefail

SOURCE="${1:-}"
DEST="${2:-}"
shift 2 || true

DRY_RUN=false
MIRROR=false
for arg in "$@"; do
    case "$arg" in
        --dry-run) DRY_RUN=true ;;
        --mirror) MIRROR=true ;;
    esac
done

if [[ -z "$SOURCE" || -z "$DEST" ]]; then
    echo "Usage: $0 <source> <dest> [--dry-run] [--mirror]"
    exit 1
fi

EXCLUDES=(".git/" "node_modules/" "*.tmp" ".DS_Store")
EXCLUDE_ARGS=()
for pattern in "${EXCLUDES[@]}"; do
    EXCLUDE_ARGS+=(--exclude="$pattern")
done

RSYNC_OPTS=(-av --human-readable --progress)
$DRY_RUN && RSYNC_OPTS+=(--dry-run)
$MIRROR && RSYNC_OPTS+=(--delete)

LOG_DIR="$HOME/.local/share/sync-tool/logs"
mkdir -p "$LOG_DIR"
LOG_FILE="$LOG_DIR/sync-$(date +%Y%m%d-%H%M%S).log"

if [[ "$DEST" == *:* ]]; then
    rsync "${RSYNC_OPTS[@]}" "${EXCLUDE_ARGS[@]}" -e ssh "$SOURCE"/ "$DEST"/ | tee "$LOG_FILE"
else
    rsync "${RSYNC_OPTS[@]}" "${EXCLUDE_ARGS[@]}" "$SOURCE"/ "$DEST"/ | tee "$LOG_FILE"
fi

echo "Sync complete. Log: $LOG_FILE"

shift 2 || true removes the first two positional arguments (source and dest) so we can loop through any remaining flags cleanly; the || true prevents set -e from killing the script if there happen to be fewer than two extra arguments.

Real-World Use Cases

• Keeping a laptop and desktop’s project folders in sync without relying on a third-party cloud service.
• Deploying static websites by syncing a local build/ folder to a remote web server.
• Mirroring backups between an on-site NAS and an off-site VPS.
• Syncing configuration/dotfiles across multiple servers you manage.

Automating Sync with Cron

*/30 * * * * /usr/local/bin/sync.sh /home/user/projects user@backup-host:/backups/projects >> /var/log/sync-cron.log 2>&1

This runs the sync every 30 minutes and appends both standard output and errors to a persistent log file.

Security Considerations

• Always use SSH key-based authentication, never password auth, for automated remote syncs — this avoids storing plaintext passwords anywhere in scripts or cron jobs.
• Restrict the SSH key’s permissions on the remote side using a forced command or a dedicated low-privilege sync user, so a compromised sync script can’t do more damage than intended.
• Be extremely cautious with --delete. Combined with a wrong source/destination order, it can wipe out data on the wrong side. Always dry-run first.
• Avoid syncing directly to a production web root without a staging step, since a partially completed sync could serve broken files mid-transfer; rsync‘s --delay-updates flag helps mitigate this by staging files before the final move.

Optimization Tips

• Use -z to compress data during transfer over slow network links (skip it on fast LANs, since compression overhead isn’t worth it there).
• --partial keeps partially transferred files if a sync is interrupted, so a resumed sync doesn’t have to start that file from scratch.
• For very large directory trees, --checksum gives more accurate change detection but is slower than the default timestamp/size comparison — only use it when you suspect timestamp-based detection is unreliable.

Troubleshooting

• “Permission denied (publickey)” — your SSH key isn’t set up correctly on the remote host; re-run ssh-copy-id or check ~/.ssh/authorized_keys on the remote side.
• Sync seems to recopy everything every time — this usually means timestamps aren’t being preserved; verify you’re using -a (archive mode), which includes -t (preserve times).
• Extra nested folder appears at destination — you forgot the trailing slash on the source path; revisit Step 1’s explanation.
• “rsync: command not found” on the remote host — rsync must be installed on both ends for remote sync to work, not just locally.

Common Mistakes to Avoid

• Running --delete without testing via --dry-run first.
• Forgetting trailing slashes and ending up with unexpected nested directories.
• Syncing over an unencrypted transport instead of SSH for sensitive data.
• Hardcoding absolute paths instead of using variables, making the script inflexible for reuse.

Frequently Asked Questions

Is rsync safe to interrupt mid-transfer? Yes, especially with --partial — a subsequent run will resume rather than starting over, since rsync only transfers differences.

Can I sync in both directions (bidirectional sync)? Plain rsync is one-directional. For true bidirectional sync with conflict resolution, look into tools like unison, though you can approximate simple two-way sync by running this script in both directions carefully.

How is this different from just using scp? scp copies everything every time; rsync only transfers what’s changed, making repeated syncs dramatically faster, especially on large directory trees.

Summary

We built a flexible Bash sync tool on top of rsync that supports local and remote destinations, dry runs, exclusions, mirroring with deletion, and logging — all through a single consistent interface. The core insight is that Bash doesn’t need to reinvent syncing logic; it just needs to wrap a powerful tool like rsync in sensible defaults and safety rails so you can trust it to run unattended.

References

• rsync Official Manual
• OpenSSH Documentation
• Bash Reference Manual — GNU Project