How to Automate Backup Tasks in Bash

I lost a week of work exactly once, back when I trusted “I’ll back it up manually later” as an actual strategy. That was the last time. Since then, every machine and server I’ve managed has had an automated Bash backup script running quietly in the background, and I’ve refined the same basic approach over the years into something I now trust completely. In this article, I’ll walk you through building a real, production-worthy backup automation system in Bash, from the simplest version to one with rotation, remote storage, and proper logging.

What a Good Backup Script Actually Needs

Before writing any code, it’s worth being clear about what separates a “toy” backup script from one you can actually rely on:

• It should run unattended, without any manual intervention.
• It should compress data to save space.
• It should rotate old backups so disk usage doesn’t grow forever.
• It should log what happened, so failures are visible rather than silent.
• It should verify the backup actually succeeded, not just assume it did.
• Ideally, it should copy backups somewhere other than the same disk they came from.

Step 1: A Basic Backup Script

Save this as backup.sh:

#!/usr/bin/env bash
set -euo pipefail

SOURCE_DIR="/home/user/projects"
BACKUP_DIR="/mnt/backups"
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
BACKUP_FILE="backup_${TIMESTAMP}.tar.gz"

mkdir -p "$BACKUP_DIR"

tar czf "${BACKUP_DIR}/${BACKUP_FILE}" -C "$(dirname "$SOURCE_DIR")" "$(basename "$SOURCE_DIR")"

echo "Backup created: ${BACKUP_DIR}/${BACKUP_FILE}"

Make it executable and run it:

chmod +x backup.sh
./backup.sh

Explaining the Basic Script

• TIMESTAMP=$(date +%Y%m%d_%H%M%S) generates a sortable, unique timestamp for each backup file, so multiple runs never overwrite each other.
• tar czf creates (c) a gzip-compressed (z) archive file (f), bundling everything under $SOURCE_DIR into a single .tar.gz file.
• Using -C "$(dirname "$SOURCE_DIR")" "$(basename "$SOURCE_DIR")" changes into the parent directory before archiving, so the resulting tarball contains a clean relative path (like projects/...) instead of an absolute path baked into every entry, which matters if you ever need to extract the backup somewhere else.

Step 2: Adding Logging

Silent scripts are dangerous — if a backup fails, you want to know about it, ideally without having to remember to check manually.

#!/usr/bin/env bash
set -euo pipefail

SOURCE_DIR="/home/user/projects"
BACKUP_DIR="/mnt/backups"
LOG_FILE="/var/log/backup.log"
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
BACKUP_FILE="backup_${TIMESTAMP}.tar.gz"

log() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

mkdir -p "$BACKUP_DIR"

log "Starting backup of $SOURCE_DIR"

if tar czf "${BACKUP_DIR}/${BACKUP_FILE}" -C "$(dirname "$SOURCE_DIR")" "$(basename "$SOURCE_DIR")"; then
    log "Backup succeeded: ${BACKUP_DIR}/${BACKUP_FILE}"
else
    log "Backup FAILED"
    exit 1
fi

Step 3: Verifying Backup Integrity

A backup that completes without error isn’t necessarily a good backup. I always verify the archive is actually readable before considering the job done:

if tar tzf "${BACKUP_DIR}/${BACKUP_FILE}" > /dev/null 2>&1; then
    log "Backup integrity verified."
else
    log "WARNING: Backup file appears corrupted!"
    exit 1
fi

tar tzf lists the contents of the archive without extracting it, which is a fast way to confirm the file isn’t truncated or corrupted. If this fails, something went wrong during archive creation, even if the initial tar czf command reported success.

Step 4: Rotating Old Backups

Without cleanup, a nightly backup job will eventually fill up the disk. Here’s how I handle rotation, keeping only the most recent N backups:

#!/usr/bin/env bash
set -euo pipefail

BACKUP_DIR="/mnt/backups"
RETENTION_DAYS=14
LOG_FILE="/var/log/backup.log"

log() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

log "Removing backups older than $RETENTION_DAYS days"
find "$BACKUP_DIR" -name "backup_*.tar.gz" -type f -mtime "+$RETENTION_DAYS" -print -delete | while read -r removed; do
    log "Removed old backup: $removed"
done

find ... -mtime "+$RETENTION_DAYS" matches files whose modification time is older than the given number of days, and -delete removes them directly. The -print flag before -delete ensures each removed file’s name is echoed first, which I pipe into the loop purely for logging purposes.

Step 5: The Complete Backup Script

Putting it all together into one script I’d actually trust in production:

#!/usr/bin/env bash
set -euo pipefail

SOURCE_DIR="/home/user/projects"
BACKUP_DIR="/mnt/backups"
LOG_FILE="/var/log/backup.log"
RETENTION_DAYS=14
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
BACKUP_FILE="backup_${TIMESTAMP}.tar.gz"

log() {
    echo "$(date '+%Y-%m-%d %H:%M:%S') - $1" | tee -a "$LOG_FILE"
}

trap 'log "Backup script exited unexpectedly."' ERR

mkdir -p "$BACKUP_DIR"
log "=== Backup job started ==="

if tar czf "${BACKUP_DIR}/${BACKUP_FILE}" -C "$(dirname "$SOURCE_DIR")" "$(basename "$SOURCE_DIR")"; then
    log "Backup archive created: ${BACKUP_FILE}"
else
    log "ERROR: Backup archive creation failed."
    exit 1
fi

if tar tzf "${BACKUP_DIR}/${BACKUP_FILE}" > /dev/null 2>&1; then
    log "Backup integrity verified."
else
    log "ERROR: Backup archive failed integrity check."
    exit 1
fi

find "$BACKUP_DIR" -name "backup_*.tar.gz" -type f -mtime "+$RETENTION_DAYS" -print -delete | while read -r removed; do
    log "Removed old backup: $removed"
done

log "=== Backup job completed successfully ==="

The trap 'log "..."' ERR line ensures that if any command fails unexpectedly (thanks to set -e), the failure gets logged before the script exits, giving you a clear record even of unanticipated failures.

Step 6: Syncing Backups to a Remote Server

Local backups protect you from accidental deletion, but not from hardware failure or theft. I always sync backups off-site too:

REMOTE_HOST="backupuser@backup.example.com"
REMOTE_PATH="/backups/projects/"

if rsync -avz --partial "${BACKUP_DIR}/${BACKUP_FILE}" "${REMOTE_HOST}:${REMOTE_PATH}"; then
    log "Backup synced to remote server."
else
    log "WARNING: Remote sync failed."
fi

I deliberately don’t exit 1 on a failed remote sync in this example, since I’d rather keep the local backup and just flag the remote sync issue for follow-up, rather than treating the whole job as failed when the local backup itself succeeded.

Automating with Cron

Once the script is solid, scheduling it is the easy part. Edit your crontab with crontab -e:

0 2 * * * /home/user/scripts/backup.sh >> /var/log/backup_cron.log 2>&1

This runs the backup every night at 2 AM, redirecting both standard output and standard error to a log file so anything the script prints (beyond its own internal logging) is also captured.

Real-World Use Cases

• Nightly backups of application data or databases, keeping a rolling window of recent snapshots.
• Pre-deployment safety backups, automatically snapshotting a directory or database right before a risky deployment or migration runs.
• Configuration backups, archiving /etc or other system configuration directories on a schedule, so server configuration drift can always be rolled back.
• Personal data protection, backing up a home directory or documents folder to an external drive or NAS on a schedule.

Automation Example: Database Backup Variant

Adapting the same pattern for a MySQL/MariaDB database instead of files:

#!/usr/bin/env bash
set -euo pipefail

DB_NAME="myapp_production"
BACKUP_DIR="/mnt/backups/db"
TIMESTAMP=$(date +%Y%m%d_%H%M%S)
BACKUP_FILE="${DB_NAME}_${TIMESTAMP}.sql.gz"

mkdir -p "$BACKUP_DIR"

if mysqldump "$DB_NAME" | gzip > "${BACKUP_DIR}/${BACKUP_FILE}"; then
    echo "Database backup created: ${BACKUP_DIR}/${BACKUP_FILE}"
else
    echo "Database backup FAILED"
    exit 1
fi

The same rotation, logging, and remote-sync patterns shown earlier apply equally well to this database variant with minimal changes.

Best Practices

• Always verify backups (via tar tzf or an equivalent check), not just assume success because the primary command exited with status 0.
• Log every run, including successes, so you have a clear audit trail rather than only discovering problems when a failure notification (if any) fires.
• Keep at least one copy of backups off the original machine, whether that’s a remote server, cloud storage, or a physically separate drive.
• Rotate old backups automatically so disk usage doesn’t grow unbounded, but keep enough history to recover from a problem discovered days or weeks later.
• Test your restore process periodically. A backup you’ve never tried to restore from is a backup you don’t actually know works.

Security Considerations

• Restrict permissions on the backup directory and files (chmod 600 or 700 as appropriate) since backups often contain sensitive data.
• Never store database credentials in plaintext inside the script itself; use a .my.cnf file with restricted permissions for MySQL, or environment variables sourced from a secured location.
• If syncing to a remote server, use SSH key-based authentication rather than passwords, and consider a dedicated backup user with minimal permissions on the remote side.
• Encrypt backups containing sensitive data before storing or transferring them (see the file encryption tool covered separately), especially if they’re going to third-party cloud storage.

Optimization Tips

• For very large datasets, consider incremental backups (only backing up changed files) using rsync with --link-dest for efficient space usage across multiple backup snapshots, rather than full compressed archives every time.
• Schedule backups during low-usage hours to minimize the performance impact of compression and disk I/O on production workloads.
• For database backups, check whether your database supports a faster, more consistent backup method than a plain mysqldump, such as physical backups for very large databases.

Troubleshooting Common Issues

Backup script runs manually but not via cron — Cron jobs run with a minimal environment and no interactive shell; use absolute paths for every command and file, and avoid relying on environment variables that are only set in your interactive shell session.

“No space left on device” during backup creation — Check available disk space before starting the backup, and verify your rotation logic is actually deleting old backups as expected; a rotation bug can silently cause backups to accumulate indefinitely.

Backup completes but integrity check fails — This usually points to disk I/O errors, a process being killed mid-write, or insufficient disk space during the write itself; check system logs (dmesg, /var/log/syslog) for related disk errors.

Remote sync consistently times out — Check network connectivity and firewall rules between the two hosts, and consider adding --timeout and retry logic to the rsync command (see the file transfer utility article for a retry pattern).

Common Mistakes to Avoid

• Assuming a backup succeeded just because the script didn’t print an error, instead of explicitly verifying the archive’s integrity.
• Storing backups only on the same disk or server as the original data, leaving no protection against hardware failure or theft.
• Never testing the restore process, only the backup process — a backup is only as good as your ability to actually recover from it.
• Forgetting to rotate old backups, leading to a full disk at the worst possible moment.
• Hardcoding credentials directly in the script rather than using a secured configuration file or environment variables.

Frequently Asked Questions

How often should I run automated backups? It depends on how much data loss you can tolerate. For active production systems, nightly (or even hourly for critical databases) is common. For personal projects, daily or weekly is often sufficient.

Should backups be full or incremental? Full backups are simpler to manage and restore from, but consume more storage and take longer as data grows. Incremental backups (like rsync with hard-linking) save significant space and time for large datasets, at the cost of slightly more complex restore procedures.

How do I know if my backup script is actually working without checking manually every day? Add alerting — have the script send an email, Slack message, or push notification on failure specifically, so you only hear from it when something needs your attention, rather than needing to check logs proactively every day.

Is compressing backups always a good idea? Usually yes for reducing storage costs and transfer time, but compression does add CPU overhead and time to the backup process itself. For extremely large datasets on time-sensitive backup windows, weigh compression benefits against the added processing time.

Summary

A trustworthy backup system doesn’t need to be complicated — it needs to be automated, verified, logged, and tested. Starting from a basic tar czf command, I’ve walked through adding logging, integrity verification, rotation, and off-site syncing, building up to a script I’d genuinely rely on for real data. The single most important habit, beyond any specific script detail, is testing your restore process occasionally — because a backup that’s never been restored from is really just a hypothesis.

References

• GNU Tar Manual
• rsync Official Documentation
• Cron and Crontab — GNU/Linux Documentation