How to Create a Bash File Archiving Tool

Archiving is the flip side of unarchiving: bundling files and directories into a single, portable, often compressed container. It’s one of the oldest tasks in Unix administration, and tar has been the standard answer for decades — but a well-built wrapper script makes archiving faster, more consistent, and far less error-prone for daily use, backups, and automation.

This article builds a flexible Bash archiving tool from the ground up, covering compression choices, naming conventions, exclusions, and the automation patterns that make archiving genuinely “set and forget.”

Why Wrap tar in a Script?

Typing tar -czvf backup-$(date +%F).tar.gz --exclude='.git' --exclude='node_modules' ./project correctly, every time, from memory, is asking for typos. A script:

• Standardizes naming (timestamps, project names).
• Applies consistent exclusion rules across runs.
• Reduces the archive command to a single memorable invocation.
• Can be extended with compression level control, checksums, and destination management.

Step 1: A Basic Archiving Script

#!/usr/bin/env bash
set -euo pipefail

SRC="$1"
NAME="${2:-$(basename "$SRC")}"
TIMESTAMP=$(date +%Y%m%d-%H%M%S)
OUTPUT="${NAME}-${TIMESTAMP}.tar.gz"

if [[ ! -e "$SRC" ]]; then
    echo "Error: source '$SRC' does not exist."
    exit 1
fi

tar -czf "$OUTPUT" "$SRC"

echo "Created archive: $OUTPUT"
echo "Size: $(du -h "$OUTPUT" | cut -f1)"

Usage:

chmod +x archive.sh
./archive.sh ./project

Output:

Created archive: project-20260728-144512.tar.gz
Size: 4.2M

How It Works

• tar -czf creates (c) an archive, compresses with gzip (z), and writes to the given file (f).
• "${2:-$(basename "$SRC")}" is Bash parameter expansion: if a second argument (custom name) isn’t given, it defaults to the basename of the source path.
• du -h reports human-readable disk usage of the resulting archive, giving instant feedback on the compression result.

Step 2: Adding Exclusions

Real projects usually have directories that shouldn’t be archived — .git, node_modules, __pycache__, build artifacts:

#!/usr/bin/env bash
set -euo pipefail

SRC="$1"
NAME="${2:-$(basename "$SRC")}"
TIMESTAMP=$(date +%Y%m%d-%H%M%S)
OUTPUT="${NAME}-${TIMESTAMP}.tar.gz"

EXCLUDES=(
    "--exclude=.git"
    "--exclude=node_modules"
    "--exclude=__pycache__"
    "--exclude=*.log"
)

tar -czf "$OUTPUT" "${EXCLUDES[@]}" "$SRC"

echo "Created archive: $OUTPUT (exclusions applied)"

Storing exclusions in a Bash array (EXCLUDES=(...)) and expanding it with "${EXCLUDES[@]}" keeps each --exclude pattern as a properly separated argument — critical, since a plain string with spaces would be misinterpreted by tar.

Step 3: Choosing Compression Algorithms

Different compression tools trade speed for ratio:

A script can expose this as an option:

#!/usr/bin/env bash
set -euo pipefail

SRC="$1"
COMPRESSION="${2:-gzip}"
NAME="${3:-$(basename "$SRC")}"
TIMESTAMP=$(date +%Y%m%d-%H%M%S)

case "$COMPRESSION" in
    gzip)  FLAG="z"; EXT="tar.gz"  ;;
    bzip2) FLAG="j"; EXT="tar.bz2" ;;
    xz)    FLAG="J"; EXT="tar.xz"  ;;
    none)  FLAG="";  EXT="tar"     ;;
    *)
        echo "Unknown compression: $COMPRESSION (use gzip|bzip2|xz|none)"
        exit 1
        ;;
esac

OUTPUT="${NAME}-${TIMESTAMP}.${EXT}"
tar -c${FLAG}f "$OUTPUT" "$SRC"

echo "Created $OUTPUT using $COMPRESSION compression"
echo "Size: $(du -h "$OUTPUT" | cut -f1)"

Run it with ./archive.sh ./project xz to get maximum compression at the cost of speed, or ./archive.sh ./project none for a raw, uncompressed tarball when speed matters more than size (common for local, short-lived backups).

Step 4: Adding Integrity Verification

A serious archiving tool should let you verify the archive wasn’t corrupted:

CHECKSUM=$(sha256sum "$OUTPUT" | awk '{print $1}')
echo "$CHECKSUM  $OUTPUT" > "${OUTPUT}.sha256"
echo "Checksum saved: ${OUTPUT}.sha256"

Later, integrity can be confirmed with:

sha256sum -c "${OUTPUT}.sha256"

Output on success:

project-20260728-144512.tar.gz: OK

This is especially important for archives moved across networks or stored long-term, where silent corruption is a real risk.

Step 5: Full-Featured Archiving Script

Putting it all together:

#!/usr/bin/env bash
set -euo pipefail

usage() {
    echo "Usage: $0 <source> [-c compression] [-o output_dir] [-e exclude_pattern ...]"
    exit 1
}

SRC=""
COMPRESSION="gzip"
OUTDIR="."
EXCLUDES=()

SRC="$1"; shift || usage

while getopts ":c:o:e:" opt; do
    case "$opt" in
        c) COMPRESSION="$OPTARG" ;;
        o) OUTDIR="$OPTARG" ;;
        e) EXCLUDES+=("--exclude=$OPTARG") ;;
        *) usage ;;
    esac
done

[[ -e "$SRC" ]] || { echo "Source not found: $SRC"; exit 1; }
mkdir -p "$OUTDIR"

case "$COMPRESSION" in
    gzip)  FLAG="z"; EXT="tar.gz"  ;;
    bzip2) FLAG="j"; EXT="tar.bz2" ;;
    xz)    FLAG="J"; EXT="tar.xz"  ;;
    none)  FLAG="";  EXT="tar"     ;;
    *) echo "Unknown compression: $COMPRESSION"; exit 1 ;;
esac

NAME="$(basename "$SRC")"
TIMESTAMP=$(date +%Y%m%d-%H%M%S)
OUTPUT="${OUTDIR}/${NAME}-${TIMESTAMP}.${EXT}"

tar -c${FLAG}f "$OUTPUT" "${EXCLUDES[@]}" "$SRC"

CHECKSUM=$(sha256sum "$OUTPUT" | awk '{print $1}')
echo "$CHECKSUM  $(basename "$OUTPUT")" > "${OUTPUT}.sha256"

echo "Archive created: $OUTPUT"
echo "Size: $(du -h "$OUTPUT" | cut -f1)"
echo "Checksum: ${OUTPUT}.sha256"

Example:

./archive.sh ./project -c xz -o ./backups -e .git -e node_modules

This uses getopts to parse named flags (-c, -o, -e), a more scalable pattern than positional arguments once a script grows past two or three parameters. -e can be repeated to add multiple exclusion patterns, each appended to the EXCLUDES array.

Real-World Use Cases

• Nightly backups: Cron a call to this script against a home directory or database dump folder.
• Pre-deployment snapshots: Archive the current production directory before deploying new code, as a rollback safety net.
• Log archiving: Compress and move rotated logs older than a threshold into cold storage.
• Release packaging: Build distributable .tar.gz releases of a project for GitHub Releases or internal distribution.

Automation Example: Scheduled Backup via Cron

# crontab -e
0 2 * * * /home/user/scripts/archive.sh /home/user/data -c gzip -o /mnt/backups >> /home/user/backup.log 2>&1

This runs the archiving script every night at 2 AM, redirecting both stdout and stderr into a log file for later review.

Best Practices

• Always timestamp archive filenames to avoid accidental overwrites.
• Store exclusions in an array, never a single string, to avoid word-splitting bugs.
• Generate and store checksums alongside every archive meant for long-term storage or transfer.
• Prefer xz for archival (space matters more, time is less critical) and gzip for frequent, quick backups (speed matters more).
• Test restores periodically — an untested backup is not a backup.

Security Considerations

• Archives of sensitive directories (SSH keys, credentials, database dumps) should be encrypted, not just compressed — see the companion encryption utility article for how to pipe archives through gpg or openssl.
• Set restrictive permissions on archive output directories (chmod 700) since a .tar.gz of a home directory can contain highly sensitive data in plaintext.
• Be cautious archiving symlinks (tar follows them by default unless -h is omitted) — this can pull in unexpected content or, in adversarial contexts, be used for path traversal on extraction.

Optimization Tips

• Use pigz or pbzip2 for multi-core compression speedup on large archives: tar -cf - "$SRC" | pigz > "$OUTPUT".
• For very large directories, consider --exclude-from=file.txt to manage a long exclusion list in a separate file rather than many repeated -e flags.
• Use --one-file-system when archiving a mount point to avoid unintentionally archiving other mounted filesystems nested inside it.

Troubleshooting

• **”tar: Removing leading /' from member names"**: This is a warning, not an error — tar` strips absolute paths for portability by default; harmless in most cases.
• Archive is unexpectedly large: Check exclusions are actually being applied; tar --exclude patterns must match relative to the archived path, not absolute paths.
• Checksum mismatch after transfer: Indicates corruption in transit; re-transfer using a method that verifies integrity (e.g., rsync -c).

Common Mistakes

• Using a bare string for multiple exclude patterns instead of an array, causing tar to receive a single malformed argument.
• Forgetting to mkdir -p the output directory before writing the archive there.
• Not accounting for filenames with spaces or special characters when scripting around the archive step.

FAQs

Should I always compress archives? Not always — for very large binary files that are already compressed (e.g., video, images), further compression provides little benefit and just costs CPU time. Uncompressed tar (none) may be faster in those cases.

How do I archive multiple separate directories into one file? Pass multiple source paths to tar, or list them in the SRC handling loop: tar -czf "$OUTPUT" dir1 dir2 dir3.

Can this be combined with the unarchiving tool from the companion article? Yes — together they form a symmetric pair: one bundles, the other extracts, both using the same naming and compression conventions.

Summary

A well-designed archiving script turns a routine but detail-sensitive task into a single reliable command. By layering in configurable compression, exclusion handling, checksums, and sensible defaults, the script scales from “quick manual backup” to “scheduled, auditable, production-grade archival process” without needing anything beyond standard Bash and tar.

References

• GNU Tar Manual
• GNU Bash Reference Manual
• GNU Coreutils: sha256sum
• pigz project page