Offensive Security Researcher & Quantum Cryptography Analyst

How to Create a Bash File Comparer

Abdul Wahab Junaid
How to Create a Bash File Comparer

How to Create a Bash File Comparer

Diffing text files is one thing — I covered that in the Bash File Diff Tool article in this series. But sometimes what I actually need is a different question entirely: “are these two files identical?” Not a line-by-line breakdown, just a fast, reliable yes-or-no answer, ideally one that works just as well on binaries, images, and archives as it does on text. That’s what pushed me to build a dedicated file comparer, and this article walks through exactly how it works.

Comparer vs. Diff Tool — What’s the Actual Difference?

It’s worth being precise about this distinction, since the two tools solve related but different problems:

If you need both, use them together: the comparer tells you that something changed, and the diff tool (for text files) tells you what changed.

Prerequisites

sudo apt install coreutils diffutils

sha256sum, md5sum, and cmp are all part of GNU Coreutils and are installed by default on virtually every Linux system.

Step 1: The Simplest Possible Comparer — Checksums

#!/usr/bin/env bash
set -euo pipefail

FILE_A="${1:-}"
FILE_B="${2:-}"

if [[ -z "$FILE_A" || -z "$FILE_B" ]]; then
    echo "Usage: $0 <file-A> <file-B>"
    exit 1
fi

if [[ ! -f "$FILE_A" || ! -f "$FILE_B" ]]; then
    echo "Error: both arguments must be existing files."
    exit 1
fi

HASH_A=$(sha256sum "$FILE_A" | awk '{print $1}')
HASH_B=$(sha256sum "$FILE_B" | awk '{print $1}')

if [[ "$HASH_A" == "$HASH_B" ]]; then
    echo "IDENTICAL: files have the same content."
else
    echo "DIFFERENT: files have different content."
fi

How This Works

Step 2: Speeding Things Up With a Size Pre-Check

Hashing large files takes time. Before we bother hashing anything, we can rule out a mismatch instantly if the file sizes differ:

SIZE_A=$(stat -c "%s" "$FILE_A" 2>/dev/null || stat -f "%z" "$FILE_A")
SIZE_B=$(stat -c "%s" "$FILE_B" 2>/dev/null || stat -f "%z" "$FILE_B")

if [[ "$SIZE_A" != "$SIZE_B" ]]; then
    echo "DIFFERENT: file sizes don't match ($SIZE_A vs $SIZE_B bytes)."
    exit 0
fi

echo "Sizes match ($SIZE_A bytes). Verifying content with checksums..."

HASH_A=$(sha256sum "$FILE_A" | awk '{print $1}')
HASH_B=$(sha256sum "$FILE_B" | awk '{print $1}')

if [[ "$HASH_A" == "$HASH_B" ]]; then
    echo "IDENTICAL: files have the same content."
else
    echo "DIFFERENT: files have different content despite matching size."
fi

This is a classic short-circuit optimization: two files with different sizes can never be identical, so there’s no point computing an expensive hash in that case — we save real time on large files that are obviously different.

Step 3: Using cmp for a Faster First-Byte-Difference Check

If you don’t actually need a cryptographic guarantee and just want the fastest possible comparison, cmp is a great alternative to hashing — it compares byte-by-byte and stops at the very first difference it finds, rather than reading the entire file:

if cmp -s "$FILE_A" "$FILE_B"; then
    echo "IDENTICAL (verified via byte comparison)."
else
    DIFF_POINT=$(cmp "$FILE_A" "$FILE_B" 2>&1 | awk '{print $NF}')
    echo "DIFFERENT: first difference found near byte $DIFF_POINT."
fi

cmp -s runs silently and just sets an exit code (0 for identical, 1 for different), which is perfect for use inside an if condition. Without -s, cmp prints the byte and line number of the first mismatch, which we capture for a more informative message.

Step 4: Extending to Directory-Wide Comparison

Let’s build a mode that compares every matching file between two directory trees, useful for verifying a backup or sync operation actually worked correctly:

compare_directories() {
    local dir_a="$1"
    local dir_b="$2"
    local identical=0
    local different=0
    local missing=0

    while IFS= read -r -d '' file_a; do
        rel_path="${file_a#"$dir_a"/}"
        file_b="${dir_b}/${rel_path}"

        if [[ ! -f "$file_b" ]]; then
            echo "MISSING in $dir_b: $rel_path"
            ((missing++))
            continue
        fi

        if cmp -s "$file_a" "$file_b"; then
            ((identical++))
        else
            echo "DIFFERS: $rel_path"
            ((different++))
        fi
    done < <(find "$dir_a" -type f -print0)

    echo ""
    echo "Summary: $identical identical, $different different, $missing missing"
}

Breaking Down the Directory Comparison

Full Combined Script

#!/usr/bin/env bash
set -euo pipefail

PATH_A="${1:-}"
PATH_B="${2:-}"

if [[ -z "$PATH_A" || -z "$PATH_B" ]]; then
    echo "Usage: $0 <path-A> <path-B>"
    exit 1
fi

if [[ ! -e "$PATH_A" || ! -e "$PATH_B" ]]; then
    echo "Error: one or both paths do not exist."
    exit 1
fi

if [[ -d "$PATH_A" && -d "$PATH_B" ]]; then
    identical=0; different=0; missing=0
    while IFS= read -r -d '' file_a; do
        rel_path="${file_a#"$PATH_A"/}"
        file_b="${PATH_B}/${rel_path}"
        if [[ ! -f "$file_b" ]]; then
            echo "MISSING in $PATH_B: $rel_path"
            ((missing++))
        elif cmp -s "$file_a" "$file_b"; then
            ((identical++))
        else
            echo "DIFFERS: $rel_path"
            ((different++))
        fi
    done < <(find "$PATH_A" -type f -print0)
    echo ""
    echo "Summary: $identical identical, $different different, $missing missing"

elif [[ -f "$PATH_A" && -f "$PATH_B" ]]; then
    SIZE_A=$(stat -c "%s" "$PATH_A" 2>/dev/null || stat -f "%z" "$PATH_A")
    SIZE_B=$(stat -c "%s" "$PATH_B" 2>/dev/null || stat -f "%z" "$PATH_B")

    if [[ "$SIZE_A" != "$SIZE_B" ]]; then
        echo "DIFFERENT: sizes don't match ($SIZE_A vs $SIZE_B bytes)."
        exit 0
    fi

    if cmp -s "$PATH_A" "$PATH_B"; then
        echo "IDENTICAL: files match byte-for-byte."
    else
        echo "DIFFERENT: files have the same size but different content."
    fi
else
    echo "Error: both arguments must be the same type (both files or both directories)."
    exit 1
fi

Real-World Use Cases

Automation Ideas

Verify a nightly sync job actually resulted in matching directories, alerting on any mismatch:

0 4 * * * /usr/local/bin/comparer.sh /data/live /data/replica | grep -qE "DIFFERS|MISSING" && echo "Sync verification failed!" | mail -s "Sync Mismatch Detected" you@example.com

Security Considerations

Optimization Tips

Troubleshooting

Common Mistakes to Avoid

Frequently Asked Questions

Which is better for comparison: cmp, diff, or checksums? cmp is fastest for a single pairwise “are these identical?” check. diff is better when you need to see the actual content differences in text files. Checksums (sha256sum) are best when you need a portable fingerprint to verify against a value stored or published elsewhere, without having both files present at once.

Can I compare a local file against a remote file without downloading it fully? Yes, if the remote system can provide a checksum — many download mirrors publish .sha256 files alongside downloads specifically for this purpose, letting you verify integrity without a full local diff.

Does this handle very large files (multi-gigabyte) efficiently? cmp handles large files reasonably well since it stops at the first difference. Full SHA-256 hashing of very large files takes proportionally longer since it must read the entire file — the size pre-check in Step 2 helps avoid unnecessary full hashes.

Summary

We built a Bash file comparer that uses size pre-checks, byte-level comparison via cmp, and cryptographic checksums via sha256sum to reliably determine whether files or entire directory trees are identical — working equally well on text and binary content. The key distinction to remember from this article is that a comparer answers whether something changed, while a diff tool answers what changed — and knowing which question you’re actually asking will save you time reaching for the right tool.

References

Exit mobile version