How to Create a Bash File Comparer

Diffing text files is one thing — I covered that in the Bash File Diff Tool article in this series. But sometimes what I actually need is a different question entirely: “are these two files identical?” Not a line-by-line breakdown, just a fast, reliable yes-or-no answer, ideally one that works just as well on binaries, images, and archives as it does on text. That’s what pushed me to build a dedicated file comparer, and this article walks through exactly how it works.

Comparer vs. Diff Tool — What’s the Actual Difference?

It’s worth being precise about this distinction, since the two tools solve related but different problems:

• A diff tool shows you what changed, line by line, and only really makes sense for text content.
• A comparer answers whether two files are the same, using checksums or byte-level comparison, and works equally well on any file type — text, binary, images, executables, archives.

If you need both, use them together: the comparer tells you that something changed, and the diff tool (for text files) tells you what changed.

Prerequisites

sudo apt install coreutils diffutils

sha256sum, md5sum, and cmp are all part of GNU Coreutils and are installed by default on virtually every Linux system.

Step 1: The Simplest Possible Comparer — Checksums

#!/usr/bin/env bash
set -euo pipefail

FILE_A="${1:-}"
FILE_B="${2:-}"

if [[ -z "$FILE_A" || -z "$FILE_B" ]]; then
    echo "Usage: $0 <file-A> <file-B>"
    exit 1
fi

if [[ ! -f "$FILE_A" || ! -f "$FILE_B" ]]; then
    echo "Error: both arguments must be existing files."
    exit 1
fi

HASH_A=$(sha256sum "$FILE_A" | awk '{print $1}')
HASH_B=$(sha256sum "$FILE_B" | awk '{print $1}')

if [[ "$HASH_A" == "$HASH_B" ]]; then
    echo "IDENTICAL: files have the same content."
else
    echo "DIFFERENT: files have different content."
fi

How This Works

• sha256sum "$FILE_A" computes a SHA-256 cryptographic hash of the file’s contents and prints it alongside the filename, separated by whitespace. awk '{print $1}' grabs just the hash itself, discarding the filename.
• Comparing two SHA-256 hashes is a mathematically reliable way to determine whether two files are byte-for-byte identical — the probability of two different files producing the same hash (a “collision”) is astronomically small, making this approach far more trustworthy than comparing file sizes or timestamps alone.
• This approach works identically well on text files, images, videos, executables, or archives, because hashing operates on raw bytes, not on any particular file format.

Step 2: Speeding Things Up With a Size Pre-Check

Hashing large files takes time. Before we bother hashing anything, we can rule out a mismatch instantly if the file sizes differ:

SIZE_A=$(stat -c "%s" "$FILE_A" 2>/dev/null || stat -f "%z" "$FILE_A")
SIZE_B=$(stat -c "%s" "$FILE_B" 2>/dev/null || stat -f "%z" "$FILE_B")

if [[ "$SIZE_A" != "$SIZE_B" ]]; then
    echo "DIFFERENT: file sizes don't match ($SIZE_A vs $SIZE_B bytes)."
    exit 0
fi

echo "Sizes match ($SIZE_A bytes). Verifying content with checksums..."

HASH_A=$(sha256sum "$FILE_A" | awk '{print $1}')
HASH_B=$(sha256sum "$FILE_B" | awk '{print $1}')

if [[ "$HASH_A" == "$HASH_B" ]]; then
    echo "IDENTICAL: files have the same content."
else
    echo "DIFFERENT: files have different content despite matching size."
fi

This is a classic short-circuit optimization: two files with different sizes can never be identical, so there’s no point computing an expensive hash in that case — we save real time on large files that are obviously different.

Step 3: Using cmp for a Faster First-Byte-Difference Check

If you don’t actually need a cryptographic guarantee and just want the fastest possible comparison, cmp is a great alternative to hashing — it compares byte-by-byte and stops at the very first difference it finds, rather than reading the entire file:

if cmp -s "$FILE_A" "$FILE_B"; then
    echo "IDENTICAL (verified via byte comparison)."
else
    DIFF_POINT=$(cmp "$FILE_A" "$FILE_B" 2>&1 | awk '{print $NF}')
    echo "DIFFERENT: first difference found near byte $DIFF_POINT."
fi

cmp -s runs silently and just sets an exit code (0 for identical, 1 for different), which is perfect for use inside an if condition. Without -s, cmp prints the byte and line number of the first mismatch, which we capture for a more informative message.

Step 4: Extending to Directory-Wide Comparison

Let’s build a mode that compares every matching file between two directory trees, useful for verifying a backup or sync operation actually worked correctly:

compare_directories() {
    local dir_a="$1"
    local dir_b="$2"
    local identical=0
    local different=0
    local missing=0

    while IFS= read -r -d '' file_a; do
        rel_path="${file_a#"$dir_a"/}"
        file_b="${dir_b}/${rel_path}"

        if [[ ! -f "$file_b" ]]; then
            echo "MISSING in $dir_b: $rel_path"
            ((missing++))
            continue
        fi

        if cmp -s "$file_a" "$file_b"; then
            ((identical++))
        else
            echo "DIFFERS: $rel_path"
            ((different++))
        fi
    done < <(find "$dir_a" -type f -print0)

    echo ""
    echo "Summary: $identical identical, $different different, $missing missing"
}

Breaking Down the Directory Comparison

• find "$dir_a" -type f -print0 prints each found file path separated by a null byte (\0) instead of a newline. Combined with read -r -d '' (reading until a null delimiter), this is the standard, robust way to handle filenames that might contain spaces, newlines, or other unusual characters — a normal newline-based loop would break on such filenames.
• rel_path="${file_a#"$dir_a"/}" uses Bash parameter expansion to strip the dir_a prefix from the full path, leaving just the relative path — this lets us construct the corresponding path inside dir_b for comparison.
• We use cmp -s here rather than full SHA-256 hashing for speed, since directory comparisons often involve many files and cmp‘s early-exit behavior on the first difference makes it noticeably faster in aggregate.

Full Combined Script

#!/usr/bin/env bash
set -euo pipefail

PATH_A="${1:-}"
PATH_B="${2:-}"

if [[ -z "$PATH_A" || -z "$PATH_B" ]]; then
    echo "Usage: $0 <path-A> <path-B>"
    exit 1
fi

if [[ ! -e "$PATH_A" || ! -e "$PATH_B" ]]; then
    echo "Error: one or both paths do not exist."
    exit 1
fi

if [[ -d "$PATH_A" && -d "$PATH_B" ]]; then
    identical=0; different=0; missing=0
    while IFS= read -r -d '' file_a; do
        rel_path="${file_a#"$PATH_A"/}"
        file_b="${PATH_B}/${rel_path}"
        if [[ ! -f "$file_b" ]]; then
            echo "MISSING in $PATH_B: $rel_path"
            ((missing++))
        elif cmp -s "$file_a" "$file_b"; then
            ((identical++))
        else
            echo "DIFFERS: $rel_path"
            ((different++))
        fi
    done < <(find "$PATH_A" -type f -print0)
    echo ""
    echo "Summary: $identical identical, $different different, $missing missing"

elif [[ -f "$PATH_A" && -f "$PATH_B" ]]; then
    SIZE_A=$(stat -c "%s" "$PATH_A" 2>/dev/null || stat -f "%z" "$PATH_A")
    SIZE_B=$(stat -c "%s" "$PATH_B" 2>/dev/null || stat -f "%z" "$PATH_B")

    if [[ "$SIZE_A" != "$SIZE_B" ]]; then
        echo "DIFFERENT: sizes don't match ($SIZE_A vs $SIZE_B bytes)."
        exit 0
    fi

    if cmp -s "$PATH_A" "$PATH_B"; then
        echo "IDENTICAL: files match byte-for-byte."
    else
        echo "DIFFERENT: files have the same size but different content."
    fi
else
    echo "Error: both arguments must be the same type (both files or both directories)."
    exit 1
fi

Real-World Use Cases

• Verifying a downloaded file wasn’t corrupted, by comparing its checksum against a known-good reference.
• Confirming a backup or sync actually completed correctly, using the directory comparison mode.
• Deduplication workflows, identifying files with identical content but different names before cleaning up storage.
• Software release verification, confirming a locally built binary matches an officially published one.

Automation Ideas

Verify a nightly sync job actually resulted in matching directories, alerting on any mismatch:

0 4 * * * /usr/local/bin/comparer.sh /data/live /data/replica | grep -qE "DIFFERS|MISSING" && echo "Sync verification failed!" | mail -s "Sync Mismatch Detected" you@example.com

Security Considerations

• SHA-256 is appropriate for integrity verification; MD5 and SHA-1 are not, for security-sensitive contexts — both are considered cryptographically broken for adversarial scenarios (though still fine for simple accidental-corruption checks). Default to SHA-256 or better when the comparison matters for security.
• A matching checksum only proves content is identical — it says nothing about permissions, ownership, or extended attributes, which might matter in security-sensitive contexts like verifying restored system files.
• Be cautious comparing files across different filesystems with different line-ending conventions (like a repo checked out on Windows vs. Linux) — byte-level comparison will report these as “different” even though the visible text content might look the same.

Optimization Tips

• Use the size pre-check (Step 2) before any expensive hashing — it’s a near-zero-cost filter that immediately rules out obvious mismatches.
• cmp is generally faster than computing full SHA-256 hashes for a single pairwise comparison, since it can stop at the first difference; hashing is more valuable when you need a portable fingerprint to compare against a reference stored elsewhere (like a published checksum file).
• For directory comparisons involving thousands of files, consider parallelizing with xargs -P to compare multiple file pairs concurrently on multi-core systems.

Troubleshooting

• Checksums differ even though the files “look” identical when opened — check for invisible differences like line-ending style (CRLF vs LF), trailing whitespace, or byte-order marks (BOM) at the start of text files.
• “cmp: EOF on file_a” — this specific message means the files have different lengths and one ended before a difference was found in the shared portion; it usually indicates one file was truncated or is a partial/incomplete copy.
• Directory comparison reports “MISSING” for files that clearly exist — double check for symlinks or case-sensitivity mismatches between the two directory trees, especially when comparing across different filesystems.

Common Mistakes to Avoid

• Using file modification timestamps as a proxy for “identical content” — timestamps can differ even when content is byte-for-byte the same, and can match by coincidence when content actually differs.
• Choosing MD5 for anything security-sensitive out of habit; it’s fine for basic accidental-corruption checks but not for verifying authenticity against a malicious actor.
• Forgetting that directory comparison needs to check both directions — a file present in B but missing from A won’t be caught if you only iterate over A’s file list (extend the script to check both directions if that matters for your use case).

Frequently Asked Questions

Which is better for comparison: cmp, diff, or checksums? cmp is fastest for a single pairwise “are these identical?” check. diff is better when you need to see the actual content differences in text files. Checksums (sha256sum) are best when you need a portable fingerprint to verify against a value stored or published elsewhere, without having both files present at once.

Can I compare a local file against a remote file without downloading it fully? Yes, if the remote system can provide a checksum — many download mirrors publish .sha256 files alongside downloads specifically for this purpose, letting you verify integrity without a full local diff.

Does this handle very large files (multi-gigabyte) efficiently? cmp handles large files reasonably well since it stops at the first difference. Full SHA-256 hashing of very large files takes proportionally longer since it must read the entire file — the size pre-check in Step 2 helps avoid unnecessary full hashes.

Summary

We built a Bash file comparer that uses size pre-checks, byte-level comparison via cmp, and cryptographic checksums via sha256sum to reliably determine whether files or entire directory trees are identical — working equally well on text and binary content. The key distinction to remember from this article is that a comparer answers whether something changed, while a diff tool answers what changed — and knowing which question you’re actually asking will save you time reaching for the right tool.

References

• GNU Coreutils Manual (sha256sum, cmp)
• GNU Diffutils Manual
• Bash Reference Manual — GNU Project