Offensive Security Researcher & Quantum Cryptography Analyst

How to Create a Bash File Joiner

Abdul Wahab Junaid
How to Create a Bash File Joiner

How to Create a Bash File Joiner

If you’ve ever split a large video, backup archive, or log file into smaller chunks, you already know the pain of putting those pieces back together. I’ve dealt with this problem more times than I’d like to admit, especially when moving large files across systems with strict upload limits. Instead of relying on third-party GUI tools, I ended up writing my own Bash file joiner, and honestly, it’s one of the simplest yet most useful scripts I’ve built.

In this article, I’ll walk you through everything I know about creating a Bash file joiner, from the basic concept to a production-ready script you can actually rely on.

What Is a File Joiner and Why Bash Is a Great Fit

A file joiner is a utility that takes multiple file parts (usually named something like file.part001, file.part002, etc.) and concatenates them back into a single, original file. This is the reverse operation of a file splitter.

Bash is a great fit for this task because:

The Core Concept

At its heart, joining files in Bash is deceptively simple:

cat file.part001 file.part002 file.part003 > original_file.zip

That’s it. The cat command reads each file in sequence and writes the combined byte stream to a new file. The real engineering work is in making this reliable: handling arbitrary numbers of parts, verifying integrity, and giving the user good feedback.

Building a Basic Bash File Joiner

Let’s start with a beginner-friendly version.

#!/bin/bash

# join.sh - A basic file joiner

output_file="joined_output.bin"

cat part_* > "$output_file"

echo "Files joined into $output_file"

Here’s what’s happening internally:

This works fine for quick, one-off jobs, but it has weaknesses: no error handling, no validation, and it assumes alphabetical order matches the correct part order.

A More Robust Version

Here’s an intermediate script that adds proper argument handling and safety checks.

#!/bin/bash

set -euo pipefail

usage() {
    echo "Usage: $0 <output_file> <part1> <part2> [part3 ...]"
    exit 1
}

if [ "$#" -lt 2 ]; then
    usage
fi

output_file="$1"
shift

if [ -e "$output_file" ]; then
    read -rp "Output file '$output_file' already exists. Overwrite? (y/n): " confirm
    if [[ "$confirm" != "y" ]]; then
        echo "Aborted."
        exit 1
    fi
fi

: > "$output_file"  # truncate/create output file

for part in "$@"; do
    if [ ! -f "$part" ]; then
        echo "Error: part file '$part' not found." >&2
        exit 1
    fi
    cat "$part" >> "$output_file"
    echo "Joined: $part"
done

echo "All parts joined successfully into $output_file"

How This Script Works Internally

Automatically Detecting and Ordering Parts

Manually listing part files gets tedious. Here’s a version that automatically detects parts based on a naming pattern and sorts them numerically instead of alphabetically (which matters once you pass part010, part011, etc.).

#!/bin/bash

set -euo pipefail

prefix="$1"
output_file="$2"

mapfile -t parts < <(find . -maxdepth 1 -name "${prefix}*" -print | sort -V)

if [ "${#parts[@]}" -eq 0 ]; then
    echo "No parts found matching prefix '$prefix'" >&2
    exit 1
fi

: > "$output_file"

for part in "${parts[@]}"; do
    cat "$part" >> "$output_file"
done

echo "Joined ${#parts[@]} parts into $output_file"

The key detail here is sort -V, which performs a “version sort.” This ensures part2 comes before part10, unlike a plain alphabetical sort where part10 would incorrectly come before part2.

Verifying Integrity After Joining

Joining files is only half the job — you also want to confirm the result matches the original. I always add a checksum step when I’m dealing with anything important.

sha256sum joined_output.bin

If you had the original file’s checksum saved before splitting, compare it like this:

echo "expected_checksum  joined_output.bin" | sha256sum -c -

This tells you instantly whether the reassembled file is byte-for-byte identical to the original.

Real-World Use Cases

Automation Example

Here’s how I integrate a joiner into a cron-based automation that watches a folder for completed part sets and joins them automatically:

#!/bin/bash

set -euo pipefail

watch_dir="/data/incoming"
output_dir="/data/processed"

for prefix in $(find "$watch_dir" -name "*.part001" -exec basename {} .part001 \;); do
    output_file="$output_dir/${prefix}.bin"
    cat "$watch_dir/${prefix}".part* > "$output_file"
    echo "$(date): Joined $prefix into $output_file" >> /var/log/joiner.log
done

This can run every few minutes via cron to automatically process incoming file parts without manual intervention.

Best Practices

Security Considerations

Optimization Tips

Troubleshooting Common Issues

Problem: The joined file is corrupted or won’t open. Check that all parts were included and in the correct order. Run sha256sum on the result and compare it against the original checksum.

Problem: “Argument list too long” error. This happens when using a glob like part_* with an extremely large number of files. Switch to find combined with sort -V and a loop instead of relying on shell globbing.

Problem: Parts joined in the wrong order. This is almost always due to alphabetical vs. numeric sorting issues. Use sort -V instead of the default sort.

Problem: Permission denied when writing output. Check that you have write permissions in the output directory, and that the disk isn’t full.

Common Mistakes to Avoid

Frequently Asked Questions

Can I join files without knowing the original filename? Yes. You can name the output file however you like — the joiner doesn’t need to know the original name unless it was stored in a manifest file during splitting.

Does the joining order really matter? Absolutely. Files must be joined in the exact order they were split, or the resulting file will be corrupted.

Can this handle binary files like videos or ZIP archives? Yes. cat operates on raw bytes, so it works identically for text and binary files.

What if I don’t know how many parts there are? Use a glob pattern or find with a prefix match, then sort the results — no need to know the exact count in advance.

Is there a size limit to how large a joined file can be? The only real limit is your filesystem’s maximum file size and available disk space.

Summary

Building a Bash file joiner is one of those small utilities that pays for itself the first time you need it. Starting from a simple cat part1 part2 > output, you can grow the script into something that automatically detects parts, sorts them correctly, verifies integrity with checksums, and integrates cleanly into automated pipelines. The key lessons I’ve learned: always sort numerically, always verify with a checksum, and always handle missing files gracefully before you commit to writing output.

References

Exit mobile version