Offensive Security Researcher & Quantum Cryptography Analyst

How to Create a Bash File Compression Tool

Abdul Wahab Junaid
How to Create a Bash File Compression Tool

How to Create a Bash File Compression Tool

Compression is one of those things that feels simple until you actually need to do it well. Sure, everyone knows tar -czf archive.tar.gz folder/, but there’s a lot more nuance once you start caring about compression ratio, speed, format compatibility, and automation. I ended up building my own Bash compression tool after getting tired of manually picking flags every time, and in this article I’ll walk you through the entire process — from the simplest possible version to a genuinely useful, production-ready script.

Why Build a Custom Compression Tool

The honest answer: convenience and consistency. I wanted one command that could:

Let’s build it step by step.

Prerequisites

sudo apt install tar gzip bzip2 xz-utils zip p7zip-full

Step 1: The Basic Compression Script

#!/usr/bin/env bash
set -euo pipefail

SOURCE="${1:-}"
FORMAT="${2:-gz}"

if [[ -z "$SOURCE" ]]; then
    echo "Usage: $0 <file-or-folder> [gz|bz2|xz|zip]"
    exit 1
fi

if [[ ! -e "$SOURCE" ]]; then
    echo "Error: '$SOURCE' does not exist."
    exit 1
fi

BASENAME=$(basename -- "$SOURCE")

case "$FORMAT" in
    gz)
        tar -czf "${BASENAME}.tar.gz" "$SOURCE"
        OUTPUT="${BASENAME}.tar.gz"
        ;;
    bz2)
        tar -cjf "${BASENAME}.tar.bz2" "$SOURCE"
        OUTPUT="${BASENAME}.tar.bz2"
        ;;
    xz)
        tar -cJf "${BASENAME}.tar.xz" "$SOURCE"
        OUTPUT="${BASENAME}.tar.xz"
        ;;
    zip)
        zip -qr "${BASENAME}.zip" "$SOURCE"
        OUTPUT="${BASENAME}.zip"
        ;;
    *)
        echo "Error: Unsupported format '$FORMAT'. Choose gz, bz2, xz, or zip."
        exit 1
        ;;
esac

echo "Created: $OUTPUT"

How This Works

Step 2: Showing Compression Ratio

Knowing how much space you actually saved is genuinely useful, so let’s add that:

ORIGINAL_SIZE=$(du -sb "$SOURCE" | cut -f1)
COMPRESSED_SIZE=$(stat -c "%s" "$OUTPUT" 2>/dev/null || stat -f "%z" "$OUTPUT")

RATIO=$(awk "BEGIN { printf \"%.1f\", ($ORIGINAL_SIZE - $COMPRESSED_SIZE) / $ORIGINAL_SIZE * 100 }")

echo "Original size:   $(numfmt --to=iec-i --suffix=B "$ORIGINAL_SIZE")"
echo "Compressed size: $(numfmt --to=iec-i --suffix=B "$COMPRESSED_SIZE")"
echo "Space saved:     ${RATIO}%"

Explaining This Section

Step 3: Excluding Unwanted Files and Folders

This is where a custom tool really starts to shine over typing raw tar commands. Let’s add an exclude list:

EXCLUDES=(".git" "node_modules" "*.log" "__pycache__")

EXCLUDE_ARGS=()
for pattern in "${EXCLUDES[@]}"; do
    EXCLUDE_ARGS+=(--exclude="$pattern")
done

tar -czf "${BASENAME}.tar.gz" "${EXCLUDE_ARGS[@]}" "$SOURCE"

The Mechanics

Step 4: Putting It All Together

Here’s the complete, polished script:

#!/usr/bin/env bash
set -euo pipefail

SOURCE="${1:-}"
FORMAT="${2:-gz}"
EXCLUDES=(".git" "node_modules" "*.log" "__pycache__")

if [[ -z "$SOURCE" || ! -e "$SOURCE" ]]; then
    echo "Usage: $0 <file-or-folder> [gz|bz2|xz|zip]"
    exit 1
fi

BASENAME=$(basename -- "$SOURCE")
EXCLUDE_ARGS=()
for pattern in "${EXCLUDES[@]}"; do
    EXCLUDE_ARGS+=(--exclude="$pattern")
done

case "$FORMAT" in
    gz)  OUTPUT="${BASENAME}.tar.gz"; tar -czf "$OUTPUT" "${EXCLUDE_ARGS[@]}" "$SOURCE" ;;
    bz2) OUTPUT="${BASENAME}.tar.bz2"; tar -cjf "$OUTPUT" "${EXCLUDE_ARGS[@]}" "$SOURCE" ;;
    xz)  OUTPUT="${BASENAME}.tar.xz"; tar -cJf "$OUTPUT" "${EXCLUDE_ARGS[@]}" "$SOURCE" ;;
    zip) OUTPUT="${BASENAME}.zip"; zip -qr "$OUTPUT" "$SOURCE" -x ".git/*" "node_modules/*" ;;
    *) echo "Unsupported format: $FORMAT"; exit 1 ;;
esac

ORIGINAL_SIZE=$(du -sb "$SOURCE" | cut -f1)
COMPRESSED_SIZE=$(stat -c "%s" "$OUTPUT" 2>/dev/null || stat -f "%z" "$OUTPUT")
RATIO=$(awk "BEGIN { printf \"%.1f\", ($ORIGINAL_SIZE - $COMPRESSED_SIZE) / $ORIGINAL_SIZE * 100 }")

echo "Created: $OUTPUT"
echo "Original size:   $(numfmt --to=iec-i --suffix=B "$ORIGINAL_SIZE")"
echo "Compressed size: $(numfmt --to=iec-i --suffix=B "$COMPRESSED_SIZE")"
echo "Space saved:     ${RATIO}%"

Sample output:

Created: myproject.tar.gz
Original size:   128MiB
Compressed size: 31MiB
Space saved:     75.8%

Real-World Use Cases

Automation Example

Compress and timestamp a project folder nightly:

0 1 * * * /usr/local/bin/compress.sh /var/www/myproject gz && mv myproject.tar.gz /backups/myproject-$(date +\%F).tar.gz

This runs at 1 AM, compresses the project directory, and renames the output with the current date for easy identification later.

Security Considerations

Optimization Tips

Troubleshooting

Common Mistakes to Avoid

Frequently Asked Questions

Which compression format should I use by default? For general Linux-to-Linux use, tar.gz is a safe, fast, widely-compatible default. Use .zip when sharing with Windows users, and .xz when you need maximum compression and don’t mind the extra time.

Can I encrypt the archive at the same time as compressing it? Yes — pipe the output through gpg: tar -czf - folder/ | gpg -c -o archive.tar.gz.gpg. This compresses and symmetrically encrypts in one pipeline.

How do I compress without including the parent directory structure? cd into the folder first, then compress its contents with . as the source, rather than compressing the folder from outside.

Summary

We built a flexible Bash compression tool that supports multiple formats, reports compression ratios, and intelligently excludes files you never want archived. The underlying lesson is that tar‘s consistent flag structure (-c, -z/-j/-J, -f) makes it easy to build a single script that adapts to different compression algorithms, and that small usability touches — like human-readable size output — make a world of difference in a tool you’ll use daily.

References

Exit mobile version