hashdeep: A tool for computing and verifying hash values of files in a directory

Every digital forensics investigation I’ve worked on starts with the same non-negotiable step: proving that the evidence I’m analyzing hasn’t changed since I acquired it. That’s where Hashdeep comes in. It’s a set of cross-platform tools for computing hashes of arbitrary numbers of files, recursively through entire directory trees, and — critically — for comparing sets of hashes against each other to detect additions, deletions, or modifications. Here’s everything I’ve learned using it in real forensic and incident response work.

What Hashdeep Is and How It Works

Hashdeep (part of the broader “md5deep” package, which now supports multiple algorithms) is a command-line utility that:

1. Recursively walks a directory tree
2. Computes one or more cryptographic hashes (MD5, SHA-1, SHA-256, SHA-512, Tiger, Whirlpool) for every file
3. Outputs results in a standard format that can later be used for audit mode — comparing a known-good hash set against a target directory to flag matches, new files, missing files, and modified files

Internally it’s written in C, using the OpenSSL/hash libraries for computation and a multi-threaded engine for speed on large datasets. This makes it noticeably faster than looping sha256sum over files individually in a shell script.

Installing Hashdeep

On Debian/Ubuntu/Kali:

sudo apt update
sudo apt install -y hashdeep

Verify:

hashdeep -h

On macOS via Homebrew:

brew install hashdeep

Basic Syntax

hashdeep [options] [FILES/DIRECTORIES]

Key options I use constantly:

hashdeep -r /evidence/case001/         # recursive hashing of a directory
hashdeep -c sha256 -r /evidence/       # specify algorithm(s)
hashdeep -r /evidence/ > case001.hashes.txt   # save output to a file
hashdeep -a -k known.hashes -r /target/       # audit mode against a known set
hashdeep -X exclude_known.txt -r /target/     # negative match mode

Real Example

Generating a baseline hash set for a directory of extracted evidence:

$ hashdeep -c sha256 -r /evidence/case001/ > case001_baseline.txt
$ cat case001_baseline.txt
%%%% HASHDEEP-1.0
%%%% size,sha256,filename
##
## Invoked from: /home/analyst
## $ hashdeep -c sha256 -r /evidence/case001/
##
1048576,3a7bd3e2360a3d...,/evidence/case001/disk.img
2048,f5c8e1a0b9d2c3...,/evidence/case001/notes.txt

Later, during audit mode, I re-hash the same directory and compare:

$ hashdeep -a -k case001_baseline.txt -r /evidence/case001/
hashdeep: Audit failed
   Input files examined: 2
  Known files expecting: 2
        Files matched: 1
Files partially matched: 0
            Files moved: 0
        New files found: 0
  Known files not found: 1

That “Known files not found” line is exactly the alarm bell I’m looking for — it tells me instantly that a file changed, was deleted, or was replaced since the baseline was taken, which is critical evidence in itself during an investigation.

Real-World Use Cases

Chain of custody verification — I hash every piece of digital evidence immediately upon acquisition (disk images, memory dumps, extracted files) and store those hashes in the case file (often pasted directly into CherryTree, as covered in that article).

Malware/incident response triage — comparing hashes of a compromised system’s binaries against a known-clean baseline from a gold image to spot tampered system files.

Data exfiltration/integrity monitoring — periodically re-hashing a monitored directory to detect unauthorized file changes on a critical server.

Integration with Other Tools

• Guymager/dd — after imaging a disk, I immediately hashdeep the resulting image file(s) alongside the tool’s own built-in hash verification for a second, independent confirmation.
• Autopsy — hash sets generated with Hashdeep can be imported as known-file hash databases for filtering in Autopsy’s triage workflow.
• Bulk_extractor — I run bulk_extractor and hashdeep in the same evidence-processing pipeline, one for content carving, the other for integrity assurance.

Performance and Troubleshooting

• Use -p <size> to control piecewise hashing (splitting large files into blocks) when you need to detect partial modification within a huge file like a disk image.
• On very large evidence sets, hashing is I/O bound; running from a fast SSD write-blocker setup rather than a slow USB 2.0 device makes a dramatic difference in processing time.
• A mistake I made early in my career: forgetting the -r flag and assuming subdirectories were included by default — they are not; without -r, hashdeep only processes files in the exact paths given.

Best Practices

• Always generate and store the baseline hash file itself with its own hash and store it separately (or sign it) so the baseline can’t be silently altered.
• Use SHA-256 or better for anything with legal/evidentiary weight — MD5 and SHA-1 are still supported but are considered weak for collision resistance.
• Document the exact command line used to generate hashes directly in your case notes for reproducibility.

FAQ

What’s the difference between hashdeep and md5deep? They’re part of the same project family; md5deep computes a single algorithm’s hash, while hashdeep supports multiple algorithms and adds the audit/comparison mode.

Can Hashdeep detect renamed files? Yes — in audit mode it can identify a “moved” file when the hash matches but the path/name changed.

Is Hashdeep suitable for legal/court-admissible evidence handling? It’s widely used in the forensics community for exactly that purpose, provided it’s part of a properly documented chain-of-custody process.

Summary

Hashdeep is one of those unglamorous but absolutely essential tools — it’s the mathematical proof that the evidence I analyzed today is the same evidence I collected on day one. I run it as one of the very first commands in nearly every forensic case I open.

References

• Official SourceForge project (md5deep/hashdeep): https://github.com/jessek/hashdeep
• Man page: man hashdeep