bulk_extractor: A tool for extracting useful data from large data sets, such as disk images

bulk_extractor is a high-speed digital forensics tool that scans a disk image, file, or directory of files and extracts useful information — such as email addresses, credit card numbers, URLs, domain names, phone numbers, EXIF metadata, search engine queries, and even fragments of encrypted or compressed data — without parsing or relying on the underlying file system structure. Because it never needs to fully understand a file system, bulk_extractor can process corrupted, damaged, or unusual file systems that would defeat file-system-aware tools like The Sleuth Kit.

Developed originally by Simson Garfinkel, bulk_extractor uses a collection of “scanners,” each responsible for recognizing a specific type of artifact, and runs them all in parallel across the input data using multiple threads, making it extremely fast even on very large images (multi-terabyte drives). It also includes a companion GUI/analysis tool, bulk_extractor viewer (BEViewer), and a bulk_diff.py/bulk_extractor histogram feature for identifying commonly repeated artifacts (useful for spotting exfiltrated data or repeated malicious indicators).

Key capabilities:

• Parallelized, file-system-independent scanning of raw data (disk images, memory dumps, files, directories)
• Extraction of email addresses, URLs, domains, phone numbers, credit card numbers, IP addresses
• EXIF metadata extraction from embedded JPEG images
• Extraction of Windows Registry artifacts, Internet history, and cache fragments
• Base64/gzip/zip decompression and re-scanning of decompressed content (“recursive” scanning)
• Histogram generation showing frequency of found artifacts (e.g., most common email address)
• Word list generation for password-cracking dictionary support (wordlist scanner)
• Optional restart/resume capability for extremely long-running jobs on huge images

Installation

Kali Linux (via APT):

sudo apt update
sudo apt install bulk-extractor -y

Verify installation:

bulk_extractor -V

Expected output:

bulk_extractor 2.0.4

Manual build from source (latest features):

sudo apt install build-essential libssl-dev libewf-dev flex byacc autoconf automake libtool -y
git clone https://github.com/simsong/bulk_extractor.git
cd bulk_extractor
./bootstrap.sh
./configure
make
sudo make install

Install the BEViewer GUI (optional, for reviewing results graphically):

sudo apt install bulk-extractor-gui -y

Syntax

bulk_extractor [options] -o <output_dir> <input_image_or_file>

Command-Line Options

Basic Usage

Step 1 — Run a full default scan on a disk image:

bulk_extractor -o /home/claude/be_output evidence.dd

Expected output:

bulk_extractor version: 2.0.4
Input file: evidence.dd
Output directory: /home/claude/be_output
Disk Size: 10737418240
Threads: 8
1: 0.00% Done  (00:00:00 elapsed)
...
100.00% Done. Elapsed time: 00:04:12
All Threads Finished!

Step 2 — Review generated feature files:

ls /home/claude/be_output/

Expected output:

email.txt        url.txt        ccn.txt         domain.txt
exif.txt         telephone.txt  ip.txt          report.xml

Step 3 — Inspect a specific feature file:

cat /home/claude/be_output/email.txt | head -5

Expected output:

# BANNER FILE NOT PROVIDED (-b option)
# EMAIL RFC822
# Feature-Recorder: email
1048576	suspect@example.com	suspect@example.com
2097152	admin@company.org	admin@company.org

Practical Examples with Output

Example 1: Basic scan of an evidence image

bulk_extractor -o /home/claude/case1_be evidence.dd

Output:

Phase 1: Scanning image.
100.00% Done. Files created: 42
Elapsed time: 00:05:33

Example 2: Extracting only email and URL artifacts

bulk_extractor -x all -e email -e url -o /home/claude/email_url_only evidence.dd

Output:

Scanners enabled: email, url
Found 214 email addresses, 892 URLs.

Example 3: Listing available scanners

bulk_extractor -h

Output:

Scanners:
  accts       - Credit card / financial account numbers
  email       - Email addresses
  exif        - EXIF metadata from JPEGs
  gps         - GPS coordinates
  net         - Network packets (pcap-like)
  url         - URLs and web history
  wordlist    - Generate password-cracking word list
  winlnk      - Windows shortcut (.lnk) files
  winprefetch - Windows prefetch files

Example 4: Scanning with a custom thread count for performance tuning

bulk_extractor -j 16 -o /home/claude/fast_scan evidence.dd

Output:

Threads: 16
100.00% Done. Elapsed time: 00:02:47

Example 5: Extracting EXIF metadata from an image-heavy evidence set

bulk_extractor -E exif -o /home/claude/exif_only evidence.dd
cat /home/claude/exif_only/exif.txt | head -3

Output:

1572864  {"Make":"Apple","Model":"iPhone 13","GPSLatitude":"31.5204","GPSLongitude":"74.3587"}

Example 6: Generating a password wordlist from disk content

bulk_extractor -e wordlist -o /home/claude/wordlist_scan evidence.dd
wc -l /home/claude/wordlist_scan/wordlist.txt

Output:

48213 /home/claude/wordlist_scan/wordlist.txt

Example 7: Scanning a directory of loose files instead of a disk image

bulk_extractor -R -o /home/claude/dir_scan /home/claude/extracted_files/

Output:

Processing directory recursively: /home/claude/extracted_files/
Files processed: 3,204
100.00% Done.

Example 8: Generating histograms for common artifacts

ls /home/claude/case1_be/*histogram*

Output:

email_histogram.txt  domain_histogram.txt  url_histogram.txt

head -5 /home/claude/case1_be/email_histogram.txt

Output:

n=214	suspect@example.com
n=87	admin@company.org
n=42	noreply@service.com

Example 9: Extracting network artifacts (packets embedded in memory/disk)

bulk_extractor -E net -o /home/claude/net_scan memdump.raw
cat /home/claude/net_scan/packets.pcap | file -

Output:

/dev/stdin: tcpdump capture file (little-endian)

Example 10: Using a custom regex search for a specific case number pattern

echo 'CASE-[0-9]{6}' > custom_regex.txt
bulk_extractor -r custom_regex.txt -o /home/claude/custom_regex_scan evidence.dd
cat /home/claude/custom_regex_scan/*.txt | grep CASE

Output:

2097152  CASE-004821  CASE-004821

Example 11: Reviewing the final XML report summary

grep -A2 "feature_files" /home/claude/case1_be/report.xml

Output:

<feature_files>
  <email>214</email>
  <url>892</url>
  <ccn>3</ccn>
</feature_files>

Example 12: Resuming/checking a long-running job’s progress

tail -f /home/claude/case1_be/bulk_extractor.log

Output:

[Thread 3] Processing offset 4831838208 of 10737418240 (45.0%)

Common Use Cases

• Rapid triage on large images: Extracting actionable intelligence (emails, URLs, credit card numbers) from terabyte-scale disk images without needing to fully parse the file system.
• Data breach / PII exposure investigations: Locating credit card numbers, SSNs, and personal data patterns across a compromised system.
• Corporate espionage cases: Extracting email addresses and domains that reveal communication with competitor organizations.
• Password cracking preparation: Generating custom wordlists from a suspect’s own disk content to feed into tools like John the Ripper or Hashcat.
• Damaged/corrupted file system analysis: When the file system is too damaged for TSK/Autopsy to parse, bulk_extractor can still recover useful artifacts.
• Network/malware artifact recovery: Extracting embedded network packets or GPS metadata relevant to a case.

Automation with Bash

#!/bin/bash
# bulk_extractor_batch.sh - Automated bulk_extractor triage across evidence

EVIDENCE_DIR="$1"
OUTBASE="be_results_$(date +%Y%m%d_%H%M%S)"

if [ -z "$EVIDENCE_DIR" ]; then
    echo "Usage: $0 <evidence_directory>"
    exit 1
fi

mkdir -p "$OUTBASE"

for img in "$EVIDENCE_DIR"/*.dd "$EVIDENCE_DIR"/*.E01; do
    [ -e "$img" ] || continue
    name=$(basename "$img")
    echo "[*] Running bulk_extractor on $name..."
    bulk_extractor -j 8 -o "$OUTBASE/${name}_be" "$img"

    echo "[*] Summary for $name:"
    for f in email url ccn domain telephone; do
        count=$(wc -l < "$OUTBASE/${name}_be/${f}.txt" 2>/dev/null || echo 0)
        echo "    $f: $count entries"
    done
done

echo "[*] bulk_extractor batch complete. Results in $OUTBASE/"

Run:

chmod +x bulk_extractor_batch.sh
./bulk_extractor_batch.sh ./evidence_images/

Tips and Best Practices

1. Always specify a fresh, non-existent output directory with -o — bulk_extractor will refuse to overwrite an existing one by default.
2. Use -j to tune thread count to match your CPU core count for maximum throughput on large images.
3. Disable unnecessary scanners (-x) when you know exactly what artifact types you need — this speeds up processing significantly.
4. Review the histogram files first — they often reveal the most investigatively significant (most frequent) artifacts immediately.
5. Use the BEViewer GUI for large result sets; manually grepping through megabytes of feature files is inefficient for broad review.
6. Cross-reference exif.txt GPS coordinates with mapping tools to geolocate where photos were taken.
7. Combine bulk_extractor results with Scalpel/Foremost carved files — bulk_extractor complements file carvers by finding non-file artifacts like emails and URLs that carving alone would miss.
8. For very large images, run on a system with ample RAM and fast (SSD/NVMe) storage — bulk_extractor is I/O-bound in addition to CPU-bound.

Troubleshooting

References

• Official GitHub repository: https://github.com/simsong/bulk_extractor
• bulk_extractor user manual (PDF): https://github.com/simsong/bulk_extractor/wiki
• Kali Linux bulk_extractor tool page: https://www.kali.org/tools/bulk-extractor/
• Garfinkel, S., “Digital forensics XML and the DFXML toolset” (Digital Investigation Journal)
• DFRWS bulk_extractor papers: https://dfrws.org