waybackurls: Complete Guide to Historical URL Enumeration Using Kali Linux

Abdul Wahab Junaid

4 weeks ago

waybackurls: Complete Guide to Historical URL Enumeration Using Kali Linux

waybackurls is a small, focused Go utility created by Tom Hudson (tomnomnom) that fetches all known URLs for a given domain (including subdomains) directly from the Internet Archive’s Wayback Machine CDX API. It is one of the earliest and most widely used tools in the modern bug bounty recon toolkit, valued for its simplicity — it does exactly one thing well: pull historical URL records for a domain from web.archive.org. It is frequently chained with other tools (gau, httpx, gf, qsreplace) to build parameter-testing and endpoint-discovery pipelines.

How to Install

# Kali Linux
sudo apt update
sudo apt install waybackurls -y
waybackurls -h

# Or via Go
go install github.com/tomnomnom/waybackurls@latest

Syntax

echo <domain> | waybackurls [options]
cat domains.txt | waybackurls [options]
waybackurls <domain>

All Command-Line Options

Option	Description
`-dates`	Include the timestamp/date each URL was archived
`-no-subs`	Exclude subdomains; restrict to the exact domain given
`-get-versions`	Fetch all archived versions/snapshots of a specific URL (used with a single URL input)

waybackurls is intentionally minimalist. Unlike gau, it has very few flags — its power comes from Unix-pipeline composition with tools like sort, uniq, grep, httpx, and gf.

Basic Usage (Expected Output in Bash)

$ echo "testphp.vulnweb.com" | waybackurls

Output:

http://testphp.vulnweb.com/
http://testphp.vulnweb.com/login.php
http://testphp.vulnweb.com/search.php?test=query
http://testphp.vulnweb.com/artists.php?artist=1
http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-1/
http://testphp.vulnweb.com/comment.php?aid=2

Practical Examples with Output

Example 1 — Include archive timestamps

$ echo "testphp.vulnweb.com" | waybackurls -dates

Output:

2019-03-14T02:11:07 http://testphp.vulnweb.com/login.php
2020-07-22T09:45:12 http://testphp.vulnweb.com/search.php?test=query

Example 2 — Restrict to exact domain, excluding subdomains

$ echo "example.com" | waybackurls -no-subs

Output:

http://example.com/
http://example.com/about
http://example.com/contact

Example 3 — Bulk domain list processing

$ cat domains.txt | waybackurls > all-wayback-urls.txt
$ wc -l all-wayback-urls.txt

Output:

8452 all-wayback-urls.txt

Example 4 — Deduplicate and sort results

$ echo "testphp.vulnweb.com" | waybackurls | sort -u

Output:

http://testphp.vulnweb.com/
http://testphp.vulnweb.com/artists.php?artist=1
http://testphp.vulnweb.com/comment.php?aid=1
http://testphp.vulnweb.com/login.php

Example 5 — Extract only parameterized URLs

$ echo "testphp.vulnweb.com" | waybackurls | grep '=' | sort -u

Output:

http://testphp.vulnweb.com/artists.php?artist=1
http://testphp.vulnweb.com/comment.php?aid=1
http://testphp.vulnweb.com/listproducts.php?cat=2

Example 6 — Filter for specific file extensions (potential sensitive files)

$ echo "example.com" | waybackurls | grep -E '\.(bak|sql|env|log|config)$'

Output:

http://example.com/backup/db_backup.sql
http://example.com/.env
http://example.com/debug.log

Example 7 — Chain with httpx to find live URLs

$ echo "testphp.vulnweb.com" | waybackurls | httpx -silent -mc 200

Output:

http://testphp.vulnweb.com/login.php
http://testphp.vulnweb.com/search.php?test=query

Example 8 — Combine with gf patterns for XSS-prone URLs

$ echo "testphp.vulnweb.com" | waybackurls | gf xss

Output:

http://testphp.vulnweb.com/search.php?test=query
http://testphp.vulnweb.com/guestbook.php?name=test

Example 9 — Combine with qsreplace for automated payload injection

$ echo "testphp.vulnweb.com" | waybackurls | grep '=' | qsreplace "<script>alert(1)</script>"

Output:

http://testphp.vulnweb.com/artists.php?artist=<script>alert(1)</script>
http://testphp.vulnweb.com/comment.php?aid=<script>alert(1)</script>

Common Use Cases

Passive historical URL discovery without generating direct traffic to the live target.
Surfacing old, forgotten, or deprecated endpoints and parameters for testing.
Building large seed lists for XSS/SQLi/SSRF fuzzing pipelines via gf and qsreplace.
Identifying accidentally archived sensitive files (backups, config files, logs).
Complementing gau, katana, and hakrawler for maximum URL coverage in recon.

Automation with Bash

#!/bin/bash
# waybackurls-recon.sh — passive recon + sensitive file discovery pipeline

DOMAIN="$1"
OUTDIR="wayback-recon-$(date +%Y%m%d)"
mkdir -p "$OUTDIR"

echo "[*] Fetching Wayback URLs for $DOMAIN..."
echo "$DOMAIN" | waybackurls | sort -u > "$OUTDIR/all-urls.txt"

echo "[*] Searching for potentially sensitive archived files..."
grep -E '\.(bak|sql|env|log|config|zip|tar\.gz)$' "$OUTDIR/all-urls.txt" > "$OUTDIR/sensitive-files.txt"

echo "[*] Extracting parameterized URLs for fuzzing..."
grep '=' "$OUTDIR/all-urls.txt" > "$OUTDIR/param-urls.txt"

echo "[*] Checking which are still live..."
cat "$OUTDIR/all-urls.txt" | httpx -silent -mc 200 -o "$OUTDIR/live-urls.txt"

echo "[+] Recon complete."
echo "    Total URLs: $(wc -l < "$OUTDIR/all-urls.txt")"
echo "    Sensitive candidates: $(wc -l < "$OUTDIR/sensitive-files.txt")"
echo "    Live URLs: $(wc -l < "$OUTDIR/live-urls.txt")"

Tips and Best Practices

Always deduplicate (sort -u) — the Wayback CDX API frequently returns the same URL from multiple snapshot timestamps.
Grep for sensitive extensions (.bak, .sql, .env, .log) early — these are common high-value findings.
Combine with gau for broader source coverage (Common Crawl, OTX, URLScan) since waybackurls only queries the Wayback Machine.
Use -no-subs when you specifically want to scope results to a single host rather than the whole organization.
Chain into httpx immediately to discard dead archived URLs and focus only on what’s currently reachable.

Troubleshooting

Issue	Cause	Fix
No results returned	Domain never archived by Wayback Machine	Try `gau` instead for broader source coverage
Command not found	Go bin path not in PATH	Add `$(go env GOPATH)/bin` to `PATH`
Extremely large output on big domains	No filtering applied	Pipe through `grep`/`sort -u` to narrow results
API rate limiting / slow responses	Wayback CDX API throttling on large queries	Add delays between bulk domain requests
Duplicate near-identical URLs	Multiple archive snapshots of same page	Deduplicate with `sort -u` or `anew`

References

Official GitHub repository: https://github.com/tomnomnom/waybackurls
Wayback Machine CDX API docs: https://archive.org/help/wayback_api.php
Kali tool page: https://www.kali.org/tools/waybackurls/
tomnomnom’s tool suite: https://github.com/tomnomnom