Imagine backing up a 500 GB file server every single night, in full, forever. Not only would that consume enormous amounts of storage, it would also take hours to complete each time — most of which is spent re-copying files that haven’t changed since yesterday. There’s a smarter way: incremental backups, which only copy the data that has actually changed since the last backup.
This article explains incremental backups from first principles — how they differ from full and differential backups, how to implement them on Linux using rsync, tar, and modern tools like restic and borgbackup, and how to structure a real backup rotation strategy around them.
Full vs. Incremental vs. Differential Backups
Understanding these three backup types — and the tradeoffs between them — is essential before implementing any backup strategy.
| Backup Type | What It Copies | Storage Use | Restore Speed | Backup Speed |
|---|---|---|---|---|
| Full | Everything, every time | Highest | Fastest (single restore point) | Slowest |
| Incremental | Only changes since the last backup (full or incremental) | Lowest | Slowest (must replay full + every increment) | Fastest |
| Differential | Only changes since the last full backup | Medium | Medium (full + one differential) | Medium |
flowchart LR
subgraph Full Backup Strategy
F1[Full: Sun] --> F2[Full: Mon] --> F3[Full: Tue]
end
flowchart LR
subgraph Incremental Backup Strategy
I1[Full: Sun] --> I2[Incremental: Mon changes only] --> I3[Incremental: Tue changes only] --> I4[Incremental: Wed changes only]
endWith incremental backups, Monday’s backup only contains what changed between Sunday and Monday. Tuesday’s backup only contains what changed between Monday and Tuesday — and so on. To fully restore Wednesday’s state, you need the original full backup plus every incremental backup in the chain, applied in order.
Why Choose Incremental Backups?
| Benefit | Explanation |
|---|---|
| Massive storage savings | Only changed data is stored after the initial full backup |
| Much faster backup runs | Less data to copy each time means shorter backup windows |
| Lower bandwidth usage | Critical when backing up to remote/offsite/cloud storage |
| More frequent backups possible | Since each run is fast and cheap, you can back up more often (e.g., hourly) |
The Tradeoff
The main downside is restore complexity and speed — recovering data means restoring the last full backup, then replaying every subsequent incremental backup in the correct order. A broken link anywhere in that chain can make later backups unusable.
Method 1: Incremental Backups with rsync and Hard Links
A classic and elegant technique uses rsync‘s --link-dest option combined with hard links, popularized by tools like rsnapshot. Each backup looks like a complete snapshot from the outside, but unchanged files are actually just hard links to the previous backup — using almost no extra disk space for unchanged data.
flowchart TD
A[backup.2026-07-22 Full snapshot] -->|Hard links for unchanged files| B[backup.2026-07-23]
B -->|Hard links for unchanged files| C[backup.2026-07-24]
C -->|Only new/changed files use real disk space| D[Result: Each folder looks complete, but disk usage stays low]
Example Script
#!/bin/bash
SOURCE="/var/www"
BACKUP_ROOT="/backup/snapshots"
DATE=$(date +%F)
LATEST_LINK="$BACKUP_ROOT/latest"
mkdir -p "$BACKUP_ROOT"
rsync -av --delete \
--link-dest="$LATEST_LINK" \
"$SOURCE/" "$BACKUP_ROOT/$DATE/"
# Update the "latest" pointer for next time
rm -f "$LATEST_LINK"
ln -s "$BACKUP_ROOT/$DATE" "$LATEST_LINK"
Each day’s folder ($BACKUP_ROOT/2026-07-24/) appears to contain a full copy of the source directory — you can cd into it and browse it like any normal folder — but unchanged files are hard-linked to yesterday’s backup, so they consume zero additional disk space.
Method 2: Incremental Backups with tar
tar has native support for incremental backups using a “snapshot file” that tracks file metadata between runs.
First Backup (Full)
tar --create \
--file=/backup/full-2026-07-20.tar \
--listed-incremental=/backup/snapshot.snar \
/var/www
Subsequent Backups (Incremental)
tar --create \
--file=/backup/incr-2026-07-21.tar \
--listed-incremental=/backup/snapshot.snar \
/var/www
The --listed-incremental snapshot file (snapshot.snar) remembers exactly what was backed up last time — each subsequent run only includes files that are new or modified since the last snapshot update.
Restoring a Full Chain
cd /restore-target
tar --extract --listed-incremental=/dev/null --file=/backup/full-2026-07-20.tar
tar --extract --listed-incremental=/dev/null --file=/backup/incr-2026-07-21.tar
tar --extract --listed-incremental=/dev/null --file=/backup/incr-2026-07-22.tar
Notice the extracts must happen in order — full, then each incremental in sequence.
Method 3: Modern Tools — restic and borgbackup
Modern backup tools like restic and borgbackup handle incremental logic automatically, with added benefits like deduplication and encryption built in — no manual snapshot management required.
restic Example
# Initialize a backup repository (once)
restic init --repo /backup/restic-repo
# Run a backup (restic automatically only stores new/changed data)
restic backup /var/www --repo /backup/restic-repo
# List all snapshots
restic snapshots --repo /backup/restic-repo
# Restore a specific snapshot
restic restore latest --target /restore --repo /backup/restic-repo
borgbackup Example
# Initialize a repository
borg init --encryption=repokey /backup/borg-repo
# Create an incremental backup (borg deduplicates automatically)
borg create /backup/borg-repo::backup-{now} /var/www
# List all backups
borg list /backup/borg-repo
# Restore a specific archive
borg extract /backup/borg-repo::backup-2026-07-24
Both tools use content-based deduplication — meaning even if a large file is only partially modified, only the changed chunks are stored, not the entire file again. This is more efficient than traditional file-level incremental approaches.
Comparison Table: Incremental Backup Tools
| Tool | Deduplication | Encryption Built In | Complexity | Best For |
|---|---|---|---|---|
rsync + --link-dest | File-level (via hard links) | No (add manually) | Medium | Simple snapshot-style backups |
tar --listed-incremental | No | No | Medium-High | Environments requiring plain tar archives |
restic | Block-level | Yes (built in) | Low | Modern, secure automated backups |
borgbackup | Block-level | Yes (built in) | Low | Modern, space-efficient backups |
Real-World Example: Backing Up a Growing Log/Data Server
Scenario: A company has a data ingestion server accumulating about 50 GB of new data files per day, with a total dataset of 5 TB. A full backup every night would take hours and consume enormous storage.
Solution using restic:
# Weekly full-equivalent baseline handled automatically by restic's dedup engine
0 1 * * * restic backup /data --repo /backup/restic-repo --tag daily
# Prune old snapshots weekly, keeping a sensible retention policy
0 3 * * 0 restic forget --repo /backup/restic-repo --keep-daily 14 --keep-weekly 8 --keep-monthly 12 --prune
Because restic deduplicates at the block level, even though 50 GB of new files are added daily, if those files share content with existing data (common in log formats, database exports, etc.), the actual stored increment can be far smaller than 50 GB — saving significant storage over time.
Cisco Example: Incremental Configuration Backups
While Cisco devices don’t have a native “incremental backup” concept for configs (configs are usually small enough for full backups), the same principle applies when managing large numbers of devices. A common pattern is to back up configs daily but only alert or store a new version if the configuration actually changed — effectively an application-level incremental strategy:
# Simplified logic: only save a new config backup if it differs from the last one
import hashlib
def config_changed(new_config, last_backup_path):
with open(last_backup_path, 'rb') as f:
old_hash = hashlib.sha256(f.read()).hexdigest()
new_hash = hashlib.sha256(new_config.encode()).hexdigest()
return old_hash != new_hash
This pattern — hash comparison to detect actual changes — is the same underlying idea that makes incremental backups efficient: don’t store what hasn’t changed.
Python Example: A Custom Incremental Backup Script Using Hashes
import os
import hashlib
import shutil
import json
from datetime import date
SOURCE_DIR = "/var/www"
BACKUP_DIR = "/backup/incremental"
MANIFEST_FILE = os.path.join(BACKUP_DIR, "manifest.json")
def file_hash(path):
h = hashlib.sha256()
with open(path, 'rb') as f:
while chunk := f.read(8192):
h.update(chunk)
return h.hexdigest()
def load_manifest():
if os.path.exists(MANIFEST_FILE):
with open(MANIFEST_FILE) as f:
return json.load(f)
return {}
def save_manifest(manifest):
with open(MANIFEST_FILE, 'w') as f:
json.dump(manifest, f)
def run_incremental_backup():
manifest = load_manifest()
today_dir = os.path.join(BACKUP_DIR, str(date.today()))
os.makedirs(today_dir, exist_ok=True)
changed_count = 0
for root, _, files in os.walk(SOURCE_DIR):
for name in files:
full_path = os.path.join(root, name)
rel_path = os.path.relpath(full_path, SOURCE_DIR)
current_hash = file_hash(full_path)
if manifest.get(rel_path) != current_hash:
dest_path = os.path.join(today_dir, rel_path)
os.makedirs(os.path.dirname(dest_path), exist_ok=True)
shutil.copy2(full_path, dest_path)
manifest[rel_path] = current_hash
changed_count += 1
save_manifest(manifest)
print(f"Incremental backup complete: {changed_count} files changed")
run_incremental_backup()
This script only copies files whose SHA-256 hash has changed since the last run — the core principle behind every incremental backup tool, written from scratch to illustrate exactly how they work under the hood.
Designing a Retention Strategy (GFS: Grandfather-Father-Son)
A common industry-standard rotation scheme for incremental backups is GFS:
| Level | Frequency | Retention |
|---|---|---|
| Son (daily) | Every day | Keep 7 daily backups |
| Father (weekly) | Every Sunday | Keep 4 weekly backups |
| Grandfather (monthly) | First of the month | Keep 12 monthly backups |
flowchart TD
A[Daily Incrementals - Son] --> B[Weekly Rollup - Father]
B --> C[Monthly Rollup - Grandfather]This gives you fine-grained recent recovery points (daily, for the last week) while still retaining longer-term history (monthly, for the last year) without keeping 365 individual daily backups forever.
Best Practices
- Always pair incrementals with periodic full backups — an unbroken chain of 200 incrementals with no recent full backup is fragile and slow to restore.
- Test the entire restore chain periodically, not just the latest incremental — a broken link anywhere breaks the whole chain.
- Use tools with built-in deduplication (
restic,borgbackup) for the best balance of simplicity and efficiency. - Automate retention/pruning so incremental chains don’t grow forever.
- Monitor backup job success/failure with alerts — a silently failing incremental chain can go unnoticed for weeks.
- Encrypt incremental backups, especially when stored offsite or in the cloud.
- Document the exact restore procedure, including the order of operations, so anyone on the team can execute a recovery under pressure.
Troubleshooting
| Symptom | Likely Cause | Fix |
|---|---|---|
| Restore is missing recent files | An incremental in the chain was skipped or corrupted | Verify all incrementals were applied in the correct order |
| Backup chain keeps growing indefinitely | No periodic full backup / no pruning policy | Schedule regular full backups; configure retention/pruning |
tar incremental restore fails | Snapshot file (.snar) lost or corrupted | Rebuild backup chain starting from a new full backup |
restic/borgbackup restore is slow | Very deep history with excessive small snapshots | Prune old snapshots regularly per your retention policy |
| Disk space still growing fast despite incrementals | Source data has high change rate (not much overlap) | Consider differential or full+dedup strategy instead |
Summary
Incremental backups are the practical answer to the real-world constraints of storage, bandwidth, and time. By only capturing what has changed since the last backup — whether through rsync hard-linking, tar‘s snapshot tracking, or modern deduplicating tools like restic and borgbackup — you can back up far more frequently, use dramatically less storage, and still maintain a complete, restorable history of your data. The key is pairing incrementals with a sound retention strategy and testing restores regularly, so the speed and efficiency gains never come at the cost of reliability.
