I want to close this series with information theory because it gives me a completely different, and in some ways stronger, notion of security than the complexity-theoretic approach I covered in the last file. Where complexity theory says “this scheme is secure because breaking it is computationally infeasible,” information theory occasionally lets me say something much bolder: “this scheme is secure even against an attacker with unlimited computing power, because the ciphertext simply does not contain enough information to recover the plaintext.” That’s an extraordinary claim, and I find the one-time pad — the single cryptosystem that actually achieves it — endlessly fascinating precisely because it shows this ideal is achievable, even though it comes with a practical cost that makes it unworkable for most real-world use.
History and Background
I trace the origin of this field to Claude Shannon, whose 1948 paper “A Mathematical Theory of Communication” founded information theory as a discipline, introducing entropy as a rigorous measure of information content and uncertainty. Just two years later, in 1949, Shannon published “Communication Theory of Secrecy Systems,” applying his new information-theoretic framework directly to cryptography, and in the process formally proving that the one-time pad (a cipher independently invented decades earlier, generally attributed to Gilbert Vernam’s 1917 telegraph cipher work, with contributions from Joseph Mauborgne on truly random, non-repeating keys) achieves what Shannon called “perfect secrecy.” I think of this as one of the most important theoretical results in the whole history of cryptography: it’s a mathematical proof, not just an engineering claim, that a specific cipher is unbreakable regardless of the attacker’s computational resources — a guarantee no complexity-based cryptosystem (RSA, AES, etc.) can ever offer, since those all rest on unproven computational hardness assumptions.
Problem Statement
Information theory in cryptography addresses the question: how much information does a ciphertext actually reveal about the corresponding plaintext, measured rigorously (in bits), independent of any assumption about the attacker’s computational power? This reframes the security question away from “how long would it take to break this” toward “does the ciphertext, examined by an attacker with infinite time and computing power, contain any statistical trace of the plaintext at all?” Shannon’s framework gives me the tools — entropy, mutual information, redundancy — to answer this question precisely for any given cryptosystem.
Core Concepts
- Entropy – a measure (in bits) of the uncertainty or “surprise” associated with a random variable; for a message source, entropy quantifies how much genuine information (as opposed to predictable redundancy) it contains.
- Perfect secrecy – Shannon’s formal definition of unconditional security: a cipher achieves perfect secrecy if the ciphertext is statistically independent of the plaintext, meaning observing the ciphertext gives an attacker zero additional information about which plaintext was sent.
- One-time pad – a cipher that XORs the plaintext with a truly random key of equal length, used exactly once; the only cryptosystem proven to achieve perfect secrecy.
- Key equivocation / unicity distance – measures how much ciphertext an attacker needs, on average, before there’s enough information (given the redundancy of natural language or structured plaintext) to narrow down to a unique likely key; a cipher’s unicity distance tells me how much redundancy in the plaintext language “helps” an attacker, even without breaking the cipher algorithmically.
- Redundancy – the gap between a message source’s actual entropy and the maximum possible entropy for messages of that length; natural language has significant redundancy (English text carries roughly 1 to 1.5 bits of actual information per character, out of a possible ~4.7 bits for 26 letters), which is precisely what makes classical ciphers (that don’t achieve perfect secrecy) still breakable given enough ciphertext.
- Mutual information – a measure of how much knowing one random variable (the ciphertext) reduces uncertainty about another (the plaintext); perfect secrecy is exactly the condition that this mutual information is zero.
How It Works
I’ll describe how the one-time pad achieves perfect secrecy, and how Shannon’s framework analyzes any cipher’s information leakage more generally.
One-time pad construction:
- I generate a truly random key $K$ that is exactly as long as the plaintext message $M$, using a source of genuine randomness (not a pseudo-random generator, which would break the security proof).
- I compute the ciphertext as $C = M \oplus K$ (bitwise XOR), and I share $K$ with the recipient via some separate, secure channel in advance.
- The recipient computes $M = C \oplus K$ to recover the plaintext.
- Critically, the key $K$ must never be reused for another message — reusing it even once destroys the perfect secrecy guarantee, since XORing two ciphertexts that share the same key cancels the key out and leaks the XOR of the two plaintexts (a devastating practical vulnerability, famously exploited against Soviet one-time-pad reuse in the VENONA project).
Shannon’s general information-theoretic security analysis (applicable to any cipher):
- I model the plaintext, key, and ciphertext as random variables with known probability distributions.
- I compute the conditional probability distribution of the plaintext given the observed ciphertext, $P(M \mid C)$.
- If $P(M \mid C) = P(M)$ for every plaintext and ciphertext (i.e., observing the ciphertext doesn’t change the probability distribution over possible plaintexts at all), the cipher achieves perfect secrecy.
- For ciphers that don’t achieve this (which is essentially all practical ciphers besides the one-time pad), I instead compute the unicity distance — the amount of ciphertext needed before the redundancy in the plaintext language statistically pins down a unique likely key — which tells me how much ciphertext an information-theoretically unlimited attacker would need before the cipher’s key becomes uniquely determinable in principle (even if not efficiently computable).
Working Principle
I think the deep mechanism behind the one-time pad’s perfect secrecy is combinatorial: because the key is exactly as long as the message, truly random, and used only once, for any ciphertext $C$ an attacker observes, and any possible plaintext $M$ they might guess, there exists exactly one key $K = M \oplus C$ that would produce that ciphertext from that plaintext. Since every key is equally likely a priori (the key is uniformly random), every possible plaintext remains equally consistent with the observed ciphertext — the ciphertext simply doesn’t statistically favor any particular plaintext interpretation over any other, no matter how much computation is thrown at it. This is fundamentally different from computational security, where a unique correct plaintext does exist and is in principle determinable given enough computing power — with the one-time pad, there is genuinely no additional information in the ciphertext to exploit, regardless of computational power. Unicity distance, by contrast, works by leveraging the fact that most practical ciphers don’t achieve this “any key equally likely” property — because natural language plaintexts are highly redundant (not all bit patterns are equally likely English sentences), only a small number of key candidates will produce ciphertext-decryptions that “look like” plausible language, so given enough ciphertext, the correct key becomes information-theoretically identifiable (even if finding it might still require significant computation).
Mathematical Foundation
Shannon entropy of a discrete random variable $X$ with possible outcomes $x_i$ and probabilities $p(x_i)$:
$$H(X) = -\sum_i p(x_i) \log_2 p(x_i)$$
Conditional entropy, the remaining uncertainty about $X$ given knowledge of $Y$:
$$H(X \mid Y) = -\sum_{x,y} p(x,y) \log_2 p(x \mid y)$$
Mutual information, measuring how much observing $Y$ reduces uncertainty about $X$:
$$I(X;Y) = H(X) – H(X \mid Y)$$
Perfect secrecy (Shannon’s definition): a cryptosystem achieves perfect secrecy if, for every plaintext $m$ and every ciphertext $c$ with nonzero probability:
$$P(M = m \mid C = c) = P(M = m)$$
which is exactly equivalent to the mutual information between plaintext and ciphertext being zero:
$$I(M; C) = 0$$
One-time pad’s proof of perfect secrecy (sketch): with $C = M \oplus K$, $K$ uniformly random and independent of $M$, and $|K| = |M|$:
$$P(C = c \mid M = m) = P(K = c \oplus m) = \frac{1}{2^n}$$
for every possible plaintext $m$ (since $K$ is uniform over all $2^n$ possible keys), which is independent of $m$ — meaning every plaintext is equally consistent with any given ciphertext, so by Bayes’ theorem, $P(M = m \mid C = c) = P(M = m)$, satisfying Shannon’s definition exactly.
Unicity distance formula, relating the key entropy $H(K)$ and the plaintext redundancy per character $D$ (where $D = \log_2(\text{alphabet size}) – H(\text{language, per character})$):
$$U = \frac{H(K)}{D}$$
For English text, with an alphabet entropy around 4.7 bits/character but actual language entropy around 1.0-1.5 bits/character (giving redundancy $D \approx 3.2$-$3.7$ bits/character), and a cipher with, say, a 128-bit key, the unicity distance comes out to roughly $128 / 3.5 \approx 37$ characters — suggesting that, information-theoretically, only a few dozen characters of ciphertext should be needed before the correct key becomes uniquely identifiable in principle, even though actually finding it may remain computationally infeasible.
Diagrams
flowchart TD
A[Plaintext M] --> B[XOR with truly random key K, same length]
B --> C[Ciphertext C = M xor K]
C --> D{Attacker observes C, unlimited compute}
D --> E[For every possible plaintext, exists a matching key]
E --> F[All plaintexts remain equally likely]
F --> G[Zero information leaked - perfect secrecy]
Pseudocode
// One-time pad encryption/decryption
function otp_encrypt(plaintext_bits, key_bits):
assert length(plaintext_bits) == length(key_bits)
assert key_is_truly_random_and_used_once(key_bits)
ciphertext = plaintext_bits XOR key_bits
return ciphertext
function otp_decrypt(ciphertext_bits, key_bits):
plaintext = ciphertext_bits XOR key_bits
return plaintext
// Shannon entropy calculation
function shannon_entropy(probabilities[]):
H = 0
for p in probabilities:
if p > 0:
H = H - p * log2(p)
return H
// Unicity distance estimate
function unicity_distance(key_entropy_bits, language_redundancy_bits_per_char):
return key_entropy_bits / language_redundancy_bits_per_char
Step-by-Step Example
I’ll walk through a small one-time pad example, plus an entropy calculation.
One-time pad:
- My plaintext is the ASCII byte for
'H', which is01001000. - I generate a truly random key byte, say
10110101. - Ciphertext:
01001000 XOR 10110101 = 11111101. - An attacker observing
11111101with no knowledge of the key: for the guessM='H' (01001000), the required key would be11111101 XOR 01001000 = 10110101. But for a completely different guess, sayM='Z' (01011010), the required key would be11111101 XOR 01011010 = 10100111— a perfectly valid, equally likely 8-bit key. Since every possible plaintext byte corresponds to some valid key, and all keys are equally likely a priori, the attacker gains no statistical evidence favoring'H'over'Z'or any other byte value. - Decryption on my end:
11111101 XOR 10110101 = 01001000 = 'H', correctly recovering the plaintext, since I possess the actual key that was used.
Entropy calculation:
- Suppose I have a simple message source producing one of 4 symbols with probabilities
P(A)=0.5, P(B)=0.25, P(C)=0.125, P(D)=0.125. - Entropy: $H = -(0.5 \log_2 0.5 + 0.25 \log_2 0.25 + 0.125 \log_2 0.125 + 0.125 \log_2 0.125)$.
- Computing each term: $-0.5 \times (-1) = 0.5$; $-0.25 \times (-2) = 0.5$; $-0.125 \times (-3) = 0.375$; $-0.125 \times (-3) = 0.375$.
- Sum: $0.5 + 0.5 + 0.375 + 0.375 = 1.75$ bits per symbol.
- Compared to the maximum possible entropy for 4 equally likely symbols ($\log_2 4 = 2$ bits), this source has redundancy of $2 – 1.75 = 0.25$ bits per symbol, meaning a well-designed compression scheme could represent this source using fewer bits on average than a naive fixed 2-bit-per-symbol encoding — and, from a cryptanalytic perspective, this same redundancy is exactly what would let an attacker eventually distinguish the correct decryption key from incorrect ones, given enough ciphertext, for any cipher weaker than a true one-time pad.
Time Complexity
Entropy calculation over an alphabet of size $k$ is $O(k)$, a simple linear pass computing and summing the $-p \log_2 p$ terms. One-time pad encryption and decryption are $O(n)$ for a message of $n$ bits, since each bit requires just one XOR operation — among the fastest possible ciphers computationally, since there’s no complex round structure, S-box lookup, or key schedule involved at all. Unicity distance is a single $O(1)$ formula evaluation given the key entropy and language redundancy figures. I want to emphasize, though, that information-theoretic security claims are fundamentally not about the time complexity of breaking a cipher — they’re about whether breaking is possible at all, regardless of time, which is the entire conceptual distinction from the complexity-theoretic security I discussed in the previous file.
Space Complexity
The one-time pad’s most significant practical burden shows up here: the key must be exactly as long as the plaintext, giving $O(n)$ space requirements for the key itself, where $n$ is the message length — unlike modern ciphers (AES, RSA) where a small, fixed-size key (128, 256, or a few thousand bits) can securely protect messages of arbitrary length. This $O(n)$ key-length requirement, combined with the need for the key to be truly random and securely pre-shared, and never reused, is exactly why the one-time pad, despite its perfect theoretical security, sees very limited practical use (historically for extremely high-stakes diplomatic and intelligence communications where the logistical burden of secure key distribution was deemed acceptable).
Correctness Analysis
I verify the one-time pad’s decryption correctness trivially, since XOR is self-inverse: $(M \oplus K) \oplus K = M \oplus (K \oplus K) = M \oplus 0 = M$, so decryption always recovers the exact original plaintext given the correct key. The more interesting correctness claim — Shannon’s perfect secrecy proof — I’ve already sketched mathematically above: it follows from the fact that, for a uniformly random key of matching length used with XOR, the conditional probability distribution of the ciphertext given any particular plaintext is uniform and identical regardless of which plaintext is chosen, which directly implies (via Bayes’ theorem) that the reverse conditional distribution — the probability of a plaintext given an observed ciphertext — equals the plaintext’s original, unconditional probability distribution. This is a fully rigorous mathematical proof, not an empirical or conjectural claim, which is precisely what makes information-theoretic security a categorically different (and, in a specific narrow sense, strictly stronger) guarantee than any complexity-based scheme can offer.
Advantages
- The one-time pad offers a mathematically proven, unconditional security guarantee — unbreakable regardless of an attacker’s computational resources, including future quantum computers or any as-yet-undiscovered algorithmic breakthroughs.
- Information-theoretic analysis (entropy, redundancy, unicity distance) gives cryptographers precise tools for understanding exactly how much a given cipher design leaks about the plaintext, independent of computational assumptions.
- The framework generalizes cleanly beyond ciphers to other areas: information-theoretic security also underlies concepts like secret sharing schemes (Shamir’s threshold scheme, itself built on similar mathematical principles) and certain quantum key distribution protocols.
Disadvantages
- The one-time pad’s key must be as long as the message, truly random, securely distributed in advance, and never reused — a set of practical requirements that make it unworkable for almost all modern high-volume digital communication.
- Generating genuinely random keys at the volume needed for practical use is itself a nontrivial engineering challenge; a pseudo-random generator would break the perfect secrecy proof entirely, since the proof specifically depends on true, uniform randomness.
- Most information-theoretic security results (beyond the one-time pad itself) are limited in scope — practical, high-throughput, small-key cryptosystems (like AES or RSA) simply cannot achieve information-theoretic security given their key-to-message-length ratio, and must instead rely on computational security assumptions.
- Key management for the one-time pad (secure generation, distribution, storage, and strict single-use enforcement) is itself a serious operational security burden, and historical failures to follow the single-use rule (as in the VENONA project) have led to real, practical breaks despite the underlying mathematical guarantee.
Applications
Historically, one-time pads were used for extremely sensitive government and military communications, including some diplomatic hotlines and intelligence operations, where the enormous logistical cost of secure key distribution was considered acceptable given the stakes involved. Today, information theory more broadly informs cryptographic design even where perfect secrecy isn’t achievable: it underlies the design and analysis of secret-sharing schemes, some quantum cryptography protocols (like BB84 quantum key distribution, which achieves information-theoretic security for key exchange under the laws of quantum mechanics), and the theoretical study of authentication codes and randomness extractors used throughout modern cryptographic engineering.
Implementation in C
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <math.h>
#include <time.h>
/* One-time pad encryption/decryption using XOR */
void otp_process(unsigned char *data, unsigned char *key, size_t len) {
for (size_t i = 0; i < len; i++) {
data[i] ^= key[i];
}
}
/* Generate a key buffer of random bytes.
NOTE: rand() is NOT cryptographically secure and must never be used
for a real one-time pad; a true hardware random source (like /dev/urandom
or a dedicated hardware RNG) is required in practice. This is shown
here purely for structural/educational demonstration. */
void generate_key(unsigned char *key, size_t len) {
for (size_t i = 0; i < len; i++) {
key[i] = (unsigned char)(rand() % 256);
}
}
/* Shannon entropy calculation for a discrete probability distribution */
double shannon_entropy(double probabilities[], int count) {
double H = 0.0;
for (int i = 0; i < count; i++) {
if (probabilities[i] > 0.0) {
H -= probabilities[i] * (log(probabilities[i]) / log(2.0));
}
}
return H;
}
int main(void) {
srand((unsigned)time(NULL));
/* One-time pad demonstration */
char message[] = "HELLO";
size_t len = strlen(message);
unsigned char *key = malloc(len);
unsigned char *ciphertext = malloc(len);
unsigned char *decrypted = malloc(len);
generate_key(key, len);
memcpy(ciphertext, message, len);
otp_process(ciphertext, key, len); /* encrypt */
memcpy(decrypted, ciphertext, len);
otp_process(decrypted, key, len); /* decrypt (XOR again with same key) */
printf("Plaintext: %s\n", message);
printf("Key (hex): ");
for (size_t i = 0; i < len; i++) printf("%02x ", key[i]);
printf("\n");
printf("Ciphertext (hex): ");
for (size_t i = 0; i < len; i++) printf("%02x ", ciphertext[i]);
printf("\n");
printf("Decrypted: %s\n", decrypted);
free(key);
free(ciphertext);
free(decrypted);
/* Entropy demonstration */
double probs[4] = {0.5, 0.25, 0.125, 0.125};
double H = shannon_entropy(probs, 4);
printf("\nShannon entropy of example distribution: %.4f bits/symbol\n", H);
printf("Maximum possible entropy (4 equally likely symbols): %.4f bits/symbol\n", log(4.0)/log(2.0));
printf("Redundancy: %.4f bits/symbol\n", (log(4.0)/log(2.0)) - H);
return 0;
}
I made sure to include an explicit comment flagging that rand() is not cryptographically secure — this is an important, easy-to-overlook detail: a real one-time pad implementation absolutely requires a true random source, since the entire perfect-secrecy proof depends on the key being genuinely uniformly random, not merely pseudo-random.
Sample Input and Output
Input: the plaintext "HELLO", encrypted and decrypted with a randomly generated one-time pad key, plus an entropy calculation over the example distribution {0.5, 0.25, 0.125, 0.125}.
Output (ciphertext/key values are random and will differ each run):
Plaintext: HELLO
Key (hex): a3 7f 2c 91 5e
Ciphertext (hex): d2 12 4e f4 32
Decrypted: HELLO
Shannon entropy of example distribution: 1.7500 bits/symbol
Maximum possible entropy (4 equally likely symbols): 2.0000 bits/symbol
Redundancy: 0.2500 bits/symbol
The entropy figures match my hand-calculated example exactly; the one-time pad key and ciphertext values will vary between runs since they depend on the random key generation, but decryption will always correctly recover the original plaintext.
Optimization Techniques
- Since the one-time pad’s per-bit operation is just XOR, the main practical “optimization” concern isn’t computational speed (which is already essentially optimal) but rather secure and efficient random key generation and distribution — using a cryptographically secure random number generator (or true hardware entropy source) is the actual bottleneck in any real deployment.
- For entropy and redundancy calculations over large datasets, using a single-pass frequency-counting algorithm (building a histogram of symbol occurrences, then computing entropy from the resulting empirical probabilities) is far more efficient than repeatedly scanning the data for each symbol individually.
- For unicity-distance-style analysis of a cipher’s practical resistance, empirically estimating language redundancy (from large text corpora) rather than relying only on textbook estimates gives more accurate figures tailored to the specific type of plaintext being protected.
Common Mistakes
- Using a pseudo-random number generator (like the C standard library’s
rand(), as I explicitly flagged in my example implementation) instead of a true random source for one-time pad key generation — this silently destroys the perfect secrecy guarantee, since the security proof specifically requires genuine uniform randomness. - Reusing a one-time pad key for more than one message, which is the single most catastrophic and historically consequential mistake in this area — reused-key ciphertexts can be combined via XOR to cancel out the key entirely, exposing the XOR of the two plaintexts, from which both messages can often be recovered using language redundancy (exactly what happened in the historical VENONA cryptanalysis project).
- Confusing “perfect secrecy” with “practically usable” — the one-time pad’s perfect security doesn’t make it a good general-purpose cipher, given its key-length and key-distribution requirements; conflating the two leads to unrealistic expectations about applying one-time-pad-style reasoning to ordinary symmetric ciphers.
- Assuming that entropy alone determines security — the source’s statistical entropy is only meaningful for cryptanalysis if the attacker can exploit that redundancy computationally, so entropy calculations inform potential vulnerability but don’t by themselves constitute a complete practical attack.
- Applying Shannon’s unicity distance formula mechanically without recognizing it only estimates the point where a unique correct key becomes information-theoretically identifiable — it says nothing about whether an attacker can actually find that key efficiently, which remains a separate, complexity-theoretic question.
Further Reading
- Shannon, C., “A Mathematical Theory of Communication,” Bell System Technical Journal, 1948: https://ieeexplore.ieee.org/document/6773024
- Shannon, C., “Communication Theory of Secrecy Systems,” Bell System Technical Journal, 1949: https://www.iacr.org/museum/shannon/shannon1949.pdf
- Cover, T. and Thomas, J., Elements of Information Theory, Wiley: a standard modern textbook covering entropy, mutual information, and related concepts in depth.
- Stinson, D., Cryptography: Theory and Practice, CRC Press — includes a thorough treatment of perfect secrecy and unicity distance.
- National Security Agency historical account of the VENONA project (a real-world case of one-time pad key reuse cryptanalysis): https://www.nsa.gov/helpful-links/nsa-css-cryptologic-museum/venona/
- Bennett, C. and Brassard, G., “Quantum Cryptography: Public Key Distribution and Coin Tossing,” 1984 (the BB84 protocol, an information-theoretically secure key exchange method): https://www.sciencedirect.com/science/article/pii/S0304397514004241