Stream Cipher in Cryptography: RC4, ChaCha20, and Keystream Generation Explained

Every time a browser opens a secure website, a chat app sends a message, or a Wi-Fi router encrypts a packet, there is a good chance a stream cipher is quietly doing the work behind the scenes. Stream ciphers are one of the two great families of symmetric encryption (the other being block ciphers), and understanding how they generate a keystream and combine it with plaintext is one of the most useful things anyone learning cryptography can do. This article walks through stream ciphers from the ground up, covering the mathematics, the internal design of RC4 and ChaCha20, real attacks that have broken deployed systems, and the best practices that keep modern implementations safe.

What Is a Stream Cipher?

A stream cipher is a symmetric encryption algorithm that encrypts data one bit or byte at a time, rather than in fixed-size blocks. It works by generating a pseudorandom sequence of bits called a keystream, derived from a secret key (and usually a nonce or initialization vector), and then combining that keystream with the plaintext — almost always using the XOR operation.

Mathematically, encryption and decryption are symmetric and elegant:

$$ C_i = P_i \oplus K_i $$

$$ P_i = C_i \oplus K_i $$

Here, $P_i$ is the $i$-th bit (or byte) of plaintext, $K_i$ is the corresponding bit of the keystream, and $C_i$ is the resulting ciphertext bit. Because XOR is its own inverse, the same keystream is used for both encryption and decryption.

This design makes stream ciphers extremely fast and lightweight — ideal for real-time applications like voice calls, video streaming, and wireless communication — but it also means their entire security rests on one property: the keystream must never repeat, and it must be indistinguishable from true randomness to an attacker who does not know the key.

Stream Ciphers vs. Block Ciphers

Block ciphers can actually be turned into stream ciphers by running them in modes like CTR (Counter) or OFB (Output Feedback), which shows that the boundary between the two categories is more about mode of operation than a hard architectural wall.

Anatomy of a Stream Cipher: The Keystream Generator

The heart of any stream cipher is its keystream generator (KSG) — a deterministic algorithm that takes a secret key $K$ and, usually, a nonce $N$, and produces an arbitrarily long sequence of pseudorandom bytes:

$$ KSG(K, N) \rightarrow K_1, K_2, K_3, \ldots, K_n $$

A good keystream generator must satisfy several properties:

1. Long period — the sequence should not repeat for an astronomically long time.
2. Statistical randomness — the output should pass standard randomness tests (frequency test, run test, autocorrelation test).
3. Unpredictability — given any prefix of the keystream, an attacker without the key should not be able to predict the next bit with a probability meaningfully better than 0.5.
4. Key/nonce sensitivity — a tiny change in the key or nonce should produce a completely uncorrelated keystream (the avalanche effect).

Formally, unpredictability is often expressed through the next-bit test: a generator is cryptographically secure if no probabilistic polynomial-time adversary can guess bit $K_{i+1}$ given $K_1, \ldots, K_i$ with probability significantly greater than:

$$ Pr[\text{guess correct}] = \frac{1}{2} + \epsilon $$

where $\epsilon$ is negligible.

Stream ciphers fall into two broad architectural categories:

• Synchronous stream ciphers — the keystream depends only on the key and nonce, not on the plaintext or ciphertext. RC4 and ChaCha20 are both synchronous.
• Self-synchronizing (asynchronous) stream ciphers — the keystream depends partly on previously generated ciphertext, giving some resilience to bit loss but weaker error containment. Cipher Feedback (CFB) mode is a classic example.

RC4: The Classic (and Now Deprecated) Stream Cipher

RC4 was designed by Ron Rivest in 1987 and remained a proprietary trade secret until it leaked onto Usenet in 1994. Its simplicity made it enormously popular — it powered SSL/TLS, WEP, and WPA for over two decades — but that same simplicity eventually became its downfall.

RC4 Internal State

RC4 maintains an internal permutation array $S$ of 256 bytes (values 0–255), plus two index pointers $i$ and $j$. The algorithm has two phases:

Phase 1: Key Scheduling Algorithm (KSA)

The KSA initializes the permutation array using the secret key:

for i from 0 to 255:
    S[i] = i

j = 0
for i from 0 to 255:
    j = (j + S[i] + key[i mod key_length]) mod 256
    swap(S[i], S[j])

Phase 2: Pseudo-Random Generation Algorithm (PRGA)

Once the array is scrambled, the PRGA produces the keystream byte by byte:

i = 0
j = 0
while generating keystream:
    i = (i + 1) mod 256
    j = (j + S[i]) mod 256
    swap(S[i], S[j])
    K = S[(S[i] + S[j]) mod 256]
    output K

Each output byte $K$ is then XORed with the corresponding plaintext byte to produce ciphertext.

Why RC4 Was Broken

Despite its elegance, RC4 suffers from several well-documented weaknesses:

• Biased keystream bytes: Statistical analysis (Mantin and Shamir, 2001; AlFardan et al., 2013) showed that certain byte positions in the RC4 keystream — especially the second output byte — are biased toward specific values far more often than a truly random stream would produce. Attackers exploiting millions of encrypted sessions with the same plaintext (as in HTTPS cookies) could recover plaintext through these biases.
• Weak key schedule (Fluhrer-Mantin-Shamir attack): In WEP, RC4 was used with short, predictable IVs. Certain “weak keys” leaked information about the secret key in the first few keystream bytes, letting attackers recover a WEP key after capturing a modest number of packets.
• Related-key vulnerabilities: Because WEP concatenated a short IV directly with the static key, related keys produced correlated keystreams, catastrophically undermining security.
• No built-in nonce misuse resistance: reusing the same key/IV combination twice produces the same keystream, letting attackers XOR two ciphertexts together to cancel the keystream and recover a stream of XORed plaintexts, from which both plaintexts can often be recovered using frequency analysis.

By 2015, RFC 7465 formally prohibited RC4 in TLS, and IETF/NIST guidance uniformly recommends against RC4 in any new system. It is retained here purely as a teaching example of stream cipher design — not for use in production.

ChaCha20: The Modern Replacement

ChaCha20, designed by Daniel J. Bernstein in 2008 as an evolution of his earlier Salsa20 cipher, is now the stream cipher of choice for modern protocols such as TLS 1.3, WireGuard, and Signal. It was standardized (paired with the Poly1305 MAC) in RFC 8439.

ChaCha20 Design Philosophy

Unlike RC4’s byte-oriented permutation table, ChaCha20 is built around a 512-bit internal state arranged as a 4×4 matrix of 32-bit words, and it produces keystream in 64-byte blocks using only three simple operations: addition modulo $2^{32}$, XOR, and bitwise rotation — often abbreviated as ARX (Add-Rotate-XOR). This makes it extremely fast in software, without needing lookup tables that can leak information through cache-timing side channels — a real problem that affected some AES implementations.

The ChaCha20 State Matrix

The 512-bit initial state is laid out as:

$$ \begin{bmatrix} c_0 & c_1 & c_2 & c_3 \ k_0 & k_1 & k_2 & k_3 \ k_4 & k_5 & k_6 & k_7 \ b & n_0 & n_1 & n_2 \end{bmatrix} $$

Where:

• $c_0 \ldots c_3$ are fixed constants (“expa”, “nd 3”, “2-by”, “te k” in ASCII)
• $k_0 \ldots k_7$ are the 256-bit (8-word) secret key
• $b$ is a 32-bit block counter
• $n_0, n_1, n_2$ form a 96-bit nonce

The Quarter Round Function

The core primitive is the quarter round, applied to four 32-bit words $(a, b, c, d)$:

a = a + b;  d = d XOR a;  d = rotate_left(d, 16)
c = c + d;  b = b XOR c;  b = rotate_left(b, 12)
a = a + b;  d = d XOR a;  d = rotate_left(d, 8)
c = c + d;  b = b XOR c;  b = rotate_left(b, 7)

ChaCha20 applies 20 total rounds (hence the name) — 10 iterations alternating between “column rounds” (applied to the matrix columns) and “diagonal rounds” (applied to the diagonals). This alternation ensures fast and thorough diffusion: a single bit flip anywhere in the key or nonce cascades to affect roughly half the output bits after just a few rounds, satisfying the strict avalanche criterion.

After 20 rounds, the final state is added word-wise (mod $2^{32}$) to the original input state, and the resulting 512 bits (64 bytes) become one keystream block:

$$ \text{Block}_i = \text{ChaChaRounds}(State_i) + State_i \pmod{2^{32}} $$

The block counter $b$ increments for each subsequent 64-byte block, allowing the cipher to seek to any position in the keystream — a property RC4 lacks, and one that is very useful for parallel processing and random access (e.g., disk encryption).

Why ChaCha20 Is Considered Stronger

• No known practical biases: unlike RC4, ChaCha20’s keystream has withstood over a decade of cryptanalysis; the best known attacks work only against reduced-round variants (e.g., 7 out of 20 rounds), leaving a large security margin.
• Constant-time by design: because it uses only addition, rotation, and XOR (no table lookups or branches dependent on secret data), it naturally resists cache-timing side-channel attacks.
• Nonce structure prevents casual reuse: the 96-bit nonce plus 32-bit counter gives a huge space, and protocols built on ChaCha20 (like RFC 7539’s TLS cipher suite) mandate nonce uniqueness per key.
• Performance on devices without AES hardware acceleration: ChaCha20 was specifically designed to run fast in pure software, making it a strong choice for mobile devices and embedded systems lacking AES-NI instructions.

Keystream Generation: A Side-by-Side Comparison

Common Attacks Against Stream Ciphers

Understanding stream cipher security means understanding how they fail. The most important attack classes are:

1. Keystream Reuse (Two-Time Pad Attack)

If the same key and nonce are ever reused, the keystream repeats. XORing two ciphertexts encrypted under the same keystream cancels it out:

$$ C_1 \oplus C_2 = (P_1 \oplus K) \oplus (P_2 \oplus K) = P_1 \oplus P_2 $$

Once an attacker has $P_1 \oplus P_2$, statistical and linguistic analysis (crib-dragging) can often recover both plaintexts. This is precisely what happened in real-world failures like Microsoft’s early Point-to-Point Tunneling Protocol (PPTP) implementations and certain misuse cases of the one-time pad.

2. Related-Key and Weak-Key Attacks

If a cipher’s key schedule doesn’t sufficiently mix related keys (as in RC4/WEP), differences in the key can propagate into predictable differences in the keystream, allowing statistical key recovery — the basis of the Fluhrer-Mantin-Shamir attack.

3. Distinguishing Attacks

Cryptanalysts try to find statistical distinguishers — patterns that let them tell a cipher’s keystream apart from true randomness with better-than-chance probability. Any successful distinguisher, even a small one, is treated as a serious cryptographic weakness because it can often be amplified into a full break.

4. Side-Channel Attacks

Even a mathematically perfect stream cipher can be broken through implementation flaws — timing variations, power consumption analysis, or cache access patterns. This is a major reason ChaCha20’s simple ARX operations (which avoid data-dependent table lookups) are preferred over algorithms whose implementations can leak information through S-box lookups.

Best Practices for Using Stream Ciphers

1. Never reuse a key/nonce pair. This single rule prevents the most catastrophic class of stream cipher failures.
2. Use standardized, vetted constructions. Prefer ChaCha20-Poly1305 or AES-CTR with a proper AEAD (Authenticated Encryption with Associated Data) mode over ad-hoc designs.
3. Always authenticate ciphertext. Stream ciphers provide confidentiality only — without a MAC (like Poly1305 or HMAC), ciphertext can be bit-flipped by an attacker without detection, since XOR-based encryption has no built-in integrity checking.
4. Avoid RC4 entirely in new systems; it is formally deprecated in TLS and SSH.
5. Manage nonces carefully in distributed systems. Use counters, and never allow two machines to independently pick nonces without coordination when sharing a key.
6. Rotate keys periodically even when nonces are managed correctly, to limit the amount of data exposed if a key is later compromised.

Real-World Applications

• TLS 1.3 uses ChaCha20-Poly1305 as an alternative cipher suite to AES-GCM, particularly valuable on mobile devices without AES hardware acceleration.
• WireGuard VPN uses ChaCha20-Poly1305 as its sole symmetric cipher, prized for its speed and constant-time safety.
• Google’s early adoption of ChaCha20-Poly1305 in Chrome and Android significantly improved HTTPS performance on phones before AES-NI was universal in mobile chipsets.
• Legacy systems like older Wi-Fi (WEP) and some VPN implementations historically relied on RC4, contributing to well-documented real-world breaches.

Common Mistakes When Implementing Stream Ciphers

• Reusing nonces across sessions or restarting a counter without also rotating the key.
• Using a stream cipher without any message authentication, leaving ciphertext malleable.
• Rolling a custom keystream generator instead of using audited, standardized designs.
• Truncating or predicting nonces in low-entropy environments (embedded systems, IoT).
• Assuming “fast” and “simple” always means “secure” — RC4’s simplicity was exactly what made its biases so exploitable.

Frequently Asked Questions

Is RC4 still safe to use anywhere? No. RC4 is formally banned in TLS by RFC 7465 and is discouraged in every modern protocol due to statistical biases in its keystream and historical key-recovery attacks against WEP.

Why does ChaCha20 use 20 rounds specifically? Cryptanalysis of reduced-round variants (Salsa20/ChaCha20 with fewer rounds) has shown that meaningful attacks only work up to around 7–8 rounds. Using 20 rounds provides a large security margin beyond the best-known attacks.

Can a stream cipher be used for random number generation? Yes — many cryptographically secure pseudorandom number generators (CSPRNGs) are, at their core, stream cipher keystream generators seeded with a truly random key.

Is ChaCha20 faster than AES? On hardware with AES-NI acceleration, AES-GCM is typically faster. On hardware without such acceleration (many phones, IoT devices, older CPUs), ChaCha20 usually outperforms AES significantly in pure software.

Do stream ciphers provide integrity, not just confidentiality? No. Plain stream ciphers only provide confidentiality. Integrity requires a separate MAC (e.g., Poly1305) or an AEAD construction combining both.

Summary

Stream ciphers encrypt data by XORing plaintext with a pseudorandom keystream derived from a secret key and nonce. RC4 pioneered simple, software-friendly stream cipher design but was ultimately undone by statistical biases and weak key scheduling, leading to its deprecation. ChaCha20 addresses these weaknesses with a modern ARX-based design, a seekable block-counter architecture, and a decade-plus track record of resisting cryptanalysis, making it the preferred choice in protocols like TLS 1.3 and WireGuard. Regardless of which cipher is used, the golden rules remain the same: never reuse a key/nonce pair, always pair encryption with authentication, and prefer standardized, peer-reviewed constructions over custom designs.

References

• RFC 8439 — ChaCha20 and Poly1305 for IETF Protocols, Internet Engineering Task Force.
• RFC 7465 — Prohibiting RC4 Cipher Suites, Internet Engineering Task Force.
• Fluhrer, S., Mantin, I., Shamir, A. — Weaknesses in the Key Scheduling Algorithm of RC4, Selected Areas in Cryptography, 2001.
• AlFardan, N., Bernstein, D. J., Paterson, K., Poettering, B., Schuldt, J. — On the Security of RC4 in TLS, USENIX Security Symposium, 2013.
• Bernstein, D. J. — ChaCha, a variant of Salsa20, Document ID: 4027b5256e17b9796842e6d0f68b0b8e, 2008.
• NIST Special Publication 800-38A — Recommendation for Block Cipher Modes of Operation.
• National Institute of Standards and Technology (NIST) — FIPS 197: Advanced Encryption Standard (AES) (for comparative context with block ciphers).