Whenever I explain cryptography to someone new to the field, I start by taking apart the word “cryptosystem” itself. It’s not one thing — it’s an assembly of parts that only works securely when every piece is doing its job correctly. In this article, I’ll walk through each of those parts: keys, algorithms, and protocols, plus the supporting components that tie them together, from the math underneath to the way they show up in real systems.
Defining a Cryptosystem
Formally, a cryptosystem is typically defined as a five-tuple:
$$(\mathcal{P}, \mathcal{C}, \mathcal{K}, \mathcal{E}, \mathcal{D})$$
where $\mathcal{P}$ is the plaintext space, $\mathcal{C}$ is the ciphertext space, $\mathcal{K}$ is the key space, $\mathcal{E}$ is the family of encryption functions, and $\mathcal{D}$ is the family of decryption functions, such that for every key $k \in \mathcal{K}$:
$$D_k(E_k(p)) = p \quad \text{for all } p \in \mathcal{P}$$
This formalism, first popularized in modern form through Shannon’s foundational work, captures the essential requirement of any cryptosystem: encryption must be reversible with the correct key, and — critically — computationally infeasible to reverse without it.
Component 1: Keys
The key is the secret parameter that controls the behavior of the encryption and decryption functions. Per Kerckhoffs’s principle, a cryptosystem should remain secure even if everything about the system is public except the key — security should never depend on obscurity.
Symmetric Keys
In symmetric cryptography, the same key $k$ is used for both encryption and decryption:
$$C = E_k(P), \qquad P = D_k(C)$$
This is efficient and fast (AES, ChaCha20), but it requires a secure way to share the key between parties beforehand — the classic “key distribution problem.”
Asymmetric (Public/Private) Key Pairs
In asymmetric cryptography, each party has a mathematically related key pair: a public key $k_{pub}$ that can be shared openly, and a private key $k_{priv}$ that must be kept secret.
$$C = E_{k_{pub}}(P), \qquad P = D_{k_{priv}}(C)$$
RSA, based on the difficulty of factoring large semiprime numbers, and elliptic curve cryptography (ECC), based on the difficulty of the elliptic curve discrete logarithm problem, are the two dominant families in use today. ECC achieves comparable security to RSA with much shorter key lengths — a 256-bit ECC key offers roughly the same security strength as a 3072-bit RSA key.
Key Length and Security Strength
| Symmetric Key Size | Approximate RSA Equivalent | Approximate ECC Equivalent |
|---|---|---|
| 80 bits (deprecated) | 1024 bits | 160 bits |
| 112 bits | 2048 bits | 224 bits |
| 128 bits | 3072 bits | 256 bits |
| 256 bits | 15360 bits | 512 bits |
(Approximate equivalence per NIST SP 800-57 guidance.)
Key Management Lifecycle
A key isn’t just generated and used — it moves through a full lifecycle: generation, distribution, storage, rotation, and eventual destruction. Weaknesses at any stage of this lifecycle (predictable random number generation, insecure storage, indefinite reuse) routinely undermine otherwise strong cryptography.
Component 2: Algorithms
The algorithm defines the actual mathematical transformation applied to the plaintext. Algorithms generally fall into a few major families:
Block Ciphers
Block ciphers operate on fixed-size chunks of data (e.g., 128 bits for AES), transforming each block through multiple rounds of substitution and permutation operations — a design pattern known as a substitution-permutation network (SPN). AES, standardized in FIPS 197, is the dominant modern block cipher, supporting 128, 192, and 256-bit keys.
Stream Ciphers
Stream ciphers generate a pseudorandom keystream $k_s$ that is combined with plaintext bit-by-bit or byte-by-byte, typically via XOR:
$$C_i = P_i \oplus k_{s,i}$$
ChaCha20 is a widely deployed modern stream cipher, valued for its high performance in software without dedicated hardware acceleration.
Hash Functions
A cryptographic hash function $H$ maps arbitrary-length input to a fixed-length output, with three essential properties: preimage resistance, second-preimage resistance, and collision resistance. SHA-256 and the SHA-3 family (based on the Keccak sponge construction) are current standards.
Message Authentication Codes (MACs)
A MAC combines a hash-like structure with a secret key to provide both integrity and authenticity: $T = \text{MAC}_k(M)$. HMAC (Hash-based MAC) is the most widely used construction, combining a hash function with a key in a specific nested structure defined in RFC 2104.
Digital Signature Algorithms
Digital signatures use asymmetric keys to provide authenticity, integrity, and non-repudiation. RSA-PSS, ECDSA, and EdDSA (particularly Ed25519) are the current standards, each defined by specific signing and verification algorithms over their respective mathematical structures.
Component 3: Protocols
A protocol defines how keys and algorithms are actually used together in a real interaction between parties — the sequencing, negotiation, and authentication steps that turn raw cryptographic primitives into a usable secure system.
Key Exchange Protocols
Diffie-Hellman key exchange lets two parties establish a shared secret over an insecure channel without ever transmitting the secret itself. Each party computes:
$$A = g^a \bmod p, \qquad B = g^b \bmod p$$
and then derives the same shared secret:
$$s = B^a \bmod p = A^b \bmod p = g^{ab} \bmod p$$
The security of this exchange rests on the computational difficulty of the discrete logarithm problem — recovering $a$ from $A$, $g$, and $p$.
Transport Protocols
TLS (Transport Layer Security) is the protocol most people interact with daily, layering a handshake (which negotiates algorithms and authenticates the server via certificates), a key exchange (commonly ephemeral Diffie-Hellman for forward secrecy), and a record layer (which applies the negotiated symmetric cipher and MAC/AEAD construction to actual application data).
Authentication Protocols
Protocols like Kerberos use symmetric cryptography and trusted third parties (a Key Distribution Center) to authenticate users and services within a network without repeatedly transmitting passwords.
How the Components Fit Together: A Worked Example
Consider a typical HTTPS connection:
- Protocol negotiation — client and server agree on a TLS version and cipher suite.
- Key exchange — an ephemeral elliptic-curve Diffie-Hellman exchange establishes a shared secret.
- Authentication — the server proves its identity using a digital signature verified against a certificate chain rooted in a trusted certificate authority.
- Key derivation — the shared secret is passed through a key derivation function (like HKDF) to produce separate keys for encryption and integrity.
- Symmetric encryption — application data is encrypted using an AEAD cipher (commonly AES-GCM or ChaCha20-Poly1305), which combines confidentiality and integrity in a single construction.
Every one of the components discussed above — key, algorithm, and protocol — plays a distinct, necessary role in that single connection.
Supporting Components Worth Knowing
- Random Number Generators (RNGs/CSPRNGs) — cryptographic security depends on truly unpredictable key material; a weak RNG has historically undermined otherwise-strong systems.
- Padding schemes — used to fit plaintext into fixed block sizes (like PKCS#7) or to add semantic security to asymmetric encryption (like OAEP for RSA).
- Key Derivation Functions (KDFs) — such as HKDF or PBKDF2, used to derive one or more cryptographic keys from a shared secret or password.
- Certificate Authorities and PKI — the trust infrastructure that binds public keys to verified identities.
Best Practices for Combining Components
- Never mix and match cryptographic primitives outside of vetted, standardized protocols — subtle interactions between components are a common source of real-world vulnerabilities.
- Prefer AEAD (Authenticated Encryption with Associated Data) modes over separately combining encryption and a MAC, since the correct combination order is easy to get wrong (encrypt-then-MAC is the only generally safe manual construction).
- Use ephemeral key exchange where possible to achieve forward secrecy — protecting past sessions even if a long-term key is later compromised.
- Rely on established libraries (OpenSSL, libsodium, BoringSSL) rather than reimplementing primitives.
Common Mistakes
- Treating “algorithm strength” as the whole picture while neglecting key management or protocol design.
- Using ECB mode for block ciphers, which fails to hide data patterns because identical plaintext blocks always produce identical ciphertext blocks.
- Failing to authenticate a key exchange, leaving it open to man-in-the-middle interception.
- Hardcoding keys directly into source code or configuration files.
FAQs
What’s the difference between an algorithm and a protocol? An algorithm is a mathematical transformation (like AES encryption of a single block). A protocol is the sequence of steps, message exchanges, and rules that define how algorithms and keys are used together between two or more parties.
Why do we need both symmetric and asymmetric cryptography? Asymmetric cryptography solves the key-distribution problem elegantly but is computationally expensive. Symmetric cryptography is fast but requires a shared secret. Most real systems use asymmetric cryptography to establish a shared secret, then switch to fast symmetric cryptography for bulk data.
Is a longer key always a stronger component? Only relative to the specific algorithm’s structure — comparing raw key lengths across different algorithm families (like RSA vs. AES) is meaningless without accounting for their different mathematical foundations.
What role do digital certificates play? They bind a public key to a verified identity, issued and signed by a trusted certificate authority, allowing one party to confirm they’re really communicating with who they think they are.
Summary
A cryptosystem is best understood as three interlocking components: keys that provide the secret parameter, algorithms that define the mathematical transformation, and protocols that govern how everything is actually used together in practice. Weakness in any one component — a short key, a broken cipher mode, an unauthenticated exchange — can undermine the security of the whole system, regardless of how strong the other components are. Understanding each piece individually, and how they interact in real systems like TLS, is the foundation for building or evaluating any secure system.
References
- NIST FIPS 197 — Advanced Encryption Standard (AES).
- NIST SP 800-57 Part 1 — Recommendation for Key Management.
- RFC 2104 — HMAC: Keyed-Hashing for Message Authentication.
- RFC 8446 — The Transport Layer Security (TLS) Protocol Version 1.3.
- Diffie, W., & Hellman, M. (1976). New Directions in Cryptography. IEEE Transactions on Information Theory.
- Shannon, C. (1949). Communication Theory of Secrecy Systems.