Introduction to Redis: Data Structures, Caching, and Pub/Sub Messaging

The first time I plugged Redis into a project, it was to solve a simple problem: my application’s database was getting hammered by repeated reads of the same data, and response times were creeping up. I added a Redis cache in front of it, and within an afternoon, average response time dropped by more than 80%. That was years ago, and since then I’ve used Redis for far more than caching — session storage, rate limiting, real-time leaderboards, job queues, and pub/sub messaging between services. In this article, I’ll walk through what Redis actually is, the data structures that make it so versatile, and how to use it well.

What Is Redis, Really?

Redis stands for Remote Dictionary Server, and at its core, it’s an in-memory data store. That single fact — that data lives in RAM rather than on disk — is what makes Redis so fast. Reads and writes typically complete in well under a millisecond, because there’s no disk seek time involved, and Redis is single-threaded for command execution, which avoids a lot of the locking complexity that multi-threaded databases have to deal with.

Redis is often labeled a “key-value store,” but that undersells it. Unlike simpler key-value stores that only support storing strings, Redis supports several rich data structures natively, and it gives you atomic operations on all of them. This is a big part of why Redis has become the default choice for so many caching, messaging, and real-time use cases.

Redis’s Core Data Structures

Strings

Strings are the simplest Redis data type — a key maps to a single value, which can be text, a serialized JSON blob, or even binary data up to 512MB. Strings support atomic operations like INCR and DECR, which makes them perfect for counters — think page view counts, rate limiters, or vote tallies. Because the increment is atomic at the Redis engine level, you don’t need to worry about race conditions when multiple clients increment the same counter simultaneously.

Lists

Redis lists are ordered collections of strings, implemented internally as linked lists (with an optimized “listpack” encoding for smaller lists). You can push and pop from either end in constant time, which makes lists a natural fit for queues and stacks. A common pattern I’ve used is a simple job queue: producers LPUSH jobs onto a list, and workers BRPOP (a blocking pop) to pick up jobs as they arrive, without needing to poll constantly.

Sets

Sets store unique, unordered collections of strings, with fast membership testing and support for operations like union, intersection, and difference. I’ve used sets for things like tracking unique visitors to a page in a given day, or for tag systems where I need to quickly answer “which items have both tag A and tag B” using SINTER.

Sorted Sets

Sorted sets (often called ZSETs) are one of my favorite Redis structures. Every member of a sorted set has an associated score, and Redis keeps the set ordered by that score automatically. This makes sorted sets perfect for leaderboards — ZADD to add or update a player’s score, ZREVRANGE to get the top N players, and ZRANK to find a specific player’s rank, all in logarithmic time. I’ve also used sorted sets for time-based data, using a timestamp as the score, to build things like “recent activity” feeds that stay sorted without any extra work.

Hashes

Hashes let you store field-value pairs under a single key, similar to a small object or dictionary. They’re ideal for representing structured records — a user profile, for example — where you want to update or read individual fields (like just the email field) without pulling and re-serializing an entire JSON blob. Hashes are also more memory-efficient than storing many separate string keys for related fields.

Streams

Redis Streams, added in Redis 5.0, are an append-only log data structure designed for event and message data, similar in spirit to a lightweight Kafka. Streams support consumer groups, which let multiple consumers cooperatively process a stream of events, each message going to exactly one consumer in the group. I’ve used streams for building event-sourcing style systems where I need durable, ordered event logs with the ability to replay from any point.

Other Structures

Redis also offers HyperLogLog for approximate cardinality counting (great for “how many unique users” at massive scale with minimal memory), Geospatial indexes for location-based queries, and Bitmaps for compact boolean flag storage across large populations of items.

Redis as a Caching Layer

Caching is still probably the most common Redis use case, and for good reason. The typical pattern is called cache-aside: when your application needs data, it first checks Redis. If the data is there (a cache hit), it returns immediately. If not (a cache miss), the application queries the primary database, then writes the result into Redis before returning it, so the next request is fast.

Key things I’ve learned about doing this well:

Set sensible TTLs. Every cached key should generally have an expiration time (EXPIRE or SET ... EX), so stale data doesn’t live forever if your invalidation logic misses an edge case.

Think about cache invalidation. The famous saying is that cache invalidation is one of the two hard problems in computer science. When underlying data changes, you need a strategy — whether that’s actively deleting the cache key on write, or relying on short TTLs to bound staleness.

Watch out for cache stampedes. If a popular cache key expires and a flood of concurrent requests all miss at once, they can all hit your database simultaneously. Techniques like locking around cache regeneration, or staggering TTLs, help avoid this.

Use appropriate eviction policies. Redis supports several eviction policies (like allkeys-lru or volatile-lru) for when memory fills up. Choosing the right one for your workload matters — a pure cache usually wants allkeys-lru, while a mixed workload with some persistent data might want volatile-lru so only keys with a TTL get evicted.

Pub/Sub Messaging

Redis’s publish/subscribe feature lets clients subscribe to named channels and receive messages published to those channels in real time. A publisher calls PUBLISH channel message, and every currently subscribed client receives it instantly.

I’ve used pub/sub for things like broadcasting real-time notifications to connected websocket clients across multiple application server instances — when one server needs to notify a user, it publishes to a channel, and whichever server holds that user’s websocket connection picks it up and forwards it.

It’s important to understand Redis pub/sub’s limitations, though. Messages are fire-and-forget — if no client is subscribed to a channel when a message is published, that message is lost. There’s no persistence and no replay. For use cases where you need guaranteed delivery or the ability to replay missed messages, Redis Streams (or a dedicated message broker) is the better choice.

Redis Persistence Briefly

While Redis is primarily an in-memory store, it does offer persistence options — RDB snapshots and AOF logs — so data can survive a restart. I go into much more depth on this in a companion article on Redis persistence, but the short version is: Redis is not purely ephemeral, and you can configure it for durability guarantees depending on your tolerance for potential data loss versus performance.

Practical Example: Rate Limiting with Redis

A pattern I use constantly is API rate limiting with Redis strings and TTLs:

INCR rate_limit:user123
EXPIRE rate_limit:user123 60

If INCR returns a value greater than my allowed threshold within that 60-second window, I reject the request. Because INCR is atomic, this works correctly even under heavy concurrent load without any additional locking.

Real-World Use Cases

Beyond caching, I’ve deployed Redis for:

• Session storage for web applications, since sessions need fast reads/writes and can tolerate being ephemeral.
• Leaderboards in gaming and gamified applications, using sorted sets.
• Distributed locks, using the SET key value NX EX seconds pattern to implement simple mutual exclusion across services.
• Job queues, using lists or streams to coordinate background workers.
• Real-time analytics, using HyperLogLog for approximate unique counts and sorted sets for trending content.

Advantages and Limitations

Redis’s biggest advantage is raw speed combined with a genuinely useful set of data structures that map naturally onto common application problems. It’s also relatively simple to operate compared to some distributed databases, especially for smaller deployments.

The limitations are real, though. Because Redis is primarily memory-bound, your dataset needs to fit (or mostly fit) in RAM, which can get expensive at very large scale. Redis Cluster provides horizontal scaling and sharding, but it introduces operational complexity, especially around multi-key operations that span shards. And while persistence options exist, Redis is fundamentally optimized for speed over the strongest durability guarantees you’d get from a disk-oriented database.

Security Considerations

Redis historically shipped with no authentication by default, and misconfigured, publicly exposed Redis instances have been a real source of security incidents. Best practice today includes enabling requirepass or Redis ACLs (introduced in Redis 6, which allow fine-grained per-user permissions), binding Redis to private networks rather than exposing it publicly, enabling TLS for data in transit, and disabling or renaming dangerous commands like FLUSHALL and CONFIG in production environments.

Comparing Redis to Other NoSQL Options

Compared to Cassandra or DynamoDB, Redis trades some durability and horizontal scale for extreme low-latency performance and richer in-place data structures. Compared to Memcached, its closest caching competitor, Redis offers far more data structures and features (pub/sub, persistence, scripting via Lua) while Memcached remains simpler and, in some pure-cache benchmarks, marginally more memory-efficient for basic string caching.

Best Practices I Follow

• Choose the right data structure for the job rather than defaulting to strings with JSON everywhere — hashes, sorted sets, and lists often fit better and perform better.
• Set TTLs on cache keys by default.
• Monitor memory usage and configure an eviction policy deliberately, not by accident.
• Use Redis Streams instead of pub/sub when you need durability or replay.
• Secure every Redis instance with authentication, network isolation, and TLS.
• Avoid extremely large values or huge single keys (like massive lists or hashes), which can cause latency spikes since Redis commands are largely single-threaded.

Redis Data Structure Server vs. Simple Cache

I want to spend a bit more time on why I think of Redis as a “data structure server” rather than just a cache, because that framing changes how I use it day to day. A pure cache is stateless from the application’s perspective — you can wipe it and regenerate everything from the source of truth. But once I started using sorted sets for leaderboards, hashes for session objects, and streams for lightweight event pipelines, Redis stopped being disposable and started holding data that genuinely mattered on its own. That shift is worth recognizing early in any project, because it changes your persistence configuration, your backup strategy, and your operational expectations dramatically. A Redis instance holding only regenerable cache data can be treated casually; a Redis instance holding active job queues or session state needs the same operational seriousness as any primary datastore.

Lua Scripting and Atomic Multi-Step Operations

One feature I haven’t touched on yet that deserves mention is Redis’s support for server-side Lua scripting via EVAL. Individual Redis commands are already atomic, but sometimes an operation needs several steps to be atomic together — check a value, and only if it meets some condition, modify a different key. Wrapping this logic in a Lua script and sending it to Redis for server-side execution guarantees the whole sequence runs as a single atomic unit, since Redis executes the entire script without interleaving other clients’ commands in between.

I’ve used this pattern for more sophisticated rate limiting than a simple INCR/EXPIRE pair can achieve — implementing a proper sliding-window or token-bucket algorithm requires reading and updating multiple pieces of state together, and a Lua script executed atomically on the Redis server avoids the race conditions that would otherwise occur if that logic ran as several separate round trips from the application.

Redis Cluster and Horizontal Scaling

For datasets or throughput requirements that outgrow a single Redis instance, Redis Cluster provides automatic sharding across multiple nodes, using a hash-slot mechanism (16,384 slots distributed across the cluster’s master nodes) rather than the consistent-hashing-ring approach used by Cassandra or DynamoDB. Each key is mapped to a slot based on a hash of the key (or a specific “hash tag” within the key, if you want to force related keys onto the same slot for multi-key operations).

A real operational nuance with Redis Cluster: multi-key operations (like MGET across several keys, or a Lua script touching multiple keys) generally require all involved keys to live in the same hash slot. This means cluster-aware applications often need to deliberately co-locate related keys using hash tags — for example, naming keys user:{123}:profile and user:{123}:settings so the {123} portion is what actually gets hashed, guaranteeing both keys land on the same node and can be operated on together atomically.

Monitoring Redis in Production

A few metrics I watch closely on every Redis deployment: memory usage relative to the configured maxmemory limit, since approaching that ceiling triggers eviction (or, if no eviction policy is set, write failures); the eviction rate itself, which tells me whether my cache is actually sized appropriately for the working set; hit rate, which tells me whether the cache is actually earning its keep or mostly missing and falling through to the database anyway; and replication lag, if I’m running read replicas, since a lagging replica serving stale reads can quietly cause subtle application bugs that are difficult to trace back to their root cause.

Redis Data Types in Practice: A Quick Decision Guide

When I’m modeling a new feature against Redis, I run through a short mental checklist to pick the right structure. If I just need a single value with atomic increment/decrement, a string is enough. If I need an ordered collection I’ll push to and pop from either end, a list fits. If I need fast membership checks and set algebra, a set is the answer. If I need anything ranked or sorted by a score — leaderboards, priority queues, time-ordered feeds — a sorted set almost always beats trying to hand-roll that logic with strings. If I’m representing a structured object with several fields I’ll want to read or update individually, a hash keeps things tidy and memory-efficient compared to spreading those fields across separate top-level keys. And if I need a durable, replayable, multi-consumer event log, streams are the right tool rather than trying to force that shape onto a list.

This decision process matters because picking the wrong structure often means recreating structure-specific behavior manually in application code — for instance, using a plain string to store JSON and manually parsing, modifying, and re-serializing it on every update, when a hash would have let Redis handle field-level updates natively and atomically.

Expiration and TTL Mechanics

Nearly every Redis use case I’ve described benefits from setting a time-to-live on keys, and it’s worth understanding how Redis actually implements expiration under the hood. Rather than continuously scanning every key to check if it’s expired, Redis uses a combination of lazy expiration (a key is checked and removed if expired the moment it’s accessed) and active expiration (a background process periodically samples a random selection of keys with TTLs set and proactively removes any that have expired). This hybrid approach keeps expiration overhead low without letting expired keys linger indefinitely in memory just because nothing happened to request them again.

One subtlety worth knowing: replication of expiration is handled carefully so that replicas don’t independently decide a key has expired ahead of the primary — instead, the primary is responsible for the authoritative expiration decision and propagates the deletion to replicas explicitly, avoiding a class of subtle inconsistency that could otherwise arise from clock drift between primary and replica nodes.

Final Thoughts

Redis earns its popularity because it solves real, common problems — caching, real-time coordination, and lightweight messaging — with an elegant set of tools and genuinely excellent performance. It’s not a replacement for your primary database, but as a complementary layer, it’s one of the most useful pieces of infrastructure I reach for on nearly every project I build.