Offensive Security Researcher & Quantum Cryptography Analyst

How to Listen to Docker Events on Your Docker Hosts: Real-Time Event Monitoring Guide

Abdul Wahab Junaid
How to Listen to Docker Events on Your Docker Hosts

How to Listen to Docker Events on Your Docker Hosts

Most people find out about docker events after something’s already gone wrong — a container kept restarting overnight, or an image got pulled and nobody knows when or by whom. I started using it proactively instead: piping it into a small alerting script so I get notified the moment a container dies, rather than finding out from a user complaint an hour later. This guide covers docker events from basic usage through filtering, JSON output for scripting, and wiring it into real monitoring pipelines.

What docker events Actually Is

The Docker daemon maintains an internal event stream for nearly everything that happens to containers, images, volumes, networks, and the daemon itself. docker events is a client that subscribes to that stream in real time over the Docker Engine API (GET /events) — it’s push-based, not polling, so events appear the instant they happen with no delay.

Event categories include:

Basic Usage

docker events

This blocks and streams events live. Open a second terminal and generate some activity:

docker run -d --name events-demo alpine sleep 300

Back in the first terminal, expected output:

2026-07-29T10:40:12.123456789Z container create a1b2c3d4... (image=alpine, name=events-demo)
2026-07-29T10:40:12.234567890Z network connect 8f7e6d5c... (container=a1b2c3d4..., name=bridge)
2026-07-29T10:40:12.345678901Z container start a1b2c3d4... (image=alpine, name=events-demo)

Stop it and watch the corresponding events:

docker stop events-demo

2026-07-29T10:41:05.111222333Z container kill a1b2c3d4... (signal=15)
2026-07-29T10:41:05.222333444Z container die a1b2c3d4... (exitCode=143)
2026-07-29T10:41:05.333444555Z container stop a1b2c3d4...

Note the exitCode in the die event — this is one of the most operationally useful fields, letting you distinguish a clean shutdown (0) from a crash (non-zero) without inspecting the container separately.

Filtering Events

Use --filter to narrow the stream — essential once you have more than a couple of containers running, or the output becomes unreadable noise.

By event type:

docker events --filter event=die

By container:

docker events --filter container=events-demo

By image:

docker events --filter image=alpine

By label:

docker events --filter label=environment=production

Combine multiple filters (AND logic between different filter keys, OR logic within the same key):

docker events --filter event=die --filter event=oom --filter label=environment=production

Expected behavior: this streams only die or oom events, restricted to containers labeled environment=production.

Time-Bounded Queries (Not Just Live Streaming)

docker events isn’t only for live tailing — you can query a historical window using --since and --until, exactly like docker logs:

docker events --since "2026-07-29T09:00:00" --until "2026-07-29T10:00:00"

Expected output: every daemon event that occurred in that one-hour window, useful for reconstructing what happened during an incident after the fact — assuming the daemon was running continuously (this reads live from the daemon’s in-memory buffer, not a persisted log, so events before a daemon restart are gone; more on that below).

Relative durations work too:

docker events --since 30m

JSON Output for Scripting

Raw text output isn’t great for programmatic consumption. Use --format:

docker events --format '{{json .}}'

Expected output — one JSON object per line:

{"status":"die","id":"a1b2c3d4...","from":"alpine","Type":"container","Action":"die","Actor":{"ID":"a1b2c3d4...","Attributes":{"exitCode":"137","image":"alpine","name":"events-demo"}},"time":1753784465,"timeNano":1753784465123456789}

Pipe into jq to build a real alerting one-liner:

docker events --filter event=die --format '{{json .}}' | jq -r '"\(.Actor.Attributes.name) exited with code \(.Actor.Attributes.exitCode) at \(.time)"'

Expected output:

events-demo exited with code 137 at 1753784465

Exit code 137 specifically means the container was killed via SIGKILL (128 + 9) — commonly from an OOM kill or a forced docker kill. Combine with the oom event filter to distinguish memory kills from manual ones.

Building a Real Alert Script

Here’s a practical example — a bash script that watches for any container dying with a non-zero exit code and posts to a webhook (e.g., Slack):

#!/usr/bin/env bash
# watch-crashes.sh
WEBHOOK_URL="https://hooks.slack.com/services/XXX/YYY/ZZZ"

docker events --filter event=die --format '{{json .}}' | while read -r event; do
  exit_code=$(echo "$event" | jq -r '.Actor.Attributes.exitCode')
  name=$(echo "$event" | jq -r '.Actor.Attributes.name')

  if [ "$exit_code" != "0" ]; then
    message="⚠️ Container *${name}* exited with code ${exit_code}"
    curl -s -X POST -H 'Content-type: application/json' \
      --data "{\"text\":\"${message}\"}" \
      "$WEBHOOK_URL"
  fi
done

chmod +x watch-crashes.sh
nohup ./watch-crashes.sh &

Run it as a systemd service for durability:

# /etc/systemd/system/docker-crash-watch.service
[Unit]
Description=Watch Docker container crashes and alert
After=docker.service
Requires=docker.service

[Service]
ExecStart=/usr/local/bin/watch-crashes.sh
Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable --now docker-crash-watch.service

Health Check Events

If your containers define a HEALTHCHECK, Docker emits health_status events, which are extremely useful for catching “the process is running but the app is broken” scenarios that die events won’t cover:

docker run -d --name health-demo \
  --health-cmd="curl -f http://localhost/ || exit 1" \
  --health-interval=5s \
  nginx:alpine

docker events --filter event=health_status --filter container=health-demo

Expected output over time:

2026-07-29T10:50:05.000000000Z container health_status: healthy a1b2c3d4... (name=health-demo)

If you then break the container’s health check target, you’d see a transition to health_status: unhealthy — a strong signal to alert on and one that plain die monitoring would completely miss.

Docker Compose and Events

docker events works daemon-wide regardless of whether containers were started via Compose, docker run, or Swarm — Compose doesn’t have its own separate event bus. Filter by Compose project label to scope to one stack:

docker events --filter label=com.docker.compose.project=myapp

Internals: Where This Data Comes From

The Docker daemon (dockerd) maintains an in-memory ring buffer of recent events (default capacity: 1000 events) that the Engine API’s /events endpoint streams from over a long-lived HTTP connection. This has two important consequences:

  1. Events are not persisted across daemon restarts. If dockerd restarts, the buffer is cleared — --since queries can’t reach further back than the daemon’s current uptime plus whatever’s still in the buffer.
  2. High-volume event bursts can evict old events from the buffer before a slow consumer reads them, though this is rare in normal operation — it matters mainly on hosts running very high container churn (e.g., CI runners spinning up hundreds of short-lived containers per minute).

For durable, long-term event history, you need to actively consume the stream into external storage — there’s no built-in persistence layer, which is exactly why the alert-script pattern above (or shipping into a time-series/log system) matters for anything beyond live debugging.

Security Considerations

Troubleshooting

SymptomCauseFix
docker events shows nothingNo activity since command started, or filters too narrowTrigger a test action (docker run, docker stop) in another terminal
Missing events from before a certain timeDaemon restarted, clearing the in-memory bufferShip events externally going forward; historical gap is unrecoverable
Script consuming events stops receiving new onesLong-lived connection dropped (daemon restart, network blip)Wrap the consumer in a restart loop (systemd Restart=always, as shown above)
permission denied on socketUser not in docker groupAdd user to group or run with sudo

Real-World Deployment Notes

The alert-script pattern in this guide is genuinely how I’ve bootstrapped monitoring on small Docker fleets before a full observability stack was justified — it’s a few lines of bash and gets you meaningful signal on day one. As the fleet grows, I’ve typically graduated this into a small always-on consumer service (rather than a bash loop) that normalizes events into a message queue (Kafka, RabbitMQ) so multiple downstream systems — alerting, audit logging, autoscaling triggers — can consume the same event stream independently instead of each polling the Docker socket separately. Also worth knowing: on Kubernetes, the equivalent isn’t docker events at all but the Kubernetes Events API (kubectl get events or watching via the API server) — the underlying container runtime events still exist, but orchestration-level tooling on K8s abstracts you away from talking to the container engine directly.

Summary

docker events turns Docker from something you have to actively check on into something that tells you when it needs attention. The combination of --filter for signal-to-noise control, --format '{{json .}}' for scriptability, and the fact that it’s a genuine live push stream (not polling) makes it a small but high-leverage piece of any container monitoring setup — especially for catching OOM kills and unhealthy states that simpler “is the container running” checks miss entirely.

References

Exit mobile version