Offensive Security Researcher & Quantum Cryptography Analyst

How to Run a Weave Network on AWS: Docker Container Networking and Overlay Setup Guide

Abdul Wahab Junaid
How to Run a Weave Network on AWS

How to Run a Weave Network on AWS

When I first started running Docker containers across multiple EC2 instances, I hit the same wall almost every DevOps engineer hits: containers on one host couldn’t talk to containers on another host without a mess of manual port mapping and NAT rules. That’s the problem Weave Net was built to solve, and in this guide I’m going to walk you through exactly how I set up a Weave overlay network on AWS, from the underlying theory to the working multi-host deployment.

Why Overlay Networking Matters on AWS

By default, Docker creates an isolated bridge network (docker0) on each host. Containers on the same host can reach each other through this bridge, but containers on different hosts are invisible to one another unless you expose ports and route traffic through the host’s public or private IP. On AWS, where I’m usually running instances across multiple Availability Zones inside a VPC, this becomes painful fast — especially once I’m running more than a handful of services that need to discover and talk to each other dynamically.

An overlay network solves this by creating a virtual network that spans multiple hosts. Every container gets its own IP address on this virtual network, and traffic between containers is encapsulated and routed transparently across the underlying AWS network, regardless of which subnet or AZ the host actually lives in. Weave Net was one of the earliest and most reliable tools to implement this for Docker.

How Weave Net Works Internally

Weave creates a router process (weaver) on every host that participates in the network. Each router:

Because the routers gossip topology information to each other rather than requiring a central controller, the network is resilient — if one host goes down, the others keep working, and Weave will automatically re-route around a lost peer.

Prerequisites

Before starting, I made sure I had:

Security Group Rule Example

Type: Custom TCP
Port Range: 6783
Source: <security-group-id-of-your-cluster>

Type: Custom UDP
Port Range: 6783-6784
Source: <security-group-id-of-your-cluster>

I always scope the source to the cluster’s own security group rather than opening it to 0.0.0.0/0, since Weave traffic is unencrypted by default unless you enable the built-in encryption.

Step 1: Launch and Prepare the EC2 Instances

# On each instance
sudo apt-get update
sudo apt-get install -y docker.io
sudo systemctl enable --now docker
sudo usermod -aG docker ubuntu

Log out and back in so the group membership takes effect, then confirm Docker is running:

docker version

Expected output includes both a Client and Server section confirming the daemon is active.

Step 2: Install Weave Net

Weave ships as a single self-installing script that pulls down the weaveworks/weave-kube or weaveworks/weave container image and wraps it in a CLI tool called weave.

sudo curl -L git.io/weave -o /usr/local/bin/weave
sudo chmod a+x /usr/local/bin/weave

Verify it installed correctly:

weave version

Step 3: Launch the Weave Router on Each Host

On the first instance (I’ll call it host1, with private IP 10.0.1.10):

weave launch

This pulls the Weave router image and starts it as a container named weave. On the second instance (host2), I tell Weave to connect to the first host so they discover each other:

weave launch 10.0.1.10

You can add more peers later at any time:

weave connect 10.0.1.10

Check that the peers have found each other:

weave status

Expected output looks like this:

Version: 2.8.1
Service: router
Protocol: weave 1..2
Peers: 2 (with 2 established connections)
Connections: 1 (1 established)

Step 4: Set the Docker Environment to Use Weave

Weave provides a helper that configures your shell so docker run automatically attaches new containers to the Weave network:

eval "$(weave env)"

From this point on, any container you launch with docker run in this shell session gets attached to the Weave network automatically and receives an IP from Weave’s IPAM range (10.32.0.0/12 by default).

Step 5: Run Containers and Test Cross-Host Connectivity

On host1:

docker run --name pingme -d busybox sleep 3600
weave ps pingme

weave ps shows the container’s Weave-assigned IP and MAC address:

pingme 7a:0c:1f:3e:aa:01 10.40.0.2/12

On host2, launch another container and ping the first one by its container name — Weave provides built-in DNS-based service discovery:

eval "$(weave env)"
docker run -it --rm busybox ping -c 3 pingme.weave.local

Expected output:

PING pingme.weave.local (10.40.0.2): 56 data bytes
64 bytes from 10.40.0.2: seq=0 ttl=64 time=1.203 ms
64 bytes from 10.40.0.2: seq=1 ttl=64 time=0.998 ms
64 bytes from 10.40.0.2: seq=2 ttl=64 time=1.045 ms

That confirms two containers on two separate EC2 instances, in two different security-group-protected subnets, are talking to each other over the overlay as if they were on the same LAN.

Running Weave with Docker Compose

Weave doesn’t integrate natively with Compose’s networks: key the way Docker’s own overlay driver does, so in practice I either use the weave env trick before running docker compose up, or I install Weave as a proper Docker network plugin:

docker plugin install weaveworks/net-plugin:latest_release
docker network create --driver=weaveworks/net-plugin:latest_release --attachable weave-net

Then in docker-compose.yml:

version: "3.9"
services:
  web:
    image: nginx:alpine
    networks:
      - weave-net
  api:
    image: myorg/api:latest
    networks:
      - weave-net

networks:
  weave-net:
    external: true

Running docker compose up -d on each host attaches every service to the same overlay, and DNS-based discovery (web, api) works exactly as it would in a single-host Compose setup.

Security: Encrypting Weave Traffic

Since AWS inter-instance traffic can traverse infrastructure I don’t fully control (especially across AZs), I enable Weave’s built-in encryption using a pre-shared password:

weave launch --password mySuperSecretPassword123

All peers must use the same password. Internally, Weave uses NaCl (Curve25519, Salsa20, Poly1305) for encrypting both the gossip control traffic and the data-plane packets, so this adds real cryptographic protection rather than relying solely on the AWS network boundary.

Monitoring and Troubleshooting

A few commands I reach for constantly when something isn’t working:

weave status connections   # shows peer connection health
weave status ipam          # shows IP allocation across the cluster
weave report                # dumps a detailed JSON status report
docker logs weave           # router-level logs

Common issues I’ve run into:

Best Practices I Follow

Summary

Weave Net gives you a production-capable overlay network across AWS EC2 instances without needing an external SDN controller, custom routing tables, or a service mesh just to let containers find each other. Once the routers are peered, container-to-container communication and DNS-based discovery just work, whether your containers are on the same instance or three Availability Zones apart. For anything beyond a couple of test hosts, I’d recommend enabling encryption and monitoring weave status as part of your regular health checks.

References

Exit mobile version