How to Run a Weave Network on AWS: Docker Container Networking and Overlay Setup Guide

When I first started running Docker containers across multiple EC2 instances, I hit the same wall almost every DevOps engineer hits: containers on one host couldn’t talk to containers on another host without a mess of manual port mapping and NAT rules. That’s the problem Weave Net was built to solve, and in this guide I’m going to walk you through exactly how I set up a Weave overlay network on AWS, from the underlying theory to the working multi-host deployment.

Why Overlay Networking Matters on AWS

By default, Docker creates an isolated bridge network (docker0) on each host. Containers on the same host can reach each other through this bridge, but containers on different hosts are invisible to one another unless you expose ports and route traffic through the host’s public or private IP. On AWS, where I’m usually running instances across multiple Availability Zones inside a VPC, this becomes painful fast — especially once I’m running more than a handful of services that need to discover and talk to each other dynamically.

An overlay network solves this by creating a virtual network that spans multiple hosts. Every container gets its own IP address on this virtual network, and traffic between containers is encapsulated and routed transparently across the underlying AWS network, regardless of which subnet or AZ the host actually lives in. Weave Net was one of the earliest and most reliable tools to implement this for Docker.

How Weave Net Works Internally

Weave creates a router process (weaver) on every host that participates in the network. Each router:

• Maintains a partial view of the network topology and gossips updates to its peers.
• Creates a network bridge (weave) on the host, attached to a Linux kernel bridge.
• Captures traffic destined for other Weave-connected containers and encapsulates it using either UDP (with optional fast datapath via VXLAN) or a sleeve mode that falls back to userspace forwarding when fast datapath isn’t possible (for example, across networks that block UDP or drop VXLAN packets).
• Assigns IP addresses to containers using an internal IP Address Management (IPAM) allocator, so you don’t need an external DHCP server for the overlay.

Because the routers gossip topology information to each other rather than requiring a central controller, the network is resilient — if one host goes down, the others keep working, and Weave will automatically re-route around a lost peer.

Prerequisites

Before starting, I made sure I had:

• Two or more AWS EC2 instances running Ubuntu 22.04 (Amazon Linux works too, but the package names differ slightly).
• Docker Engine 20.10+ installed on each instance.
• Security group rules allowing traffic between the instances on TCP/UDP port 6783 (Weave control and data plane) and UDP port 6784 (used for fast datapath in some configurations).
• SSH access to both instances.

Security Group Rule Example

Type: Custom TCP
Port Range: 6783
Source: <security-group-id-of-your-cluster>

Type: Custom UDP
Port Range: 6783-6784
Source: <security-group-id-of-your-cluster>

I always scope the source to the cluster’s own security group rather than opening it to 0.0.0.0/0, since Weave traffic is unencrypted by default unless you enable the built-in encryption.

Step 1: Launch and Prepare the EC2 Instances

# On each instance
sudo apt-get update
sudo apt-get install -y docker.io
sudo systemctl enable --now docker
sudo usermod -aG docker ubuntu

Log out and back in so the group membership takes effect, then confirm Docker is running:

docker version

Expected output includes both a Client and Server section confirming the daemon is active.

Step 2: Install Weave Net

Weave ships as a single self-installing script that pulls down the weaveworks/weave-kube or weaveworks/weave container image and wraps it in a CLI tool called weave.

sudo curl -L git.io/weave -o /usr/local/bin/weave
sudo chmod a+x /usr/local/bin/weave

Verify it installed correctly:

weave version

Step 3: Launch the Weave Router on Each Host

On the first instance (I’ll call it host1, with private IP 10.0.1.10):

weave launch

This pulls the Weave router image and starts it as a container named weave. On the second instance (host2), I tell Weave to connect to the first host so they discover each other:

weave launch 10.0.1.10

You can add more peers later at any time:

weave connect 10.0.1.10

Check that the peers have found each other:

weave status

Expected output looks like this:

Version: 2.8.1
Service: router
Protocol: weave 1..2
Peers: 2 (with 2 established connections)
Connections: 1 (1 established)

Step 4: Set the Docker Environment to Use Weave

Weave provides a helper that configures your shell so docker run automatically attaches new containers to the Weave network:

eval "$(weave env)"

From this point on, any container you launch with docker run in this shell session gets attached to the Weave network automatically and receives an IP from Weave’s IPAM range (10.32.0.0/12 by default).

Step 5: Run Containers and Test Cross-Host Connectivity

On host1:

docker run --name pingme -d busybox sleep 3600
weave ps pingme

weave ps shows the container’s Weave-assigned IP and MAC address:

pingme 7a:0c:1f:3e:aa:01 10.40.0.2/12

On host2, launch another container and ping the first one by its container name — Weave provides built-in DNS-based service discovery:

eval "$(weave env)"
docker run -it --rm busybox ping -c 3 pingme.weave.local

Expected output:

PING pingme.weave.local (10.40.0.2): 56 data bytes
64 bytes from 10.40.0.2: seq=0 ttl=64 time=1.203 ms
64 bytes from 10.40.0.2: seq=1 ttl=64 time=0.998 ms
64 bytes from 10.40.0.2: seq=2 ttl=64 time=1.045 ms

That confirms two containers on two separate EC2 instances, in two different security-group-protected subnets, are talking to each other over the overlay as if they were on the same LAN.

Running Weave with Docker Compose

Weave doesn’t integrate natively with Compose’s networks: key the way Docker’s own overlay driver does, so in practice I either use the weave env trick before running docker compose up, or I install Weave as a proper Docker network plugin:

docker plugin install weaveworks/net-plugin:latest_release
docker network create --driver=weaveworks/net-plugin:latest_release --attachable weave-net

Then in docker-compose.yml:

version: "3.9"
services:
  web:
    image: nginx:alpine
    networks:
      - weave-net
  api:
    image: myorg/api:latest
    networks:
      - weave-net

networks:
  weave-net:
    external: true

Running docker compose up -d on each host attaches every service to the same overlay, and DNS-based discovery (web, api) works exactly as it would in a single-host Compose setup.

Security: Encrypting Weave Traffic

Since AWS inter-instance traffic can traverse infrastructure I don’t fully control (especially across AZs), I enable Weave’s built-in encryption using a pre-shared password:

weave launch --password mySuperSecretPassword123

All peers must use the same password. Internally, Weave uses NaCl (Curve25519, Salsa20, Poly1305) for encrypting both the gossip control traffic and the data-plane packets, so this adds real cryptographic protection rather than relying solely on the AWS network boundary.

Monitoring and Troubleshooting

A few commands I reach for constantly when something isn’t working:

weave status connections   # shows peer connection health
weave status ipam          # shows IP allocation across the cluster
weave report                # dumps a detailed JSON status report
docker logs weave           # router-level logs

Common issues I’ve run into:

• Peers won’t connect: almost always a security group blocking TCP/UDP 6783. Double-check both directions.
• Fast datapath falls back to sleeve mode: this happens when the underlying network drops VXLAN UDP packets, which some older AWS NAT configurations do. Sleeve mode still works, just with slightly higher latency.
• DNS resolution fails between containers: confirm both containers were launched with weave env active, and that weave status dns shows registered entries.

Best Practices I Follow

• Always pin the Weave version explicitly in automation scripts instead of relying on “latest,” since router protocol versions must match across peers.
• Use a dedicated security group for the Weave cluster ports so it’s never accidentally left open to the internet.
• Turn on encryption for anything crossing AZ boundaries.
• Combine Weave with AWS CloudWatch agent shipping docker logs weave output for centralized visibility.
• If you’re moving toward Kubernetes, know that Weave also ships a CNI plugin (Weave Net for Kubernetes), which uses the same underlying router but integrates with kubelet instead of the Docker CLI directly.

Summary

Weave Net gives you a production-capable overlay network across AWS EC2 instances without needing an external SDN controller, custom routing tables, or a service mesh just to let containers find each other. Once the routers are peered, container-to-container communication and DNS-based discovery just work, whether your containers are on the same instance or three Availability Zones apart. For anything beyond a couple of test hosts, I’d recommend enabling encryption and monitoring weave status as part of your regular health checks.

References

• Docker networking overview: https://docs.docker.com/network/
• Weaveworks Weave Net documentation (archived project docs): https://www.weave.works/docs/net/latest/overview/
• Docker plugin system: https://docs.docker.com/engine/extend/
• CNCF Cloud Native Interactive Landscape: https://landscape.cncf.io/