How to Set Up an IPS in Linux: A Step-by-Step Guide

The first time an intrusion prevention system I configured actually blocked a real brute-force attempt in production, I finally understood why security teams treat IPS deployment as a serious project rather than a checkbox. Getting it right requires understanding what you’re actually building, not just running an installer.

This guide walks through setting up a genuinely functional IPS on Linux using Suricata, one of the most widely deployed open-source options, along with the concepts you need to configure it well rather than just turn it on.

IDS vs IPS: A Critical Distinction

• IDS (Intrusion Detection System) — monitors traffic passively (often via a mirrored/SPAN port) and generates alerts. It does not block anything on its own.
• IPS (Intrusion Prevention System) — sits inline in the traffic path and can actively drop or reject malicious packets in real time.

The same engine (Suricata, Snort) can typically run in either mode — the difference is deployment: passive tap/mirror for IDS, inline bridge/routed mode for IPS.

flowchart LR
    A[Incoming Traffic] --> B{IPS Engine}
    B -->|Matches malicious signature| C[Drop/Reject Packet]
    B -->|Clean traffic| D[Forward to Destination]
    B --> E[Log Alert to SIEM]

Choosing an IPS Engine

This guide uses Suricata, given its native multi-threaded inline mode and active rule ecosystem via Emerging Threats.

Prerequisites

• A Linux system (Ubuntu 22.04/24.04 used in examples) with at least two network interfaces if deploying in inline bridge mode
• Root/sudo access
• Basic familiarity with iptables/nftables and network interfaces

Step 1: Install Suricata

sudo add-apt-repository ppa:oisf/suricata-stable
sudo apt update
sudo apt install suricata -y
suricata --build-info | head -20

Verify the install includes NFQUEUE support, which is required for inline IPS mode on Linux:

suricata --build-info | grep -i nfqueue

Step 2: Choose an Inline Deployment Mode

Two common inline architectures on Linux:

1. NFQUEUE mode — integrates with iptables/nftables, redirecting packets to Suricata for inspection before allowing them to continue through the kernel’s network stack. Simpler to deploy on a single-interface gateway/router host.
2. AF_PACKET IPS mode (bridge) — Suricata bridges two interfaces directly, inspecting and forwarding packets itself. Higher throughput, common on dedicated IPS appliances.

This guide uses NFQUEUE mode, since it’s the most accessible for a single Linux gateway/firewall host.

Step 3: Configure iptables to Redirect Traffic to Suricata

# Redirect forwarded traffic to NFQUEUE for inspection
sudo iptables -I FORWARD -j NFQUEUE --queue-num 0

# Optionally also inspect traffic destined to/from the host itself
sudo iptables -I INPUT -j NFQUEUE --queue-num 0
sudo iptables -I OUTPUT -j NFQUEUE --queue-num 0

Make sure IP forwarding is enabled if this host acts as a router/gateway:

sudo sysctl -w net.ipv4.ip_forward=1
echo "net.ipv4.ip_forward=1" | sudo tee -a /etc/sysctl.conf

Step 4: Configure Suricata for Inline (IPS) Mode

Edit /etc/suricata/suricata.yaml:

# Set the run mode to NFQUEUE
nfq:
  mode: accept
  repeat-mark: 1
  repeat-mask: 1
  route-queue: 2

# Ensure the default action for drop-capable rules is enabled

Run Suricata explicitly in NFQUEUE IPS mode:

sudo suricata -c /etc/suricata/suricata.yaml -q 0

The -q 0 flag tells Suricata to read from NFQUEUE number 0, matching the iptables rule configured above.

Step 5: Update and Enable Rulesets

Suricata’s detection power comes from its rules. Use suricata-update (bundled with modern installs) to fetch community rulesets:

sudo suricata-update
sudo suricata-update list-sources
sudo suricata-update enable-source et/open   # Emerging Threats Open ruleset
sudo suricata-update

This populates /var/lib/suricata/rules/suricata.rules, referenced by the main config.

Step 6: Configure Rule Actions for Blocking

By default, many community rules are written with an alert action (detection only). For true IPS behavior, specific rules need a drop action. You can create a local rule file for custom, actively-blocking rules:

sudo nano /etc/suricata/rules/local.rules

# Example: block a known malicious IP outright
drop ip 203.0.113.50 any -> any any (msg:"Blocked known malicious IP"; sid:1000001; rev:1;)

# Example: drop traffic matching a brute-force SSH pattern
drop tcp any any -> $HOME_NET 22 (msg:"Possible SSH brute force"; flow:to_server; threshold: type threshold, track by_src, count 5, seconds 60; sid:1000002; rev:1;)

Reference this file in suricata.yaml under rule-files:, then restart Suricata.

Step 7: Test the Configuration

Use a controlled, safe test signature to confirm blocking works — Suricata ships with a built-in test string for this purpose:

curl http://testmynids.org/uid/index.html

This should trigger a known Emerging Threats test rule and, if configured with a drop action, actually be blocked. Check logs:

sudo tail -f /var/log/suricata/fast.log
sudo tail -f /var/log/suricata/eve.json

Step 8: Run Suricata as a Persistent Service

sudo systemctl enable suricata
sudo systemctl start suricata
sudo systemctl status suricata

Edit /etc/default/suricata to ensure it launches with the correct NFQUEUE flags on boot (RUN_MODE=nfqueue in some distro packaging, or a custom systemd override otherwise).

Integrating with a SIEM

For real operational value, IPS logs (eve.json, in structured JSON format) should feed into a centralized SIEM like Elastic Stack (via Filebeat) or Splunk, enabling correlation, dashboards, and alerting beyond local log files.

# Example Filebeat module for Suricata
sudo filebeat modules enable suricata
sudo filebeat setup
sudo systemctl restart filebeat

Tuning: Avoiding False Positives

An IPS that blocks legitimate traffic is often worse than no IPS at all — it creates outages and erodes trust in the system. Best practices for tuning:

• Start new rulesets in alert-only mode, monitor for false positives over 1-2 weeks, then selectively convert well-validated rules to drop.
• Use suppression lists for known-good traffic that triggers benign matches.
• Apply thresholding (as shown in the SSH example above) to avoid over-triggering on legitimate bursty traffic.
• Regularly review and prune outdated or noisy rules.

Common Mistakes

• Deploying broad drop rules across an entire ruleset immediately, causing unexpected outages.
• Forgetting net.ipv4.ip_forward=1, breaking routing entirely on a gateway deployment.
• Not testing NFQUEUE failure behavior — if Suricata crashes, does traffic fail open (pass through) or fail closed (blocked entirely)? This should be a deliberate decision (iptables -j NFQUEUE --queue-bypass allows fail-open behavior).
• Neglecting to update rulesets regularly, leaving the IPS blind to newly disclosed threats.
• Running IPS inline without adequate hardware sizing, introducing unacceptable latency on high-throughput links.

Frequently Asked Questions

Will an IPS slow down my network? Inline inspection introduces some latency, but on properly sized hardware with multi-threaded engines like Suricata, the impact is typically negligible for most enterprise workloads. Under-provisioned deployments can become a bottleneck.

Can I run Suricata in both IDS and IPS mode simultaneously on different interfaces? Yes — many deployments run IDS (AF_PACKET, passive) on some interfaces for visibility while running IPS (NFQUEUE, inline) at a specific chokepoint, like the network perimeter.

What happens if Suricata crashes while in inline mode? By default, without --queue-bypass, traffic through the NFQUEUE rule stalls (fail-closed). This is a deliberate resilience/security tradeoff that should be explicitly decided based on your risk tolerance.

Is Suricata free to use in production? Yes, Suricata is open-source (GPLv2) and free for commercial and production use, maintained by the Open Information Security Foundation (OISF).

Summary and Recommendations

Building a working IPS on Linux is entirely achievable with open-source tooling like Suricata, but doing it well requires understanding inline deployment architecture, careful rule tuning to avoid false positives, and integration with centralized logging for real operational value. Start in alert-only mode, tune deliberately, then move to active blocking.

Further reading:

• Suricata Official Documentation
• Emerging Threats Ruleset
• NIST SP 800-94 — Guide to Intrusion Detection and Prevention Systems