Offensive Security Researcher & Quantum Cryptography Analyst

How to Troubleshoot Firewall Policies on Cisco ASA Firewalls: Step-by-Step Guide

Abdul Wahab Junaid
How to Troubleshoot Firewall Policies on Cisco ASA Firewalls

How to Troubleshoot Firewall Policies on Cisco ASA Firewalls

“Traffic is being blocked and nobody knows why” is one of the most common tickets a network or security engineer will ever get, and Cisco ASA firewalls have a very specific, learnable methodology for tracking it down. This guide walks through ASA policy troubleshooting from the fundamentals of how the ASA evaluates traffic, through the exact CLI commands used to diagnose real problems, to the mistakes that cause the most wasted troubleshooting time.

How the ASA Evaluates Traffic: The Order of Operations

Before troubleshooting anything, it helps to know the actual order the ASA applies checks to a packet, because a block at any one stage looks similar from the outside but requires a completely different fix:

  1. Interface security levels and NAT — the ASA applies NAT rules and determines routing/interface path.
  2. Access Control Lists (ACLs) — interface-applied ACLs permit or deny traffic explicitly.
  3. Stateful inspection / connection table — the ASA tracks connections in a state table; return traffic for an already-permitted flow is allowed without needing a matching ACL entry in the reverse direction.
  4. Application inspection (MPF – Modular Policy Framework) — deep packet inspection for protocols like FTP, SIP, ESMTP, which may rewrite embedded IPs or enforce protocol conformance.
  5. VPN / crypto processing — if traffic matches a VPN policy, IPsec/SSL processing happens here.

Most real-world “traffic is blocked” tickets are actually ACL or NAT problems, but a meaningful minority are inspection/state-table issues that look identical from the user’s perspective, which is exactly why a structured, layer-by-layer approach saves time.

Understanding ASA ACL Fundamentals

Unlike router ACLs, ASA ACLs are typically applied per-interface in the in direction and evaluated top-down, first-match-wins — same fundamental logic as IOS, but ASA syntax and the concept of “security levels” adds a layer worth understanding.

By default (without any ACL applied), the ASA allows traffic from a higher security-level interface to a lower one (e.g., inside → outside) and blocks the reverse. Once an ACL is applied to an interface, that ACL’s explicit rules take over completely for traffic entering that interface — the security-level default no longer applies to permit/deny decisions for that interface, though it still matters for other things like same-security-level traffic and NAT.

Lab Topology

Step 1 — Confirm the Problem with packet-tracer

This is the single most valuable ASA troubleshooting command that exists. It simulates a packet through the entire policy stack and reports exactly where it would be dropped, if anywhere.

ASAFW1# packet-tracer input inside tcp 10.10.10.50 12345 192.168.50.20 443

Expected output for a working flow (trimmed):

Phase: 1
Type: ROUTE-LOOKUP
Subtype: Resolve Egress Interface
Result: ALLOW
...
Phase: 2
Type: ACCESS-LIST
Subtype:
Result: ALLOW
Config:
access-group INSIDE_IN in interface inside
access-list INSIDE_IN extended permit tcp 10.10.10.0 255.255.255.0 192.168.50.0 255.255.255.0 eq 443
...
Result:
input-interface: inside
input-status: up
input-line-status: up
output-interface: dmz
output-status: up
output-line-status: up
Action: allow

If instead the traffic is blocked, the output shows exactly which phase and rule caused the drop:

Phase: 2
Type: ACCESS-LIST
Subtype:
Result: DROP
Config:
access-group INSIDE_IN in interface inside
access-list INSIDE_IN extended deny ip any any
...
Action: drop
Drop-reason: (acl-drop) Flow is denied by configured rule

This single command tells you immediately whether the problem is an ACL, NAT, routing, or inspection issue — always run this first before digging through raw configuration.

Step 2 — Review the Relevant ACL

ASAFW1# show access-list INSIDE_IN

Expected output (trimmed):

access-list INSIDE_IN; 4 elements
access-list INSIDE_IN line 1 extended permit tcp 10.10.10.0 255.255.255.0 192.168.50.0 255.255.255.0 eq https (hitcnt=142) 0x8a3f21c4
access-list INSIDE_IN line 2 extended permit icmp any any (hitcnt=0) 0x9b4e12a1
access-list INSIDE_IN line 3 extended deny ip any any log (hitcnt=88) 0x1c2d3e4f

The hitcnt field is invaluable — it shows exactly how many times each rule has matched traffic. A rule you expect to be matching with hitcnt=0 tells you traffic isn’t reaching that rule at all (often because an earlier, broader rule is matching first), while a growing hitcnt on a deny rule confirms real traffic is being blocked there.

Step 3 — Check NAT Configuration

NAT misconfiguration is the second most common cause of “traffic blocked” tickets that actually aren’t ACL problems at all.

ASAFW1# show nat detail

Expected output shows each NAT rule and, critically, translation hit counts:

Manual NAT Policies (Section 1)
1 (inside) to (dmz) source static INSIDE-NET INSIDE-NET destination static DMZ-NET DMZ-NET
    translate_hits = 142, untranslate_hits = 140

If translate_hits stays at zero while you know traffic is being generated, the NAT rule isn’t matching — check source/destination object definitions for typos or overly narrow scoping.

Check active translations for a specific host:

ASAFW1# show xlate | include 10.10.10.50

Step 4 — Check the Connection Table

ASAFW1# show conn address 10.10.10.50

Expected output for an established, permitted connection:

TCP dmz  192.168.50.20:443 inside  10.10.10.50:52011, idle 0:00:02, bytes 4832, flags UIO

The flags field decodes connection state — U (up), I (inbound data), O (outbound data) indicate a healthy bidirectional flow. If no connection entry exists at all despite the client attempting to connect, the traffic is being dropped before it ever reaches the state table (almost always an ACL or routing issue caught earlier).

Step 5 — Check Application Inspection Policy (MPF)

Some protocols (FTP, SIP, ESMTP, DNS) are affected by default inspection policies that can rewrite payloads or enforce protocol conformance, occasionally causing unexpected drops for otherwise-permitted traffic — particularly for protocols carrying embedded IP addresses (like FTP’s PORT command) that need translation-aware inspection to work through NAT.

ASAFW1# show run policy-map
ASAFW1# show service-policy

Expected show service-policy output includes per-inspection-engine packet counts, which can reveal if a specific inspection engine (e.g., inspect ftp) is dropping packets that a plain ACL check wouldn’t have caught:

Class-map: inspection_default
  Inspect: ftp, packet 245, drop 12, reset-drop 0

Step 6 — Check Syslog for Explicit Deny Messages

The ASA logs every ACL deny (if log is set on the rule, which is default behavior for the implicit deny) with a specific, well-documented message ID:

ASAFW1# show logging | include 106023

Expected output:

%ASA-4-106023: Deny tcp src inside:10.10.10.50/52011 dst dmz:192.168.50.20/443 by access-group "INSIDE_IN" [0x1c2d3e4f, 0x0]

Message %ASA-4-106023 (or %ASA-6-106100 for logged permit/deny with the log keyword) is the canonical “here’s exactly what got blocked and by which ACL” log line — searching for this pattern during an active incident is often faster than manually reviewing ACLs.

Real-World Enterprise Scenario: New Application Breaks After a “Simple” Firewall Change

A common scenario: an application team reports their new microservice, added last week, can’t reach an internal API on a non-standard port (TCP 8443). Here’s the methodical path:

  1. Run packet-tracer with the exact source, destination, and port reported. This immediately confirms whether it’s even an ASA issue versus a problem elsewhere (app misconfiguration, DNS, routing on another device).
  2. If packet-tracer shows an ACL drop, check show access-list <name> for the relevant rule — often the issue is that TCP 8443 was never added to an existing object-group used by the rule.
  3. If NAT is in play (e.g., traffic crossing from a public-facing DMZ), verify show nat detail hit counts to confirm the correct translation is actually being applied to this specific new source/destination pair.
  4. If ACL and NAT both look correct but the connection still fails, check show conn for a half-open or reset connection, and check show service-policy for an inspection engine that might be interfering with the specific protocol/port.
  5. Once identified, make the object-group/ACL update, then immediately re-run the same packet-tracer command to confirm the fix before telling the application team it’s resolved — never assume a config change worked without re-testing.

Optimization and Best Practices

Common Configuration Mistakes

Troubleshooting Checklist Summary

  1. packet-tracer — pinpoint exactly which phase drops the traffic.
  2. show access-list <name> — confirm rule order and hit counts.
  3. show nat detail and show xlate — confirm translation is occurring as expected.
  4. show conn — confirm connection state and flags.
  5. show service-policy — check for inspection-engine interference.
  6. show logging | include 106023 (or relevant syslog IDs) — find explicit deny events in real time.

FAQs

What’s the fastest way to confirm if the ASA is even the problem? packet-tracer with the exact 5-tuple (protocol, source IP/port, destination IP/port) reported by the user — if it shows Action: allow end-to-end, the ASA isn’t the blocking point and troubleshooting should move to routing, the destination host’s own firewall, or the application itself.

Does packet-tracer actually send real traffic? No — it’s a simulation using the current configuration and routing/NAT tables; it doesn’t generate real packets on the wire, which makes it safe to run in production at any time without side effects.

Why does a connection work initially but then get dropped mid-session? This is often a connection or idle timeout being hit (show run timeout), an inspection engine resetting a non-conformant long-lived connection, or in cluster/failover setups, a state sync issue after a failover event — check show conn timestamps and show failover state if applicable.

Can two ACL rules with overlapping scope cause unpredictable behavior? No — ASA ACL evaluation is deterministic, always first-match top-down, so overlapping rules behave predictably (the first matching rule wins) even if it’s not the outcome you intended. The fix is always reordering, not treating it as random behavior.

Summary

Troubleshooting ASA firewall policy issues comes down to walking the same order the ASA itself uses to process a packet: routing and NAT, then ACLs, then the stateful connection table, then application inspection. packet-tracer is the tool that collapses all of that into a single command showing exactly where and why traffic is being dropped, and it should be the first thing run on almost any “traffic is blocked” ticket. From there, show access-list hit counts, show nat detail translation counts, show conn state, and syslog message 106023 round out a methodology that turns what feels like a mystery into a five-minute diagnosis in the vast majority of cases.

References

Exit mobile version