Basics of Converged Storage Network Technologies

For most of my career, storage networks and general data networks were two completely separate worlds — different switches, different cabling, different teams, different vendors. Converged networking was the technology shift that started collapsing that wall, and understanding it properly matters whether you are designing a new data center or just trying to make sense of what “FCoE” or “unified fabric” actually means on a spec sheet. This article covers the core converged storage networking technologies from the ground up.

What Does “Converged” Mean in Storage Networking?

Convergence, in this context, means running storage traffic (which traditionally required its own dedicated, lossless Fibre Channel fabric) and general IP/Ethernet data traffic over the same physical network infrastructure — the same switches, the same cabling — instead of maintaining two entirely separate physical networks.

The motivation is straightforward: reducing the number of physical switches, cables, HBAs/NICs, and the associated cost, power, cooling, and management overhead of running parallel networks.

Why This Was Hard: The Lossless Requirement

Traditional Ethernet is a “best effort” network — it is designed to tolerate and recover from occasional packet loss via TCP retransmission. Fibre Channel, by contrast, was designed as a lossless fabric — it uses buffer-to-buffer credits to prevent a sender from ever transmitting faster than the receiver can accept, so frames are essentially never dropped due to congestion under normal operation.

This difference matters enormously for storage traffic. Block storage protocols are latency-sensitive and historically assumed a lossless transport; running FC-like storage traffic over standard “lossy” Ethernet without modification would introduce unacceptable retransmission-driven latency and jitter. Solving this problem is what the entire suite of Data Center Bridging (DCB) enhancements to Ethernet was built to address.

Data Center Bridging (DCB): The Foundation of Convergence

DCB is a set of IEEE Ethernet extensions that make standard Ethernet capable of supporting lossless, storage-grade traffic alongside regular IP traffic:

1. Priority Flow Control (PFC) — IEEE 802.1Qbb

Instead of pausing an entire link when congested (as classic Ethernet PAUSE frames do), PFC allows pausing individual traffic classes/priorities independently. This means storage traffic (assigned to a specific priority) can be made lossless while regular IP traffic on the same physical link continues operating normally, even under congestion.

2. Enhanced Transmission Selection (ETS) — IEEE 802.1Qaz

ETS allows bandwidth to be allocated by traffic class, guaranteeing a minimum bandwidth percentage to storage traffic even when the link is congested with other traffic, while still allowing classes to burst above their guarantee when bandwidth is available.

Example ETS bandwidth allocation on a 10GbE converged link:
  FCoE storage traffic:  50% guaranteed
  Regular LAN traffic:   30% guaranteed
  Management traffic:    20% guaranteed

3. Data Center Bridging Exchange (DCBX)

A discovery and negotiation protocol (built on LLDP) that lets switches and endpoints automatically exchange and agree on PFC, ETS, and other DCB configuration parameters, so they do not have to be manually configured identically on every device.

FCoE (Fibre Channel over Ethernet)

FCoE is the technology that directly encapsulates native Fibre Channel frames inside Ethernet frames, allowing FC traffic to travel across a DCB-enabled Ethernet network while preserving the FC protocol itself (zoning, WWPNs, FLOGI, etc.) unchanged above the encapsulation layer.

FCoE Frame Structure (simplified):
[ Ethernet Header ] [ FCoE Header ] [ FC Frame (unchanged) ] [ FCS ]

Key architectural components:

• CNA (Converged Network Adapter) — a host adapter that presents itself to the OS as both a NIC and an FC HBA simultaneously, over a single physical port and cable.
• FCF (Fibre Channel Forwarder) — a switch function that bridges FCoE traffic into a native FC fabric (or handles it natively within an FCoE-only network segment).
• VN_Port / VF_Port / VE_Port — FCoE equivalents of standard FC port types (N_Port, F_Port, E_Port), operating over the Ethernet/DCB transport.

Important limitation: FCoE requires a lossless, DCB-capable network end to end — it is not designed to be routed across standard, non-DCB IP networks. This restricted FCoE’s practical deployment largely to the access/aggregation layer within a single data center, and adoption plateaued as iSCSI and NVMe-oF alternatives matured.

iSCSI: The Original IP-Based Block Storage Protocol

Unlike FCoE, iSCSI encapsulates SCSI commands inside standard TCP/IP packets, meaning it runs over completely ordinary Ethernet/IP networks without requiring DCB or any special lossless configuration (though DCB/jumbo frames can still improve performance).

iSCSI Protocol Stack:
[ SCSI Commands ] -> [ iSCSI Layer ] -> [ TCP ] -> [ IP ] -> [ Ethernet ]

Because iSCSI rides on ordinary routable IP, it can traverse standard routers and even WANs (with appropriate latency/performance considerations), which FCoE fundamentally cannot do without gateway translation back to native FC.

NVMe-oF (NVMe over Fabrics): The Modern Convergence Layer

NVMe-oF extends the low-latency, highly parallel NVMe protocol (originally designed for local PCIe-attached SSDs) across a network fabric, and it comes in several transport flavors relevant to convergence:

• NVMe/FC — NVMe commands transported over traditional Fibre Channel fabrics, coexisting with legacy SCSI-based FCP traffic on the same physical fabric.
• NVMe/TCP — NVMe commands transported over standard TCP/IP, requiring no special lossless network configuration at all, making it the most “converged-friendly” and easiest to deploy at scale of the NVMe-oF transports.
• NVMe/RoCE (RDMA over Converged Ethernet) — NVMe commands transported using RDMA (Remote Direct Memory Access) over a DCB-enabled lossless Ethernet network, delivering extremely low latency by allowing direct memory-to-memory transfers that bypass much of the traditional OS networking stack.

Comparing Converged Technologies

Unified Fabric Architecture Example

A converged data center network design typically looks like this at the access layer:

Server (CNA or standard NIC)
     |
     v
Top-of-Rack Converged Switch (DCB-enabled)
     |-- Storage traffic (FCoE/NVMe-RoCE) --> FCF/Native FC Fabric --> Storage Array
     |-- LAN traffic (standard Ethernet)   --> Core/Aggregation Switch --> Rest of network

A single pair of redundant converged switches and a single pair of CNAs per server can replace what would traditionally have required separate Ethernet NICs, FC HBAs, Ethernet switches, and FC switches — a substantial reduction in cabling, switch ports, and power/cooling footprint.

Advantages of Converged Storage Networking

• Reduced cabling, adapters, and switch port count, lowering both capital and operational cost
• Simplified physical data center topology and reduced power/cooling footprint
• Ability to dynamically allocate bandwidth between storage and LAN traffic via ETS rather than statically over-provisioning two separate networks
• Easier for organizations to consolidate storage and network administration skill sets over time

Disadvantages and Challenges

• Added configuration complexity (DCB/PFC/ETS must be correctly and consistently configured across every hop)
• Convergence increases the “blast radius” of a single network failure, since both storage and LAN traffic now share the same physical infrastructure — proper redundancy design becomes even more critical
• FCoE specifically saw declining vendor investment and market adoption as iSCSI and NVMe/TCP matured, since those alternatives achieve much of the same cost/cabling benefit without requiring DCB end-to-end
• Requires cross-team expertise — network engineers need to understand storage protocol requirements, and storage engineers need to understand Ethernet/DCB configuration

Real-World Enterprise Trend

In practice, many organizations that adopted FCoE in the early-to-mid 2010s have since migrated toward either continuing with native FC for their highest-performance tier alongside iSCSI/NFS for general workloads, or adopting NVMe/TCP for new high-performance deployments — largely because NVMe/TCP delivers strong performance without requiring the DCB configuration discipline that FCoE demands end-to-end. Native FC fabrics, meanwhile, remain extremely common in large enterprises precisely because they avoid the DCB dependency entirely, since lossless behavior is inherent to the FC protocol itself.

Common Mistakes

1. Deploying FCoE without consistent DCB configuration on every switch hop, leading to unexpected packet loss and severe storage performance problems that are difficult to diagnose because they masquerade as “flaky storage.”
2. Underestimating the shared-fate risk of full convergence — a converged switch failure now potentially impacts both storage and general network connectivity simultaneously, so redundant, diverse-path design becomes non-negotiable.
3. Assuming iSCSI performance is inherently inferior to FC/FCoE — on modern 10/25/100GbE networks with jumbo frames and proper TCP tuning, iSCSI and especially NVMe/TCP can be highly competitive with traditional FC for many workloads.
4. Treating NVMe/RoCE as a drop-in replacement without addressing lossless network requirements, since RoCE still fundamentally depends on the same PFC/DCB discipline that FCoE required.

FAQs

Q: Is FCoE still relevant today? It is still deployed in existing environments, but new deployments increasingly favor iSCSI, NVMe/TCP, or continuing with native FC, since those approaches avoid FCoE’s strict end-to-end DCB requirement.

Q: Does converged networking mean I no longer need a SAN? No — convergence changes the physical transport layer, not the fundamental need for shared, protected, centrally managed storage. A converged network can still connect hosts to a traditional SAN array; it just does so over shared physical infrastructure rather than dedicated FC cabling.

Q: What is the easiest converged/IP-based protocol to deploy today for high performance? NVMe/TCP is generally considered the easiest modern option, since it delivers strong NVMe-level performance over completely standard, unmodified TCP/IP networks with no DCB configuration requirement.

Planning Checklist for a Converged Network Deployment

Before rolling out any converged design, I always work through this list with the network team:

• Confirm every switch, adapter, and driver in the path actually supports the DCB features required (PFC, ETS, DCBX) — not just “supports 10GbE,” since DCB support varies significantly between switch models even within the same vendor line
• Map out which traffic classes need guaranteed bandwidth and agree on ETS percentages before go-live, not after storage performance complaints start
• Plan redundant, physically diverse paths for converged links, since a single converged switch failure now affects both storage and LAN traffic simultaneously
• Validate CNA firmware and driver versions against the array vendor’s interoperability matrix before deployment — mismatched firmware is a very common source of early instability in converged environments
• Decide early whether NVMe/TCP is a better fit than FCoE or NVMe/RoCE for new deployments, given its simpler operational model

Migration Considerations From Legacy FC to Converged/IP Storage

Many organizations do not do a clean rip-and-replace; instead they run a hybrid architecture for years. A typical phased approach:

1. New server refreshes get CNAs or standard NICs capable of both LAN and iSCSI/NVMe-TCP traffic, while legacy servers keep dedicated FC HBAs
2. New storage arrays are deployed with multi-protocol front ends (FC, iSCSI, NVMe/TCP, NVMe/FC all available from the same array), allowing gradual host migration without a forced cutover
3. Legacy FC fabric is maintained for existing latency-sensitive workloads while new workloads default to IP-based protocols
4. Over several hardware refresh cycles, the dedicated FC fabric footprint shrinks as it becomes viable to retire, rather than being replaced in a single risky cutover event

This gradual approach is far more common in real enterprise environments than a single “flag day” migration, largely because storage migrations of any kind carry real risk and most organizations prefer to de-risk them across multiple budget and maintenance cycles.

Summary

Converged storage networking exists to solve a real, practical problem: running lossless, latency-sensitive storage traffic and best-effort IP traffic over the same physical infrastructure instead of two separate, expensive parallel networks. Technologies like FCoE and NVMe/RoCE achieve this by extending Ethernet with lossless capabilities via DCB (PFC, ETS, DCBX), while iSCSI and NVMe/TCP take the alternative path of running storage protocols over completely standard, unmodified IP networks. Understanding the trade-offs between these approaches — deployment complexity, routability, and performance — is essential for designing a modern data center network that meets both storage and general networking needs efficiently.

References

• SNIA — Converged networking and NVMe-oF technical documents, snia.org
• Cisco — Unified Fabric and FCoE design guides, cisco.com
• IEEE — 802.1Qbb (PFC) and 802.1Qaz (ETS) standards, ieee.org
• NVM Express, Inc. — NVMe-oF specification, nvmexpress.org
• Dell EMC — Converged infrastructure and NVMe-oF documentation, dell.com/support
• Broadcom (Brocade/Emulex) — FCoE and CNA technical documentation, broadcom.com