Storage devices don’t just need capacity and speed — they need a way to talk to the rest of the system. That’s the job of storage interfaces and protocols: the physical connections and command sets that let a CPU, controller, or network fetch and write data to a drive. This article works through the major storage interfaces and protocols, from local direct-attach connections to network-based storage protocols.
Local Storage Interfaces
SATA (Serial ATA)
The long-standing standard for consumer HDDs and SSDs, using the AHCI (Advanced Host Controller Interface) command protocol.
| Version | Max Speed |
|---|---|
| SATA I | 1.5 Gbps (~150 MB/s) |
| SATA II | 3 Gbps (~300 MB/s) |
| SATA III | 6 Gbps (~600 MB/s) |
SAS (Serial Attached SCSI)
Used predominantly in enterprise servers and storage arrays. SAS drives are typically more reliable, support dual-porting (for redundant controller paths), and use the mature SCSI command set.
| Feature | SATA | SAS |
|---|---|---|
| Command set | ATA | SCSI |
| Dual-port support | No | Yes |
| Max speed (12G SAS) | 6 Gbps | 12 Gbps |
| Typical use | Consumer, entry servers | Enterprise servers, storage arrays |
| Cabling | Point-to-point | Supports expanders for many drives per controller |
SAS and SATA drives can often physically connect to the same SAS backplane, but SATA controllers cannot accept SAS drives — SAS is backward-compatible with SATA, not the reverse.
NVMe (Non-Volatile Memory Express)
Covered in depth in the companion solid-state storage article — NVMe runs the storage protocol directly over PCIe, purpose-built for flash’s low-latency, high-parallelism characteristics, replacing the older AHCI approach designed around spinning disks.
SCSI (Historical Foundation)
The original SCSI (Small Computer System Interface) parallel bus predates SAS, and its command set lives on conceptually inside SAS, Fibre Channel, and iSCSI today — all three transport the same underlying SCSI command architecture over different physical/network layers.
Network Storage Protocols
iSCSI (Internet SCSI)
Encapsulates SCSI commands inside TCP/IP, letting servers access block storage over an ordinary Ethernet network rather than requiring dedicated Fibre Channel hardware.
graph LR
Host[Server: iSCSI Initiator] -->|TCP/IP over Ethernet| Target[Storage Array: iSCSI Target]
# Linux iSCSI initiator setup
sudo apt install open-iscsi
sudo iscsiadm -m discovery -t sendtargets -p 10.0.0.50:3260
sudo iscsiadm -m node --login
lsblk
| Feature | Detail |
|---|---|
| Transport | TCP/IP over standard Ethernet |
| Hardware needed | Standard NICs (or dedicated iSCSI HBAs for offload) |
| Typical speed | 1/10/25/100 GbE |
| Cost | Lower than FC — reuses existing Ethernet infrastructure |
Fibre Channel (FC)
A dedicated, purpose-built storage network technology, covered in depth in the companion Fibre Channel troubleshooting article. Known for very low latency and lossless, deterministic delivery, at the cost of requiring dedicated switches, HBAs, and cabling.
FCoE (Fibre Channel over Ethernet)
Encapsulates native FC frames inside Ethernet frames, letting FC traffic run over a converged Ethernet network (requiring DCB — Data Center Bridging extensions for lossless delivery), rather than a fully separate FC fabric.
| Feature | Fibre Channel | iSCSI | FCoE |
|---|---|---|---|
| Transport | Dedicated FC fabric | Standard TCP/IP | Ethernet with DCB |
| Latency | Lowest | Higher (TCP/IP overhead) | Low, close to native FC |
| Hardware | FC HBAs, FC switches | Standard NICs | CNAs (Converged Network Adapters), DCB switches |
| Complexity | High (separate skill set) | Lower (reuses IP knowledge) | Moderate |
| Typical adoption today | Still common in large enterprises | Very common, cost-effective | Declining in favor of native iSCSI/NVMe-oF |
NFS (Network File System)
A file-level protocol (not block-level) allowing Linux/Unix systems to mount remote directories as if they were local.
sudo mount -t nfs 10.0.0.60:/export/data /mnt/data
SMB/CIFS (Server Message Block)
The Windows-native file-sharing protocol, also widely supported on Linux (via Samba) and macOS.
sudo mount -t cifs //10.0.0.60/share /mnt/share -o username=user,password=pass
NVMe-oF (NVMe over Fabrics)
Extends NVMe’s low-latency, high-queue-depth model across a network — using RDMA (RoCE, iWARP), Fibre Channel, or TCP as the transport — bringing near-local NVMe performance to networked storage.
graph LR
Host[Server: NVMe-oF Initiator] -->|RDMA / FC / TCP| Target[NVMe-oF Storage Target]
| NVMe-oF Transport | Description |
|---|---|
| NVMe/TCP | Runs over standard TCP/IP networks, easiest to deploy |
| NVMe/RoCE | RDMA over Converged Ethernet, very low latency, needs DCB |
| NVMe/FC | Runs NVMe commands over existing Fibre Channel fabrics |
Protocol Comparison Summary
| Protocol | Access Type | Transport | Typical Latency | Common Use Case |
|---|---|---|---|---|
| SATA/SAS | Block (local) | Direct-attach | Very low | Local server/workstation storage |
| NVMe | Block (local) | PCIe | Lowest | High-performance local flash |
| iSCSI | Block (network) | TCP/IP | Moderate | Cost-effective SAN over Ethernet |
| Fibre Channel | Block (network) | Dedicated FC fabric | Very low | High-performance enterprise SAN |
| FCoE | Block (network) | Converged Ethernet | Low | Converged FC/Ethernet infrastructure |
| NFS | File (network) | TCP/IP | Moderate | Linux/Unix file sharing |
| SMB/CIFS | File (network) | TCP/IP | Moderate | Windows file sharing |
| NVMe-oF | Block (network) | TCP/RDMA/FC | Very low | High-performance networked flash |
Practical Examples
Checking Local Storage Interface Type on Linux
lsblk -d -o NAME,TRAN,SIZE,MODEL
Example output:
NAME TRAN SIZE MODEL
sda sas 1.8T HGST-SAS-Drive
nvme0n1 nvme 1.0T Samsung SSD 980 PRO
Cisco: Verifying iSCSI/FCoE Configuration Context
Router# show interface fc1/1
Router# show fcoe database
Python: Enumerate Block Devices and Their Transport
import subprocess
import json
result = subprocess.run(
["lsblk", "-J", "-o", "NAME,TRAN,SIZE,MODEL"],
capture_output=True, text=True
)
data = json.loads(result.stdout)
for device in data["blockdevices"]:
print(f"{device['name']}: transport={device.get('tran')}, size={device['size']}")
Best Practices
- Choose iSCSI for cost-effective SAN deployments reusing existing Ethernet skills and infrastructure; choose Fibre Channel when the lowest, most deterministic latency is required and budget/skills support it.
- Use dedicated VLANs (and ideally dedicated switches/NICs) for iSCSI traffic to isolate it from general LAN traffic and avoid contention.
- For NVMe-oF, prefer RoCE or native FC transport for latency-critical workloads; NVMe/TCP is the easiest entry point when ultra-low latency isn’t the top priority.
- Always use dual-pathing/multipathing regardless of protocol — a single network or fabric path is a single point of failure.
- Keep file-level (NFS/SMB) and block-level (iSCSI/FC) storage separated conceptually — they solve different problems and shouldn’t be conflated in architecture planning.
Troubleshooting
| Symptom | Likely Cause | Fix |
|---|---|---|
| iSCSI target not discovered | Firewall blocking port 3260, wrong IP | Verify iscsiadm discovery, check firewall rules |
| SAS drive not recognized | Wrong controller (SATA-only), cabling issue | Verify HBA/controller supports SAS |
| NVMe-oF high latency | Wrong transport chosen for workload | Consider RoCE/FC instead of TCP for latency-sensitive needs |
| NFS mount slow | Network congestion, server-side load | Check network path and NFS server performance |
| FCoE traffic drops | DCB not properly configured | Verify DCB/PFC (Priority Flow Control) settings on switches |
Further Reading
- T10 Technical Committee — SCSI Standards
- RFC 3720 — iSCSI
- NVM Express NVMe-oF Specification
- SNIA Networking Storage Resources
- Red Hat: Configuring an iSCSI Target
Conclusion
Every storage protocol represents a different tradeoff between performance, cost, and complexity — from local SATA/SAS/NVMe interfaces to network-based iSCSI, Fibre Channel, FCoE, and the newer NVMe-oF standards. Choosing the right one comes down to understanding your workload’s latency and throughput requirements against your budget and existing infrastructure skill set.
