XDP Deep Dive: eXpress Data Path for High-Performance Packet Processing at the Driver Layer

Modern cloud infrastructure demands packet processing at rates of 100Gbps and beyond — a scale where traditional kernel networking stacks struggle to keep up. XDP (eXpress Data Path), built on top of eBPF technology, provides a programmable, high-performance packet processing framework that operates at the lowest point in the software stack: the network interface card (NIC) driver layer. This article explores the XDP architecture from the ground up, examining its execution model, action types, map integrations, and real-world production deployments that achieve millions of packets per second per core.

1. The Need for a Faster Data Path

The Linux kernel networking stack, while robust and feature-rich, introduces significant latency and overhead for each packet:

• Memory allocation: Every packet requires a sk_buff allocation (including the ~230-byte metadata structure)
• Context switches: Hardware interrupt → softirq → network stack → protocol layers → userspace
• Layer traversal: Each layer (Ethernet → IP → TCP/UDP → socket buffer) adds processing overhead
• Copy operations: Data must traverse kernel-userspace boundaries for applications

At 10Gbps line rate, a system processing minimum-size (64-byte) packets must handle ~14.88 million packets per second (Mpps). Each sk_buff allocation alone can consume hundreds of CPU cycles, making it nearly impossible to reach line rate with standard stack processing on commodity hardware.

2. XDP Architecture: Processing Before the Stack

XDP inserts an eBPF hook at the earliest possible point in the RX path — directly inside the NIC driver's receive function, before the kernel allocates an sk_buff. This enables zero-copy, zero-allocation packet processing capable of handling 25+ Mpps per core.

// --- XDP execution point in the RX path ---
// NIC Driver -> [XDP Hook (eBPF)] -> sk_buff allocation -> netif_receive_skb() -> Protocol Stack
//
// XDP sees raw packet data in a BPF context:
struct xdp_md {
    __u32 data;               // Start of packet data
    __u32 data_end;           // End of packet data
    __u32 data_meta;          // Metadata pointer
    __u32 rx_queue_index;     // Incoming RX queue index
    __u32 ingress_ifindex;    // Interface index
};

2.1 Three Attachment Modes

Mode	Description	Performance
Native XDP	Driver-level hook (ndo_xdp_xmit). Requires driver support (i400e, mlx5, ixgbe, etc.)	Highest — runs before sk_buff allocation
Generic XDP	Fallback in core network stack. Allocating sk_buff first, then running XDP.	Moderate — useful for unsupported NICs
Offloaded XDP	Program loaded directly into NIC firmware (Netronome/Corstone SmartNIC).	Absolute maximum — BPF runs on NIC hardware

3. XDP Actions: Directing Packet Fate

Every XDP program must return one of five action codes that determines the kernel's handling of the packet:

3.1 XDP_ABORTED (0)

Indicates a program error. The packet is dropped, and a trace event is recorded for debugging. This should never happen in production code — it indicates verifier-detected issues or runtime exceptions.

3.2 XDP_PASS

Pass the packet to the normal kernel networking stack for standard processing. Use this when XDP cannot handle the packet (e.g., non-IPv4 traffic, packets that need full stack inspection) or as a learning mode action.

3.3 XDP_DROP

Drop the packet immediately in the driver. This is the fastest possible dispose action — ideal for DDoS mitigation where unwanted packets must be discarded as early as possible.

3.4 XDP_TX

Transmit the (possibly modified) packet back out through the same NIC it arrived on. Useful for Layer 2/L3 rewriting (MAC address translation), simple load balancers that rewrite destination and bounce back, and reflection testing.

3.5 XDP_REDIRECT

The most powerful action — redirect the packet to another network interface (cross-NIC forwarding), a different CPU core for further processing (via XSKMAP to AF_XDP sockets), or a remote CPU via cpumap for userspace consumption. Enables building complex forwarding planes without leaving the driver context.

4. BPF Maps for XDP: State and Communication

XDP programs leverage BPF maps to maintain state and communicate with userspace. The most critical map types for XDP are:

4.1 DEVMAP (BPF_MAP_TYPE_DEVMAP)

Alongside bpf_redirect_map(), DEVMAP enables XDP programs to forward packets to other NIC interfaces. The map keys are interface indices, and values are bpf_devmap_val structures containing the target interface and CPU information.

// --- DEVMAP definition and redirect usage ---
struct {
    __uint(type, BPF_MAP_TYPE_DEVMAP);
    __uint(max_entries, 64);
    __type(key, __u32);
    __type(value, struct bpf_devmap_val);
} devmap SEC(".maps");

// In the XDP program:
SEC("xdp")
int xdp_l2fwd(struct xdp_md *ctx) {
    __u32 key = DST_IFINDEX;
    return bpf_redirect_map(&devmap, key, XDP_PASS);
}

4.2 CPUMAP (BPF_MAP_TYPE_CPUMAP)

For cases where packets need to be processed by specific CPU cores (e.g., for RSS queue steering), CPUMAP redirects packets to per-CPU socket queues. This is the foundation for XDP-based userspace packet processing pipelines.

4.3 XSKMAP (BPF_MAP_TYPE_XSKMAP)

AF_XDP sockets combined with XSKMAP enable zero-copy packet delivery from XDP directly to userspace applications, completely bypassing the kernel networking stack. This is how high-performance DPDK-like processing is achieved while retaining the safety and programmability of the kernel.

4.4 Array and Hash Maps

Standard BPF array and hash maps are ubiquitous for configuration data (allowlists, blocklists, rate limits), statistics (packet counters per flow), and routing tables (longest-prefix-match lookups).

5. Building a Production DDoS Mitigation System

XDP's primary production success story is in high-performance DDoS mitigation. A typical implementation uses hash maps to track per-source counters and drops packets from IPs exceeding thresholds.

// --- Simplified DDoS XDP filter ---
#define MAX_RULES 4096
struct {
    __uint(type, BPF_MAP_TYPE_HASH);
    __uint(max_entries, MAX_RULES);
    __type(key, __u32);     // Source IP
    __type(value, __u64);   // Bytes per second counter
} blocklist SEC(".maps");

SEC("xdp")
int xdp_ddos_filter(struct xdp_md *ctx) {
    void *data_end = (void *)(long)ctx->data_end;
    void *data = (void *)(long)ctx->data;
    struct ethhdr *eth = data;
    struct iphdr *ip;

    // Bounds check (required by verifier)
    if ((void *)(eth + 1) > data_end)
        return XDP_PASS;

    if (eth->h_proto != bpf_htons(ETH_P_IP))
        return XDP_PASS;  // Let stack handle non-IPv4

    ip = (void *)(eth + 1);
    if ((void *)(ip + 1) > data_end)
        return XDP_PASS;

    // Check blocklist
    __u32 src_ip = bpf_ntohl(ip->saddr);
    __u64 *blocked = bpf_map_lookup_elem(&blocklist, &src_ip);
    if (blocked && *blocked > THRESHOLD)
        return XDP_DROP;  // Drop in driver layer

    return XDP_PASS;
}

Production results: Cloudflare reported processing 100Gbps of DDoS attack traffic using XDP on a single server, dropping 6M+ malicious packets per second per core with just 10% CPU utilization.

6. AF_XDP: Zero-Copy Userspace Access

AF_XDP is the mechanism that allows userspace applications to receive packets directly from XDP with zero memory copies. The UMEM (User Memory Manager) model works as follows:

1. Allocation: Userspace allocates a contiguous memory region (UMEM), divided into equal-sized frames.
2. Ring buffers: Four rings connect kernel and userspace: Fill ring (userspace gives empty frames to kernel), Completion ring (kernel returns used frames after TX), RX ring (kernel delivers received packets), TX ring (userspace submits frames for transmission).
3. Zero-copy: Packet data is written directly into UMEM by the NIC (via DMA), with both kernel and userspace accessing the same memory.

This architecture achieves remarkable performance: Facebook's Katran L4 load balancer processes 100M+ packets per second using XDP + AF_XDP.

7. XDP vs DPDK vs AF_PACKET

Aspect	XDP + eBPF	DPDK	AF_PACKET
Flexibility	High — reprogrammable at runtime, verifier-safe	Moderate — requires application restart for changes	Low — raw socket capture
Performance	10-25 Mpps/core	25-80 Mpps/core	1-3 Mpps/core
Driver Support	Requires XDP-capable driver (~30 NIC drivers supported)	Requires DPDK-compatible NIC	Works with any NIC
Safety	Kernel verifies program safety — no crashes	Unsafe — bug causes kernel panic or data corruption	Safe but slow
Memory Model	Shared kernel/userspace (AF_XDP)	Hugepages + userspace drivers	sk_buff copy to userspace
Deployment Complexity	Low — works with existing kernel	High — requires DPDK setup, hugepages, NUMA tuning	Minimal
Update in Production	Hot-reload eBPF program without downtime	Requires service restart and reinitialization	N/A

8. Advanced Techniques

8.1 Packet Header Rewrite

XDP can modify packet headers in-place using safe helper functions. The verifier ensures only safe modifications are allowed:

// --- NAT-like rewrite using bpf_xdp_adjust_head ---
SEC("xdp")
int xdp_nat(struct xdp_md *ctx) {
    void *data_end = (void *)(long)ctx->data_end;
    void *data = (void *)(long)ctx->data;

    struct ethhdr *eth = data;
    struct iphdr *ip = (void *)(eth + 1);

    // Shrink packet headroom for encapsulation
    if (bpf_xdp_adjust_head(ctx, -(int)sizeof(struct gre_hdr)) < 0)
        return XDP_DROP;

    // Rewrite GRE header, then update IPs
    // ... (bounds-checked operations)

    return XDP_TX;  // Send modified packet back out
}

8.2 Tail Calls (bpf_tail_call)

Large XDP programs can be decomposed into smaller chained programs using tail calls. Each tail call replaces the current program's context atomically, with no function call overhead. This enables modular XDP programs where each stage handles a specific protocol.

8.3 Batch Processing

The latest XDP APIs support batch processing, reducing per-packet overhead by processing multiple packets in a single system call. This is critical for reaching maximum throughput on modern NICs.

9. Practical Deployment Checklist

Before deploying XDP in production, ensure:

• Driver support: Verify NIC driver supports native XDP (check via ip link set dev <iface> xdp)
• Ring buffer sizing: Set net.core.netdev_budget=60000 (default 300 may limit XDP performance)
• NUMA awareness: XDP program runs on the CPU handling the NIC's RX queue. Pin IRQs to local NUMA nodes.
• Hugepages (AF_XDP): AF_XDP requires hugepages for UMEM allocation. Configure /proc/sys/vm/nr_hugepages.
• Verifier compliance: All XDP programs must pass the BPF verifier. Strict bounds checks on every pointer dereference are mandatory.
• Fallback strategy: Always have a generic XDP or iptables fallback for production safety.

10. Production Case Studies

Cloudflare ddos mitigation: StackArmor system uses XDP to drop 6M+ packets/sec per core during volumetric attacks. The eBPF program maintains per-source-IP counters in hash maps, with automatic threshold detection and blocking.

Facebook Katran: Layer 4 load balancer handles 100M+ pps using XDP + AF_XDP. Uses consistent hashing in eBPF maps to distribute connections to backend servers without connection state.

LinkedIn Traffic Shaping: XDP-based traffic policers use token bucket algorithms implemented in BPF maps for per-flow rate limiting at 100Gbps.

11. Conclusion

XDP represents a paradigm shift in how we approach high-performance networking in the Linux kernel. By extending the kernel's programmable capabilities to the NIC driver layer, it achieves DPDK-class performance with kernel-class safety, flexibility, and operational simplicity. Whether you're building DDoS mitigation systems, layer 4 load balancers, network observability platforms, or programmable firewalls, XDP provides a robust, production-proven foundation that continues to evolve with each kernel release.

The synergy of eBPF programs, BPF maps, AF_XDP zero-copy delivery, and kernel-bypass techniques ensures that XDP will remain at the center of cloud-native networking innovation for years to come.