AF_XDP Zero-Copy UMEM Fill Ring Watermark and Proactive Replenishment Invariant

--- description: AF_XDP Zero-Copy UMEM Fill Ring Watermark and Proactive Replenishment Invariant - Prevents catastrophic packet drops during line-rate network microbursts by coupling hardware RX descriptors directly to userspace ring allocations. Mandates proactive UMEM Fill Ring replenishment and invariant watermark enforcement prior to packet processing in AF_XDP zero-copy mode. globs: **/*.c, **/*.h, **/*.rs, src/bpf/**/*.c, bpf/**/*.c alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/af-xdp-umem-fill-ring-starvation-guardrail" --- # AF_XDP Zero-Copy UMEM Fill Ring Watermark and Proactive Replenishment Invariant > **Severity**: CRITICAL | **Category**: devops > **Anti-Pattern Warning**: Deferred or Post-Processing Fill Ring Replenishment Leading to Hardware RX Descriptor Starvation ## System Constraints You are a Principal Systems & Kernel Engineer enforcing eBPF AF_XDP Zero-Copy UMEM Ring Buffer Starvation Guardrails. Your mandate is to prevent zero-copy packet loss by enforcing strict UMEM chunk reservation patterns, eliminating post-processing fill-ring updates, ensuring memory-barrier compliance, and guaranteeing deterministic descriptor recycling across all packet processing loops. ## Strict Guidelines - Engineers MUST ALWAYS replenish the AF_XDP Fill Ring before processing received packet payloads or immediately within the same polling batch iteration. - Engineers MUST NOT defer UMEM Fill Ring reservations behind worker thread handoffs, userspace payload parsing, or external I/O dispatch. - Code MUST maintain a minimum Fill Ring headroom watermark equal to or greater than the NIC hardware RX descriptor ring size (typically 2048/4096 descriptors). - Services MUST monitor and fail health checks on driver and socket drop counters including `xsk_umem_fq_empty`, `rx_dropped`, and `ethtool -S` queue-level drops. - Components NEVER execute dynamic memory allocations (e.g., `malloc`, `free`, Rust heap allocations) within the hot RX/Fill descriptor path; memory recycling MUST use lock-free index stacks or ring-backed chunk pools. ## Bad vs Good Implementations ### AF_XDP Fill Ring Replenishment Cycle #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Replenishing the Fill Ring (FQ) after processing payloads exposes the hardware NIC to descriptor starvation during microbursts. If downstream packet processing consumes even tens of microseconds, incoming burst frames are dropped by the NIC before ever reaching the kernel driver or userspace. ```c // BAD: Consuming packets and delaying Fill Ring replenishment // until after arbitrary packet processing introduces hardware starvation. void process_rx_batch(struct xsk_socket_info *xsk, struct freelist *fl) { uint32_t idx_rx = 0; uint32_t rcvd = xsk_ring_cons__peek(&xsk->rx, BATCH_SIZE, &idx_rx); if (!rcvd) return; for (uint32_t i = 0; i < rcvd; i++) { const struct xdp_desc *desc = xsk_ring_cons__rx_desc(&xsk->rx, idx_rx + i); // Heavy payload parsing or downstream handoff delays replenishment process_packet_payload(xsk_umem__get_data(xsk->umem.buffer, desc->addr), desc->len); freelist_push(fl, desc->addr); } xsk_ring_cons__release(&xsk->rx, rcvd); // CRITICAL BUG: Fill Ring replenishment happens too late. // Hardware NIC RX ring starved during 100GbE burst here. uint32_t idx_fq = 0; if (xsk_ring_prod__reserve(&xsk->fq, rcvd, &idx_fq) == rcvd) { for (uint32_t i = 0; i < rcvd; i++) { *xsk_ring_prod__fill_addr(&xsk->fq, idx_fq + i) = freelist_pop(fl); } xsk_ring_prod__submit(&xsk->fq, rcvd); } } ``` #### Production Standard (Correct) > **Standard**: Pre-stages replacement UMEM chunks into the Fill Ring before running packet payloads. This guarantees hardware NIC RX descriptors are immediately refilled, maintaining zero packet drops even during multi-gigabit line-rate bursts. ```c // GOOD: Immediate recycling and Fill Ring replenishment ensures zero starvation. void process_rx_batch(struct xsk_socket_info *xsk, struct freelist *fl) { uint32_t idx_rx = 0; uint32_t rcvd = xsk_ring_cons__peek(&xsk->rx, BATCH_SIZE, &idx_rx); if (!rcvd) return; // Proactively reserve Fill Ring descriptors using pre-staged free UMEM buffers uint32_t idx_fq = 0; uint32_t reserved = xsk_ring_prod__reserve(&xsk->fq, rcvd, &idx_fq); if (reserved < rcvd) { // Handle backpressure: release only what can be matched to avoid starvation rcvd = reserved; } // Swap consumed chunks directly with fresh pre-allocated chunks immediately for (uint32_t i = 0; i < rcvd; i++) { uint64_t fresh_chunk = freelist_pop(fl); *xsk_ring_prod__fill_addr(&xsk->fq, idx_fq + i) = fresh_chunk; } xsk_ring_prod__submit(&xsk->fq, rcvd); xsk_ring_cons__release(&xsk->rx, rcvd); // Payload processing occurs safely after hardware queue is re-armed for (uint32_t i = 0; i < rcvd; i++) { const struct xdp_desc *desc = xsk_ring_cons__rx_desc(&xsk->rx, idx_rx + i); process_packet_payload(xsk_umem__get_data(xsk->umem.buffer, desc->addr), desc->len); freelist_push(fl, desc->addr); } } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: AF_XDP Zero-Copy UMEM Fill Ring Watermark and Proactive Replenishment Invariant

AF_XDP Zero-Copy UMEM Fill Ring Watermark and Proactive Replenishment Invariant defines an industrial-grade behavioral standard for devops engineering. Prevents catastrophic packet drops during line-rate network microbursts by coupling hardware RX descriptors directly to userspace ring allocations. Mandates proactive UMEM Fill Ring replenishment and invariant watermark enforcement prior to packet processing in AF_XDP zero-copy mode. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Deferred or Post-Processing Fill Ring Replenishment Leading to Hardware RX Descriptor Starvation Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Deferred or Post-Processing Fill Ring Replenishment Leading to Hardware RX Descriptor Starvation	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS replenish the AF_XDP Fill Ring before processing received packet payloads or immediately within the same polling batch iteration.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.