eBPF XDP Layer-4 Direct Server Return (DSR) State Synchronization Guardrail

--- description: eBPF XDP Layer-4 Direct Server Return (DSR) State Synchronization Guardrail - Enforces deterministic, synchronized conntrack state management across distributed XDP ingress routers deploying Layer-4 DSR load balancing. Prevents silent packet drops and TCP reset storms caused by asymmetric routing and stale Maglev/consistent-hash lookups during rapid autoscaling events. globs: **/*.c, **/*.h, pkg/loadbalancer/**/*.go, src/bpf/**/*.rs, infra/xdp/**/*.yaml alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/ebpf-xdp-dsr-conntrack-sync-guardrail" --- # eBPF XDP Layer-4 Direct Server Return (DSR) State Synchronization Guardrail > **Severity**: CRITICAL | **Category**: devops > **Anti-Pattern Warning**: Local-only in-memory eBPF conntrack table updates without cluster-wide cross-node sync or consistent encapsulation fallbacks. ## System Constraints You are a Principal Systems & Network Infrastructure Architect enforcing invariants for eBPF XDP Layer-4 Direct Server Return (DSR) state consistency. In L4 DSR topologies, return traffic bypasses load balancers, meaning asymmetric packet delivery will corrupt active TCP streams if ingress routers diverge in their conntrack state during backend scale-down or node reallocation. You must strictly reject any eBPF XDP implementation that relies solely on isolated per-node BPF maps or naive 5-tuple consistent hashing without active cross-node state replication, IP-in-IP/GUE encapsulation metadata fallback, or TCP sequence/timestamp tracking. ## Strict Guidelines - Engineers MUST ALWAYS propagate connection established/terminated events across all ingress nodes using high-throughput ring buffers (BPF_MAP_TYPE_RINGBUF) combined with a distributed peer-sync bus (e.g., multicast, Raft, or Cilium-style BPF map sync). - Engineers MUST NOT mutate or evict connection-tracking entries purely on consistent hash recalculations without verifying active TCP FIN/RST or checking TCP sequence tracking invariants. - Code SHOULD encode backend session affinity metadata directly into the encapsulation headers (e.g., Generic UDP Encapsulation [GUE] private flags or Geneve option TLVs) to allow secondary XDP forwarding nodes to recover target backend identity statelessly. - Services MUST implement graceful backend draining with synchronized tombstone timers before purging nodes from the XDP Maglev/rendezvous lookup tables. - Components NEVER drop asymmetric packets without checking distributed synchronization fallback maps or inspecting embedded DSR encapsulation headers. ## Bad vs Good Implementations ### Handling Ingress Packet Routing: Naive Local Map vs. Enriched DSR Conntrack Lookup with Sync Fallback #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: The code maintains an isolated local conntrack table per ingress node. During rapid scale-up/scale-down, ECMP re-routing directs mid-stream packets to an adjacent router that lacks this entry. The adjacent router computes a different backend via Maglev rebalancing, dispatching existing traffic to an incorrect target, which generates RST packets back to client due to DSR. ```c // Anti-Pattern: Local eBPF hash-table lookup with naive Maglev recalculation. // If the node table changes or packets arrive via an asymmetric ECMP path, conntrack fails // and falls back to a newly computed backend, resetting established TCP sessions. struct bpf_map_def SEC("maps") ct_map = { .type = BPF_MAP_TYPE_HASH, .key_size = sizeof(struct ipv4_ct_tuple), .value_size = sizeof(struct ct_entry), .max_entries = 1048576, }; SEC("xdp") int xdp_dsr_lb(struct xdp_md *ctx) { void *data = (void *)(long)ctx->data; void *data_end = (void *)(long)ctx->data_end; struct iphdr *iph = data + sizeof(struct ethhdr); if ((void *)(iph + 1) > data_end) return XDP_DROP; struct ipv4_ct_tuple tuple = { iph->saddr, iph->daddr, 0, 0, iph->protocol }; struct ct_entry *entry = bpf_map_lookup_elem(&ct_map, &tuple); if (!entry) { // Re-hashing dynamically during backend churn routes mid-stream TCP packets // to a brand-new backend, inducing immediate TCP RSTs in DSR topology. __u32 backend_id = get_maglev_backend(iph->saddr); struct ct_entry new_entry = { .backend_id = backend_id }; bpf_map_update_elem(&ct_map, &tuple, &new_entry, BPF_ANY); return forward_dsr(ctx, backend_id); } return forward_dsr(ctx, entry->backend_id); } ``` #### Production Standard (Correct) > **Standard**: The solution checks whether the inbound packet is a new connection or an in-flight stream. Mid-stream packets without local conntrack entries trigger a distributed state fallback rather than naive consistent-hash re-evaluation. Newly established flows emit events into a high-performance ring buffer to maintain state sync across all ingress nodes. ```c // Production Guardrail: Replicated state lookup, RingBuf event broadcast, // and stateless GUE header fallback for cross-node DSR consistency. struct { __uint(type, BPF_MAP_TYPE_LRU_HASH); __uint(max_entries, 2097152); __type(key, struct ipv4_ct_tuple); __type(value, struct ct_entry); } ct_map SEC(".maps"); struct { __uint(type, BPF_MAP_TYPE_RINGBUF); __uint(max_entries, 16777216); } ct_sync_events SEC(".maps"); SEC("xdp") int xdp_dsr_lb(struct xdp_md *ctx) { void *data = (void *)(long)ctx->data; void *data_end = (void *)(long)ctx->data_end; struct iphdr *iph = data + sizeof(struct ethhdr); if ((void *)(iph + 1) > data_end) return XDP_DROP; struct ipv4_ct_tuple tuple = { .src_ip = iph->saddr, .dst_ip = iph->daddr, .proto = iph->protocol }; struct ct_entry *entry = bpf_map_lookup_elem(&ct_map, &tuple); if (!entry) { // If mid-stream TCP (SYN flag absent), avoid blind Maglev allocation. // Query distributed sync cache or inspect encapsulated GUE context before rerouting. if (!is_tcp_syn_packet(iph, data_end)) { __u32 fallback_backend = lookup_distributed_sync_map(&tuple); if (fallback_backend != 0) return forward_dsr_gue(ctx, fallback_backend); return XDP_PASS; // Escalate to L7 mesh for uncorrupted connection recovery } __u32 backend_id = get_maglev_backend(iph->saddr); struct ct_entry new_entry = { .backend_id = backend_id, .state = CT_STATE_SYN_RECEIVED }; bpf_map_update_elem(&ct_map, &tuple, &new_entry, BPF_ANY); // Asynchronously broadcast new connection mapping to peer ingress nodes struct ct_event *evt = bpf_ringbuf_reserve(&ct_sync_events, sizeof(*evt), 0); if (evt) { evt->tuple = tuple; evt->backend_id = backend_id; evt->event_type = CT_EVENT_NEW; bpf_ringbuf_submit(evt, 0); } return forward_dsr_gue(ctx, backend_id); } return forward_dsr_gue(ctx, entry->backend_id); } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: eBPF XDP Layer-4 Direct Server Return (DSR) State Synchronization Guardrail

eBPF XDP Layer-4 Direct Server Return (DSR) State Synchronization Guardrail defines an industrial-grade behavioral standard for devops engineering. Enforces deterministic, synchronized conntrack state management across distributed XDP ingress routers deploying Layer-4 DSR load balancing. Prevents silent packet drops and TCP reset storms caused by asymmetric routing and stale Maglev/consistent-hash lookups during rapid autoscaling events. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Local-only in-memory eBPF conntrack table updates without cluster-wide cross-node sync or consistent encapsulation fallbacks. Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Local-only in-memory eBPF conntrack table updates without cluster-wide cross-node sync or consistent encapsulation fallbacks.	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS propagate connection established/terminated events across all ingress nodes using high-throughput ring buffers (BPF_MAP_TYPE_RINGBUF) combined with a distributed peer-sync bus (e.g., multicast, Raft, or Cilium-style BPF map sync).

Engineers MUST ALWAYS propagate connection established/terminated events across all ingress nodes using high-throughput ring buffers (BPF_MAP_TYPE_RINGBUF) combined with a distributed peer-sync bus (e.g., multicast, Raft, or Cilium-style BPF map sync).

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.