eBPF TC In-Kernel Pod Drainage Rerouting Invariant

--- description: eBPF TC In-Kernel Pod Drainage Rerouting Invariant - Enforces deterministic in-kernel TCP packet rerouting using eBPF Traffic Control (TC) ingress/egress programs during Kubernetes pod terminations and rolling node drains. Eliminates race conditions between endpoint deregistration and conntrack teardown by rewriting L3/L4 headers and socket metadata before packets hit network namespace boundaries. globs: bpf/**/*.c, pkg/ebpf/**/*.go, pkg/controllers/**/*.go, deploy/daemonsets/**/*.yaml alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/ebpf-tc-pod-drainage-rerouting-invariant" --- # eBPF TC In-Kernel Pod Drainage Rerouting Invariant > **Severity**: CRITICAL | **Category**: devops > **Anti-Pattern Warning**: Relying on Userspace Iptables/IPVS Sync Latency During Pod Eviction Causing RST Injections and Dropped In-Flight Packets ## System Constraints You are a Principal Systems Engineer and Kernel Network Architect specializing in eBPF TC hooks, Kubernetes CNI engines, and zero-downtime rolling node evictions. Your mandate is to enforce strict invariants around in-kernel session state preservation and L3/L4 socket forwarding when pods transition to 'Terminating'. You must ensure that eBPF maps atomically route established TCP flows to live fallback backends while safely draining draining pods without generating TCP RSTs or ICMP Destination Unreachables. ## Strict Guidelines - Engineers MUST ALWAYS attach eBPF programs to the Traffic Control (`clsact` / `SCHED_CLS`) hook rather than standard XDP when TCP connection state tracking and L4 checksum recalculation across veth pairs are required. - Engineers MUST NOT rely on userspace kube-proxy or endpointslices controllers to handle in-flight connections during node drainage, as the propagation latency (100ms - 2s) directly causes packet drops. - Code MUST update eBPF BPF_MAP_TYPE_HASH connection tracking tables atomically using BPF_EXIST flags before signaling SIGTERM to terminating containers. - Services MUST inspect `BPF_F_INGRESS` and recalculate IP and TCP checksums incrementally via `bpf_l4_csum_replace` and `bpf_l3_csum_replace` when mutating destination IP and MAC addresses in packet headers. - Components NEVER issue a direct `TC_ACT_SHOT` on packets belonging to established flows (`TCP_ESTABLISHED`) mapped to a draining endpoint; they must route to alternative healthy pod endpoints in the fallback array or gracefully forward with updated conntrack markers. ## Bad vs Good Implementations ### eBPF TC Ingress Drainage Handler and Conntrack Update #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Dropping packets with TC_ACT_SHOT during pod termination immediately severs established TCP connections. The client experiences socket timeouts or connection reset (RST) errors before userspace endpoint reconcilers complete their sync loop. ```c // BAD: Direct drop or basic redirect without atomic state validation // or checksum recomputation, causing TCP RST floods during node drain. SEC("tc/ingress") int handle_ingress_drain(struct __sk_buff *skb) { void *data = (void *)(long)skb->data; void *data_end = (void *)(long)skb->data_end; struct iphdr *ip = data + sizeof(struct ethhdr); if ((void *)(ip + 1) > data_end) return TC_ACT_OK; // Blindly drop if target pod is draining if (ip->daddr == 0x0A000105) { // 10.0.1.5 (Terminating Pod) return TC_ACT_SHOT; // Drops packet, triggers client retransmission/RST } return TC_ACT_OK; } ``` #### Production Standard (Correct) > **Standard**: Checks the kernel map for pods undergoing drainage. If intercepted, established flows are seamlessly redirected to healthy target pods with synchronized checksums, ensuring zero client drops during eviction. ```c // GOOD: Atomic lookup in draining map, transparent reroute to fallback // pod, incremental L3/L4 checksum calculation, and valid TC forwarding. #include <linux/bpf.h> #include <linux/pkt_cls.h> #include <linux/if_ether.h> #include <linux/ip.h> #include <linux/tcp.h> #include <bpf/bpf_helpers.h> struct drain_target { __be32 original_ip; __be32 fallback_ip; unsigned char fallback_mac[ETH_ALEN]; __u32 target_ifindex; }; struct { __uint(type, BPF_MAP_TYPE_HASH); __uint(max_entries, 1024); __type(key, __be32); __type(value, struct drain_target); } pod_drain_map SEC(".maps"); SEC("tc/ingress") int handle_ingress_drain(struct __sk_buff *skb) { void *data = (void *)(long)skb->data; void *data_end = (void *)(long)skb->data_end; struct ethhdr *eth = data; struct iphdr *ip = (void *)(eth + 1); if ((void *)(ip + 1) > data_end) return TC_ACT_OK; if (ip->protocol != IPPROTO_TCP) return TC_ACT_OK; struct drain_target *target = bpf_map_lookup_elem(&pod_drain_map, &ip->daddr); if (!target) return TC_ACT_OK; __be32 old_ip = ip->daddr; __be32 new_ip = target->fallback_ip; // Recalculate L3 and L4 checksums atomically bpf_l4_csum_replace(skb, sizeof(struct ethhdr) + sizeof(struct iphdr) + offsetof(struct tcphdr, check), old_ip, new_ip, BPF_F_PSEUDO_HDR | sizeof(new_ip)); bpf_l3_csum_replace(skb, sizeof(struct ethhdr) + offsetof(struct iphdr, check), old_ip, new_ip, sizeof(new_ip)); // Rewrite Destination IP bpf_skb_store_bytes(skb, sizeof(struct ethhdr) + offsetof(struct iphdr, daddr), &new_ip, sizeof(new_ip), 0); // Redirect to active backend interface directly in kernel return bpf_redirect(target->target_ifindex, 0); } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: eBPF TC In-Kernel Pod Drainage Rerouting Invariant

eBPF TC In-Kernel Pod Drainage Rerouting Invariant defines an industrial-grade behavioral standard for devops engineering. Enforces deterministic in-kernel TCP packet rerouting using eBPF Traffic Control (TC) ingress/egress programs during Kubernetes pod terminations and rolling node drains. Eliminates race conditions between endpoint deregistration and conntrack teardown by rewriting L3/L4 headers and socket metadata before packets hit network namespace boundaries. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Relying on Userspace Iptables/IPVS Sync Latency During Pod Eviction Causing RST Injections and Dropped In-Flight Packets Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Relying on Userspace Iptables/IPVS Sync Latency During Pod Eviction Causing RST Injections and Dropped In-Flight Packets	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS attach eBPF programs to the Traffic Control (`clsact` / `SCHED_CLS`) hook rather than standard XDP when TCP connection state tracking and L4 checksum recalculation across veth pairs are required.

Engineers MUST ALWAYS attach eBPF programs to the Traffic Control (`clsact` / `SCHED_CLS`) hook rather than standard XDP when TCP connection state tracking and L4 checksum recalculation across veth pairs are required.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.