Linux eBPF TCX Egress Scheduling and EDT Bufferbloat Prevention

--- description: Linux eBPF TCX Egress Scheduling and EDT Bufferbloat Prevention - Enforces BPF link-managed TCX (BPF_TCX_EGRESS) kernel-level packet scheduling on container veth interfaces using Earliest Departure Time (EDT) pacing. Prevents multi-tenant TCP bufferbloat and noisy-neighbor latency degradation by obsoleting legacy clsact netlink filter management in favor of deterministic skb->tstamp egress regulation. globs: bpf/**/*.c, pkg/bpf/**/*.go, cmd/daemon/**/*.go, src/bpf/**/*.rs alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/ebpf-tcx-egress-traffic-shaping-guardrail" --- # Linux eBPF TCX Egress Scheduling and EDT Bufferbloat Prevention > **Severity**: CRITICAL | **Category**: devops > **Anti-Pattern Warning**: Attaching Legacy Unmanaged Netlink Clsact Filters Without skb->tstamp EDT Pacing ## System Constraints You are a Principal Systems and Kernel Engineer enforcing production invariants for Linux eBPF tcx traffic scheduling and container egress control. Mandate kernel 6.6+ TCX link semantics, Earliest Departure Time (EDT) packet pacing via skb->tstamp, and deterministic multi-tenant rate limiting across all Kubernetes container veth interfaces. ## Strict Guidelines - Engineers MUST ALWAYS attach egress scheduling programs via BPF links (bpf_link_create with BPF_TCX_EGRESS) instead of executing legacy 'tc filter add dev veth... clsact' netlink shell commands. - Engineers MUST ALWAYS implement Earliest Departure Time (EDT) packet scheduling by setting skb->tstamp based on per-tenant token bucket horizons, paired with a host-side FQ (Fair Queuing) qdisc. - Code MUST NEVER drop packets inside eBPF simply for rate throttling; rate limits MUST be achieved by advancing skb->tstamp into future departure horizons, dropping ONLY when delivery latency exceeds max bufferbloat thresholds (buffer horizon cap). - Services MUST query and bind to the specific container network interface ifindex dynamically upon CNI pod setup, maintaining pinned BPF link paths under /sys/fs/bpf to prevent orphaned attachments. - Components NEVER fall back to legacy unbounded pfifo_fast queues on container host-facing veth pairs, as this reintroduces tail-drop oscillations and catastrophic TCP bufferbloat. ## Bad vs Good Implementations ### eBPF TCX Egress EDT Pacing vs Legacy TC Filter Drop #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Legacy cls_bpf programs that drop packets on burst thresholds destroy TCP congestion window scaling and fail to coordinate with the hardware/kernel packet scheduler. Using raw TC_ACT_SHOT without pacing creates extreme tail latencies and CPU overhead. ```c #include <linux/bpf.h> #include <linux/pkt_cls.h> #include <bpf/bpf_helpers.h> // ANTI-PATTERN: Legacy cls_bpf filter dropping packets directly on rate overflow // Lacks EDT skb->tstamp calculation, causing massive TCP throughput collapse & retries SEC("classifier") int legacy_egress_shape(struct __sk_buff *skb) { __u64 now = bpf_ktime_get_ns(); __u32 key = 0; // Crude rate limiting causing bursty drops instead of kernel pacing __u64 *last_tx = bpf_map_lookup_elem(&rate_map, &key); if (last_tx && (now - *last_tx) < 1000000) { return TC_ACT_SHOT; // Violates EDT; causes immediate TCP retransmissions } if (last_tx) *last_tx = now; return TC_ACT_OK; } ``` #### Production Standard (Correct) > **Standard**: Attaches directly to tcx/egress with BPF link management. It implements Earliest Departure Time (EDT) by projecting departure timestamps (skb->tstamp) using per-tenant transmission rates. sch_fq handles actual hardware-timed transmission, eliminating bufferbloat without premature drops. ```c #include <vmlinux.h> #include <bpf/bpf_helpers.h> #include <bpf/bpf_endian.h> #define TCX_NEXT -1 #define TCX_PASS 0 #define TCX_DROP 2 #define MAX_HORIZON_NS 50000000ULL // 50ms drop horizon struct tenant_cfg { __u64 rate_bps; __u64 t_last; }; struct { __uint(type, BPF_MAP_TYPE_HASH); __uint(max_entries, 1024); __type(key, __u32); // tenant/pod ID __type(value, struct tenant_cfg); } tenant_rates SEC(".maps"); SEC("tcx/egress") int tcx_egress_edt_pacer(struct __sk_buff *skb) { __u64 now = bpf_ktime_get_ns(); __u32 tenant_id = skb->mark; // Propagated tenant boundary mark struct tenant_cfg *cfg = bpf_map_lookup_elem(&tenant_rates, &tenant_id); if (!cfg || cfg->rate_bps == 0) return TCX_PASS; __u64 len_bytes = skb->len; __u64 emit_delay_ns = (len_bytes * 8 * 1000000000ULL) / cfg->rate_bps; __u64 departure_time = cfg->t_last; if (departure_time < now) departure_time = now; departure_time += emit_delay_ns; // Prevent bufferbloat by rejecting packets scheduled beyond max horizon if (departure_time - now > MAX_HORIZON_NS) return TCX_DROP; cfg->t_last = departure_time; skb->tstamp = departure_time; // Paced departure handed to sch_fq return TCX_PASS; } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: Linux eBPF TCX Egress Scheduling and EDT Bufferbloat Prevention

Linux eBPF TCX Egress Scheduling and EDT Bufferbloat Prevention defines an industrial-grade behavioral standard for devops engineering. Enforces BPF link-managed TCX (BPF_TCX_EGRESS) kernel-level packet scheduling on container veth interfaces using Earliest Departure Time (EDT) pacing. Prevents multi-tenant TCP bufferbloat and noisy-neighbor latency degradation by obsoleting legacy clsact netlink filter management in favor of deterministic skb->tstamp egress regulation. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Attaching Legacy Unmanaged Netlink Clsact Filters Without skb->tstamp EDT Pacing Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Attaching Legacy Unmanaged Netlink Clsact Filters Without skb->tstamp EDT Pacing	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS attach egress scheduling programs via BPF links (bpf_link_create with BPF_TCX_EGRESS) instead of executing legacy 'tc filter add dev veth... clsact' netlink shell commands.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.