eBPF BPF-Ringbuf Adaptive Backpressure Shedding and Dynamic Map Compaction

--- description: eBPF BPF-Ringbuf Adaptive Backpressure Shedding and Dynamic Map Compaction - Enforces strict memory budget invariants, adaptive telemetry shedding, and dynamic state-map compaction within eBPF kernel programs. Prevents ring buffer exhaustion cascades, kernel memory locking stalls, and high-frequency softirq lockups during high-throughput L7 socket and network audit bursts. globs: **/*.bpf.c, **/*.bpf.h, bpf/**/*.c, src/bpf/**/*.c, pkg/ebpf/**/*.go, src/agent/**/*.rs alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/ebpf-ringbuf-backpressure-compaction-standard" --- # eBPF BPF-Ringbuf Adaptive Backpressure Shedding and Dynamic Map Compaction > **Severity**: CRITICAL | **Category**: devops > **Anti-Pattern Warning**: Unchecked Ringbuf Allocation with Unbounded Hash Map Retention Causing Kernel Lockups and Blind Telemetry Cascades ## System Constraints You are a Principal Systems Engineer and Linux Kernel Specialist enforcing production invariants for eBPF ring buffer telemetry and kernel map lifecycle management. You must eliminate unchecked memory reservations, boundless map growths, and unmitigated packet capture cascades under heavy L7 load. ## Strict Guidelines - Engineers MUST ALWAYS inspect the return pointer of `bpf_ringbuf_reserve()` and invoke fallback dynamic aggregation into `BPF_MAP_TYPE_PERCPU_ARRAY` maps upon reservation failure or saturation. - Engineers MUST NOT allocate dynamically unbounded `BPF_MAP_TYPE_HASH` maps for volatile L7 connection tracking; all transient connection trackers MUST utilize bounded `BPF_MAP_TYPE_LRU_HASH` or active generational compaction maps. - Programs MUST implement atomic watermark pressure checks via a synchronized read-mostly control array map to trigger kernel-side adaptive sampling BEFORE attempting payload serialization. - Code MUST explicitly call `bpf_ringbuf_discard()` with `BPF_RB_NO_WAKEUP` flags if a reservation cannot be finalized, avoiding ring buffer fragmentation and zombie allocations. - Engineers MUST NEVER hold socket audit contexts across unbounded tracepoints without explicit lease lifetimes or generation-based eviction checks. - Kernel programs MUST enforce a deterministic byte ceiling for variable-length payload buffers using verifier-bounded masks (e.g., `len &= 0x3FFF`) to eliminate ring buffer head starvation. ## Bad vs Good Implementations ### Unbounded Reserve vs. Adaptive Backpressure with Fallback Compaction #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Failing to check kernel-userspace backpressure causes abrupt allocation failures during socket bursts. Dropping events silently without updating fallback aggregation counters blindspots incident response, and lack of dynamic shedding exhausts ring buffer memory without adaptive telemetry recovery. ```c // BAD: Blind ring buffer reservation without watermark awareness or fallback aggregation. // Triggers OOM allocation failures and silent telemetry dropping during microbursts. #include <vmlinux.h> #include <bpf/bpf_helpers.h> struct event_t { __u32 pid; __u32 len; char payload[4096]; }; struct { __uint(type, BPF_MAP_TYPE_RINGBUF); __uint(max_entries, 1 << 24); } telemetry_rb SEC(".maps"); SEC("sockops") int trace_sock_burst(struct bpf_sock_ops *skops) { struct event_t *evt; // Blind reserve: If userspace lags, this consistently fails and drops L7 state silently evt = bpf_ringbuf_reserve(&telemetry_rb, sizeof(*evt), 0); if (!evt) { return 0; // Telemetry loss without audit trail or in-kernel aggregation } evt->pid = bpf_get_current_pid_tgid() >> 32; evt->len = skops->bytes_received; bpf_ringbuf_submit(evt, 0); return 0; } ``` #### Production Standard (Correct) > **Standard**: Implements two-tiered load shedding. A control map conveys userspace consumption lag; during saturation, telemetry shifts from high-overhead ringbuf serialization to per-CPU aggregation counters. Reservation failures trigger fallback compaction, and low-latency flags prevent softirq wakeup cascades. ```c // GOOD: Watermark-driven adaptive shedding, bounded LRU state, and per-CPU fallback compaction. #include <vmlinux.h> #include <bpf/bpf_helpers.h> #define MAX_PAYLOAD_MASK 0x0FFF // Bounded to 4KB for verifier safety struct audit_event { __u32 pid; __u32 bytes; __u8 shed_level; }; struct compaction_stat { __u64 dropped_events; __u64 accumulated_bytes; }; struct { __uint(type, BPF_MAP_TYPE_RINGBUF); __uint(max_entries, 1 << 24); } telemetry_rb SEC(".maps"); struct { __uint(type, BPF_MAP_TYPE_PERCPU_ARRAY); __uint(max_entries, 1); __type(key, __u32); __type(value, struct compaction_stat); } fallback_compaction_map SEC(".maps"); struct { __uint(type, BPF_MAP_TYPE_ARRAY); __uint(max_entries, 1); __type(key, __u32); __type(value, __u32); // 0 = Nominal, 1 = Under Pressure, 2 = Heavy Backpressure } backpressure_ctrl_map SEC(".maps"); SEC("sockops") int trace_sock_burst(struct bpf_sock_ops *skops) { __u32 zero = 0; __u32 *pressure_level = bpf_map_lookup_elem(&backpressure_ctrl_map, &zero); __u32 current_level = pressure_level ? *pressure_level : 0; __u32 rx_bytes = skops->bytes_received & MAX_PAYLOAD_MASK; // Level 2: Shed events in-kernel, aggregate to per-cpu map without ringbuf thrashing if (current_level >= 2) { struct compaction_stat *stat = bpf_map_lookup_elem(&fallback_compaction_map, &zero); if (stat) { stat->dropped_events++; stat->accumulated_bytes += rx_bytes; } return 0; } struct audit_event *evt = bpf_ringbuf_reserve(&telemetry_rb, sizeof(*evt), 0); if (!evt) { // Adaptive fallback compaction on ringbuf saturation struct compaction_stat *stat = bpf_map_lookup_elem(&fallback_compaction_map, &zero); if (stat) { stat->dropped_events++; stat->accumulated_bytes += rx_bytes; } return 0; } evt->pid = bpf_get_current_pid_tgid() >> 32; evt->bytes = rx_bytes; evt->shed_level = (__u8)current_level; // Suppress wakeups when backpressure flag is active to reduce IRQ storm __u64 flags = (current_level == 1) ? BPF_RB_NO_WAKEUP : 0; bpf_ringbuf_submit(evt, flags); return 0; } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: eBPF BPF-Ringbuf Adaptive Backpressure Shedding and Dynamic Map Compaction

eBPF BPF-Ringbuf Adaptive Backpressure Shedding and Dynamic Map Compaction defines an industrial-grade behavioral standard for devops engineering. Enforces strict memory budget invariants, adaptive telemetry shedding, and dynamic state-map compaction within eBPF kernel programs. Prevents ring buffer exhaustion cascades, kernel memory locking stalls, and high-frequency softirq lockups during high-throughput L7 socket and network audit bursts. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Unchecked Ringbuf Allocation with Unbounded Hash Map Retention Causing Kernel Lockups and Blind Telemetry Cascades Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Unchecked Ringbuf Allocation with Unbounded Hash Map Retention Causing Kernel Lockups and Blind Telemetry Cascades	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS inspect the return pointer of `bpf_ringbuf_reserve()` and invoke fallback dynamic aggregation into `BPF_MAP_TYPE_PERCPU_ARRAY` maps upon reservation failure or saturation.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.