eBPF Continuous Profiler DWARF Frame-Budget & 5% CPU Ceiling Guardrail

--- description: eBPF Continuous Profiler DWARF Frame-Budget & 5% CPU Ceiling Guardrail - Enforces a strict 5% host CPU utilization ceiling and deterministic frame-budget quotas for eBPF continuous stack unwinding across high-density container nodes. Prevents kernel-space unwind loops from dominating sampled PMU interrupts, preemptively truncating stack walking and dynamically attenuating sample frequencies to eliminate latency spikes. globs: **/*.bpf.c, **/*.bpf.h, pkg/profiler/**/*.go, src/profiler/**/*.rs, deploy/daemonsets/**/*.yaml alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/ebpf-dwarf-unwinder-frame-budget-throttle-guardrail" --- # eBPF Continuous Profiler DWARF Frame-Budget & 5% CPU Ceiling Guardrail > **Severity**: CRITICAL | **Category**: devops > **Anti-Pattern Warning**: Unbounded DWARF Unwind Loops and Static PMU Sampling Inducing Host Kernel Latency Spikes ## System Constraints You are a Principal Systems Engineer and Kernel Observability Architect enforcing safety and performance invariants for eBPF continuous profilers utilizing DWARF stack unwinding. You must ensure all eBPF unwinders adhere to deterministic CPU cycles, hard-capped frame quotas, adaptive sample frequency back-off, and strict isolation so unwinding overhead never exceeds 5% of any CPU core in production. ## Strict Guidelines - Engineers MUST ALWAYS enforce a hard ceiling on unwound stack frames (e.g., maximum 128 frames) per sample inside the eBPF unwinder loop. - Engineers MUST ALWAYS measure elapsed runtime or cycle consumption per sample using bpf_ktime_get_ns() or hardware cycle counters and abort immediately if the execution budget is exceeded. - Engineers MUST NOT allow unbounded loop bounds or variable-depth recursion when resolving DWARF Call Frame Information (CFI) expressions or table lookups in kernel context. - Code SHOULD dynamically scale down PMU sampling frequency (e.g., from 99 Hz down to 19 Hz) when aggregate profiler agent CPU usage exceeds 3% on any individual CPU core. - Services MUST emit a STACK_TRUNCATED_FRAME_BUDGET sentinel marker and increment a Prometheus counter whenever an unwinding trace is aborted due to budget exhaustion. - Components NEVER execute blocking operations, unpinned memory allocations, or unbounded hash map lookups within perf_event sampling probe handlers. ## Bad vs Good Implementations ### eBPF Kernel DWARF Unwinder Budget Throttling #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: The loop lacks a deterministic iteration boundary and execution time guard. Deeply nested microservices or corrupt CFI tables will trap the core in an interrupt handler loop, exceeding the 5% CPU budget and inducing host-wide tail latency spikes. ```c // Unbounded unwinder loop without execution cycle checks or frame limits SEC("perf_event") int profile_sample(struct bpf_perf_event_data *ctx) { struct stack_trace trace = {}; __u64 ip = ctx->regs.ip; __u64 sp = ctx->regs.sp; // ANTI-PATTERN: Loops without frame limit or elapsed time check. // Can monopolize PMU interrupt handler and exhaust CPU under deep stacks or recursion. while (ip != 0) { trace.frames[trace.depth++] = ip; struct unwind_row *row = bpf_map_lookup_elem(&unwind_tables, &ip); if (!row) break; // Unchecked register recovery ip = *(unsigned long *)(sp + row->cfa_offset); sp = sp + row->cfa_offset; } bpf_perf_event_output(ctx, &events, BPF_F_CURRENT_CPU, &trace, sizeof(trace)); return 0; } ``` #### Production Standard (Correct) > **Standard**: Enforces deterministic loop bounds (#pragma unroll with MAX_STACK_DEPTH) and a strict 15-microsecond execution time limit per sample. Automatically flags truncated traces and updates drop metrics without blocking the kernel thread. ```c // Bounded unwinder with hard frame limit, cycle budget, and truncation telemetry #define MAX_STACK_DEPTH 128 #define MAX_SAMPLE_NS 15000 // 15 microseconds budget per sample SEC("perf_event") int profile_sample(struct bpf_perf_event_data *ctx) { struct stack_trace trace = {}; __u64 start_time = bpf_ktime_get_ns(); __u64 ip = ctx->regs.ip; __u64 sp = ctx->regs.sp; #pragma unroll for (int i = 0; i < MAX_STACK_DEPTH; i++) { if (ip == 0) break; // Enforce hard execution time ceiling (5% of 99Hz interval is ~500us budget across samples) if ((bpf_ktime_get_ns() - start_time) > MAX_SAMPLE_NS) { trace.flags |= STACK_TRUNCATED_TIME_BUDGET; __sync_fetch_and_add(&throttled_samples_count, 1); break; } trace.frames[i] = ip; trace.depth++; struct unwind_row *row = bpf_map_lookup_elem(&unwind_tables, &ip); if (!row) break; if (bpf_probe_read_user(&ip, sizeof(ip), (void *)(sp + row->cfa_offset)) < 0) { break; } sp = sp + row->cfa_offset; } bpf_perf_event_output(ctx, &events, BPF_F_CURRENT_CPU, &trace, sizeof(trace)); return 0; } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: eBPF Continuous Profiler DWARF Frame-Budget & 5% CPU Ceiling Guardrail

eBPF Continuous Profiler DWARF Frame-Budget & 5% CPU Ceiling Guardrail defines an industrial-grade behavioral standard for devops engineering. Enforces a strict 5% host CPU utilization ceiling and deterministic frame-budget quotas for eBPF continuous stack unwinding across high-density container nodes. Prevents kernel-space unwind loops from dominating sampled PMU interrupts, preemptively truncating stack walking and dynamically attenuating sample frequencies to eliminate latency spikes. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Unbounded DWARF Unwind Loops and Static PMU Sampling Inducing Host Kernel Latency Spikes Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Unbounded DWARF Unwind Loops and Static PMU Sampling Inducing Host Kernel Latency Spikes	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS enforce a hard ceiling on unwound stack frames (e.g., maximum 128 frames) per sample inside the eBPF unwinder loop.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.