Deterministic Dynamic Drop-Sampling & Ring Buffer Backpressure Guardrails for eBPF Observability

Q: How does adopting this rule prevent AI hallucinations?

This rule explicitly declares what the AI must avoid (Unthrottled Static Ingestion Leading to Silent Kernel Ring Buffer Drops and Out-of-Memory Collector Cascades) and prescribes verified positive examples. Grounding the LLM with negative and positive pairs eliminates guesswork.

Q: Which AI coding environments are supported?

This specification is multi-platform compatible: Cursor (.cursor/rules/*.mdc & .cursorrules), Anthropic Claude Code (CLAUDE.md), Windsurf (.windsurfrules), Codex / Cline (AGENTS.md), and GitHub Copilot (.github/copilot-instructions.md).

Q: Can I combine this rule with other technology stacks?

Yes. Use the Interactive Stack Builder or download an Agent Pack to merge this rule with other frontend, backend, database, security, and DevOps rules without syntax collision.

--- description: Deterministic Dynamic Drop-Sampling & Ring Buffer Backpressure Guardrails for eBPF Observability - Enforces adaptive drop-sampling and closed-loop backpressure regulation between user-space telemetry collectors and kernel eBPF ring buffers. Guarantees deterministic backpressure signaling during multi-gigabit partition storms to prevent unmonitored ring buffer exhaustion and kernel event loss. globs: src/collector/**/*.ts, src/observability/**/*.ts, src/agent/**/*.ts, daemon/**/*.ts alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/ebpf-ring-buffer-backpressure-sampling-guardrails" --- # Deterministic Dynamic Drop-Sampling & Ring Buffer Backpressure Guardrails for eBPF Observability > **Severity**: CRITICAL | **Category**: devops > **Anti-Pattern Warning**: Unthrottled Static Ingestion Leading to Silent Kernel Ring Buffer Drops and Out-of-Memory Collector Cascades ## System Constraints You are a Principal Software Architect enforcing eBPF Network Observability Drop-Sampling Rate Guardrails. Your mandate is to eliminate unthrottled event emissions from kernel-space eBPF ring buffers (BPF_MAP_TYPE_RINGBUF / BPF_MAP_TYPE_PERF_EVENT_ARRAY) and enforce dynamic, hysteresis-governed drop-sampling algorithms in user-space daemon controllers. All eBPF collector loops must establish strict watermarks, atomically synchronize sampling factors to kernel config maps, and preserve monotonic drop metrics under partition storms. ## Strict Guidelines - Engineers MUST ALWAYS implement closed-loop watermark monitoring that dynamically adjusts sampling rates before kernel ring buffers breach 70% capacity. - Engineers MUST NOT emit events from kernel-space without evaluating an atomic sampling map configured by user-space backpressure loops. - Services MUST reconcile kernel-side drop counters (incremented when events are dropped or skipped) alongside ingested records to guarantee accurate loss attribution. - Code SHOULD employ dual-threshold hysteresis (e.g., scale-down at 75% capacity, restore at 40% capacity) to prevent rapid oscillation of sampling rates during traffic bursts. - Components NEVER allow unbounded batch sizes or unbounded consumer processing queues without deterministic backpressure propagation to the kernel configuration map. ## Bad vs Good Implementations ### Naive Static Ring Buffer Ingestion vs. Closed-Loop Dynamic Drop-Sampling Controller #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: The naive implementation polls the ring buffer without checking capacity watermarks or consumer queue latency. During network bursts, user-space HTTP/sink dispatch bottlenecks, causing the kernel BPF ring buffer to exhaust its allocated pages, dropping critical network forensics without adjusting kernel sampling or emitting drop counters. ```typescript import { RingBuffer, BpfMap } from 'bpf-binding-lib'; interface NetworkTraceEvent { srcIp: string; dstIp: string; bytes: number; } export class EbpfNetworkCollector { private ringBuffer: RingBuffer; constructor(ringBufferFd: number) { this.ringBuffer = new RingBuffer(ringBufferFd); } public startIngestion(): void { // ANTI-PATTERN: Consumes continuously without monitoring buffer lag or consumer backpressure. // In multi-gigabit storms, user-space queue swells, processing stalls, and kernel drops frames silently. this.ringBuffer.poll(-1, (data: Buffer) => { const event = this.deserialize(data); this.forwardToTelemetryPipeline(event); }); } private deserialize(data: Buffer): NetworkTraceEvent { return { srcIp: '10.0.0.1', dstIp: '10.0.0.2', bytes: data.readUInt32LE(0) }; } private forwardToTelemetryPipeline(event: NetworkTraceEvent): void { // Heavy processing causes user-space queue buildup without kernel feedback fetch('https://telemetry-sink.internal/ingest', { method: 'POST', body: JSON.stringify(event), }); } } ``` #### Production Standard (Correct) > **Standard**: The production pattern measures ring buffer consumer lag against strict high/low watermarks (70%/40% hysteresis). Upon detecting saturation, it immediately informs kernel space by atomically updating a BPF map configuration, causing the eBPF hook to drop-sample at the packet source before events enter the ring buffer, preserving system stability. ```typescript import { RingBuffer, BpfMap } from 'bpf-binding-lib'; interface ConfigMapKey { id: number; } interface ConfigMapValue { sampleRateDivisor: number; } export class ResilientEbpfCollector { private ringBuffer: RingBuffer; private configMap: BpfMap<ConfigMapKey, ConfigMapValue>; private readonly highWatermarkRatio = 0.70; private readonly lowWatermarkRatio = 0.40; private currentDivisor = 1; private isThrottled = false; constructor(ringBufferFd: number, configMapFd: number) { this.ringBuffer = new RingBuffer(ringBufferFd); this.configMap = new BpfMap<ConfigMapKey, ConfigMapValue>(configMapFd); } public startIngestion(): void { const pollIntervalMs = 50; setInterval(() => { this.evaluateBackpressureAndRegulate(); }, pollIntervalMs); this.ringBuffer.poll(100, (data: Buffer) => { const event = this.deserialize(data); this.enqueueForBatchProcessing(event); }); } private evaluateBackpressureAndRegulate(): void { const totalBytes = this.ringBuffer.getCapacityBytes(); const unconsumedBytes = this.ringBuffer.getUnconsumedBytes(); const usageRatio = unconsumedBytes / totalBytes; if (!this.isThrottled && usageRatio >= this.highWatermarkRatio) { // Saturated: scale sampling divisor up exponentially (e.g., 1 -> 16) this.isThrottled = true; this.currentDivisor = Math.min(this.currentDivisor * 4, 128); this.applyKernelSamplingDivisor(this.currentDivisor); } else if (this.isThrottled && usageRatio <= this.lowWatermarkRatio) { // Hysteresis recovery: reduce sampling divisor this.currentDivisor = Math.max(1, Math.floor(this.currentDivisor / 2)); if (this.currentDivisor === 1) { this.isThrottled = false; } this.applyKernelSamplingDivisor(this.currentDivisor); } } private applyKernelSamplingDivisor(divisor: number): void { // Atomically write new divisor to BPF array map so kernel drops at source this.configMap.update({ id: 0 }, { sampleRateDivisor: divisor }); } private deserialize(data: Buffer): { bytes: number } { return { bytes: data.readUInt32LE(0) }; } private enqueueForBatchProcessing(_event: { bytes: number }): void { // Deterministic batch pipeline handling } } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: Deterministic Dynamic Drop-Sampling & Ring Buffer Backpressure Guardrails for eBPF Observability

Deterministic Dynamic Drop-Sampling & Ring Buffer Backpressure Guardrails for eBPF Observability defines an industrial-grade behavioral standard for devops engineering. Enforces adaptive drop-sampling and closed-loop backpressure regulation between user-space telemetry collectors and kernel eBPF ring buffers. Guarantees deterministic backpressure signaling during multi-gigabit partition storms to prevent unmonitored ring buffer exhaustion and kernel event loss. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Unthrottled Static Ingestion Leading to Silent Kernel Ring Buffer Drops and Out-of-Memory Collector Cascades Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Unthrottled Static Ingestion Leading to Silent Kernel Ring Buffer Drops and Out-of-Memory Collector Cascades	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS implement closed-loop watermark monitoring that dynamically adjusts sampling rates before kernel ring buffers breach 70% capacity.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.