AgentRules.org
RulesStacksClientsAuditorPrunerSelf-HealRFCsBenchmarkCLICI/CDAuthorsSubmit
This specification is maintained by the AgentRules Working Group. Have enhancements or want to claim maintainership?Submit Specification →
devopscritical PriorityID: otel-ebpf-profiling-collector-invariants

OTel eBPF Profiler Host-Protection and Resource Guardrails

Enforces deterministic host stability constraints for OpenTelemetry continuous profiling eBPF collectors deployed across high-density multi-tenant compute nodes. Guarantees CPU unwind sampling frequency ceilings (<=19Hz), off-host symbolization offloading, and kernel BPF ringbuffer drop policies to bound telemetry overhead below 1% CPU and 200MB memory. Prevents host kernel lockups, slab memory exhaustion, and CPU throttling under high container density.

Prohibited Anti-Pattern: Unbounded Sampling Frequencies, On-Host DWARF Parsing, and Blocking BPF Ringbuffers Causing Host Memory Exhaustion

Export Ready-to-Drop Configuration

Select format below to copy code or download file directly.

---
description: OTel eBPF Profiler Host-Protection and Resource Guardrails - Enforces deterministic host stability constraints for OpenTelemetry continuous profiling eBPF collectors deployed across high-density multi-tenant compute nodes. Guarantees CPU unwind sampling frequency ceilings (<=19Hz), off-host symbolization offloading, and kernel BPF ringbuffer drop policies to bound telemetry overhead below 1% CPU and 200MB memory. Prevents host kernel lockups, slab memory exhaustion, and CPU throttling under high container density.
globs: deployments/**/*.yaml, k8s/**/*.yaml, helm/**/*.yaml, charts/**/values.yaml, config/otel/**/*.yaml
alwaysApply: true
verified_by: "agentrules.org"
license: "CC-BY-4.0"
source: "https://agentrules.org/rules/otel-ebpf-profiling-collector-invariants"
---

# OTel eBPF Profiler Host-Protection and Resource Guardrails

> **Severity**: CRITICAL | **Category**: devops
> **Anti-Pattern Warning**: Unbounded Sampling Frequencies, On-Host DWARF Parsing, and Blocking BPF Ringbuffers Causing Host Memory Exhaustion

## System Constraints
You are a Principal DevOps and Linux Kernel Reliability Architect enforcing strict operational invariants for OpenTelemetry continuous profiling via eBPF collectors. Continuous profiling must never degrade host stability, starve co-located workloads, or exhaust slab/kernel memory. Mandate bounded sampling rates, non-blocking ring buffer topologies, remote symbolization offloading, and kernel safety circuit breakers across all collector configurations.

## Strict Guidelines
- Engineers MUST ALWAYS cap eBPF CPU sampling frequency between 11Hz and 19Hz per core; frequencies exceeding 49Hz MUST be rejected to avoid cache trashing on dense multi-tenant nodes.
- Engineers MUST NEVER perform DWARF or local debug symbol resolution on production hosts; raw stack addresses and frame pointers MUST be shipped to an external symbolization service.
- Collector DaemonSets MUST configure explicit Linux cgroup resource limits (< 1.5% CPU core equivalent, < 256MiB RAM limit) and drop capabilities down to only BPF, PERFMON, and SYS_RESOURCE.
- eBPF BPF map allocations MUST specify static max_entries sizes and utilize drop-new/lossy ring buffers (`BPF_F_NO_PREALLOC` avoided in latency-sensitive paths) to guarantee deterministic memory ceilings.
- Collectors MUST implement dynamic backoff or emergency unhooking when host CPU utilization surpasses 90% or host load loadavg exceeds 1.5x physical cores.
- Services MUST NEVER attach kprobes to high-frequency kernel scheduler paths (e.g., `finish_task_switch`); collectors MUST use `perf_event_open` hardware/software sampling or tracepoints exclusively.

## Bad vs Good Implementations
### OTel Continuous Profiling DaemonSet Configuration Guardrail

#### Bad Antipattern (AI Common Hallucination)
> **Why it fails**: Configuring 100Hz CPU sampling introduces excessive context-switching overhead (~5-8% host penalty). Enabling local symbolization causes unconstrained memory spikes via on-host DWARF parsing. Running with privileged: true bypasses granular capability security, and omitting resource limits risks node-wide OOM cascades.
```yaml
# Anti-Pattern: Unbounded sampling, on-host symbolization, missing circuit breakers
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: otel-profiling-agent
  namespace: telemetry
spec:
  template:
    spec:
      hostPID: true
      containers:
        - name: profiler
          image: otel/ebpf-profiler:latest
          securityContext:
            privileged: true
          env:
            - name: PROFILING_SAMPLE_RATE_HZ
              value: "100"
            - name: ENABLE_LOCAL_SYMBOLIZATION
              value: "true"
            - name: MAP_STACK_DEPTH
              value: "256"
          # No resource limits or circuit breakers defined
```

#### Production Standard (Correct)
> **Standard**: Locks CPU sampling rate to 19Hz (a prime number to eliminate resonance with periodic timers). Drops all root privileges except granular CAP_BPF/CAP_PERFMON. Offloads expensive symbolization to an external cluster service, sets strict memory/CPU limits (<= 256MiB / 200m), and activates a host-load circuit breaker at 85% CPU.
```yaml
# Production Invariant: Bounded frequency, off-host symbols, strict cgroup bounds & capabilities
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: otel-profiling-agent
  namespace: telemetry
spec:
  template:
    spec:
      hostPID: true
      containers:
        - name: profiler
          image: otel/ebpf-profiler:0.12.0
          securityContext:
            privileged: false
            readOnlyRootFilesystem: true
            capabilities:
              drop: ["ALL"]
              add: ["SYS_PTRACE", "CAP_BPF", "CAP_PERFMON"]
          resources:
            limits:
              cpu: "200m"
              memory: "256Mi"
            requests:
              cpu: "50m"
              memory: "128Mi"
          env:
            - name: PROFILING_SAMPLE_RATE_HZ
              value: "19"
            - name: ENABLE_LOCAL_SYMBOLIZATION
              value: "false"
            - name: SYMBOLIZER_ENDPOINT
              value: "http://symbol-service.telemetry.svc.cluster.local:8080"
            - name: CIRCUIT_BREAKER_CPU_THRESHOLD_PCT
              value: "85"
            - name: BPF_RING_BUFFER_PAGES
              value: "64"
```
Recommended MCP Servers (mcp-serverlist.com)

Equip agent with system tools to verify code for this stack.

Kubernetes Cluster Inspector MCP
Inspects cluster DaemonSets, cgroup allocations, and container capabilities against profiling guardrails.
Recommended Agent Skills (askills.dev)

Ready-to-use executable skill functions tailored to this stack.

eBPF Map & Program Verifier CLI
Audits active kernel BPF maps, memory footprint, and perf event sampling frequencies on Linux hosts.
Architectural Specification & Invariant Rationale

Architectural Invariant Design: OTel eBPF Profiler Host-Protection and Resource Guardrails

OTel eBPF Profiler Host-Protection and Resource Guardrails defines an industrial-grade behavioral standard for devops engineering. Enforces deterministic host stability constraints for OpenTelemetry continuous profiling eBPF collectors deployed across high-density multi-tenant compute nodes. Guarantees CPU unwind sampling frequency ceilings (<=19Hz), off-host symbolization offloading, and kernel BPF ringbuffer drop policies to bound telemetry overhead below 1% CPU and 200MB memory. Prevents host kernel lockups, slab memory exhaustion, and CPU throttling under high container density. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Unbounded Sampling Frequencies, On-Host DWARF Parsing, and Blocking BPF Ringbuffers Causing Host Memory Exhaustion Without an explicit system instruction file, developers spend hours debugging silent failures.
Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure ModeAI Symptom (The Anti-Pattern)SeverityProduction Impact & Remedy
AI Hallucination & Deprecation DriftUnbounded Sampling Frequencies, On-Host DWARF Parsing, and Blocking BPF Ringbuffers Causing Host Memory ExhaustionCritical
Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production.
Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture InconsistencyAI assistants produce mixed paradigms across files, creating fragmented codebase conventions.Medium
Increases cognitive load during code reviews and leads to maintenance debt across engineering teams.
Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.
Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

1

Engineers MUST ALWAYS cap eBPF CPU sampling frequency between 11Hz and 19Hz per core

Engineers MUST ALWAYS cap eBPF CPU sampling frequency between 11Hz and 19Hz per core; frequencies exceeding 49Hz MUST be rejected to avoid cache trashing on dense multi-tenant nodes.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.
Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.
2

Engineers MUST NEVER perform DWARF or local debug symbol resolution on production hosts

Engineers MUST NEVER perform DWARF or local debug symbol resolution on production hosts; raw stack addresses and frame pointers MUST be shipped to an external symbolization service.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.
Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.
3

Collector DaemonSets MUST configure explicit Linux cgroup resource limits (< 1.5% CPU core equivalent, < 256MiB RAM limit) and drop capabilities down to only BPF, PERFMON, and SYS_RESOURCE.

Collector DaemonSets MUST configure explicit Linux cgroup resource limits (< 1.5% CPU core equivalent, < 256MiB RAM limit) and drop capabilities down to only BPF, PERFMON, and SYS_RESOURCE.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.
Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.
4

eBPF BPF map allocations MUST specify static max_entries sizes and utilize drop-new/lossy ring buffers (`BPF_F_NO_PREALLOC` avoided in latency-sensitive paths) to guarantee deterministic memory ceilings.

eBPF BPF map allocations MUST specify static max_entries sizes and utilize drop-new/lossy ring buffers (`BPF_F_NO_PREALLOC` avoided in latency-sensitive paths) to guarantee deterministic memory ceilings.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.
Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.
5

Collectors MUST implement dynamic backoff or emergency unhooking when host CPU utilization surpasses 90% or host load loadavg exceeds 1.5x physical cores.

Collectors MUST implement dynamic backoff or emergency unhooking when host CPU utilization surpasses 90% or host load loadavg exceeds 1.5x physical cores.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.
Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.
6

Services MUST NEVER attach kprobes to high-frequency kernel scheduler paths (e.g., `finish_task_switch`)

Services MUST NEVER attach kprobes to high-frequency kernel scheduler paths (e.g., `finish_task_switch`); collectors MUST use `perf_event_open` hardware/software sampling or tracepoints exclusively.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.
Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.
Technical Questions & Answers

Frequently Asked Technical Questions

Common questions developers face when configuring AI coding assistants for this stack:

Q:Why is OTel eBPF Profiler Host-Protection and Resource Guardrails critical for AI coding assistants?

Enforces deterministic host stability constraints for OpenTelemetry continuous profiling eBPF collectors deployed across high-density multi-tenant compute nodes. Guarantees CPU unwind sampling frequency ceilings (<=19Hz), off-host symbolization offloading, and kernel BPF ringbuffer drop policies to bound telemetry overhead below 1% CPU and 200MB memory. Prevents host kernel lockups, slab memory exhaustion, and CPU throttling under high container density. Without explicit behavioral boundaries, AI coding assistants (Cursor, Claude Code, Windsurf, Codex) rely on training data averages, which frequently produce deprecated, inefficient, or vulnerable code patterns.

Q:How does adopting this rule prevent AI hallucinations?

This rule explicitly declares what the AI must avoid (Unbounded Sampling Frequencies, On-Host DWARF Parsing, and Blocking BPF Ringbuffers Causing Host Memory Exhaustion) and prescribes verified positive examples. Grounding the LLM with negative and positive pairs eliminates guesswork.

Q:Which AI coding environments are supported?

This specification is multi-platform compatible: Cursor (.cursor/rules/*.mdc & .cursorrules), Anthropic Claude Code (CLAUDE.md), Windsurf (.windsurfrules), Codex / Cline (AGENTS.md), and GitHub Copilot (.github/copilot-instructions.md).

Q:Can I combine this rule with other technology stacks?

Yes. Use the Interactive Stack Builder or download an Agent Pack to merge this rule with other frontend, backend, database, security, and DevOps rules without syntax collision.

Open Specification · CC BY 4.0·Free for Personal, Team & Commercial In-Repo Use

Citation & Open Source Attribution Guidelines

This specification is licensed under Creative Commons Attribution 4.0 International (CC BY 4.0). You are completely free to adapt and share this rule in your workspace. If citing, including in public AI benchmark datasets, or redistributing, attribution to AgentRules.org is required.

Related DEVOPS Rules

View all rules →
devops
86

Kubernetes Gateway API v1 HTTPRoute Canary and ReferenceGrant Invariants

Enforces strict conformance to Kubernetes Gateway API v1 specifications during migration from legacy Ingress controllers. Mandates explicit ReferenceGrant resources for cross-namespace routing, strict relative integer weighting for HTTPRoute canary releases, and explicit backendRef port declarations.

devops
118

ArgoCD ApplicationSet Matrix Generator Invariants & Cascade Deletion Guards

Enforces strict structural constraints on ArgoCD ApplicationSet Matrix generators combining cluster selectors and Git directories. Prevents catastrophic multi-cluster cascade pruning, accidental secret exposure through parameter interpolation, and runaway Application CR explosion via explicit sync policies, preservation finalizers, and parameter validation.

devops
40

Karpenter Resilient Topology, Interruption Queue, and Consolidation Budget Standard

Enforces deterministic disruption budgets, multi-instance-type diversification, and event-driven SQS interruption queue handling across Karpenter NodePools and EC2NodeClasses. Eliminates cascading eviction loops, instance starvation, and availability downtime during AWS Spot rebalances and underutilized node consolidation.

1 Rule in Stack

OTel