AgentRules.org
RulesStacksClientsAuditorPrunerSelf-HealRFCsBenchmarkCLICI/CDAuthorsSubmit
This specification is maintained by the AgentRules Working Group. Have enhancements or want to claim maintainership?Submit Specification →
devopscritical PriorityID: otel-ebpf-profiling-overhead-guardrail

OpenTelemetry eBPF Continuous Profiling Overhead & Symbol Cache Limiter

Enforces strict runtime bounds on OpenTelemetry eBPF CPU continuous profilers by dynamically regulating stack-walking sample rates and capping userspace symbolization memory footprints. This ensures that kernel-level profiling instrumentation never consumes more than 1% aggregate CPU cycles or causes out-of-memory kernel panics under elevated system stress.

Prohibited Anti-Pattern: Static high-frequency eBPF sampling with unbounded native symbol maps causing node throttling and OOM panics

Export Ready-to-Drop Configuration

Select format below to copy code or download file directly.

---
description: OpenTelemetry eBPF Continuous Profiling Overhead & Symbol Cache Limiter - Enforces strict runtime bounds on OpenTelemetry eBPF CPU continuous profilers by dynamically regulating stack-walking sample rates and capping userspace symbolization memory footprints. This ensures that kernel-level profiling instrumentation never consumes more than 1% aggregate CPU cycles or causes out-of-memory kernel panics under elevated system stress.
globs: **/otel-profiler*.{yaml,yml,json,toml}, **/profiling/**/*.{go,c,h,rs,ts}, deploy/k8s/**/*.{yaml,yml}, helm/**/*.{yaml,yml}
alwaysApply: true
verified_by: "agentrules.org"
license: "CC-BY-4.0"
source: "https://agentrules.org/rules/otel-ebpf-profiling-overhead-guardrail"
---

# OpenTelemetry eBPF Continuous Profiling Overhead & Symbol Cache Limiter

> **Severity**: CRITICAL | **Category**: devops
> **Anti-Pattern Warning**: Static high-frequency eBPF sampling with unbounded native symbol maps causing node throttling and OOM panics

## System Constraints
You are a Principal Observability Infrastructure Engineer enforcing OpenTelemetry Continuous Profiling eBPF Sampling Overhead Guardrails. You must mandate dynamic feedback-controlled sampling rates, strict frame-pointer and DWARF unwinding limits, bounded BPF map ring buffers, and capped resident symbol table caching across all profiling agents to guarantee continuous CPU overhead remains strictly under 1.0%.

## Strict Guidelines
- Profilers MUST implement an adaptive PID/feedback loop that reduces sampling frequency (from nominal 19Hz down to 1-5Hz) whenever profiler process CPU utilization exceeds 0.8% of any host core or aggregate system overhead surpasses 1.0%.
- Engineers MUST NOT configure static eBPF sampling intervals exceeding 19Hz on production nodes without active kernel-level overhead throttle circuits.
- Symbol tables and DWARF unwinding caches MUST have hard resident set size (RSS) memory limits enforced with an LRU eviction policy to prevent OOM termination of the agent daemon.
- eBPF BPF_MAP_TYPE_STACK_TRACE allocations MUST set explicit max_entries ceilings tuned against available locked memory (RLIMIT_MEMLOCK) to prevent kernel allocation failures.
- Agent daemons MUST expose high-precision runtime telemetry detailing dropped samples, unwinding failures, current sampling Hertz, and eBPF CPU duty cycle.

## Bad vs Good Implementations
### Configuring Adaptive Profiling Rates and Capped Symbol Memory in OpenTelemetry Profiling Agent

#### Bad Antipattern (AI Common Hallucination)
> **Why it fails**: A static 99Hz sampling rate combined with an unbounded symbol cache (-1) and large ring buffers easily exceeds the 1% CPU overhead threshold under load and risks out-of-memory node panics due to unchecked DWARF unwinding.
```yaml
# Static high-frequency configuration without adaptive rate control or cache ceilings
apiVersion: v1
kind: ConfigMap
metadata:
  name: otel-profiler-config
data:
  config.yaml: |
    profiler:
      sampling_frequency_hz: 99
      symbolication:
        cache_size: -1
      bpf:
        stack_depth: 256
        ring_buffer_pages: 1024
```

#### Production Standard (Correct)
> **Standard**: Enforces a reasonable 19Hz target frequency with automatic degradation to 3Hz if overhead crosses 1.0%, restricts stack depth to 64 frames, and constrains symbol memory to an LRU-managed 256MB boundary.
```yaml
# Adaptive throttling configuration preserving strict <= 1% total system overhead
apiVersion: v1
kind: ConfigMap
metadata:
  name: otel-profiler-config
data:
  config.yaml: |
    profiler:
      sampling:
        target_frequency_hz: 19
        min_frequency_hz: 3
        overhead_threshold_percent: 1.0
        adaptive_backoff_enabled: true
      symbolication:
        cache_type: lru
        max_memory_mb: 256
        prune_interval: 60s
      bpf:
        max_stack_depth: 64
        stack_map_max_entries: 16384
        perf_event_buffer_pages: 64
        drop_on_saturation: true
```
Recommended MCP Servers (mcp-serverlist.com)

Equip agent with system tools to verify code for this stack.

Prometheus Metrics Inspector
Queries Prometheus endpoints to monitor agent CPU overhead and dropped eBPF sample ratios in real time.
Recommended Agent Skills (askills.dev)

Ready-to-use executable skill functions tailored to this stack.

eBPF Subsystem Diagnostic CLI
Inspects running eBPF maps, programs, and locked memory utilization directly on target Linux nodes.
Architectural Specification & Invariant Rationale

Architectural Invariant Design: OpenTelemetry eBPF Continuous Profiling Overhead & Symbol Cache Limiter

OpenTelemetry eBPF Continuous Profiling Overhead & Symbol Cache Limiter defines an industrial-grade behavioral standard for devops engineering. Enforces strict runtime bounds on OpenTelemetry eBPF CPU continuous profilers by dynamically regulating stack-walking sample rates and capping userspace symbolization memory footprints. This ensures that kernel-level profiling instrumentation never consumes more than 1% aggregate CPU cycles or causes out-of-memory kernel panics under elevated system stress. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Static high-frequency eBPF sampling with unbounded native symbol maps causing node throttling and OOM panics Without an explicit system instruction file, developers spend hours debugging silent failures.
Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure ModeAI Symptom (The Anti-Pattern)SeverityProduction Impact & Remedy
AI Hallucination & Deprecation DriftStatic high-frequency eBPF sampling with unbounded native symbol maps causing node throttling and OOM panicsCritical
Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production.
Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture InconsistencyAI assistants produce mixed paradigms across files, creating fragmented codebase conventions.Medium
Increases cognitive load during code reviews and leads to maintenance debt across engineering teams.
Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.
Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

1

Profilers MUST implement an adaptive PID/feedback loop that reduces sampling frequency (from nominal 19Hz down to 1-5Hz) whenever profiler process CPU utilization exceeds 0.8% of any host core or aggregate system overhead surpasses 1.0%.

Profilers MUST implement an adaptive PID/feedback loop that reduces sampling frequency (from nominal 19Hz down to 1-5Hz) whenever profiler process CPU utilization exceeds 0.8% of any host core or aggregate system overhead surpasses 1.0%.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.
Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.
2

Engineers MUST NOT configure static eBPF sampling intervals exceeding 19Hz on production nodes without active kernel-level overhead throttle circuits.

Engineers MUST NOT configure static eBPF sampling intervals exceeding 19Hz on production nodes without active kernel-level overhead throttle circuits.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.
Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.
3

Symbol tables and DWARF unwinding caches MUST have hard resident set size (RSS) memory limits enforced with an LRU eviction policy to prevent OOM termination of the agent daemon.

Symbol tables and DWARF unwinding caches MUST have hard resident set size (RSS) memory limits enforced with an LRU eviction policy to prevent OOM termination of the agent daemon.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.
Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.
4

eBPF BPF_MAP_TYPE_STACK_TRACE allocations MUST set explicit max_entries ceilings tuned against available locked memory (RLIMIT_MEMLOCK) to prevent kernel allocation failures.

eBPF BPF_MAP_TYPE_STACK_TRACE allocations MUST set explicit max_entries ceilings tuned against available locked memory (RLIMIT_MEMLOCK) to prevent kernel allocation failures.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.
Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.
5

Agent daemons MUST expose high-precision runtime telemetry detailing dropped samples, unwinding failures, current sampling Hertz, and eBPF CPU duty cycle.

Agent daemons MUST expose high-precision runtime telemetry detailing dropped samples, unwinding failures, current sampling Hertz, and eBPF CPU duty cycle.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.
Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.
Technical Questions & Answers

Frequently Asked Technical Questions

Common questions developers face when configuring AI coding assistants for this stack:

Q:Why is OpenTelemetry eBPF Continuous Profiling Overhead & Symbol Cache Limiter critical for AI coding assistants?

Enforces strict runtime bounds on OpenTelemetry eBPF CPU continuous profilers by dynamically regulating stack-walking sample rates and capping userspace symbolization memory footprints. This ensures that kernel-level profiling instrumentation never consumes more than 1% aggregate CPU cycles or causes out-of-memory kernel panics under elevated system stress. Without explicit behavioral boundaries, AI coding assistants (Cursor, Claude Code, Windsurf, Codex) rely on training data averages, which frequently produce deprecated, inefficient, or vulnerable code patterns.

Q:How does adopting this rule prevent AI hallucinations?

This rule explicitly declares what the AI must avoid (Static high-frequency eBPF sampling with unbounded native symbol maps causing node throttling and OOM panics) and prescribes verified positive examples. Grounding the LLM with negative and positive pairs eliminates guesswork.

Q:Which AI coding environments are supported?

This specification is multi-platform compatible: Cursor (.cursor/rules/*.mdc & .cursorrules), Anthropic Claude Code (CLAUDE.md), Windsurf (.windsurfrules), Codex / Cline (AGENTS.md), and GitHub Copilot (.github/copilot-instructions.md).

Q:Can I combine this rule with other technology stacks?

Yes. Use the Interactive Stack Builder or download an Agent Pack to merge this rule with other frontend, backend, database, security, and DevOps rules without syntax collision.

Open Specification · CC BY 4.0·Free for Personal, Team & Commercial In-Repo Use

Citation & Open Source Attribution Guidelines

This specification is licensed under Creative Commons Attribution 4.0 International (CC BY 4.0). You are completely free to adapt and share this rule in your workspace. If citing, including in public AI benchmark datasets, or redistributing, attribution to AgentRules.org is required.

Related DEVOPS Rules

View all rules →
devops
86

Kubernetes Gateway API v1 HTTPRoute Canary and ReferenceGrant Invariants

Enforces strict conformance to Kubernetes Gateway API v1 specifications during migration from legacy Ingress controllers. Mandates explicit ReferenceGrant resources for cross-namespace routing, strict relative integer weighting for HTTPRoute canary releases, and explicit backendRef port declarations.

devops
118

ArgoCD ApplicationSet Matrix Generator Invariants & Cascade Deletion Guards

Enforces strict structural constraints on ArgoCD ApplicationSet Matrix generators combining cluster selectors and Git directories. Prevents catastrophic multi-cluster cascade pruning, accidental secret exposure through parameter interpolation, and runaway Application CR explosion via explicit sync policies, preservation finalizers, and parameter validation.

devops
40

Karpenter Resilient Topology, Interruption Queue, and Consolidation Budget Standard

Enforces deterministic disruption budgets, multi-instance-type diversification, and event-driven SQS interruption queue handling across Karpenter NodePools and EC2NodeClasses. Eliminates cascading eviction loops, instance starvation, and availability downtime during AWS Spot rebalances and underutilized node consolidation.

1 Rule in Stack

OpenTelemetry