Continuous eBPF Profiling & OTLP Trace Correlation Standard

Q: How does adopting this rule prevent AI hallucinations?

This rule explicitly declares what the AI must avoid (Ad-Hoc In-Process Profiling Endpoints with Disconnected Trace Context and Omitted Frame Pointers) and prescribes verified positive examples. Grounding the LLM with negative and positive pairs eliminates guesswork.

Q: Which AI coding environments are supported?

This specification is multi-platform compatible: Cursor (.cursor/rules/*.mdc & .cursorrules), Anthropic Claude Code (CLAUDE.md), Windsurf (.windsurfrules), Codex / Cline (AGENTS.md), and GitHub Copilot (.github/copilot-instructions.md).

Q: Can I combine this rule with other technology stacks?

Yes. Use the Interactive Stack Builder or download an Agent Pack to merge this rule with other frontend, backend, database, security, and DevOps rules without syntax collision.

Continuous eBPF Profiling & OTLP Trace Correlation Standard

--- description: Continuous eBPF Profiling & OTLP Trace Correlation Standard - Enforces sub-1% overhead fleet-wide continuous profiling utilizing non-invasive eBPF kernel probes mapped directly to OpenTelemetry span contexts. Mandates compiler frame-pointer preservation, non-lock sampling frequencies, and OTLP profiling signal schema convergence for deterministic code-level root cause analysis. globs: Dockerfile*, **/Dockerfile*, deployments/**/*.{yaml,yml}, infra/**/*.{tf,yaml}, src/telemetry/**/*.{ts,js,go}, src/instrumentation/**/*.{ts,js,go} alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/continuous-ebpf-profiling-otlp-convergence-standard" --- # Continuous eBPF Profiling & OTLP Trace Correlation Standard > **Severity**: CRITICAL | **Category**: devops > **Anti-Pattern Warning**: Ad-Hoc In-Process Profiling Endpoints with Disconnected Trace Context and Omitted Frame Pointers ## System Constraints You are a Principal Software Architect enforcing the Continuous eBPF Profiling & OTLP Profiling Signal Convergence Standard. Mandate that all production container runtimes preserve frame pointers, operate continuous eBPF stack samplers at prime-number frequencies below 1% CPU overhead, and bind active OpenTelemetry trace IDs and span IDs directly to eBPF stack samples using OTLP profiling signal standards. ## Strict Guidelines - Engineers MUST ALWAYS compile runtime binaries with frame-pointer preservation enabled (`-fno-omit-frame-pointer` and `-mno-omit-leaf-frame-pointer` for native runtimes; `-XX:+PreserveFramePointer` for JVM; `--perf-prof` and `--interpreted-frames-native-stack` for Node.js V8). - Services MUST NOT rely on synchronous userspace signal handlers or invasive on-demand `/debug/pprof` endpoints that trigger stop-the-world pauses, memory bloat, or latency cliffs in production. - Engineers MUST configure eBPF profiling sampling frequencies to prime-number intervals (e.g., 19 Hz, 47 Hz, or 97 Hz) to prevent harmonic resonance lock with periodic workload loops or timer interrupts. - Code SHOULD dynamically project active OpenTelemetry context (`trace_id` and `span_id`) into eBPF-accessible thread-local storage or process-pinned memory buffers to enable zero-overhead kernel-to-span attribution. - Collector pipelines MUST transform and ingest profiling data using the OpenTelemetry Profiling Data Model (OTel Profiles) over OTLP/gRPC, ensuring continuous pprof stacks are natively queryable alongside distributed traces. - Containers MUST NOT strip symbol tables (`.symtab`, `.strtab`) from production target binaries unless a remote symbolication API or continuous symbol uploader (e.g., Debuginfod, Parca, or Pyroscope symbol service) is integrated into the CI/CD pipeline. ## Bad vs Good Implementations ### Correlating Dynamic Span Context to eBPF-Accessible Process Buffers #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Synchronous Node.js V8 inspector activation freezes the event loop, introduces prohibitive latency (frequently >15% overhead), and generates detached stack traces that cannot be correlated with active OpenTelemetry trace IDs and span IDs. ```typescript import * as inspector from 'node:inspector'; import { trace } from '@opentelemetry/api'; import express from 'express'; const app = express(); const session = new inspector.Session(); session.connect(); // ANTI-PATTERN: Blocking userspace CPU profiling on request path with zero OTel span correlation app.get('/api/work', async (req, res) => { session.post('Profiler.enable', () => { session.post('Profiler.start', () => { // Heavy synchronous execution blocks event loop and introduces >15% latency overhead const result = executeHeavyTask(); session.post('Profiler.stop', (err, { profile }) => { // Profiler output is isolated, disconnected from distributed traces, and dumped ad-hoc console.log('Profile captured:', profile.nodes.length); res.json({ result }); }); }); }); }); function executeHeavyTask() { let count = 0; for (let i = 0; i < 1e7; i++) count += i; return count; } ``` #### Production Standard (Correct) > **Standard**: Aligns the runtime with eBPF out-of-band sampling by syncing TraceID and SpanID into a designated memory location. The continuous eBPF profiler captures kernel/userspace stack unwinds at a sub-1% overhead prime frequency and matches them deterministically to the active OTel span. ```typescript import { trace, context } from '@opentelemetry/api'; import express from 'express'; const app = express(); // PRODUCTION FIX: Pre-allocate a 32-byte direct C-ABI aligned buffer pinned in process memory. // Continuous eBPF profilers read this thread/process address to correlate stack traces with 0% overhead. const otelEbpfContextBuffer = Buffer.alloc(32); export function syncTraceContextToEbpf(): void { const activeSpan = trace.getSpan(context.active()); if (!activeSpan) { otelEbpfContextBuffer.fill(0); return; } const spanContext = activeSpan.spanContext(); // Write 16-byte TraceID (hex) and 8-byte SpanID (hex) into the shared memory segment otelEbpfContextBuffer.write(spanContext.traceId, 0, 16, 'hex'); otelEbpfContextBuffer.write(spanContext.spanId, 16, 8, 'hex'); } app.get('/api/work', async (req, res) => { const tracer = trace.getTracer('work-service'); await tracer.startActiveSpan('executeHeavyTask', async (span) => { try { syncTraceContextToEbpf(); const result = executeHeavyTask(); res.json({ result }); } finally { span.end(); syncTraceContextToEbpf(); } }); }); function executeHeavyTask() { let count = 0; for (let i = 0; i < 1e7; i++) count += i; return count; } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: Continuous eBPF Profiling & OTLP Trace Correlation Standard

Continuous eBPF Profiling & OTLP Trace Correlation Standard defines an industrial-grade behavioral standard for devops engineering. Enforces sub-1% overhead fleet-wide continuous profiling utilizing non-invasive eBPF kernel probes mapped directly to OpenTelemetry span contexts. Mandates compiler frame-pointer preservation, non-lock sampling frequencies, and OTLP profiling signal schema convergence for deterministic code-level root cause analysis. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Ad-Hoc In-Process Profiling Endpoints with Disconnected Trace Context and Omitted Frame Pointers Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Ad-Hoc In-Process Profiling Endpoints with Disconnected Trace Context and Omitted Frame Pointers	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS compile runtime binaries with frame-pointer preservation enabled (`-fno-omit-frame-pointer` and `-mno-omit-leaf-frame-pointer` for native runtimes

Engineers MUST ALWAYS compile runtime binaries with frame-pointer preservation enabled (`-fno-omit-frame-pointer` and `-mno-omit-leaf-frame-pointer` for native runtimes; `-XX:+PreserveFramePointer` for JVM; `--perf-prof` and `--interpreted-frames-native-stack` for Node.js V8).

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.