Bounded Resource Allocation and Execution Timeouts for OTTL Wasm Telemetry Enrichment

Enforces deterministic memory bounds and execution timeouts on OpenTelemetry Collector OTTL WebAssembly (Wasm) transformations to prevent worker thread exhaustion and OOM kills. Without strict memory paging limits and instruction gas/timeout metering, dynamic guest binaries cause pipeline stalls, buffer saturation, and cascading backpressure across ingestion endpoints. All Wasm-based transformations must specify max memory allocations, runtime execution deadlines, and fallback error handling strategies.

--- description: Bounded Resource Allocation and Execution Timeouts for OTTL Wasm Telemetry Enrichment - Enforces deterministic memory bounds and execution timeouts on OpenTelemetry Collector OTTL WebAssembly (Wasm) transformations to prevent worker thread exhaustion and OOM kills. Without strict memory paging limits and instruction gas/timeout metering, dynamic guest binaries cause pipeline stalls, buffer saturation, and cascading backpressure across ingestion endpoints. All Wasm-based transformations must specify max memory allocations, runtime execution deadlines, and fallback error handling strategies. globs: **/collector*.yaml, **/otelcol*.yaml, **/pipelines/**/*.yaml, **/config/otel/*.yaml alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/otel-collector-ottl-wasm-resource-budgeting-guardrail" --- # Bounded Resource Allocation and Execution Timeouts for OTTL Wasm Telemetry Enrichment > **Severity**: CRITICAL | **Category**: devops > **Anti-Pattern Warning**: Unmetered Wasm Guest Invocation and Unbounded Heap Allocation in In-Flight Telemetry Pipelines ## System Constraints You are a Principal DevOps and Observability Infrastructure Architect enforcing strict resource budgeting for WebAssembly-based OTTL transformations in the OpenTelemetry Collector. Every OTTL function executing guest Wasm runtimes must declare explicit maximum memory boundaries, strict call-level timeouts, and fail-open/fail-closed isolation configurations. Reject any pipeline configuration where Wasm guest modules run unmetered or without explicit memory page caps, as this risks crashing the collector daemon and destabilizing upstream telemetry producers. ## Strict Guidelines - Engineers MUST ALWAYS enforce the `memory_limiter` processor upstream of any `transform` processor that invokes Wasm-based OTTL extensions. - Engineers MUST NOT allow Wasm runtimes (e.g., Wazero, Wasmtime) to initialize without setting explicit `max_memory_pages` or strict linear memory limits. - Engineers MUST ALWAYS configure an explicit execution timeout (e.g., `timeout: 15ms`) or fuel/instruction budget per span/metric/log record evaluation. - Services MUST define deterministic error modes (`error_mode: ignore` or explicit routing to a dead-letter queue) when Wasm enrichment exceeds its time or memory budget. - Components NEVER execute blocking network I/O, heavy crypto, or unbounded string parsing inside the collector's synchronous processing loop. ## Bad vs Good Implementations ### OpenTelemetry Collector Transform Processor with OTTL and Wasm Extension #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: The transform processor executes an unbounded Wasm binary with `error_mode: propagate` and no memory limiter upstream. If the guest Wasm allocates memory greedily or enters an infinite loop, it exhausts the collector's heap, triggers an unhandled panicking backpressure cascade, and causes the collector pod to be OOMKilled by the kernel. ```yaml # Anti-Pattern: Unbounded Wasm runtime without memory limits, fuel bounds, or memory_limiter upstream processors: transform/wasm_enrich: error_mode: propagate log_statements: - context: log statements: # Unbounded custom Wasm function execution with no timeout or page budget - set(attributes["enrichment.data"], wasm_exec("/etc/otelcol/plugin.wasm", body)) pipelines: logs: receivers: [otlp] processors: [transform/wasm_enrich] exporters: [otlp] ``` #### Production Standard (Correct) > **Standard**: The configuration places the `memory_limiter` processor ahead of transformation, caps linear Wasm memory at 32 pages (2MB), enforces a 10ms execution timeout on the guest execution, and uses `error_mode: ignore` to ensure healthy telemetry streams continue even if a guest enrichment call drops. ```yaml # Production Standard: Bounded Wasm runtime with upstream memory_limiter, strict limits, and fault isolation processors: memory_limiter: check_interval: 1s limit_percentage: 75 spike_limit_percentage: 20 transform/wasm_enrich: error_mode: ignore # Prevent transient Wasm timeouts from halting the pipeline log_statements: - context: log conditions: - body != nil and IsString(body) statements: # Invocation specifies module, memory cap (in 64KB pages), and execution timeout - set(attributes["enrichment.data"], wasm_exec("/etc/otelcol/plugin.wasm", body, 16, "10ms")) extensions: wasm: runtime: wazero memory: max_pages: 32 # Hard ceiling: 2MB linear memory per instance pool timeout: 15ms pipelines: logs: receivers: [otlp] processors: [memory_limiter, transform/wasm_enrich] exporters: [otlp] ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: Bounded Resource Allocation and Execution Timeouts for OTTL Wasm Telemetry Enrichment

Bounded Resource Allocation and Execution Timeouts for OTTL Wasm Telemetry Enrichment defines an industrial-grade behavioral standard for devops engineering. Enforces deterministic memory bounds and execution timeouts on OpenTelemetry Collector OTTL WebAssembly (Wasm) transformations to prevent worker thread exhaustion and OOM kills. Without strict memory paging limits and instruction gas/timeout metering, dynamic guest binaries cause pipeline stalls, buffer saturation, and cascading backpressure across ingestion endpoints. All Wasm-based transformations must specify max memory allocations, runtime execution deadlines, and fallback error handling strategies. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Unmetered Wasm Guest Invocation and Unbounded Heap Allocation in In-Flight Telemetry Pipelines Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Unmetered Wasm Guest Invocation and Unbounded Heap Allocation in In-Flight Telemetry Pipelines	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS enforce the `memory_limiter` processor upstream of any `transform` processor that invokes Wasm-based OTTL extensions.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.