OpenTelemetry Baggage Overflow Redaction & Cross-Tenant Injection Guardrail

--- description: OpenTelemetry Baggage Overflow Redaction & Cross-Tenant Injection Guardrail - Enforces strict boundary filtering, size constraints, and key allowlisting on OpenTelemetry Baggage and TraceContext propagation across distributed service mesh hops. Prevents tenant context spoofing, memory exhaustion via unbounded header propagation, and downstream metric storage cardinality explosion. globs: src/**/tracing/**/*.{ts,js}, src/**/telemetry/**/*.{ts,js}, src/**/middleware/**/*.{ts,js}, src/**/interceptors/**/*.{ts,js} alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/otel-baggage-overflow-redaction-filter-standard" --- # OpenTelemetry Baggage Overflow Redaction & Cross-Tenant Injection Guardrail > **Severity**: CRITICAL | **Category**: devops > **Anti-Pattern Warning**: Unsanitized W3C Baggage Ingestion and Metric Tag Forwarding ## System Constraints You are a Principal Software Architect enforcing enterprise-grade OpenTelemetry propagation safety. You mandate zero-trust sanitization of distributed trace context and baggage. Unchecked deserialization of W3C baggage headers allows malicious actors to execute header-flooding attacks, overwrite authenticated tenant boundaries, and cause downstream Prometheus/Otel Collector Out-Of-Memory (OOM) failures by injecting high-cardinality baggage into metric dimensions. Ensure all propagators enforce strict key allowlists, size limits, and authenticated tenant overrides. ## Strict Guidelines - Engineers MUST ALWAYS implement a custom or wrapped `TextMapPropagator` that validates and redacts incoming Baggage headers against an explicit, immutable allowlist before committing them to the active `Context`. - Engineers MUST NOT map raw, unvalidated `Baggage` keys directly into metric attributes, span attributes, or structured log MDC fields without explicit cardinality bounds checking. - Services MUST enforce strict hard limits on baggage payload sizes: maximum 16 entries, maximum 64 bytes per key, maximum 256 bytes per value, and total baggage header size not exceeding 2048 bytes. - Services MUST NEVER permit incoming baggage items to override cryptographically verified identity contexts (such as `tenant_id`, `account_id`, or `user_id` extracted from verified JWT or mTLS assertions). - Code SHOULD explicitly strip non-standard, unverified metadata and high-risk injection vectors (CRLF characters, control characters, semicolon-separated attributes) prior to forwarding trace context to downstream microservice hops. ## Bad vs Good Implementations ### Vulnerable Direct Baggage Propagation vs. Hardened Sanitizing Baggage Propagator #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Accepts unvalidated baggage directly into the active tracing context. An attacker can supply arbitrarily large headers (HTTP header DoS), inject 'tenant_id' to hijack tenant tenancy boundaries across internal RPCs, and exhaust collector memory by generating unbounded unique metric tag values. ```typescript import { context, propagation, defaultTextMapGetter, Baggage } from '@opentelemetry/api'; import { Request, Response, NextFunction } from 'express'; // ANTI-PATTERN: Blindly extracts incoming W3C trace context and baggage. // Allows upstream clients to inject arbitrary tenant_id values, spoof boundaries, // and flood downstream trace pipelines with megabytes of high-cardinality keys. export function traceMiddleware(req: Request, res: Response, next: NextFunction) { const extractedContext = propagation.extract(context.active(), req.headers, defaultTextMapGetter); context.with(extractedContext, () => { const baggage = propagation.getBaggage(context.active()); // Blindly extracting untrusted tenant identifier const tenantId = baggage?.getEntry('tenant_id')?.value; (req as any).tenantId = tenantId; // Blindly exporting all baggage to metrics/spans creates catastrophic cardinality explosion baggage?.getAllEntries().forEach(([k, v]) => { console.log(`Setting telemetry tag ${k}=${v.value}`); }); next(); }); } ``` #### Production Standard (Correct) > **Standard**: Wraps the W3C Baggage propagator with a zero-trust sanitization layer. Strictly allowlists safe routing keys, discards untrusted tenant/user ID injection vectors, truncates abusive payload sizes, strips control characters, and caps maximum items to completely prevent downstream cardinality DoS attacks. ```typescript import { context, propagation, TextMapPropagator, TextMapGetter, TextMapSetter, Context, Baggage, createBaggage } from '@opentelemetry/api'; import { W3CBaggagePropagator } from '@opentelemetry/core'; const ALLOWED_BAGGAGE_KEYS = new Set(['traffic_origin', 'experiment_variant', 'datacenter_routing']); const MAX_BAGGAGE_ENTRIES = 8; const MAX_KEY_LENGTH = 32; const MAX_VALUE_LENGTH = 128; const CONTROL_CHAR_REGEX = /[\x00-\x1F\x7F]/; export class BoundarySafeBaggagePropagator implements TextMapPropagator { private readonly delegate = new W3CBaggagePropagator(); inject(ctx: Context, carrier: unknown, setter: TextMapSetter): void { this.delegate.inject(ctx, carrier, setter); } extract(ctx: Context, carrier: unknown, getter: TextMapGetter): Context { const rawContext = this.delegate.extract(ctx, carrier, getter); const incomingBaggage = propagation.getBaggage(rawContext); if (!incomingBaggage) return ctx; const sanitizedEntries: Record<string, { value: string }> = {}; let validCount = 0; for (const [key, entry] of incomingBaggage.getAllEntries()) { if (validCount >= MAX_BAGGAGE_ENTRIES) break; if (!ALLOWED_BAGGAGE_KEYS.has(key)) continue; if (key.length > MAX_KEY_LENGTH || entry.value.length > MAX_VALUE_LENGTH) continue; if (CONTROL_CHAR_REGEX.test(key) || CONTROL_CHAR_REGEX.test(entry.value)) continue; sanitizedEntries[key] = { value: entry.value.trim() }; validCount++; } const safeBaggage = createBaggage(sanitizedEntries); return propagation.setBaggage(ctx, safeBaggage); } fields(): string[] { return this.delegate.fields(); } } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: OpenTelemetry Baggage Overflow Redaction & Cross-Tenant Injection Guardrail

OpenTelemetry Baggage Overflow Redaction & Cross-Tenant Injection Guardrail defines an industrial-grade behavioral standard for devops engineering. Enforces strict boundary filtering, size constraints, and key allowlisting on OpenTelemetry Baggage and TraceContext propagation across distributed service mesh hops. Prevents tenant context spoofing, memory exhaustion via unbounded header propagation, and downstream metric storage cardinality explosion. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Unsanitized W3C Baggage Ingestion and Metric Tag Forwarding Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Unsanitized W3C Baggage Ingestion and Metric Tag Forwarding	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS implement a custom or wrapped `TextMapPropagator` that validates and redacts incoming Baggage headers against an explicit, immutable allowlist before committing them to the active `Context`.

Engineers MUST ALWAYS implement a custom or wrapped `TextMapPropagator` that validates and redacts incoming Baggage headers against an explicit, immutable allowlist before committing them to the active `Context`.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.