OpenTelemetry W3C Baggage Ingestion Cardinality & PII Scrubber Standard

--- description: OpenTelemetry W3C Baggage Ingestion Cardinality & PII Scrubber Standard - Prevents distributed tracing memory bloat, cardinality explosion, and regulatory data leakage by strictly validating and sanitizing W3C Baggage headers at ingress boundaries. All untrusted external baggage must be filtered through an explicit allowlist and size boundary before propagation into internal execution contexts. globs: src/middleware/**/*.{ts,js}, src/gateway/**/*.{ts,js}, src/telemetry/**/*.{ts,js}, src/tracing/**/*.{ts,js}, gateway/**/*.ts alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/opentelemetry-w3c-baggage-cardinality-pii-scrubber-standard" --- # OpenTelemetry W3C Baggage Ingestion Cardinality & PII Scrubber Standard > **Severity**: CRITICAL | **Category**: devops > **Anti-Pattern Warning**: Blind Ingestion and Unbounded Propagation of Untrusted W3C Baggage Headers ## System Constraints You are a Principal Software Architect enforcing production safety for distributed observability. You must mandate that untrusted W3C Baggage (`baggage` header) received at edge proxies, reverse proxies, and API gateways is NEVER forwarded or extracted verbatim into the OpenTelemetry context. Enforce explicit allowlists, key-count caps, maximum entry byte length limits, and strict PII pattern matching before allowing baggage to cross the boundary into downstream microservices. ## Strict Guidelines - Edge gateways MUST NEVER extract unvalidated, raw W3C baggage headers directly into active tracing contexts without passing them through an allowlist validator. - Engineers MUST configure an explicit allowlist of recognized baggage keys (e.g., `tenant_id`, `request_tier`) with strict regex validation for values. - Gateways MUST enforce a strict total baggage size limit (e.g., maximum 8192 bytes total per RFC 7230/W3C Baggage recommendations) and an internal maximum of 5 allowed keys. - Baggage values MUST be evaluated against PII patterns (emails, JWT formats, bearer tokens, credit card structures) and immediately dropped if matched. - Internal propagators MUST reconstruct a clean, newly instantiated `Baggage` object via `propagation.createBaggage()` containing only sanitized pairs, dropping all unknown metadata and properties. - Components NEVER preserve arbitrary baggage keys forwarded by external clients to downstream RPCs, Kafka headers, or HTTP calls. ## Bad vs Good Implementations ### Sanitizing and Allowlisting W3C Baggage at Gateway Middleware #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Directly extracting propagation headers without sanitizing the baggage header exposes downstream collectors and traces to memory exhaustion attacks (cardinality explosion) and propagates sensitive PII throughout the entire microservice fleet. ```typescript import { Request, Response, NextFunction } from 'express'; import { context, propagation } from '@opentelemetry/api'; export function telemetryMiddleware(req: Request, res: Response, next: NextFunction): void { // CRITICAL VULNERABILITY: Blindly extracts untrusted external baggage. // Allows attackers to inject unbounded keys (DoS/memory bloat), // high-cardinality values, and PII (tokens, emails) into downstream traces. const activeContext = propagation.extract(context.active(), req.headers); context.with(activeContext, () => { next(); }); } ``` #### Production Standard (Correct) > **Standard**: Validates every incoming baggage entry against a strict allowlist, enforces maximum key/value byte limits, drops suspected PII or bearer tokens, and guarantees maximum entry cardinality before establishing the trace context. ```typescript import { Request, Response, NextFunction } from 'express'; import { context, propagation, trace } from '@opentelemetry/api'; const ALLOWED_BAGGAGE_KEYS = new Set(['tenant_id', 'client_tier', 'datacenter']); const MAX_KEY_LENGTH = 64; const MAX_VALUE_LENGTH = 128; const MAX_BAGGAGE_ENTRIES = 5; const PII_OR_TOKEN_REGEX = /(@|bearer|ey[A-Za-z0-9-_]{10,})/i; export function telemetryMiddleware(req: Request, res: Response, next: NextFunction): void { // 1. Extract raw context (TraceParent/TraceState) const extractedContext = propagation.extract(context.active(), req.headers); const rawBaggage = propagation.getBaggage(extractedContext); // 2. Construct sanitized, bounded Baggage container let sanitizedBaggage = propagation.createBaggage(); let entryCount = 0; if (rawBaggage) { for (const [key, entry] of rawBaggage.getAllEntries()) { if (entryCount >= MAX_BAGGAGE_ENTRIES) break; const isValidKey = ALLOWED_BAGGAGE_KEYS.has(key) && key.length <= MAX_KEY_LENGTH; const isValidValue = entry.value.length <= MAX_VALUE_LENGTH && !PII_OR_TOKEN_REGEX.test(entry.value); if (isValidKey && isValidValue) { sanitizedBaggage = sanitizedBaggage.setEntry(key, { value: entry.value }); entryCount++; } } } // 3. Clear untrusted baggage from context and bind strictly sanitized baggage const cleanContext = propagation.setBaggage(extractedContext, sanitizedBaggage); context.with(cleanContext, () => { next(); }); } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: OpenTelemetry W3C Baggage Ingestion Cardinality & PII Scrubber Standard

OpenTelemetry W3C Baggage Ingestion Cardinality & PII Scrubber Standard defines an industrial-grade behavioral standard for devops engineering. Prevents distributed tracing memory bloat, cardinality explosion, and regulatory data leakage by strictly validating and sanitizing W3C Baggage headers at ingress boundaries. All untrusted external baggage must be filtered through an explicit allowlist and size boundary before propagation into internal execution contexts. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Blind Ingestion and Unbounded Propagation of Untrusted W3C Baggage Headers Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Blind Ingestion and Unbounded Propagation of Untrusted W3C Baggage Headers	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Edge gateways MUST NEVER extract unvalidated, raw W3C baggage headers directly into active tracing contexts without passing them through an allowlist validator.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.