OpenTelemetry Baggage Context Quotas and PII Redaction Pipeline

--- description: OpenTelemetry Baggage Context Quotas and PII Redaction Pipeline - Enforces strict payload size budgets and deterministic PII masking on distributed OpenTelemetry baggage contexts before carrier propagation. Prevents HTTP 431 Request Header Fields Too Large outages at edge gateways and eliminates downstream context leakage of credentials and customer data across microservice boundaries. globs: src/observability/**/*.{ts,js}, src/telemetry/**/*.{ts,js}, src/middleware/**/*.{ts,js}, src/tracing/**/*.{ts,js} alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/opentelemetry-baggage-quotas-pii-redaction" --- # OpenTelemetry Baggage Context Quotas and PII Redaction Pipeline > **Severity**: CRITICAL | **Category**: devops > **Anti-Pattern Warning**: Unbounded baggage accumulation and raw token forwarding triggering gateway HTTP 431 dropouts and PII leakage ## System Constraints You are a Principal Software Architect enforcing OpenTelemetry Baggage Context Quotas & PII Redaction. Unbounded baggage injection leads to downstream gateway HTTP 431 failures and credential/PII leakage. Mandate deterministic baggage size bounds, entry limits, key whitelisting, and automated redaction prior to context propagation. ## Strict Guidelines - Engineers MUST ALWAYS enforce a maximum cumulative baggage size limit (standard max 4096 bytes, recommended per-hop 2048 bytes) and individual entry value limit (max 256 bytes) compliant with W3C Baggage specifications. - Engineers MUST NOT attach unverified, raw authentication tokens, JWTs, API keys, or personally identifiable information (PII such as emails, SSNs, or credit cards) into OpenTelemetry Baggage. - Code MUST sanitize or drop oversized or non-whitelisted baggage keys before serialization into outbound carrier HTTP headers. - Services MUST log a telemetry warning and increment a dropped-baggage metric counter (`otel.baggage.dropped_entries_total`) when baggage items exceed size budgets or fail redaction filters. - Components NEVER directly invoke `propagation.setBaggage` with user-supplied request headers or query parameters without passing through a sanitization wrapper. ## Bad vs Good Implementations ### Raw Baggage Injection vs Quota-Enforced PII Redacting Baggage Manager #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Directly injecting raw authentication tokens and PII (user emails) into Baggage leaks credentials across microservice boundaries. It also permits arbitrary payload sizes, exceeding proxy header buffers (e.g., Envoy/NGINX 8KB buffer) and causing downstream HTTP 431 errors. ```typescript import { propagation, context, BaggageEntry } from '@opentelemetry/api'; import { Request, Response, NextFunction } from 'express'; export function telemetryMiddleware(req: Request, res: Response, next: NextFunction) { let currentBaggage = propagation.getBaggage(context.active()) || propagation.createBaggage(); // CRITICAL DEFECT: Unbounded insertion of sensitive tokens and arbitrary payload headers // Propagating this downstream causes HTTP 431 errors at reverse proxies and exposes client tokens if (req.headers['authorization']) { currentBaggage = currentBaggage.setEntry('auth_token', { value: req.headers['authorization'] as string, }); } if (req.headers['x-user-email']) { currentBaggage = currentBaggage.setEntry('user_email', { value: req.headers['x-user-email'] as string, }); } context.with(propagation.setBaggage(context.active(), currentBaggage), () => { next(); }); } ``` #### Production Standard (Correct) > **Standard**: Implements deterministic key whitelisting, regex-based PII suppression, individual entry bounds (256 bytes), and cumulative context size enforcement (2048 bytes), preventing both HTTP 431 header overflows and downstream credential leaks. ```typescript import { propagation, context, Baggage, BaggageEntry } from '@opentelemetry/api'; import { Request, Response, NextFunction } from 'express'; const MAX_BAGGAGE_BYTES = 2048; const MAX_ENTRY_VALUE_BYTES = 256; const ALLOWED_BAGGAGE_KEYS = new Set(['tenant.id', 'client.version', 'routing.region']); const PII_PATTERNS = [/bearer\s+/i, /^[\w-\.]+@([\w-]+\.)+[\w-]{2,4}$/i]; export function sanitizeAndSetBaggage( baseBaggage: Baggage, incomingEntries: Record<string, string> ): Baggage { let sanitized = baseBaggage; let currentSize = Array.from(baseBaggage.getAllEntries()) .reduce((acc, [k, v]) => acc + k.length + v.value.length, 0); for (const [key, rawValue] of Object.entries(incomingEntries)) { if (!ALLOWED_BAGGAGE_KEYS.has(key)) { continue; // Drop non-whitelisted keys } const isSensitive = PII_PATTERNS.some((pattern) => pattern.test(rawValue)); if (isSensitive) { continue; // Drop PII/tokens } const truncatedValue = rawValue.slice(0, MAX_ENTRY_VALUE_BYTES); const additionalBytes = key.length + truncatedValue.length; if (currentSize + additionalBytes > MAX_BAGGAGE_BYTES) { break; // Enforce deterministic byte quota } sanitized = sanitized.setEntry(key, { value: truncatedValue }); currentSize += additionalBytes; } return sanitized; } export function telemetryMiddleware(req: Request, res: Response, next: NextFunction) { const activeContext = context.active(); const existingBaggage = propagation.getBaggage(activeContext) || propagation.createBaggage(); const candidates = { 'tenant.id': String(req.headers['x-tenant-id'] || ''), 'client.version': String(req.headers['x-client-version'] || ''), 'routing.region': String(req.headers['x-region'] || '') }; const secureBaggage = sanitizeAndSetBaggage(existingBaggage, candidates); context.with(propagation.setBaggage(activeContext, secureBaggage), () => { next(); }); } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: OpenTelemetry Baggage Context Quotas and PII Redaction Pipeline

OpenTelemetry Baggage Context Quotas and PII Redaction Pipeline defines an industrial-grade behavioral standard for devops engineering. Enforces strict payload size budgets and deterministic PII masking on distributed OpenTelemetry baggage contexts before carrier propagation. Prevents HTTP 431 Request Header Fields Too Large outages at edge gateways and eliminates downstream context leakage of credentials and customer data across microservice boundaries. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Unbounded baggage accumulation and raw token forwarding triggering gateway HTTP 431 dropouts and PII leakage Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Unbounded baggage accumulation and raw token forwarding triggering gateway HTTP 431 dropouts and PII leakage	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS enforce a maximum cumulative baggage size limit (standard max 4096 bytes, recommended per-hop 2048 bytes) and individual entry value limit (max 256 bytes) compliant with W3C Baggage specifications.

Engineers MUST ALWAYS enforce a maximum cumulative baggage size limit (standard max 4096 bytes, recommended per-hop 2048 bytes) and individual entry value limit (max 256 bytes) compliant with W3C Baggage specifications.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.