OpenTelemetry W3C Baggage Multi-Tenant Boundary Sanitization and Cryptographic Verification

--- description: OpenTelemetry W3C Baggage Multi-Tenant Boundary Sanitization and Cryptographic Verification - Enforces boundary-level sanitization and HMAC signing of W3C baggage headers at ingress gateways to prevent cross-tenant trace spoofing and telemetry privilege escalation. Downstream services must discard unverified contextual baggage and strictly accept tenant-scoped attributes extracted from authenticated identity claims. Prevents unauthorized tenant isolation bypasses occurring through distributed context propagation mechanisms. globs: src/gateway/**/*.{ts,js}, src/middleware/**/*.{ts,js}, src/telemetry/**/*.{ts,js}, src/observability/**/*.{ts,js} alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/otel-w3c-baggage-tenant-boundary-sanitization" --- # OpenTelemetry W3C Baggage Multi-Tenant Boundary Sanitization and Cryptographic Verification > **Severity**: CRITICAL | **Category**: devops > **Anti-Pattern Warning**: Blind Ingress Propagation of Untrusted W3C Baggage Headers Without Cryptographic Authentication ## System Constraints You are a Principal Software Architect enforcing OpenTelemetry W3C Baggage Multi-Tenant Boundary Sanitization. You must strictly mandate that all external ingress points strip incoming unauthenticated W3C baggage headers. Trace context baggage across multi-tenant boundaries must only be populated from verified authentication contexts and protected using cryptographic HMAC signatures to prevent tenant spoofing inside internal microservice calls. ## Strict Guidelines - Engineers MUST ALWAYS strip incoming raw 'baggage' HTTP headers at untrusted edge boundaries before invoking OpenTelemetry propagation extractors. - Engineers MUST populate multi-tenant baggage attributes ('tenant.id', 'org.id') exclusively from cryptographically verified identity tokens (e.g., validated JWT claims) after authentication. - Gateways MUST compute and append an HMAC-SHA256 signature header or signed baggage entry ('tenant.sig') covering tenant identity and trace identifiers to seal boundary context. - Internal downstream services MUST validate the cryptographic tenant signature prior to binding tenant baggage attributes to the active OpenTelemetry context. - Engineers MUST NOT allow client-supplied baggage keys matching protected system prefixes ('tenant.*', 'auth.*', 'sec.*') to bypass boundary filters. - Telemetry interceptors SHOULD drop or quarantine any request where baggage tenant context conflicts with payload or RPC-level tenant metadata. ## Bad vs Good Implementations ### Ingress Edge Baggage Propagation Sanitization #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: The edge gateway blindly extracts and activates incoming W3C baggage headers. An attacker can inject arbitrary baggage items such as 'tenant.id=victim' or elevated role metadata, which then propagates across internal microservices and risks context spoofing or cross-tenant data contamination in downstream logs, traces, and policy decisions. ```typescript import { Request, Response, NextFunction } from 'express'; import { propagation, context } from '@opentelemetry/api'; // VULNERABLE: Direct propagation of untrusted baggage headers from external clients export function openTelemetryIngressMiddleware(req: Request, res: Response, next: NextFunction): void { // Extracts baggage directly from client headers without sanitization or signature check const extractedContext = propagation.extract(context.active(), req.headers); // Untrusted client can send: 'baggage: tenant.id=tenant-victim,role=superadmin' context.with(extractedContext, () => { next(); }); } ``` #### Production Standard (Correct) > **Standard**: Incoming untrusted baggage headers are completely purged at the edge boundary. Only traceparent is retained for distributed context continuation. The gateway injects verified tenant attributes strictly derived from authentication claims alongside an HMAC-SHA256 signature binding the tenant identity to the trace ID, guaranteeing integrity across downstream microservice calls. ```typescript import { Request, Response, NextFunction } from 'express'; import { propagation, context, trace } from '@opentelemetry/api'; import { createHmac } from 'crypto'; interface AuthenticatedUser { tenantId: string; userId: string; } const BAGGAGE_SIGNING_KEY = process.env.OTEL_BAGGAGE_HMAC_SECRET!; const PROTECTED_PREFIXES = ['tenant.', 'auth.', 'sec.']; function signTenantContext(tenantId: string, traceId: string): string { return createHmac('sha256', BAGGAGE_SIGNING_KEY) .update(`${tenantId}:${traceId}`) .digest('hex'); } export function secureBaggageGatewayMiddleware(req: Request, res: Response, next: NextFunction): void { // 1. Strip raw untrusted incoming baggage header to prevent tenant spoofing delete req.headers['baggage']; // 2. Extract traceparent only (distributed trace propagation remains valid) const baseContext = propagation.extract(context.active(), req.headers); const activeSpan = trace.getSpan(baseContext); const traceId = activeSpan ? activeSpan.spanContext().traceId : 'isolated'; // 3. Obtain verified tenant identity from validated authentication layer const authUser = (req as Request & { user?: AuthenticatedUser }).user; if (!authUser) { res.status(401).json({ error: 'Unauthenticated context at ingress boundary' }); return; } // 4. Synthesize cryptographically sealed tenant baggage const signature = signTenantContext(authUser.tenantId, traceId); const sealedBaggage = propagation.createBaggage({ 'tenant.id': { value: authUser.tenantId }, 'tenant.user_id': { value: authUser.userId }, 'tenant.sig': { value: signature } }); const secureContext = propagation.setBaggage(baseContext, sealedBaggage); // 5. Inject sanitized and signed context for downstream hops const outgoingHeaders: Record<string, string> = {}; propagation.inject(secureContext, outgoingHeaders); req.headers['baggage'] = outgoingHeaders['baggage']; context.with(secureContext, () => { next(); }); } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: OpenTelemetry W3C Baggage Multi-Tenant Boundary Sanitization and Cryptographic Verification

OpenTelemetry W3C Baggage Multi-Tenant Boundary Sanitization and Cryptographic Verification defines an industrial-grade behavioral standard for devops engineering. Enforces boundary-level sanitization and HMAC signing of W3C baggage headers at ingress gateways to prevent cross-tenant trace spoofing and telemetry privilege escalation. Downstream services must discard unverified contextual baggage and strictly accept tenant-scoped attributes extracted from authenticated identity claims. Prevents unauthorized tenant isolation bypasses occurring through distributed context propagation mechanisms. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Blind Ingress Propagation of Untrusted W3C Baggage Headers Without Cryptographic Authentication Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Blind Ingress Propagation of Untrusted W3C Baggage Headers Without Cryptographic Authentication	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS strip incoming raw 'baggage' HTTP headers at untrusted edge boundaries before invoking OpenTelemetry propagation extractors.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.