Cryptographic HMAC Provenance Verification for OTel TraceState and Baggage

--- description: Cryptographic HMAC Provenance Verification for OTel TraceState and Baggage - Enforces cryptographic HMAC signing and strict provenance verification for W3C TraceState and Baggage headers at untrusted network perimeters. Prevents trace context spoofing, cross-tenant telemetry contamination, and privilege escalation via injected context baggage across service boundaries. Ensures distributed trace contexts are authenticated before downstream propagation or authorization decisions. globs: src/middleware/**/*.{ts,js}, src/tracing/**/*.{ts,js}, src/server/**/*.{ts,js}, gateway/**/*.{ts,js} alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/otel-tracestate-hmac-provenance-invariant" --- # Cryptographic HMAC Provenance Verification for OTel TraceState and Baggage > **Severity**: CRITICAL | **Category**: devops > **Anti-Pattern Warning**: Blind Propagation of Untrusted TraceState and Baggage Ingress Headers ## System Constraints You are a Principal Security and Telemetry Architect enforcing strict cryptographic provenance on OpenTelemetry W3C distributed trace context headers. Untrusted clients and ingress boundaries routinely attempt to inject spoofed baggage items (e.g., tenant_id, role, internal_routing) and forged tracestate tokens. You must ensure that all incoming trace context across untrusted boundaries either has its cryptographic HMAC verified using constant-time comparison or is stripped and re-anchored. Never allow unverified baggage to populate authorization context, multi-tenant scoping, or internal telemetry state. ## Strict Guidelines - Edge gateways and ingress middlewares MUST NEVER trust or propagate raw inbound W3C `baggage` headers from untrusted network zones without cryptographically validating an accompanying HMAC signature. - Ingress services MUST verify the HMAC token in the `tracestate` header using a constant-time comparison algorithm (`crypto.timingSafeEqual`) to prevent timing side-channel vulnerabilities. - When HMAC verification fails or is absent at an untrusted perimeter, services MUST drop the tainted `tracestate` and `baggage` items, generate a new root span context, and log a security audit event. - Internal microservices MUST sign the canonical serialization of `(trace_id, span_id, tenant_id, timestamp)` before mutating or forwarding `tracestate` to downstream services. - Services MUST NOT read sensitive authorization attributes, user roles, or tenant bindings directly from OpenTelemetry baggage without independent cryptographically signed proof. ## Bad vs Good Implementations ### Untrusted Baggage Extraction vs. Cryptographic HMAC Ingress Guardrail #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: The middleware extracts external headers directly using standard OTel propagation without authenticating the source. An attacker can forge tenant identifiers, authorization levels, or routing metadata inside the W3C Baggage and TraceState headers, escalating privileges and corrupting downstream traces. ```typescript import { Request, Response, NextFunction } from 'express'; import { propagation, context } from '@opentelemetry/api'; export function ingressTraceContextMiddleware(req: Request, res: Response, next: NextFunction): void { // VULNERABILITY: Directly extracts unverified traceparent and baggage from untrusted client // An attacker can inject 'baggage: tenantId=corp-victim,role=superadmin' const activeContext = propagation.extract(context.active(), req.headers); context.with(activeContext, () => { // Application code later treats extracted baggage as authoritative tenant context next(); }); } ``` #### Production Standard (Correct) > **Standard**: Validates the cryptographic HMAC signature embedded in the W3C TraceState header before accepting inbound context. If the signature is missing or fails constant-time verification, the middleware safely discards the tainted trace context and baggage, preventing spoofing and privilege escalation. ```typescript import { Request, Response, NextFunction } from 'express'; import { propagation, context, ROOT_CONTEXT, trace } from '@opentelemetry/api'; import { createHmac, timingSafeEqual } from 'crypto'; const TRACESTATE_KEY_PREFIX = 'acme_sig'; const HMAC_SECRET = process.env.OTEL_TRACESTATE_HMAC_SECRET!; function verifyTraceStateProvenance(traceId: string, tenantId: string, signatureHex: string): boolean { if (!signatureHex || !traceId || !tenantId) return false; const expected = createHmac('sha256', HMAC_SECRET) .update(`${traceId}:${tenantId}`) .digest(); const actual = Buffer.from(signatureHex, 'hex'); return actual.length === expected.length && timingSafeEqual(actual, expected); } export function secureIngressTraceMiddleware(req: Request, res: Response, next: NextFunction): void { const rawTraceParent = req.headers['traceparent'] as string | undefined; const rawTraceState = req.headers['tracestate'] as string | undefined; const untrustedTenant = req.headers['x-tenant-id'] as string | undefined; // Extract HMAC signature token from W3C tracestate (vendor key format: acme_sig=<hex>) const sigMatch = rawTraceState ? new RegExp(`${TRACESTATE_KEY_PREFIX}=([^,;]+)`).exec(rawTraceState) : null; const providedSig = sigMatch ? sigMatch[1] : ''; let activeCtx = ROOT_CONTEXT; if (rawTraceParent && untrustedTenant && verifyTraceStateProvenance(rawTraceParent.split('-')[1], untrustedTenant, providedSig)) { // Provenance cryptographically verified: extract context safely activeCtx = propagation.extract(ROOT_CONTEXT, req.headers); } else { // Provenance failed or absent: purge untrusted baggage & isolate downstream trace context const newSpan = trace.getTracer('ingress-gate').startSpan('untrusted-ingress-boundary'); activeCtx = trace.setSpan(ROOT_CONTEXT, newSpan); // Explicitly sanitise inbound baggage req.headers['baggage'] = undefined; } context.with(activeCtx, () => next()); } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: Cryptographic HMAC Provenance Verification for OTel TraceState and Baggage

Cryptographic HMAC Provenance Verification for OTel TraceState and Baggage defines an industrial-grade behavioral standard for devops engineering. Enforces cryptographic HMAC signing and strict provenance verification for W3C TraceState and Baggage headers at untrusted network perimeters. Prevents trace context spoofing, cross-tenant telemetry contamination, and privilege escalation via injected context baggage across service boundaries. Ensures distributed trace contexts are authenticated before downstream propagation or authorization decisions. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Blind Propagation of Untrusted TraceState and Baggage Ingress Headers Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Blind Propagation of Untrusted TraceState and Baggage Ingress Headers	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Edge gateways and ingress middlewares MUST NEVER trust or propagate raw inbound W3C `baggage` headers from untrusted network zones without cryptographically validating an accompanying HMAC signature.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.