Immutable W3C Baggage Tenant Context Propagation Across Microservices and Message Queues

--- description: Immutable W3C Baggage Tenant Context Propagation Across Microservices and Message Queues - Enforces strict extraction, validation, and propagation of tenant boundaries via W3C Baggage across synchronous RPC and asynchronous message brokers. Prohibits direct mutation of the active OpenTelemetry context and prevents downstream tenant pollution or cross-tenant log and trace leakage. Guarantees that tenant baggage entries are cryptographically validated or strictly verified at the network perimeter before crossing distributed trace contexts. globs: src/observability/**/*.ts, src/middleware/**/*.ts, src/messaging/**/*.ts, src/transport/**/*.ts, src/services/**/*.ts alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/otel-w3c-baggage-multitenant-boundary-propagation" --- # Immutable W3C Baggage Tenant Context Propagation Across Microservices and Message Queues > **Severity**: CRITICAL | **Category**: devops > **Anti-Pattern Warning**: Ad-Hoc Span Attribute Mutation and Unvalidated Asynchronous Trace Context Bleed Across Tenant Boundaries ## System Constraints You are a Principal Software Architect enforcing production invariants for multi-tenant tracing and context propagation using OpenTelemetry and W3C Baggage. You must mandate that every synchronous RPC boundary and asynchronous message producer/consumer strictly extracts, validates against an allowlist, seals, and propagates the immutable 'tenant.id' W3C baggage entry. Direct extraction without boundary sanitization, unvalidated propagation of user-controlled baggage headers, and cross-span context leaking across message queue event loops are absolute critical security and compliance violations. ## Strict Guidelines - Services MUST ALWAYS extract incoming W3C traceparent and baggage headers using OpenTelemetry's standard `propagation.extract()` API, explicitly wrapping execution in the extracted context via `context.with()`. - Services MUST validate the extracted `tenant.id` baggage against an authenticated security principal or cryptographically verified claims token before registering it into execution context. - Engineers MUST NOT manually set raw span attributes for tenant routing without binding them to the immutable W3C Baggage context, as downstream asynchronous workers will drop unpropagated span attributes. - Message producers MUST inject trace and baggage context into message headers using `propagation.inject()` and a registered carrier adapter before publishing to message brokers like Kafka, SQS, or RabbitMQ. - Message consumers MUST NEVER reuse the global trace context across batched messages; each individual message record MUST establish its own isolated context boundary. - Baggage entries MUST be validated against strict character set constraints (RFC 7230 tokens) and MUST NOT contain personally identifiable information (PII) or secrets. - Perimeter gateways MUST drop or overwrite incoming untrusted W3C baggage headers containing 'tenant.id' or 'tenant.*' keys if not signed or authenticated at the edge proxy. ## Bad vs Good Implementations ### Async Queue Producer/Consumer: Propagating Validated Baggage Context #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Span attributes do not propagate over network boundaries. The publisher fails to inject W3C baggage and trace context into the message metadata. The consumer does not extract context, resulting in broken distributed traces and missing tenant isolation downstream. ```typescript import { trace } from '@opentelemetry/api'; import { Message, Producer, Consumer } from 'some-queue-lib'; // Bad: Setting tenant on span only; does not propagate over the wire. // Producer drops trace context; Consumer leaks execution context between messages. export async function publishMessage(producer: Producer, payload: Record<string, unknown>, tenantId: string) { const tracer = trace.getTracer('billing-service'); const span = tracer.startSpan('publish_invoice'); span.setAttribute('tenant.id', tenantId); // Span attribute does NOT propagate through message headers! await producer.send({ value: JSON.stringify(payload) }); span.end(); } export async function consumeMessages(consumer: Consumer) { consumer.on('message', async (msg: Message) => { // Anti-pattern: No context extracted from headers. Downstream traces detached from tenant origin. const tracer = trace.getTracer('billing-service'); const span = tracer.startSpan('process_invoice'); await handleInvoice(JSON.parse(msg.value.toString())); span.end(); }); } ``` #### Production Standard (Correct) > **Standard**: Explicitly binds the validated tenant ID into W3C Baggage, injects headers on message production via TextMapSetter, and extracts isolated message-level context via TextMapGetter before downstream handling. ```typescript import { context, propagation, trace, ROOT_CONTEXT, TextMapGetter, TextMapSetter } from '@opentelemetry/api'; import { Message, Producer, Consumer } from 'some-queue-lib'; const TENANT_KEY = 'tenant.id'; const TENANT_REGEX = /^[a-zA-Z0-9-_]{3,64}$/; interface QueueHeaders { [key: string]: string | undefined; } const headerSetter: TextMapSetter<QueueHeaders> = { set(carrier, key, value) { carrier[key] = value; } }; const headerGetter: TextMapGetter<QueueHeaders> = { get(carrier, key) { return carrier ? carrier[key] : undefined; }, keys(carrier) { return carrier ? Object.keys(carrier) : []; } }; export async function publishMessage( producer: Producer, payload: Record<string, unknown>, verifiedTenantId: string ): Promise<void> { if (!TENANT_REGEX.test(verifiedTenantId)) { throw new Error(`Invalid tenant ID: ${verifiedTenantId}`); } const tracer = trace.getTracer('billing-service'); const tenantBaggage = propagation.createBaggage({ [TENANT_KEY]: { value: verifiedTenantId } }); const activeContext = propagation.setBaggage(context.active(), tenantBaggage); await context.with(activeContext, async () => { const span = tracer.startSpan('publish_invoice'); try { span.setAttribute(TENANT_KEY, verifiedTenantId); const headers: QueueHeaders = {}; propagation.inject(context.active(), headers, headerSetter); await producer.send({ headers, value: JSON.stringify(payload) }); } finally { span.end(); } }); } export async function consumeMessages(consumer: Consumer): Promise<void> { consumer.on('message', async (msg: Message) => { const headers = (msg.headers || {}) as QueueHeaders; const parentContext = propagation.extract(ROOT_CONTEXT, headers, headerGetter); const incomingBaggage = propagation.getBaggage(parentContext); const tenantId = incomingBaggage?.getEntry(TENANT_KEY)?.value; if (!tenantId || !TENANT_REGEX.test(tenantId)) { throw new Error('Tenant context missing or invalid in message headers'); } await context.with(parentContext, async () => { const tracer = trace.getTracer('billing-service'); const span = tracer.startSpan('process_invoice'); span.setAttribute(TENANT_KEY, tenantId); try { await handleInvoice(JSON.parse(msg.value.toString()), tenantId); } finally { span.end(); } }); }); } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: Immutable W3C Baggage Tenant Context Propagation Across Microservices and Message Queues

Immutable W3C Baggage Tenant Context Propagation Across Microservices and Message Queues defines an industrial-grade behavioral standard for devops engineering. Enforces strict extraction, validation, and propagation of tenant boundaries via W3C Baggage across synchronous RPC and asynchronous message brokers. Prohibits direct mutation of the active OpenTelemetry context and prevents downstream tenant pollution or cross-tenant log and trace leakage. Guarantees that tenant baggage entries are cryptographically validated or strictly verified at the network perimeter before crossing distributed trace contexts. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Ad-Hoc Span Attribute Mutation and Unvalidated Asynchronous Trace Context Bleed Across Tenant Boundaries Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Ad-Hoc Span Attribute Mutation and Unvalidated Asynchronous Trace Context Bleed Across Tenant Boundaries	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Services MUST ALWAYS extract incoming W3C traceparent and baggage headers using OpenTelemetry's standard `propagation.extract()` API, explicitly wrapping execution in the extracted context via `context.with()`.

Services MUST ALWAYS extract incoming W3C traceparent and baggage headers using OpenTelemetry's standard `propagation.extract()` API, explicitly wrapping execution in the extracted context via `context.with()`.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.