OpenTelemetry Distributed Context Preservation and Baggage Scoping Guardrail

--- description: OpenTelemetry Distributed Context Preservation and Baggage Scoping Guardrail - Enforces strict distributed trace context propagation and sanitization across asynchronous task boundaries, worker threads, and message brokers. Prevents broken trace parent-child hierarchies and eliminates security vulnerabilities stemming from leaking sensitive credentials or high-cardinality state through uncontained W3C Baggage headers. globs: src/**/*.ts, src/**/*.js, src/queues/**/*.ts, src/workers/**/*.ts, src/telemetry/**/*.ts alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/opentelemetry-async-context-propagation-baggage-scoping" --- # OpenTelemetry Distributed Context Preservation and Baggage Scoping Guardrail > **Severity**: CRITICAL | **Category**: devops > **Anti-Pattern Warning**: Context Detachment Across Decoupled Pipelines and Unscoped Credential Leakage via W3C Baggage ## System Constraints You are a Principal Observability and Systems Reliability Architect enforcing OpenTelemetry context propagation standards. You must ensure distributed trace context is explicitly propagated across asynchronous boundaries, message queues, and threads using the OpenTelemetry Context and Propagation API. Furthermore, you must prevent sensitive credentials, PII, and high-cardinality data from entering W3C Baggage carriers. ## Strict Guidelines - Engineers MUST ALWAYS explicitly inject trace context into message carrier metadata before enqueueing messages or dispatching background jobs using `propagation.inject(context.active(), carrier)`. - Engineers MUST ALWAYS extract incoming trace context from received messages using `propagation.extract(ROOT_CONTEXT, carrier)` and execute subsequent operations inside `context.with(extractedContext, ...)`. - Engineers MUST NOT store credentials, auth tokens, session secrets, or PII inside OpenTelemetry `baggage`, as W3C Baggage automatically propagates unencrypted across external and downstream service boundaries. - Services MUST limit baggage entries to strict, low-cardinality tenant or routing metadata and enforce strict allowlisting prior to downstream injection. - Workers and consumers MUST NEVER create root spans for consumer tasks without attaching the extracted parent span context; doing so severs distributed traces into disconnected fragments. - Async operations crossing thread or event-loop boundaries SHOULD bind execution callbacks using `context.bind(context.active(), callback)` when native AsyncLocalStorage propagation cannot be guaranteed. ## Bad vs Good Implementations ### Decoupled Queue Producer and Consumer Context Propagation with Baggage Sanitization #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: The dispatching service creates a message without serializing the active OpenTelemetry context into the job metadata, causing trace detachment. Concurrently, sensitive authorization tokens are transmitted alongside job payloads, risking downstream leakage. The consumer initializes a disconnected root span, destroying end-to-end distributed transaction visibility. ```typescript import { trace } from '@opentelemetry/api'; import { Queue } from 'bullmq'; const orderQueue = new Queue('order-processing'); const tracer = trace.getTracer('order-service'); export async function dispatchOrder(orderId: string, authToken: string) { return tracer.startActiveSpan('dispatchOrder', async (span) => { // ANTI-PATTERN: Leaking auth tokens into W3C baggage (propagates unencrypted downstream) // ANTI-PATTERN: Not injecting trace context into message headers/attributes await orderQueue.add('process', { orderId, authToken, createdAt: Date.now() }); span.end(); }); } // Consumer Worker export async function handleOrderJob(job: { data: { orderId: string } }) { // ANTI-PATTERN: Creating a trace without extracting incoming context results in orphaned root span return tracer.startActiveSpan('handleOrderJob', async (span) => { console.log(`Processing order: ${job.data.orderId}`); span.end(); }); } ``` #### Production Standard (Correct) > **Standard**: Trace context is serialized into a carrier record using OpenTelemetry's standard `propagation.inject` and deserialized using `propagation.extract(ROOT_CONTEXT, carrier)`. Consumer span execution is wrapped in `context.with(...)` with `SpanKind.CONSUMER` to establish a direct child relationship across the asynchronous boundary. Baggage is strictly constrained to non-sensitive identifiers like `tenant.id`. ```typescript import { context, propagation, ROOT_CONTEXT, trace, SpanKind } from '@opentelemetry/api'; import { Queue } from 'bullmq'; const orderQueue = new Queue('order-processing'); const tracer = trace.getTracer('order-service'); interface JobCarrier { traceparent?: string; tracestate?: string; baggage?: string; } export async function dispatchOrder(orderId: string, tenantId: string) { return tracer.startActiveSpan('dispatchOrder', { kind: SpanKind.PRODUCER }, async (span) => { try { const carrier: JobCarrier = {}; // SAFE BAGGAGE: Only inject non-sensitive routing/tenant metadata const currentBaggage = propagation.getBaggage(context.active()) || propagation.createBaggage(); const sanitizedBaggage = currentBaggage.setEntry('tenant.id', { value: tenantId }); const contextWithBaggage = propagation.setBaggage(context.active(), sanitizedBaggage); // Explicitly inject trace context into the job carrier headers propagation.inject(contextWithBaggage, carrier); await orderQueue.add('process', { orderId, telemetryMetadata: carrier, createdAt: Date.now() }); } finally { span.end(); } }); } // Consumer Worker export async function handleOrderJob(job: { data: { orderId: string; telemetryMetadata: JobCarrier } }) { // Extract incoming trace and baggage context from carrier const extractedContext = propagation.extract(ROOT_CONTEXT, job.data.telemetryMetadata); // Execute consumer span bound to the extracted parent context return context.with(extractedContext, () => { return tracer.startActiveSpan('handleOrderJob', { kind: SpanKind.CONSUMER }, async (span) => { try { const tenantId = propagation.getBaggage(context.active())?.getEntry('tenant.id')?.value; span.setAttribute('order.id', job.data.orderId); if (tenantId) span.setAttribute('tenant.id', tenantId); // Proceed with processing within preserved distributed trace } finally { span.end(); } }); }); } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: OpenTelemetry Distributed Context Preservation and Baggage Scoping Guardrail

OpenTelemetry Distributed Context Preservation and Baggage Scoping Guardrail defines an industrial-grade behavioral standard for devops engineering. Enforces strict distributed trace context propagation and sanitization across asynchronous task boundaries, worker threads, and message brokers. Prevents broken trace parent-child hierarchies and eliminates security vulnerabilities stemming from leaking sensitive credentials or high-cardinality state through uncontained W3C Baggage headers. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Context Detachment Across Decoupled Pipelines and Unscoped Credential Leakage via W3C Baggage Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Context Detachment Across Decoupled Pipelines and Unscoped Credential Leakage via W3C Baggage	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS explicitly inject trace context into message carrier metadata before enqueueing messages or dispatching background jobs using `propagation.inject(context.active(), carrier)`.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.