OpenTelemetry Asynchronous Span Link Provenance Invariant

--- description: OpenTelemetry Asynchronous Span Link Provenance Invariant - Enforces strict decoupling of asynchronous message consumer traces from producer execution contexts using causal Span Links rather than child-of parent contexts. Prevents unbounded distributed trace DAG explosion, cascading latency reporting skew, and trace visualization collapse in fan-out/fan-in and message batching event-driven pipelines. globs: src/consumers/**/*.{ts,js,py,go,java}, src/events/**/*.{ts,js,py,go,java}, src/messaging/**/*.{ts,js,py,go,java}, src/workers/**/*.{ts,js,py,go,java}, src/jobs/**/*.{ts,js,py,go,java} alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/opentelemetry-span-link-provenance-invariant" --- # OpenTelemetry Asynchronous Span Link Provenance Invariant > **Severity**: CRITICAL | **Category**: devops > **Anti-Pattern Warning**: Subordinating Asynchronous Message Consumer Execution to Producer Parent Trace Contexts ## System Constraints You are a Principal Observability and Systems Architect enforcing the OpenTelemetry Asynchronous Span Link Provenance Invariant. In event-driven, queue-based, and stream-processing architectures (Kafka, SQS, RabbitMQ, BullMQ), consumers must NEVER inherit message trace contexts as their parent span. Consumers must instantiate a new root execution span and record producer contexts exclusively through OpenTelemetry Span Links to prevent unbounded distributed trace DAGs. ## Strict Guidelines - Engineers MUST ALWAYS initialize asynchronous consumer and batch processing spans as new root spans (ROOT_CONTEXT) or autonomous worker spans. - Engineers MUST extract producer context from message carrier headers (W3C TraceContext) and bind it strictly via OpenTelemetry `links` on the consumer span. - Engineers MUST NOT pass extracted message contexts into `trace.setSpanContext()` or `tracer.startActiveSpan(name, { context: extractedContext })` when processing decoupled jobs or events. - Services processing batches MUST attach one causal Link per aggregated message item rather than nesting processing operations under an arbitrary single message's parent context. - Components NEVER represent asynchronous eventual consistency workflows as synchronous child spans of the publishing HTTP/RPC request. ## Bad vs Good Implementations ### Decoupling Kafka/Queue Consumer Context via Span Links in TypeScript #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Adopting the extracted message trace context as the active execution parent makes the consumer an in-tree descendant of the producer. In decoupled event architectures, this inflates root trace duration by hours or days, merges thousands of consumer runs into a single unrenderable distributed trace graph, and breaks latency telemetry. ```typescript import { trace, context, defaultTextMapGetter, ROOT_CONTEXT } from '@opentelemetry/api'; async function handleMessage(message: { headers: Record<string, string>; payload: unknown }) { // Anti-Pattern: Extracting producer context and setting it as active parent const extractedContext = propagation.extract(ROOT_CONTEXT, message.headers, defaultTextMapGetter); // Flaw: Consumer span becomes a direct child of the producer span. // Causes unbounded DAG explosion, infinite trace durations, and UI collapse in fan-out pipelines. return context.with(extractedContext, async () => { const tracer = trace.getTracer('event-consumer'); const span = tracer.startSpan('process-message'); try { await processPayload(message.payload); } finally { span.end(); } }); } ``` #### Production Standard (Correct) > **Standard**: The consumer initiates a fresh span rooted at ROOT_CONTEXT with SpanKind.CONSUMER and explicitly registers the extracted producer context via the `links` property. This maintains full causal provenance and distributed queryability while preserving bounded trace DAG boundaries. ```typescript import { trace, propagation, defaultTextMapGetter, ROOT_CONTEXT, SpanKind, Link } from '@opentelemetry/api'; async function handleMessage(message: { headers: Record<string, string>; payload: unknown }) { const tracer = trace.getTracer('event-consumer'); // Extract producer context strictly to construct a causal link const extractedContext = propagation.extract(ROOT_CONTEXT, message.headers, defaultTextMapGetter); const linkedSpanContext = trace.getSpan(extractedContext)?.spanContext(); const links: Link[] = linkedSpanContext ? [{ context: linkedSpanContext }] : []; // Invariant: Consumer initiates its own execution context decoupled from producer context // Causal relationship is preserved through Span Links without DAG explosion const consumerSpan = tracer.startSpan('process-message', { kind: SpanKind.CONSUMER, links, }, ROOT_CONTEXT); return trace.withSpan(consumerSpan, async () => { try { await processPayload(message.payload); } catch (error) { consumerSpan.recordException(error as Error); throw error; } finally { consumerSpan.end(); } }); } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: OpenTelemetry Asynchronous Span Link Provenance Invariant

OpenTelemetry Asynchronous Span Link Provenance Invariant defines an industrial-grade behavioral standard for devops engineering. Enforces strict decoupling of asynchronous message consumer traces from producer execution contexts using causal Span Links rather than child-of parent contexts. Prevents unbounded distributed trace DAG explosion, cascading latency reporting skew, and trace visualization collapse in fan-out/fan-in and message batching event-driven pipelines. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Subordinating Asynchronous Message Consumer Execution to Producer Parent Trace Contexts Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Subordinating Asynchronous Message Consumer Execution to Producer Parent Trace Contexts	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS initialize asynchronous consumer and batch processing spans as new root spans (ROOT_CONTEXT) or autonomous worker spans.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.