OpenTelemetry GenAI Semantic Convention and Non-Blocking Stream Attribution Standard

--- description: OpenTelemetry GenAI Semantic Convention and Non-Blocking Stream Attribution Standard - Enforces OpenTelemetry GenAI semantic conventions across multi-agent orchestrations, mandating deterministic capture of model provenance, execution context, and token usage. Requires stream wrappers using non-buffering async generator taps to record usage metrics and complete parent/child spans without inducing I/O backpressure or latency spikes. globs: src/observability/**/*.ts, src/telemetry/**/*.ts, src/agents/**/*.ts, src/llm/**/*.ts, src/orchestration/**/*.ts alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/otel-genai-semantic-conventions-production-standard" --- # OpenTelemetry GenAI Semantic Convention and Non-Blocking Stream Attribution Standard > **Severity**: CRITICAL | **Category**: devops > **Anti-Pattern Warning**: Buffering LLM output streams in memory or omitting GenAI semantic attributes leads to latency bottlenecks and untraceable inference costs. ## System Constraints You are a Principal Software Architect enforcing OpenTelemetry GenAI Semantic Conventions across all LLM multi-agent pipelines. You mandate strict adherence to standard attribute naming (gen_ai.system, gen_ai.request.model, gen_ai.response.model, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, and gen_ai.operation.name). You must reject any telemetry design that buffers stream chunks in memory for post-hoc token counting, alters downstream stream iteration latency, or drops context propagation between multi-agent distributed spans. ## Strict Guidelines - Engineers MUST ALWAYS populate standard GenAI semantic attributes (`gen_ai.system`, `gen_ai.request.model`, `gen_ai.operation.name`) prior to invoking provider APIs. - Engineers MUST wrap streaming responses in non-buffering AsyncIterable pass-through generators that incrementally accumulate chunk metadata without delaying emission. - Engineers MUST NOT wait for entire streaming responses to buffer in memory before emitting span metrics or yielding the first token. - Services MUST propagate active trace and span contexts (`context.active()`, `trace.setSpan()`) across asynchronous boundaries, worker queues, and inter-agent task handoffs. - Code SHOULD record provider-reported token usage (`gen_ai.usage.input_tokens`, `gen_ai.usage.output_tokens`) directly from stream terminal events, or fall back to verified zero-allocation tokenizer estimates when streaming omit provider counts. - Components NEVER end a telemetry span prior to stream completion (`finally` block) or error dissemination (`span.recordException()`), ensuring spans reflect true end-to-end inference latency. ## Bad vs Good Implementations ### Streaming LLM span attribution with zero-buffering pass-through generator #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Blocks the entire consumer stream by collecting chunks synchronously in memory, destroying Time-To-First-Token (TTFT) gains. It also uses non-standard custom attributes ('tokens_estimated') instead of OpenTelemetry GenAI semantic conventions, making trace aggregation across multi-agent pipelines impossible. ```typescript import { trace } from '@opentelemetry/api'; import OpenAI from 'openai'; const tracer = trace.getTracer('llm-service'); const openai = new OpenAI(); export async function streamChatBad(prompt: string) { const span = tracer.startSpan('chat_call'); try { const response = await openai.chat.completions.create({ model: 'gpt-4o', messages: [{ role: 'user', content: prompt }], stream: true, }); // ANTI-PATTERN: Buffers entire stream to extract tokens before returning, // completely blocking downstream streaming I/O and using ad-hoc attribute keys. let fullText = ''; for await (const chunk of response) { fullText += chunk.choices[0]?.delta?.content || ''; } span.setAttribute('tokens_estimated', fullText.length / 4); span.end(); return fullText; } catch (err: any) { span.recordException(err); span.end(); throw err; } } ``` #### Production Standard (Correct) > **Standard**: Uses standard OpenTelemetry GenAI semantic attributes, injects the span into context, uses an async generator to stream chunks instantaneously to downstream consumers without backpressure, captures deterministic usage from stream metadata, and guarantees span completion in a finally block. ```typescript import { trace, SpanStatusCode, context } from '@opentelemetry/api'; import OpenAI from 'openai'; // Formal GenAI semantic conventions (v1.27.0+) const GEN_AI = { SYSTEM: 'gen_ai.system', OPERATION_NAME: 'gen_ai.operation.name', REQUEST_MODEL: 'gen_ai.request.model', RESPONSE_MODEL: 'gen_ai.response.model', INPUT_TOKENS: 'gen_ai.usage.input_tokens', OUTPUT_TOKENS: 'gen_ai.usage.output_tokens', } as const; const tracer = trace.getTracer('llm-service', '1.0.0'); const openai = new OpenAI(); export async function* streamChatGood(prompt: string): AsyncIterable<string> { const span = tracer.startSpan('chat gpt-4o', { attributes: { [GEN_AI.SYSTEM]: 'openai', [GEN_AI.OPERATION_NAME]: 'chat', [GEN_AI.REQUEST_MODEL]: 'gpt-4o', }, }); let outputTokens = 0; let responseModel = 'gpt-4o'; try { const response = await context.with(trace.setSpan(context.active(), span), () => openai.chat.completions.create({ model: 'gpt-4o', messages: [{ role: 'user', content: prompt }], stream: true, stream_options: { include_usage: true }, }) ); for await (const chunk of response) { if (chunk.model) responseModel = chunk.model; if (chunk.usage) { span.setAttribute(GEN_AI.INPUT_TOKENS, chunk.usage.prompt_tokens); span.setAttribute(GEN_AI.OUTPUT_TOKENS, chunk.usage.completion_tokens); } const delta = chunk.choices[0]?.delta?.content; if (delta) { outputTokens += 1; // Fallback metric if usage isn't provided yield delta; // Non-blocking: immediately emitted to consumer } } span.setAttribute(GEN_AI.RESPONSE_MODEL, responseModel); span.setStatus({ code: SpanStatusCode.OK }); } catch (error) { span.recordException(error as Error); span.setStatus({ code: SpanStatusCode.ERROR, message: (error as Error).message }); throw error; } finally { span.end(); } } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: OpenTelemetry GenAI Semantic Convention and Non-Blocking Stream Attribution Standard

OpenTelemetry GenAI Semantic Convention and Non-Blocking Stream Attribution Standard defines an industrial-grade behavioral standard for devops engineering. Enforces OpenTelemetry GenAI semantic conventions across multi-agent orchestrations, mandating deterministic capture of model provenance, execution context, and token usage. Requires stream wrappers using non-buffering async generator taps to record usage metrics and complete parent/child spans without inducing I/O backpressure or latency spikes. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Buffering LLM output streams in memory or omitting GenAI semantic attributes leads to latency bottlenecks and untraceable inference costs. Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Buffering LLM output streams in memory or omitting GenAI semantic attributes leads to latency bottlenecks and untraceable inference costs.	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS populate standard GenAI semantic attributes (`gen_ai.system`, `gen_ai.request.model`, `gen_ai.operation.name`) prior to invoking provider APIs.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.