OpenTelemetry GenAI Semantic Convention Streaming Token Attribution Standard

--- description: OpenTelemetry GenAI Semantic Convention Streaming Token Attribution Standard - Enforces strict compliance with OpenTelemetry GenAI semantic conventions across chunked LLM response streams by mandating monotonic token increment attribution and per-chunk latency metrics. Guarantees real-time token bucket deduction and runaway streaming execution termination before gateway buffering cascades into unrecoverable financial or compute exhaustion. globs: src/gateway/**/*.{ts,js}, src/telemetry/**/*.{ts,js}, src/services/llm/**/*.{ts,js}, src/middleware/**/*.{ts,js} alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/otel-genai-streaming-token-attribution-standard" --- # OpenTelemetry GenAI Semantic Convention Streaming Token Attribution Standard > **Severity**: CRITICAL | **Category**: devops > **Anti-Pattern Warning**: Deferred End-of-Stream Telemetry and Batch Token Attribution ## System Constraints You are a Principal Engineer enforcing the OpenTelemetry GenAI Semantic Convention Streaming Token Attribution Standard. You must ensure that all chunked LLM streams record time-to-first-token (TTFT), incremental completion token metrics (`gen_ai.client.token.usage`), and span events per chunk using standardized OpenTelemetry GenAI attributes. You must reject any architecture that aggregates token metrics only upon stream completion, as this blinds rate limiters and token buckets to runaway streams. ## Strict Guidelines - Engineers MUST emit standard OpenTelemetry GenAI span attributes (`gen_ai.system`, `gen_ai.request.model`, `gen_ai.response.model`) on the root inference span at creation. - Engineers MUST track and attribute Time To First Token (TTFT) via a distinct span event or duration metric (`gen_ai.client.operation.duration` with phase markers) on the arrival of the initial non-empty stream chunk. - Engineers MUST attribute incremental or running completion tokens monotonically during the stream lifecycle to trigger dynamic budget checks, terminating the stream via AbortController if quota limits are exceeded. - Engineers MUST NOT wait until stream termination or consumption of the final chunk to record `gen_ai.client.token.usage` or deduct from shared Redis/in-memory token buckets. - Services MUST capture final token usage via `gen_ai.usage.output_tokens` and `gen_ai.usage.input_tokens` attributes when the stream closes or aborts, ensuring trace completeness. - Components NEVER drop active OpenTelemetry context across asynchronous iterable boundaries; active context MUST be propagated to chunk processors. ## Bad vs Good Implementations ### Real-time streaming token attribution with OTel semantic conventions and early quota abort #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Telemetry attributes and token budget accounting are deferred until the stream is completely consumed. If a runaway generation occurs or an attacker keeps the stream alive, the rate limiter never receives token deductions in real-time, TTFT is lost, and OTel GenAI semantic attributes are not updated. ```typescript import { trace } from '@opentelemetry/api'; const tracer = trace.getTracer('llm-gateway'); export async function* handleStreamingInference(providerStream: AsyncIterable<{ text: string; tokens?: number }>, clientBudget: number) { const span = tracer.startSpan('gen_ai.chat'); let accumulatedText = ''; let totalTokens = 0; // ANTI-PATTERN: Streaming chunks blind to OpenTelemetry and rate limiters until the stream finishes. for await (const chunk of providerStream) { accumulatedText += chunk.text; totalTokens += chunk.tokens ?? 1; yield chunk.text; } // If the client aborted or the stream ran infinitely, telemetry and rate-limiting were bypassed entirely. span.setAttribute('gen_ai.usage.completion_tokens', totalTokens); span.end(); } ``` #### Production Standard (Correct) > **Standard**: Adheres strictly to OpenTelemetry GenAI semantic conventions by attaching `gen_ai.system` and model attributes at span creation, instrumenting Time-to-First-Token (TTFT) on initial chunk emission, updating `gen_ai.usage.output_tokens` monotonically, and deducting against real-time quotas on every chunk to terminate runaway LLM generations immediately. ```typescript import { trace, SpanStatusCode, context } from '@opentelemetry/api'; import { SEMRESATTRS_SERVICE_NAME } from '@opentelemetry/semantic-conventions'; const tracer = trace.getTracer('llm-gateway'); interface StreamChunk { text: string; tokens?: number; } interface TokenQuotaEnforcer { consumeRealtime(count: number): Promise<boolean>; } export async function* handleStreamingInference( providerStream: AsyncIterable<StreamChunk>, quotaEnforcer: TokenQuotaEnforcer, modelName: string, abortController: AbortController ): AsyncGenerator<string, void, unknown> { return await tracer.startActiveSpan('gen_ai.client.chat', { attributes: { 'gen_ai.system': 'openai', 'gen_ai.request.model': modelName, 'gen_ai.operation.name': 'chat' } }, async function* (span) { let isFirstChunk = true; const startTime = performance.now(); let accumulatedCompletionTokens = 0; try { for await (const chunk of providerStream) { if (isFirstChunk) { const ttftMs = performance.now() - startTime; span.addEvent('gen_ai.client.time_to_first_token', { 'gen_ai.response.ttft_ms': ttftMs }); isFirstChunk = false; } const chunkTokens = chunk.tokens ?? Math.max(1, Math.ceil(chunk.text.length / 4)); accumulatedCompletionTokens += chunkTokens; // Monotonic telemetry update adhering to streaming conventions span.setAttribute('gen_ai.usage.output_tokens', accumulatedCompletionTokens); // In-line quota deduction to prevent runaway token spend const allowed = await quotaEnforcer.consumeRealtime(chunkTokens); if (!allowed) { span.recordException(new Error('GenAI streaming token quota exceeded. Terminating execution.')); span.setStatus({ code: SpanStatusCode.ERROR, message: 'QuotaExceeded' }); abortController.abort(); throw new Error('Streaming execution throttled: Token quota exceeded.'); } yield chunk.text; } span.setStatus({ code: SpanStatusCode.OK }); } catch (err: any) { span.recordException(err); span.setStatus({ code: SpanStatusCode.ERROR, message: err.message }); throw err; } finally { span.setAttribute('gen_ai.usage.output_tokens', accumulatedCompletionTokens); span.end(); } }); } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: OpenTelemetry GenAI Semantic Convention Streaming Token Attribution Standard

OpenTelemetry GenAI Semantic Convention Streaming Token Attribution Standard defines an industrial-grade behavioral standard for devops engineering. Enforces strict compliance with OpenTelemetry GenAI semantic conventions across chunked LLM response streams by mandating monotonic token increment attribution and per-chunk latency metrics. Guarantees real-time token bucket deduction and runaway streaming execution termination before gateway buffering cascades into unrecoverable financial or compute exhaustion. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Deferred End-of-Stream Telemetry and Batch Token Attribution Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Deferred End-of-Stream Telemetry and Batch Token Attribution	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST emit standard OpenTelemetry GenAI span attributes (`gen_ai.system`, `gen_ai.request.model`, `gen_ai.response.model`) on the root inference span at creation.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.