OpenTelemetry GenAI v1.28 Semantic Conventions Streaming Token Ledger

--- description: OpenTelemetry GenAI v1.28 Semantic Conventions Streaming Token Ledger - Enforces OpenTelemetry GenAI Semantic Conventions v1.28 for streaming LLM calls to guarantee exact-match token accounting, cross-service cost attribution, and prompt-injection security event telemetry. Mandates that spans remain open across the entire streaming lifecycle, accumulate token deltas, capture finish reasons, and populate standard 'gen_ai.usage.*' attributes on span completion. globs: src/**/*.ts, src/**/*.tsx, services/**/*.ts, packages/**/*.ts alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/otel-genai-streaming-token-ledger-production-standard" --- # OpenTelemetry GenAI v1.28 Semantic Conventions Streaming Token Ledger > **Severity**: CRITICAL | **Category**: devops > **Anti-Pattern Warning**: Premature Span Closure and Custom Ad-Hoc Attributes Discarding Streaming Token Metrics ## System Constraints You are a Principal Software Architect enforcing the OpenTelemetry GenAI Semantic Conventions (v1.28+). You must ensure that all generative AI integrations record standard telemetry attributes on client/internal spans. Under no circumstances should spans close prematurely during streaming responses or record vendor-proprietary, non-standard token usage keys. All streaming cycles must aggregate completion tokens, handle stream errors deterministically, log security/prompt-injection metadata, and record final token usage on the parent GenAI span. ## Strict Guidelines - Engineers MUST ALWAYS span LLM invocations using OpenTelemetry SpanKind.CLIENT with the standard operation attribute 'gen_ai.operation.name: chat' or 'text_completion'. - Engineers MUST NOT close the OpenTelemetry GenAI span upon receiving the initial HTTP/gRPC stream header; the span MUST remain active until the generator emits its terminal chunk or throws. - Services MUST populate standard GenAI attributes including 'gen_ai.system', 'gen_ai.request.model', 'gen_ai.usage.input_tokens', and 'gen_ai.usage.output_tokens' matching OpenTelemetry v1.28 specifications. - Components NEVER invent proprietary attribute keys (e.g., 'llm.prompt_tokens', 'ai.tokens.total'); they MUST strictly follow 'gen_ai.*' semantic conventions. - Code SHOULD record security events or prompt-injection guardrail verdicts using Span Events named 'gen_ai.content.filter' or span attributes indicating risk classifications. - Services MUST ensure that streaming generator errors record span exceptions with 'span.recordException(err)' and set status to 'SpanStatusCode.ERROR' before ending the span. ## Bad vs Good Implementations ### Enforcing OpenTelemetry v1.28 Streaming Token Ledger and Lifecycle #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: The span closes before the stream begins yielding chunks, dropping all streaming token ledger metrics and duration data. Additionally, non-standard attributes ('llm.prompt', 'vendor') violate OpenTelemetry GenAI semantic conventions, breaking cross-service FinOps dashboards and alerting. ```typescript import { trace } from '@opentelemetry/api'; import { OpenAI } from 'openai'; const tracer = trace.getTracer('llm-service'); const openai = new OpenAI(); export async function* streamCompletionBad(prompt: string) { // ANTI-PATTERN: Span closed immediately on stream initialization. // Custom non-standard telemetry keys used. Output tokens never accounted for. const span = tracer.startSpan('openai.chat'); span.setAttribute('llm.prompt', prompt); span.setAttribute('vendor', 'openai'); const stream = await openai.chat.completions.create({ model: 'gpt-4o', messages: [{ role: 'user', content: prompt }], stream: true, }); span.end(); // Premature closure! Tokens during streaming are completely unobserved. for await (const chunk of stream) { yield chunk.choices[0]?.delta?.content || ''; } } ``` #### Production Standard (Correct) > **Standard**: The span spans the entire stream generator lifecycle. It adheres strictly to OpenTelemetry GenAI semantic conventions v1.28, configures streaming usage options, captures finish reasons, correlates tenant context for billing, catches streaming disconnects, and safely terminates in a finally block. ```typescript import { trace, SpanStatusCode, SpanKind } from '@opentelemetry/api'; import { OpenAI } from 'openai'; const tracer = trace.getTracer('llm-service', '1.28.0'); const openai = new OpenAI(); export async function* streamCompletionGood( prompt: string, tenantId: string ): AsyncGenerator<string, void, unknown> { return await tracer.startActiveSpan( 'chat gpt-4o', { kind: SpanKind.CLIENT, attributes: { 'gen_ai.operation.name': 'chat', 'gen_ai.system': 'openai', 'gen_ai.request.model': 'gpt-4o', 'gen_ai.request.temperature': 0.7, 'tenant.id': tenantId } }, async function* (span) { let outputTokens = 0; let finishReason = 'unknown'; try { const stream = await openai.chat.completions.create({ model: 'gpt-4o', messages: [{ role: 'user', content: prompt }], stream: true, stream_options: { include_usage: true } }); for await (const chunk of stream) { if (chunk.usage) { span.setAttribute('gen_ai.usage.input_tokens', chunk.usage.prompt_tokens); span.setAttribute('gen_ai.usage.output_tokens', chunk.usage.completion_tokens); } const choice = chunk.choices[0]; if (choice?.finish_reason) { finishReason = choice.finish_reason; } const delta = choice?.delta?.content; if (delta) { outputTokens += 1; // Fallback heuristic if provider omits stream usage yield delta; } } span.setAttribute('gen_ai.response.finish_reasons', [finishReason]); if (!span.spanContext()) { span.setAttribute('gen_ai.usage.output_tokens', outputTokens); } span.setStatus({ code: SpanStatusCode.OK }); } catch (err: unknown) { span.recordException(err instanceof Error ? err : new Error(String(err))); span.setStatus({ code: SpanStatusCode.ERROR, message: (err as Error).message }); throw err; } finally { span.end(); } } ); } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: OpenTelemetry GenAI v1.28 Semantic Conventions Streaming Token Ledger

OpenTelemetry GenAI v1.28 Semantic Conventions Streaming Token Ledger defines an industrial-grade behavioral standard for devops engineering. Enforces OpenTelemetry GenAI Semantic Conventions v1.28 for streaming LLM calls to guarantee exact-match token accounting, cross-service cost attribution, and prompt-injection security event telemetry. Mandates that spans remain open across the entire streaming lifecycle, accumulate token deltas, capture finish reasons, and populate standard 'gen_ai.usage.*' attributes on span completion. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Premature Span Closure and Custom Ad-Hoc Attributes Discarding Streaming Token Metrics Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Premature Span Closure and Custom Ad-Hoc Attributes Discarding Streaming Token Metrics	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS span LLM invocations using OpenTelemetry SpanKind.CLIENT with the standard operation attribute 'gen_ai.operation.name

Engineers MUST ALWAYS span LLM invocations using OpenTelemetry SpanKind.CLIENT with the standard operation attribute 'gen_ai.operation.name: chat' or 'text_completion'.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.