OpenTelemetry GenAI Semantic Streaming Span Boundaries and TTFT Instrumentation

--- description: OpenTelemetry GenAI Semantic Streaming Span Boundaries and TTFT Instrumentation - Standardizes OpenTelemetry GenAI semantic conventions across asynchronous streaming LLM completions and multi-agent pipelines. Mandates strict lifecycle span encapsulation with deterministic TTFT (time-to-first-token) event attribution, token usage recording, and inline zero-allocation PII scrubbing prior to attribute export. globs: src/observability/**/*.ts, src/agents/**/*.ts, src/llm/**/*.ts, src/instrumentation/**/*.ts alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/otel-genai-streaming-spans-boundary-standard" --- # OpenTelemetry GenAI Semantic Streaming Span Boundaries and TTFT Instrumentation > **Severity**: CRITICAL | **Category**: devops > **Anti-Pattern Warning**: Unbounded LLM Streaming Spans with Leaked Tokens, Missing TTFT Events, and Raw PII Export ## System Constraints You are a Principal Observability and Infrastructure Engineer enforcing OpenTelemetry GenAI Semantic Conventions (v1.27.0+) across distributed LLM agent streaming interfaces. You mandate precise span lifecycle boundaries, distinct time-to-first-token (TTFT) span events, deterministic token accumulation, stream error capturing, and pre-export PII redaction. Never allow un-redacted streaming chunks to persist into telemetry attributes or stream termination to occur outside of span scope. ## Strict Guidelines - Engineers MUST ALWAYS wrap the entire asynchronous iterable stream within an active OpenTelemetry span having span kind SpanKind.CLIENT or SpanKind.INTERNAL conforming to 'gen_ai.system' semantic conventions. - Engineers MUST record an explicit span event named 'gen_ai.stream.first_token' immediately upon consumption of the first non-empty text chunk to establish high-fidelity TTFT metrics. - Engineers MUST sanitize streaming tokens through an in-memory redaction engine (scrubbing PII/PHI patterns like SSNs, bearer tokens, emails) before aggregating text into span attributes or stream-completion events. - Engineers MUST record final usage metrics ('gen_ai.usage.input_tokens', 'gen_ai.usage.output_tokens') and set span status to StatusCode.OK or StatusCode.ERROR strictly within a finally block wrapping stream resolution. - Components NEVER attach chunk-level raw payload objects directly to span attributes; telemetry MUST aggregate counts and redact full outputs before attaching to 'gen_ai.completion'. ## Bad vs Good Implementations ### Naive Streaming Pipeline with Missing TTFT, Raw PII, and Leaked Span Context #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: The span begins before invocation but fails to handle consumer aborts or iteration errors inside a try/finally block, leaking spans. It emits a high-cardinality, un-redacted event for every chunk instead of standard 'gen_ai.stream.first_token', and exposes un-sanitized PII directly in telemetry attributes. ```typescript import { trace } from '@opentelemetry/api'; const tracer = trace.getTracer('llm-service'); export async function* streamLlmCompletion(prompt: string, llmClient: any) { const span = tracer.startSpan('llm.stream'); // ANTI-PATTERN: Prompt logged raw without PII scrub; span opened before iteration span.setAttribute('prompt', prompt); const stream = await llmClient.chat.completions.create({ model: 'gpt-4o', messages: [{ role: 'user', content: prompt }], stream: true, }); // ANTI-PATTERN: If the consumer terminates iteration early or throws, span leaks // ANTI-PATTERN: No TTFT event recorded on first chunk for await (const chunk of stream) { const text = chunk.choices[0]?.delta?.content || ''; span.addEvent('chunk', { text }); // High cardinality & leaks PII per token yield text; } span.end(); } ``` #### Production Standard (Correct) > **Standard**: Wraps generator execution cleanly in startActiveSpan with SpanKind.CLIENT. Employs a defensive try/catch/finally to guarantee span closure on generator cancellation, captures 'gen_ai.stream.first_token' once for TTFT calculation, accumulates output safely, and runs full-string PII redaction before attaching to span attributes. ```typescript import { trace, SpanStatusCode, SpanKind } from '@opentelemetry/api'; import { sanitizePII } from '../security/sanitizer'; const tracer = trace.getTracer('genai-agent-chain', '1.0.0'); export async function* streamLlmCompletion( prompt: string, llmClient: { chat: { completions: { create: Function } } } ): AsyncGenerator<string, void, unknown> { const sanitizedInput = sanitizePII(prompt); return yield* tracer.startActiveSpan( 'chat gpt-4o', { kind: SpanKind.CLIENT, attributes: { 'gen_ai.system': 'openai', 'gen_ai.request.model': 'gpt-4o', 'gen_ai.prompt': sanitizedInput, }, }, async function* (span) { let firstTokenReceived = false; let totalCompletion = ''; let outputTokenEstimate = 0; try { const responseStream = await llmClient.chat.completions.create({ model: 'gpt-4o', messages: [{ role: 'user', content: sanitizedInput }], stream: true, stream_options: { include_usage: true }, }); for await (const chunk of responseStream) { const content = chunk.choices?.[0]?.delta?.content ?? ''; if (!firstTokenReceived && content.length > 0) { firstTokenReceived = true; span.addEvent('gen_ai.stream.first_token', { 'gen_ai.stream.ttft_ms': Date.now() - (span as any).startTime?.[0] || 0, }); } if (content.length > 0) { totalCompletion += content; outputTokenEstimate += 1; yield content; } if (chunk.usage) { span.setAttribute('gen_ai.usage.input_tokens', chunk.usage.prompt_tokens); span.setAttribute('gen_ai.usage.output_tokens', chunk.usage.completion_tokens); } } span.setAttribute('gen_ai.completion', sanitizePII(totalCompletion)); if (!span.spanContext().traceFlags) { span.setAttribute('gen_ai.usage.output_tokens', outputTokenEstimate); } span.setStatus({ code: SpanStatusCode.OK }); } catch (error) { span.recordException(error instanceof Error ? error : new Error(String(error))); span.setStatus({ code: SpanStatusCode.ERROR, message: error instanceof Error ? error.message : 'LLM stream error', }); throw error; } finally { span.end(); } } ); } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: OpenTelemetry GenAI Semantic Streaming Span Boundaries and TTFT Instrumentation

OpenTelemetry GenAI Semantic Streaming Span Boundaries and TTFT Instrumentation defines an industrial-grade behavioral standard for devops engineering. Standardizes OpenTelemetry GenAI semantic conventions across asynchronous streaming LLM completions and multi-agent pipelines. Mandates strict lifecycle span encapsulation with deterministic TTFT (time-to-first-token) event attribution, token usage recording, and inline zero-allocation PII scrubbing prior to attribute export. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Unbounded LLM Streaming Spans with Leaked Tokens, Missing TTFT Events, and Raw PII Export Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Unbounded LLM Streaming Spans with Leaked Tokens, Missing TTFT Events, and Raw PII Export	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS wrap the entire asynchronous iterable stream within an active OpenTelemetry span having span kind SpanKind.CLIENT or SpanKind.INTERNAL conforming to 'gen_ai.system' semantic conventions.

Engineers MUST ALWAYS wrap the entire asynchronous iterable stream within an active OpenTelemetry span having span kind SpanKind.CLIENT or SpanKind.INTERNAL conforming to 'gen_ai.system' semantic conventions.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.