OpenTelemetry GenAI In-Flight PII Redaction and TTFT Telemetry Standard

--- description: OpenTelemetry GenAI In-Flight PII Redaction and TTFT Telemetry Standard - Mandates that streaming GenAI telemetry pipelines perform synchronous, token-by-token PII masking and exact Time-To-First-Token (TTFT) attribute enrichment before span events or payloads are exported. Unredacted prompt/completion text and delayed metric instrumentation violate data compliance boundaries and degrade latency profiling. globs: src/telemetry/**/*.{ts,js}, src/ai/**/*.{ts,js}, src/services/**/*.{ts,js}, src/infrastructure/llm/**/*.{ts,js} alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/opentelemetry-genai-streaming-redaction-standard" --- # OpenTelemetry GenAI In-Flight PII Redaction and TTFT Telemetry Standard > **Severity**: CRITICAL | **Category**: devops > **Anti-Pattern Warning**: Exporting raw unredacted streaming token events or deferring TTFT calculation to stream completion ## System Constraints You are a Principal Observability and Security Architect enforcing the OpenTelemetry GenAI Streaming Semantic Redaction Standard. You ensure distributed tracing across LLM pipelines captures critical performance metrics (specifically Time-To-First-Token) and strictly scrubs sensitive data (PII/secrets) before emitting spans to OpenTelemetry collectors. ## Strict Guidelines - Engineers MUST ALWAYS capture and record `gen_ai.server.time_to_first_token` (in milliseconds) immediately upon arrival of the first non-empty text chunk from the upstream provider. - Engineers MUST sanitize and redact PII/credentials in-flight using a deterministic token/sliding-window sanitizer before mutating span attributes or dispatching span events. - Engineers MUST NOT emit raw, unmasked prompt inputs or streaming completion chunks to OpenTelemetry span events or attributes. - Code SHOULD adhere strictly to OpenTelemetry GenAI Semantic Conventions (e.g., `gen_ai.system`, `gen_ai.request.model`, `gen_ai.response.finish_reasons`). - Services MUST close and export spans within a bounded error boundary, ensuring partial stream failures record error status and recorded TTFT attributes without dropping telemetry. - Components NEVER buffer full streaming responses in memory solely for post-hoc redaction when real-time streaming to downstream consumers is active. ## Bad vs Good Implementations ### In-Flight Streaming Telemetry with PII Redaction and TTFT Tracking #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Raw prompt and streaming chunks containing potential PII (SSNs, API keys, credentials) are logged directly to the span. Furthermore, Time-To-First-Token (TTFT) is completely absent, blinding performance monitoring to streaming latency bottlenecks. ```typescript import { trace } from '@opentelemetry/api'; const tracer = trace.getTracer('llm-service'); export async function streamLLMResponse(prompt: string, streamSource: AsyncIterable<{ delta?: string }>) { return tracer.startActiveSpan('gen_ai.chat', async (span) => { // ANTI-PATTERN: Raw unredacted prompt set directly in span attributes span.setAttribute('gen_ai.prompt', prompt); let fullResponse = ''; for await (const chunk of streamSource) { if (chunk.delta) { fullResponse += chunk.delta; // ANTI-PATTERN: Emitting raw sensitive chunks as span events span.addEvent('gen_ai.content.chunk', { 'chunk.text': chunk.delta }); } } // ANTI-PATTERN: TTFT is missing; full response exported unredacted span.setAttribute('gen_ai.completion', fullResponse); span.end(); }); } ``` #### Production Standard (Correct) > **Standard**: Adheres to OpenTelemetry GenAI semantic conventions. Calculates high-resolution TTFT on the first non-empty chunk, continuously sanitizes chunks in-flight before emitting span events, and ensures trace spans safely capture errors within a finally block. ```typescript import { trace, SpanStatusCode } from '@opentelemetry/api'; import { hrTimeToMilliseconds } from '@opentelemetry/core'; const tracer = trace.getTracer('llm-service'); // Minimal example of an in-flight sliding window or regex-based sanitizer function redactPII(text: string): string { return text .replace(/\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,7}\b/g, '[REDACTED_EMAIL]') .replace(/\b(?:4[0-9]{12}(?:[0-9]{3})?|5[1-5][0-9]{14})\b/g, '[REDACTED_CC]') .replace(/\b\d{3}-\d{2}-\d{4}\b/g, '[REDACTED_SSN]'); } export async function* streamLLMResponse( prompt: string, model: string, streamSource: AsyncIterable<{ delta?: string }> ): AsyncGenerator<string, void, unknown> { const span = tracer.startSpan('gen_ai.chat', { attributes: { 'gen_ai.system': 'openai', 'gen_ai.request.model': model, 'gen_ai.prompt': redactPII(prompt), }, }); const startTime = process.hrtime(); let ttftRecorded = false; let completionBuffer = ''; try { for await (const chunk of streamSource) { const text = chunk.delta ?? ''; if (!text) continue; if (!ttftRecorded) { const elapsed = process.hrtime(startTime); const ttftMs = hrTimeToMilliseconds(elapsed); span.setAttribute('gen_ai.server.time_to_first_token', ttftMs); span.addEvent('gen_ai.first_token', { 'gen_ai.ttft_ms': ttftMs }); ttftRecorded = true; } const sanitized = redactPII(text); completionBuffer += sanitized; span.addEvent('gen_ai.content.chunk', { 'gen_ai.completion.chunk': sanitized, }); // Stream sanitized output immediately to consumer yield sanitized; } span.setAttribute('gen_ai.completion', completionBuffer); span.setAttribute('gen_ai.response.finish_reasons', ['stop']); span.setStatus({ code: SpanStatusCode.OK }); } catch (error) { span.recordException(error as Error); span.setStatus({ code: SpanStatusCode.ERROR, message: (error as Error).message, }); throw error; } finally { span.end(); } } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: OpenTelemetry GenAI In-Flight PII Redaction and TTFT Telemetry Standard

OpenTelemetry GenAI In-Flight PII Redaction and TTFT Telemetry Standard defines an industrial-grade behavioral standard for devops engineering. Mandates that streaming GenAI telemetry pipelines perform synchronous, token-by-token PII masking and exact Time-To-First-Token (TTFT) attribute enrichment before span events or payloads are exported. Unredacted prompt/completion text and delayed metric instrumentation violate data compliance boundaries and degrade latency profiling. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Exporting raw unredacted streaming token events or deferring TTFT calculation to stream completion Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Exporting raw unredacted streaming token events or deferring TTFT calculation to stream completion	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS capture and record `gen_ai.server.time_to_first_token` (in milliseconds) immediately upon arrival of the first non-empty text chunk from the upstream provider.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.