OpenTelemetry GenAI Semantic Conventions Pre-Export PII Redaction Guardrail

--- description: "OpenTelemetry GenAI Semantic Conventions Pre-Export PII Redaction Guardrail - Enforces client-side redaction and structural token masking on OpenTelemetry GenAI semantic convention attributes prior to buffer commitment and remote export. Prohibits persisting raw user prompts, model completions, and system messages containing sensitive PII, PHI, or credentials into span attributes or span events." globs: "src/**/*.ts, src/**/*.js, instrumentation/**/*.ts, lib/telemetry/**/*.ts" alwaysApply: true license: "CC-BY-4.0" source: "https://agentrules.org/rules/opentelemetry-genai-pii-redaction-invariant" --- # OpenTelemetry GenAI Semantic Conventions Pre-Export PII Redaction Guardrail > **Severity**: CRITICAL | **Category**: devops > **Anti-Pattern Warning**: Attaching Raw Unsanitized LLM Prompts and Completions Directly to OpenTelemetry Span Attributes ## System Constraints You are a Principal Software Architect enforcing zero-leakage telemetry invariants under OpenTelemetry GenAI Semantic Conventions. All LLM inputs, outputs, messages, and function parameters mapped to attributes such as `gen_ai.prompt`, `gen_ai.completion`, `gen_ai.system`, and `gen_ai.content.prompt` MUST pass through an in-process, deterministic redaction and tokenization pipeline prior to span attribute assignment. Never defer PII/PHI redaction solely to downstream OpenTelemetry Collectors, as collector failures, misconfigurations, and buffer drops can leak plaintext sensitive data into unencrypted logs or transit layers. ## Strict Guidelines - Engineers MUST ALWAYS execute in-process PII/PHI and secret sanitization on prompt and completion payloads before passing them to `span.setAttribute()` or `span.addEvent()`. - Engineers MUST NOT rely exclusively on downstream OpenTelemetry Collector processors (e.g., `redactionprocessor`, `filterprocessor`) for sensitive data scrubbing. - Code MUST sanitize structured conversational payloads, masking email addresses, social security numbers, credit cards, API keys, and HIPAA identifiers with deterministic surrogates like `[REDACTED_EMAIL]` or `[REDACTED_SECRET]`. - Services MUST implement custom `SpanProcessor` hooks or wrapping tracer clients that intercept semantic attributes matching `gen_ai.*` and enforce sanitization at the boundary. - Components NEVER attach raw system prompts or user message histories to span attributes if they contain unmasked user identity markers or session-scoped authorization tokens. ## Bad vs Good Implementations ### Intercepting and Redacting GenAI Attributes at Span Creation Boundary #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Raw user prompt and completion data are mapped directly to 'gen_ai.prompt' and 'gen_ai.completion' span attributes without client-side sanitization. Sensitive data (SSN, emails, tokens) is immediately captured in the trace buffer and exported over the wire. ```typescript import { trace, Span } from '@opentelemetry/api'; const tracer = trace.getTracer('ai-inference-service'); interface ChatRequest { model: string; prompt: string; // May contain user SSN, email, or credentials } export async function recordAndExecuteInference(req: ChatRequest): Promise<string> { return tracer.startActiveSpan('gen_ai.client.chat', async (span: Span) => { // VIOLATION: Raw unredacted user prompt and completion exposed to telemetry pipeline span.setAttribute('gen_ai.system', 'openai'); span.setAttribute('gen_ai.request.model', req.model); span.setAttribute('gen_ai.prompt', req.prompt); const completion = await callUpstreamLLM(req.model, req.prompt); span.setAttribute('gen_ai.completion', completion); span.end(); return completion; }); } async function callUpstreamLLM(model: string, prompt: string): Promise<string> { return 'LLM generated output with sensitive details'; } ``` #### Production Standard (Correct) > **Standard**: Production invariant compliant implementation ensuring memory safety, deterministic execution, and zero runtime hallucination. ```typescript import { trace, Span, SpanStatusCode } from '@opentelemetry/api'; const tracer = trace.getTracer('ai-inference-service'); interface ChatRequest { model: string; prompt: string; } // Deterministic pre-export masking engine export class TelemetryRedactor { private static readonly EMAIL_REGEX = /[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+/g; private static readonly SSN_REGEX = /\b(?!000|666|9\d{2})\d{3}-(?!00)\d{2}-(?!0000)\d{4}\b/g; private static readonly API_KEY_REGEX = /(?:sk-[a-zA-Z0-9]{32,}|ghp_[a-zA-Z0-9]{36})/g; public static redact(text: string): string { if (!text) return text; return text .replace(this.EMAIL_REGEX, '[REDACTED_EMAIL]') .replace(this.SSN_REGEX, '[REDACTED_SSN]') .replace(this.API_KEY_REGEX, '[REDACTED_SECRET]'); } } export async function recordAndExecuteInference(req: ChatRequest): Promise<string> { return tracer.startActiveSpan('gen_ai.client.chat', async (span: Span) => { try { span.setAttribute('gen_ai.system', 'openai'); span.setAttribute('gen_ai.request.model', req.model); // INVARIANT: Sanitization at span boundary before buffer serialization const sanitizedPrompt = TelemetryRedactor.redact(req.prompt); span.setAttribute('gen_ai.prompt', sanitizedPrompt); const completion = await callUpstreamLLM(req.model, req.prompt); const sanitizedCompletion = TelemetryRedactor.redact(completion); span.setAttribute('gen_ai.completion', sanitizedCompletion); span.setStatus({ code: SpanStatusCode.OK }); return completion; } catch (error) { span.setStatus({ code: SpanStatusCode.ERROR, message: error instanceof Error ? TelemetryRedactor.redact(error.message) : 'Unknown error' }); throw error; } finally { span.end(); } }); } async function callUpstreamLLM(model: string, prompt: string): Promise<string> { return 'LLM generated output with [email protected]'; } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: OpenTelemetry GenAI Semantic Conventions Pre-Export PII Redaction Guardrail

OpenTelemetry GenAI Semantic Conventions Pre-Export PII Redaction Guardrail defines an industrial-grade behavioral standard for devops engineering. Enforces client-side redaction and structural token masking on OpenTelemetry GenAI semantic convention attributes prior to buffer commitment and remote export. Prohibits persisting raw user prompts, model completions, and system messages containing sensitive PII, PHI, or credentials into span attributes or span events. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Attaching Raw Unsanitized LLM Prompts and Completions Directly to OpenTelemetry Span Attributes Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Attaching Raw Unsanitized LLM Prompts and Completions Directly to OpenTelemetry Span Attributes	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS execute in-process PII/PHI and secret sanitization on prompt and completion payloads before passing them to `span.setAttribute()` or `span.addEvent()`.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.