OpenTelemetry Semantic Conventions Dynamic PII Redaction & Deterministic Exemplar Standard

--- description: OpenTelemetry Semantic Conventions Dynamic PII Redaction & Deterministic Exemplar Standard - Enforces synchronous edge-level PII token masking and deterministic log-trace exemplar correlation across the telemetry pipeline before long-term persistent storage. Mandates that all OpenTelemetry SpanProcessors and LogRecordProcessors sanitize high-cardinality and sensitive identifiers into pseudonymous HMAC hashes or redacted tokens while deterministically attaching active TraceId/SpanId exemplars to metric histograms. globs: src/telemetry/**/*.{ts,js}, src/observability/**/*.{ts,js}, src/instrumentation/**/*.{ts,js}, src/middleware/**/*.{ts,js} alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/otel-semantic-pii-redaction-exemplar-standard" --- # OpenTelemetry Semantic Conventions Dynamic PII Redaction & Deterministic Exemplar Standard > **Severity**: CRITICAL | **Category**: devops > **Anti-Pattern Warning**: Leaking Plaintext PII in Telemetry Attributes and Emitting Metrics Without Correlated Trace Exemplars ## System Constraints You are a Principal Software Architect enforcing OpenTelemetry Semantic Conventions, dynamic PII redaction, and deterministic metric exemplar linking. You must reject any code that directly records raw user emails, unhashed credit card/phone data, bearer tokens, or sensitive headers to OpenTelemetry spans, logs, or metrics. Ensure all Spans and LogRecords pass through synchronous sanitizing processors that apply cryptographically salted HMAC hashes or token masks, strictly adhere to OTEL semantic convention namespaces (e.g., `user.id`, `http.request.header.*`), and couple metric bucket updates to active distributed trace exemplars. ## Strict Guidelines - Engineers MUST ALWAYS sanitize attributes synchronously inside an OpenTelemetry `SpanProcessor` or `LogRecordProcessor` before passing spans/logs to exporters. - Engineers MUST NOT record sensitive semantic attributes (e.g., `enduser.id`, `http.request.header.authorization`, `db.statement` with raw literals) in plaintext. - Code MUST use salted cryptographic HMAC-SHA256 for pseudo-anonymizing high-cardinality identifiers that require analytical correlation across spans. - Services MUST configure OpenTelemetry metric instruments with an `ExemplarFilter.ALWAYS_ON` or `ExemplarFilter.TRACE_BASED` and an `AlignedHistogramBucketExemplarReservoir` to guarantee deterministic log-to-trace navigation. - Components NEVER rely on downstream async batch log scrubbers to clean telemetry data; redaction MUST occur in-process at the edge. ## Bad vs Good Implementations ### Edge-Level Synchronous PII Redaction & Trace-Metric Exemplar Registration #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Directly sets raw email, credit card numbers, and authorization tokens into span attributes violating data privacy standards. Furthermore, the metric is recorded without explicit exemplar association or trace correlation context, making deep telemetry debugging fragmented and compliance non-compliant. ```typescript import { trace, metrics } from '@opentelemetry/api'; const tracer = trace.getTracer('payment-service'); const meter = metrics.getMeter('payment-service'); const latencyHistogram = meter.createHistogram('payment_latency_ms'); export function handlePayment(req: any) { const span = tracer.startSpan('processPayment'); // VIO-01: Exposing plaintext PII and sensitive tokens in span attributes span.setAttribute('user.email', req.body.email); span.setAttribute('credit_card', req.body.cardNumber); span.setAttribute('authorization', req.headers.authorization); const startTime = Date.now(); try { // Execute payment logic... const duration = Date.now() - startTime; // VIO-02: Recording metric without active exemplar trace context linkage latencyHistogram.record(duration, { status: 'success' }); } finally { span.end(); } } ``` #### Production Standard (Correct) > **Standard**: Enforces synchronous edge-level token masking and cryptographic HMAC pseudonymization for PII identifiers according to semantic conventions. Employs `tracer.startActiveSpan` with `context.active()` to guarantee deterministic metric-to-trace exemplar correlation. ```typescript import { trace, metrics, context, Span, ReadableSpan, SpanProcessor } from '@opentelemetry/api'; import { createHmac } from 'crypto'; const SALT = process.env.TELEMETRY_HMAC_SALT || 'default-deterministic-telemetry-salt'; export function hashPiiIdentifier(value: string): string { return createHmac('sha256', SALT).update(value).digest('hex'); } export class DynamicPiiSanitizingSpanProcessor implements SpanProcessor { private sensitiveKeys = new Set(['authorization', 'credit_card', 'password', 'token']); onStart(span: Span): void { // Synchronous interception on span initialization } onEnd(span: ReadableSpan): void { const attributes = span.attributes; for (const key of Object.keys(attributes)) { const lower = key.toLowerCase(); if (this.sensitiveKeys.has(lower) || lower.includes('token') || lower.includes('secret')) { (attributes as Record<string, unknown>)[key] = '[REDACTED]'; } } } shutdown(): Promise<void> { return Promise.resolve(); } forceFlush(): Promise<void> { return Promise.resolve(); } } const tracer = trace.getTracer('payment-service'); const meter = metrics.getMeter('payment-service'); const latencyHistogram = meter.createHistogram('payment_latency_ms', { description: 'Payment latency with trace exemplar correlation' }); export function handlePayment(req: { body: { email: string; cardNumber: string }; headers: { authorization: string } }) { return tracer.startActiveSpan('processPayment', (span) => { try { // Sanitize conformant to OpenTelemetry Semantic Conventions span.setAttribute('user.id_hash', hashPiiIdentifier(req.body.email)); span.setAttribute('http.request.header.authorization', '[REDACTED]'); const startTime = performance.now(); // Execute payment logic... const duration = performance.now() - startTime; // TraceContext is deterministically bound as an exemplar during record call latencyHistogram.record(duration, { 'payment.provider': 'stripe' }, context.active()); } finally { span.end(); } }); } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: OpenTelemetry Semantic Conventions Dynamic PII Redaction & Deterministic Exemplar Standard

OpenTelemetry Semantic Conventions Dynamic PII Redaction & Deterministic Exemplar Standard defines an industrial-grade behavioral standard for devops engineering. Enforces synchronous edge-level PII token masking and deterministic log-trace exemplar correlation across the telemetry pipeline before long-term persistent storage. Mandates that all OpenTelemetry SpanProcessors and LogRecordProcessors sanitize high-cardinality and sensitive identifiers into pseudonymous HMAC hashes or redacted tokens while deterministically attaching active TraceId/SpanId exemplars to metric histograms. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Leaking Plaintext PII in Telemetry Attributes and Emitting Metrics Without Correlated Trace Exemplars Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Leaking Plaintext PII in Telemetry Attributes and Emitting Metrics Without Correlated Trace Exemplars	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS sanitize attributes synchronously inside an OpenTelemetry `SpanProcessor` or `LogRecordProcessor` before passing spans/logs to exporters.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.