OpenTelemetry GenAI Semantic Conventions and PII Baggage Quarantine

--- description: OpenTelemetry GenAI Semantic Conventions and PII Baggage Quarantine - Enforces strict compliance with OpenTelemetry GenAI Semantic Conventions while establishing an active quarantine perimeter around W3C baggage and span attributes. Guarantees that multi-agent contextual correlation tokens persist across distributed service hops without leaking raw prompts, model responses, or personally identifiable information (PII) into telemetry backends or egress headers. globs: src/telemetry/**/*.{ts,js}, src/agents/**/*.{ts,js}, src/middleware/**/*.{ts,js}, src/llm/**/*.{ts,js} alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/opentelemetry-genai-baggage-quarantine" --- # OpenTelemetry GenAI Semantic Conventions and PII Baggage Quarantine > **Severity**: CRITICAL | **Category**: devops > **Anti-Pattern Warning**: Indiscriminate Telemetry Baggage Injection and Raw Prompt Span Pollution Leaking Sensitive Data Across Service Hops ## System Constraints You are a Principal Software Architect enforcing OpenTelemetry GenAI Semantic Conventions (v1.26.0+) and zero-leakage baggage quarantine policies across autonomous multi-agent microservice architectures. You must mandate that LLM operations exclusively populate standard `gen_ai.*` span attributes and that baggage propagation enforces strict schema validation and sanitization filters. Any attempt to serialize raw conversational content, user personal credentials, or unredacted prompts into W3C baggage headers or persistent span attributes must be unconditionally blocked and flagged. ## Strict Guidelines - Engineers MUST ALWAYS normalize GenAI span names to standard operation formats (e.g., `chat {model_name}`, `embeddings {model_name}`). - Engineers MUST ALWAYS restrict span attributes to canonical OpenTelemetry conventions: `gen_ai.system`, `gen_ai.request.model`, `gen_ai.request.temperature`, `gen_ai.usage.input_tokens`, and `gen_ai.usage.output_tokens`. - Engineers MUST NOT store raw customer inputs, chat prompts, or LLM-generated completions in W3C Baggage or root span attributes unless routed through an audited, localized redaction pipeline and explicitly configured with opt-in content capturing flags (`gen_ai.capture_message_content=false` by default). - Services MUST implement a strict Baggage Quarantine Filter on HTTP and RPC egress/ingress that drops any key not explicitly registered on the architectural allowlist (e.g., `workflow.id`, `tenant.id`, `agent.role`). - Components NEVER forward inbound baggage entries downstream without running cryptographic integrity and schema conformance checks. ## Bad vs Good Implementations ### Autonomous Agent Baggage Injection and Tracing #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Populates non-standard span attributes with raw PII and unredacted prompt data, while propagating PII through W3C Baggage, which leaks sensitive context across microservice hops and into telemetry aggregators. ```typescript import { trace, propagation, context } from '@opentelemetry/api'; // ANTI-PATTERN: Blindly attaching raw PII, prompt context, and unsanitized parameters // to W3C baggage and non-standard span attributes. export async function executeAgentStep(userEmail: string, rawPrompt: string, model: string) { const tracer = trace.getTracer('agent-runner'); return tracer.startActiveSpan('run-agent-llm', async (span) => { // Leaking sensitive customer PII into span attributes span.setAttribute('user.email', userEmail); span.setAttribute('llm.raw_prompt', rawPrompt); // Polluting downstream HTTP headers via baggage with unbounded PII let ctx = propagation.setBaggage( context.active(), propagation.createBaggage({ 'user.email': { value: userEmail }, 'prompt.preview': { value: rawPrompt.substring(0, 100) }, }) ); try { const response = await fetch('https://agent-worker.internal/v1/step', { headers: { 'x-trace-context': 'active' } }); return await response.json(); } finally { span.end(); } }); } ``` #### Production Standard (Correct) > **Standard**: Adheres strictly to OpenTelemetry GenAI Semantic Conventions (e.g., `gen_ai.system`, `gen_ai.request.model`), replaces raw prompt payloads with deterministic hashes, and imposes a strict allowlist-based Baggage Quarantine filter to prevent PII leakage downstream. ```typescript import { trace, propagation, context, SpanKind } from '@opentelemetry/api'; import { createHash } from 'node:crypto'; // ALLOWLIST: Only explicitly approved, non-PII operational correlation keys const BAGGAGE_ALLOWLIST = new Set(['tenant.id', 'workflow.id', 'agent.id', 'agent.role']); interface AgentExecutionContext { tenantId: string; workflowId: string; agentId: string; agentRole: string; userEmail: string; rawPrompt: string; model: string; } export async function executeAgentStep(ctxParams: AgentExecutionContext) { const tracer = trace.getTracer('agent-service', '1.0.0'); // Adhere to OpenTelemetry GenAI Semantic Conventions const spanName = `chat ${ctxParams.model}`; return tracer.startActiveSpan(spanName, { kind: SpanKind.CLIENT }, async (span) => { try { const promptHash = createHash('sha256').update(ctxParams.rawPrompt).digest('hex'); // Enforce standard GenAI attribute namespace span.setAttributes({ 'gen_ai.system': 'anthropic', 'gen_ai.request.model': ctxParams.model, 'gen_ai.operation.name': 'chat', 'gen_ai.request.prompt_hash': promptHash, 'workflow.id': ctxParams.workflowId, 'tenant.id': ctxParams.tenantId, }); // Baggage Quarantine: Filter candidates strictly against the allowlist const candidateBaggage: Record<string, string> = { 'tenant.id': ctxParams.tenantId, 'workflow.id': ctxParams.workflowId, 'agent.id': ctxParams.agentId, 'agent.role': ctxParams.agentRole, // PII fields are intentionally omitted and blocked from baggage }; const sanitizedBaggageEntries = Object.entries(candidateBaggage) .filter(([key]) => BAGGAGE_ALLOWLIST.has(key)) .reduce((acc, [key, val]) => ({ ...acc, [key]: { value: val } }), {}); const sanitizedContext = propagation.setBaggage( context.active(), propagation.createBaggage(sanitizedBaggageEntries) ); return await context.with(sanitizedContext, async () => { const headers: Record<string, string> = {}; propagation.inject(context.active(), headers); const response = await fetch('https://agent-worker.internal/v1/step', { headers }); const data = await response.json(); span.setAttribute('gen_ai.response.finish_reasons', ['stop']); return data; }); } catch (error) { span.recordException(error as Error); throw error; } finally { span.end(); } }); } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: OpenTelemetry GenAI Semantic Conventions and PII Baggage Quarantine

OpenTelemetry GenAI Semantic Conventions and PII Baggage Quarantine defines an industrial-grade behavioral standard for devops engineering. Enforces strict compliance with OpenTelemetry GenAI Semantic Conventions while establishing an active quarantine perimeter around W3C baggage and span attributes. Guarantees that multi-agent contextual correlation tokens persist across distributed service hops without leaking raw prompts, model responses, or personally identifiable information (PII) into telemetry backends or egress headers. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Indiscriminate Telemetry Baggage Injection and Raw Prompt Span Pollution Leaking Sensitive Data Across Service Hops Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Indiscriminate Telemetry Baggage Injection and Raw Prompt Span Pollution Leaking Sensitive Data Across Service Hops	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS normalize GenAI span names to standard operation formats (e.g., `chat {model_name}`, `embeddings {model_name}`).

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.