NATS JetStream 2.10 Clustered Deduplication Window and Ack Wait Alignment Invariant

Enforces deterministic at-most-once and exactly-once processing semantics across NATS JetStream 2.10 distributed clusters. Guarantees that stream-level duplicate tracking windows (`duplicate_window`) strictly exceed consumer acknowledgement wait periods (`ack_wait`) by at least a safety propagation buffer, while requiring deterministic, high-entropy `Nats-Msg-Id` headers on all publisher operations. Prevents phantom message duplication cascades during network partitions, consumer retry cycles, and Raft leader re-elections.

--- description: NATS JetStream 2.10 Clustered Deduplication Window and Ack Wait Alignment Invariant - Enforces deterministic at-most-once and exactly-once processing semantics across NATS JetStream 2.10 distributed clusters. Guarantees that stream-level duplicate tracking windows (`duplicate_window`) strictly exceed consumer acknowledgement wait periods (`ack_wait`) by at least a safety propagation buffer, while requiring deterministic, high-entropy `Nats-Msg-Id` headers on all publisher operations. Prevents phantom message duplication cascades during network partitions, consumer retry cycles, and Raft leader re-elections. globs: src/**/*.service.ts, src/**/*.publisher.ts, src/**/*.consumer.ts, src/**/nats*.ts, src/**/messaging/**/*.ts alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/nats-jetstream-deduplication-ack-window-alignment" --- # NATS JetStream 2.10 Clustered Deduplication Window and Ack Wait Alignment Invariant > **Severity**: CRITICAL | **Category**: backend > **Anti-Pattern Warning**: Mismatched Duplicate Window and Consumer Ack Wait Resulting in Silent Double-Processing Cascades ## System Constraints You are a Principal Distributed Systems Engineer enforcing the NATS JetStream 2.10 deduplication window and ack wait invariant. You must reject any stream or consumer configuration where `duplicate_window` is less than or equal to `ack_wait` plus expected retry/failover latency. You must mandate deterministic `Nats-Msg-Id` header assignment derived from stable domain keys for every publication, prohibiting purely non-deterministic random UUIDs on re-tries. Ensure stream replica counts adhere to cluster quorum (R=3 or R=5) and consumers operate under explicit acknowledgement modes (`AckPolicy.Explicit`). ## Strict Guidelines - Engineers MUST ALWAYS set the stream `duplicate_window` to a value strictly greater than the consumer's `ack_wait` multiplied by max retry attempts or failover timeout (minimum recommended: `duplicate_window >= ack_wait * 1.5`). - Engineers MUST NEVER publish messages to a deduplicated JetStream subject without an explicit, deterministic `Nats-Msg-Id` header derived from domain entities (e.g., entity ID + mutation version/hash). - Engineers MUST NOT generate a new random `Nats-Msg-Id` (e.g., `randomUUID()`) inside transient network retry loops for the same logical mutation. - Services MUST configure consumers with `AckPolicy.Explicit` and define explicit `max_ack_pending` limits to prevent consumer overload and memory exhaustion. - Stream configurations MUST set `replicas: 3` (or 5) in production environments to maintain the Raft-backed deduplication sliding window across node reboots and cluster re-elections. - Consumers MUST invoke `msg.ack()` promptly upon committing domain state, or call `msg.nak(delay)` with bounded exponential backoff that does not exceed the remaining deduplication window. ## Bad vs Good Implementations ### Stream and Consumer Configuration with Mismatched Ack Window vs Invariant Compliant Alignment #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: The stream deduplication window (20s) expires before the consumer acknowledgment wait deadline (30s). When a consumer fails to ack within 30 seconds and the message redelivery or publisher retry executes, JetStream has evicted the message ID from its sliding index, executing duplicate side-effects. ```typescript import { connect, JetStreamManager, AckPolicy } from 'nats'; export async function setupOrderProcessing(jsm: JetStreamManager) { // ANTI-PATTERN: duplicate_window (20s) is SHORTER than ack_wait (30s). // If a consumer hangs or times out after 30s and retries/republishes, // the stream's deduplication cache has already expired the Msg-Id, // leading to silent double-execution in downstream subscribers. await jsm.streams.add({ name: 'ORDERS', subjects: ['orders.>'], duplicate_window: 20_000_000_000, // 20 seconds in nanoseconds }); await jsm.consumers.add('ORDERS', { durable_name: 'order-worker', ack_policy: AckPolicy.Explicit, ack_wait: 30_000_000_000, // 30 seconds: EXCEEDS deduplication window! }); } ``` #### Production Standard (Correct) > **Standard**: The deduplication window is calculated to outlive the maximum potential lifetime of an unacknowledged or retried message across failovers. Replicas are pinned to 3 for Raft deduplication state replication. ```typescript import { connect, JetStreamManager, AckPolicy, RetentionPolicy, StorageType } from 'nats'; const NANOSECONDS_PER_SECOND = 1_000_000_000; export async function setupOrderProcessing(jsm: JetStreamManager) { const ACK_WAIT_SECONDS = 30; const MAX_DELIVER = 3; // INVARIANT: duplicate_window MUST cover ack_wait * max_deliver + network propagation buffer const DEDUP_WINDOW_SECONDS = (ACK_WAIT_SECONDS * MAX_DELIVER) + 30; // 120 seconds total await jsm.streams.add({ name: 'ORDERS', subjects: ['orders.>'], storage: StorageType.File, replicas: 3, retention: RetentionPolicy.Limits, duplicate_window: DEDUP_WINDOW_SECONDS * NANOSECONDS_PER_SECOND, }); await jsm.consumers.add('ORDERS', { durable_name: 'order-worker', ack_policy: AckPolicy.Explicit, ack_wait: ACK_WAIT_SECONDS * NANOSECONDS_PER_SECOND, max_deliver: MAX_DELIVER, max_ack_pending: 1000, }); } ``` ### Publisher Deduplication Header: Random Ephemeral ID vs Deterministic Domain Key #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Generating a non-deterministic random UUID for `Nats-Msg-Id` guarantees that network retry loops will present distinct IDs to the cluster, completely bypassing JetStream's duplicate identification engine. ```typescript import { JetStreamClient, headers } from 'nats'; import { randomUUID } from 'node:crypto'; interface OrderPlacedEvent { orderId: string; tenantId: string; version: number; amount: number; } export async function publishOrder(js: JetStreamClient, event: OrderPlacedEvent): Promise<void> { // ANTI-PATTERN: Using a fresh random UUID on every publish call or retry // completely invalidates JetStream deduplication tracking. const h = headers(); h.set('Nats-Msg-Id', randomUUID()); const payload = Buffer.from(JSON.stringify(event)); await js.publish(`orders.${event.tenantId}`, payload, { headers: h }); } ``` #### Production Standard (Correct) > **Standard**: Generates a deterministic message ID using aggregate root identity and versioning. Even under severe network retry cycles or timeout reconnects, NATS JetStream identifies the payload as a duplicate, returns an ACK with `duplicate: true`, and avoids duplicate persistence. ```typescript import { JetStreamClient, headers } from 'nats'; import { createHash } from 'node:crypto'; interface OrderPlacedEvent { orderId: string; tenantId: string; version: number; amount: number; } export async function publishOrder(js: JetStreamClient, event: OrderPlacedEvent): Promise<void> { // INVARIANT: Deterministic Msg-Id bound to the domain aggregate ID and version const canonicalId = `tenant:${event.tenantId}:order:${event.orderId}:v:${event.version}`; const deterministicMsgId = createHash('sha256').update(canonicalId).digest('hex'); const h = headers(); h.set('Nats-Msg-Id', deterministicMsgId); const payload = Buffer.from(JSON.stringify(event)); // Retries will reuse the exact same deterministicMsgId, allowing JetStream to deduplicate const ack = await js.publish(`orders.${event.tenantId}`, payload, { headers: h, timeout: 5000, }); if (ack.duplicate) { // Logged as idempotent success rather than an unexpected failure console.info(`[NATS] Message deduplicated by cluster: ${deterministicMsgId}`); } } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: NATS JetStream 2.10 Clustered Deduplication Window and Ack Wait Alignment Invariant

NATS JetStream 2.10 Clustered Deduplication Window and Ack Wait Alignment Invariant defines an industrial-grade behavioral standard for backend engineering. Enforces deterministic at-most-once and exactly-once processing semantics across NATS JetStream 2.10 distributed clusters. Guarantees that stream-level duplicate tracking windows (`duplicate_window`) strictly exceed consumer acknowledgement wait periods (`ack_wait`) by at least a safety propagation buffer, while requiring deterministic, high-entropy `Nats-Msg-Id` headers on all publisher operations. Prevents phantom message duplication cascades during network partitions, consumer retry cycles, and Raft leader re-elections. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Mismatched Duplicate Window and Consumer Ack Wait Resulting in Silent Double-Processing Cascades Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Mismatched Duplicate Window and Consumer Ack Wait Resulting in Silent Double-Processing Cascades	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS set the stream `duplicate_window` to a value strictly greater than the consumer's `ack_wait` multiplied by max retry attempts or failover timeout (minimum recommended

Engineers MUST ALWAYS set the stream `duplicate_window` to a value strictly greater than the consumer's `ack_wait` multiplied by max retry attempts or failover timeout (minimum recommended: `duplicate_window >= ack_wait * 1.5`).

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.