Axum 0.8 Epoll-Aware Zero-Loss Connection Draining Standard

--- description: Axum 0.8 Epoll-Aware Zero-Loss Connection Draining Standard - Guarantees zero-loss termination of active Server-Sent Events (SSE) and HTTP/2 multiplexed streams during rolling pod replacements. Enforces coordinated dual-phase graceful shutdown via Tokio cancellation tokens and hyper connection tracking to prevent TCP RST generation and truncated stream frames. globs: src/**/*.rs alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/axum-epoll-graceful-connection-draining" --- # Axum 0.8 Epoll-Aware Zero-Loss Connection Draining Standard > **Severity**: CRITICAL | **Category**: backend > **Anti-Pattern Warning**: Naive Axum serve termination hanging on infinite SSE streams until Kubernetes SIGKILL emits TCP RSTs ## System Constraints You are a Principal Rust Systems Architect enforcing the Axum 0.8 Epoll-Aware Graceful Connection Draining Invariant. You must reject any infinite streaming handlers (SSE, WebSockets, or HTTP/2 chunked responses) that lack cooperative cancellation hooks, and reject any server bootstrap that fails to coordinate application-level stream unwinding with hyper's TCP connection draining. ## Strict Guidelines - Services MUST ALWAYS bind a parent `tokio_util::sync::CancellationToken` to OS shutdown signals (SIGINT/SIGTERM) and provide child tokens to long-lived streaming handlers. - Engineers MUST NOT rely solely on `axum::serve(...).with_graceful_shutdown(...)` for services serving Server-Sent Events (SSE) or WebSockets, as hyper will wait indefinitely for active streaming handlers to terminate. - Streaming endpoints MUST yield terminal protocol messages (e.g., SSE retry/close events or HTTP/2 end-of-stream flags) upon cancellation before closing the stream channel. - The server bootstrap MUST enforce a bounded draining timeout with `tokio::time::timeout` during hyper's graceful shutdown phase to prevent pod termination deadlocks before Kubernetes SIGKILL. - Handlers MUST NEVER leak unpolled async channels into the epoll reactor across shutdown boundaries; all client connection poll loops must honor half-close states. ## Bad vs Good Implementations ### Uncoordinated SSE Loop vs Cooperative Token-Draining Stream #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: The interval stream never yields `None`. During rolling updates, Axum's graceful shutdown hangs while awaiting this connection to complete, exhausting the Kubernetes termination grace period until the kernel kills the process and sends TCP RSTs to active clients. ```rust use axum::{routing::get, response::sse::{Event, Sse}, Router}; use tokio_stream::wrappers::IntervalStream; use tokio_stream::StreamExt; use std::time::Duration; // ANTI-PATTERN: Infinite stream has no awareness of shutdown signals. // When Kubernetes triggers SIGTERM, axum::serve waits forever on this handler. // Eventually, k8s sends SIGKILL, causing epoll destruction and emitting TCP RST to client. async fn bad_sse_handler() -> Sse<impl tokio_stream::Stream<Item = Result<Event, std::convert::Infallible>>> { let stream = IntervalStream::new(tokio::time::interval(Duration::from_secs(1))) .map(|_| Ok(Event::default().data("heartbeat"))); Sse::new(stream) } pub fn router() -> Router { Router::new().route("/events", get(bad_sse_handler)) } ``` #### Production Standard (Correct) > **Standard**: The stream observes the `CancellationToken` via `tokio::select!`. On SIGTERM, the stream emits a deterministic terminal message and finishes with `None`, allowing hyper to send `GOAWAY` / stream `FIN`, completely draining epoll references without RST. ```rust use axum::{extract::State, routing::get, response::sse::{Event, Sse}, Router}; use tokio_util::sync::CancellationToken; use async_stream::stream; use std::time::Duration; use std::convert::Infallible; #[derive(Clone)] pub struct AppState { pub shutdown_token: CancellationToken, } // PRODUCTION INVARIANT: Cooperative stream abortion on cancellation signal. // Emits a clean final payload and terminates the stream, allowing hyper // to cleanly finalize the HTTP/2 frame stream and execute a TCP half-close. async fn good_sse_handler( State(state): State<AppState>, ) -> Sse<impl tokio_stream::Stream<Item = Result<Event, Infallible>> + Send + 'static> { let token = state.shutdown_token.clone(); let s = stream! { let mut ticker = tokio::time::interval(Duration::from_secs(1)); loop { tokio::select! { _ = token.cancelled() => { // Gracefully notify client of planned reconnection yield Ok(Event::default().event("shutdown").data("reconnect")); break; } _ = ticker.tick() => { yield Ok(Event::default().data("heartbeat")); } } } }; Sse::new(s) } pub fn router(state: AppState) -> Router { Router::new().route("/events", get(good_sse_handler)).with_state(state) } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: Axum 0.8 Epoll-Aware Zero-Loss Connection Draining Standard

Axum 0.8 Epoll-Aware Zero-Loss Connection Draining Standard defines an industrial-grade behavioral standard for backend engineering. Guarantees zero-loss termination of active Server-Sent Events (SSE) and HTTP/2 multiplexed streams during rolling pod replacements. Enforces coordinated dual-phase graceful shutdown via Tokio cancellation tokens and hyper connection tracking to prevent TCP RST generation and truncated stream frames. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Naive Axum serve termination hanging on infinite SSE streams until Kubernetes SIGKILL emits TCP RSTs Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Naive Axum serve termination hanging on infinite SSE streams until Kubernetes SIGKILL emits TCP RSTs	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Services MUST ALWAYS bind a parent `tokio_util

Services MUST ALWAYS bind a parent `tokio_util::sync::CancellationToken` to OS shutdown signals (SIGINT/SIGTERM) and provide child tokens to long-lived streaming handlers.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.