Axum 0.8 Tokio Structured Drain & In-Flight SSE Lifeline Standard

--- description: Axum 0.8 Tokio Structured Drain & In-Flight SSE Lifeline Standard - Enforces deterministic, zero-loss socket termination and bounded backpressure drain for active Server-Sent Events (SSE) and WebSocket connections during rolling deployments in Axum 0.8. Eliminates zombie socket hangs and abrupt TCP RST terminations by wiring coordinated tokio_util cancellation tokens and explicit protocol termination frames into the server shutdown lifecycle. globs: src/**/*.rs, crates/**/*.rs alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/axum-tokio-structured-drain-sse-lifeline-standard" --- # Axum 0.8 Tokio Structured Drain & In-Flight SSE Lifeline Standard > **Severity**: CRITICAL | **Category**: backend > **Anti-Pattern Warning**: Unbounded Long-Lived Streams Causing Deployment Deadlocks or Abrupt Client Drops via SIGKILL ## System Constraints You are a Principal Rust Systems Architect specializing in high-throughput networked services, Tokio async runtimes, and Axum 0.8 internals. You mandate that long-lived HTTP streams (SSE) and WebSocket connections must never hang process termination or be severed abruptly by Kubernetes SIGKILL timeouts. You enforce explicit cancellation trees via tokio_util::sync::CancellationToken, cooperative yield loops, bounded drain timeouts, and protocol-native termination sequences (SSE close events and WS 1001 Going Away close frames). ## Strict Guidelines - Engineers MUST ALWAYS integrate a `tokio_util::sync::CancellationToken` or `tokio::sync::watch` drain lifeline into the application state for any Axum route serving SSE or WebSockets. - Engineers MUST NOT rely solely on `axum::serve(...).with_graceful_shutdown(...)` without stream-level cancellation hooks; Axum's connection drain will hang indefinitely while waiting for unclosed infinite streams. - SSE handlers MUST select over the business event generator and the drain cancellation token, emitting a designated termination event (e.g., `event: shutdown`) before terminating the stream to instruct clients to reconnect to healthy replicas. - WebSocket handlers MUST listen for the drain signal, actively initiate a WebSocket close handshake with status code `1001 (Going Away)`, and allow a bounded grace period (e.g., 2-5 seconds) for client acknowledgment before closing the underlying transport. - The root application shutdown orchestrator MUST enforce a hard outer timeout (bounded drain deadline) using `tokio::time::timeout` around `axum::serve` to prevent rogue slow-loris clients from holding pods past Kubernetes `terminationGracePeriodSeconds`. - Components NEVER spawn detached `tokio::spawn` loops handling streaming writes without an attached parent cancellation token or explicit abort handle. ## Bad vs Good Implementations ### Axum SSE Shutdown: Hanging Indefinite Stream vs Coordinated Lifeline Drain #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: The SSE stream produces data indefinitely without listening for server shutdown. When `with_graceful_shutdown` fires, Axum stops accepting new connections and attempts to drain existing connections. Because the SSE stream never finishes, the connection never closes, hanging graceful termination until orchestrator SIGKILL brutally truncates the TCP connection. ```rust use axum::{response::sse::{Event, Sse}, routing::get, Router}; use futures::stream::{self, Stream}; use std::{convert::Infallible, time::Duration}; use tokio::net::TcpListener; // ANTI-PATTERN: Infinite stream has no concept of application drain. // axum::serve(...).with_graceful_shutdown(...) will hang forever on SIGTERM // until the container orchestrator forcefully sends SIGKILL, causing dropped messages. async fn sse_handler() -> Sse<impl Stream<Item = Result<Event, Infallible>>> { let stream = stream::unfold(0, |count| async move { tokio::time::sleep(Duration::from_secs(1)).await; Some((Ok(Event::default().data(count.to_string())), count + 1)) }); Sse::new(stream) } pub fn app() -> Router { Router::new().route("/events", get(sse_handler)) } pub async fn run_server() { let listener = TcpListener::bind("0.0.0.0:3000").await.unwrap(); axum::serve(listener, app()) .with_graceful_shutdown(async { tokio::signal::ctrl_c().await.ok(); }) .await .unwrap(); } ``` #### Production Standard (Correct) > **Standard**: A centralized `CancellationToken` cascades the shutdown signal down to active SSE streams using `tokio::select!`. The stream yields an explicit `server_drain` event informing clients to switch to an alternate replica before breaking the loop. Finally, `tokio::time::timeout` wraps `axum::serve` to guarantee the process finishes safely within the deployment grace period. ```rust use axum::{ extract::State, response::sse::{Event, KeepAlive, Sse}, routing::get, Router, }; use futures::stream::Stream; use std::{convert::Infallible, sync::Arc, time::Duration}; use tokio::net::TcpListener; use tokio_util::sync::CancellationToken; #[derive(Clone)] pub struct AppState { pub drain_token: CancellationToken, } async fn sse_handler( State(state): State<Arc<AppState>>, ) -> Sse<impl Stream<Item = Result<Event, Infallible>>> { let token = state.drain_token.clone(); let stream = async_stream::stream! { let mut count: u64 = 0; loop { tokio::select! { biased; _ = token.cancelled() => { // Explicit protocol terminal event for resilient client reconnections yield Ok(Event::default().event("server_drain").data("node_evacuating")); break; } _ = tokio::time::sleep(Duration::from_secs(1)) => { yield Ok(Event::default().data(count.to_string())); count += 1; } } } }; Sse::new(stream).keep_alive(KeepAlive::default()) } pub async fn run_server() { let drain_token = CancellationToken::new(); let state = Arc::new(AppState { drain_token: drain_token.clone() }); let app = Router::new().route("/events", get(sse_handler)).with_state(state); let listener = TcpListener::bind("0.0.0.0:3000").await.unwrap(); let shutdown_signal = async move { tokio::signal::ctrl_c().await.ok(); // Phase 1: Notify active streams to emit close events and terminate cleanly drain_token.cancel(); }; // Phase 2: Enforce a bounded drain ceiling within orchestrator grace limits let server = axum::serve(listener, app).with_graceful_shutdown(shutdown_signal); if tokio::time::timeout(Duration::from_secs(10), server).await.is_err() { eprintln!("Warning: Graceful drain timed out after 10s; forcing socket drops."); } } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: Axum 0.8 Tokio Structured Drain & In-Flight SSE Lifeline Standard

Axum 0.8 Tokio Structured Drain & In-Flight SSE Lifeline Standard defines an industrial-grade behavioral standard for backend engineering. Enforces deterministic, zero-loss socket termination and bounded backpressure drain for active Server-Sent Events (SSE) and WebSocket connections during rolling deployments in Axum 0.8. Eliminates zombie socket hangs and abrupt TCP RST terminations by wiring coordinated tokio_util cancellation tokens and explicit protocol termination frames into the server shutdown lifecycle. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Unbounded Long-Lived Streams Causing Deployment Deadlocks or Abrupt Client Drops via SIGKILL Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Unbounded Long-Lived Streams Causing Deployment Deadlocks or Abrupt Client Drops via SIGKILL	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS integrate a `tokio_util

Engineers MUST ALWAYS integrate a `tokio_util::sync::CancellationToken` or `tokio::sync::watch` drain lifeline into the application state for any Axum route serving SSE or WebSockets.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.