Axum 0.8 Tokio Broadcast Channel Graceful Teardown Barrier

--- description: Axum 0.8 Tokio Broadcast Channel Graceful Teardown Barrier - Enforces a two-phase connection draining and broadcast channel shutdown barrier for long-lived streaming connections (SSE and WebSockets) in Axum 0.8. Prevents message loss, abrupt socket resets (RST), and half-open deadlocks during zero-downtime rolling container deployments by synchronizing HTTP server termination with consumer stream exhaustion via cancellation tokens and task trackers. globs: src/**/*.rs, crates/**/*.rs alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/axum-tokio-broadcast-graceful-teardown-barrier" --- # Axum 0.8 Tokio Broadcast Channel Graceful Teardown Barrier > **Severity**: CRITICAL | **Category**: backend > **Anti-Pattern Warning**: Abrupt Runtime Termination of Axum SSE/WebSocket Streams on SIGTERM Causing Message Drops and Client Socket Resets ## System Constraints You are a Principal Rust Architect mandating deterministic connection draining for real-time streaming interfaces in Axum 0.8. You strictly enforce two-phase teardown barriers: when receiving SIGTERM/SIGINT, the server must stop accepting ingress connections, signal application-level closure frames or end-of-stream events to all active SSE/WebSocket consumers, await buffered message transmission via Tokio task tracking, and safely drop broadcast senders only after a bounded drain phase. ## Strict Guidelines - Engineers MUST ALWAYS couple Axum 0.8's `axum::serve::with_graceful_shutdown` with a centralized `tokio_util::sync::CancellationToken` and a `tokio_util::task::TaskTracker`. - Engineers MUST NOT allow SSE or WebSocket handler loops to poll `tokio::sync::broadcast::Receiver` indefinitely without listening to a shutdown cancellation token. - Code SHOULD emit an explicit application-level terminal frame (such as a WebSocket Close frame with code 1001 Going Away, or an SSE `event: close`) before unbinding client sockets. - Services MUST bound the drain barrier timeout using `tokio::time::timeout` to prevent hung client TCP connections from indefinitely blocking Kubernetes pod eviction. - Components NEVER drop the master `tokio::sync::broadcast::Sender` until all active consumers have acknowledged the shutdown signal or the drain deadline has expired. ## Bad vs Good Implementations ### Vulnerable Streaming Handler vs. Deterministic Draining Barrier #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: When SIGTERM is received, `axum::serve` stops accepting new connections but immediately terminates pending tasks if the server future completes. The SSE stream yields only until the runtime drops it, causing lost in-flight broadcast messages, socket resets, and abrupt client disconnects without a clean closure protocol. ```rust use axum::{response::sse::{Event, Sse}, routing::get, Router}; use futures_util::stream::Stream; use std::convert::Infallible; use tokio::sync::broadcast; #[derive(Clone)] struct AppState { tx: broadcast::Sender<String>, } // ANTI-PATTERN: Streaming handler directly streams broadcast receiver. // When SIGTERM occurs, axum drops the listener and aborts running futures, // truncating unconsumed messages and dropping TCP connections with RST packets. async fn sse_handler( axum::extract::State(state): axum::extract::State<AppState>, ) -> Sse<impl Stream<Item = Result<Event, Infallible>>> { let mut rx = state.tx.subscribe(); let stream = async_stream::stream! { while let Ok(msg) = rx.recv().await { yield Ok(Event::default().data(msg)); } }; Sse::new(stream) } pub async fn run_server() { let (tx, _) = broadcast::channel(1024); let app = Router::new().route("/events", get(sse_handler)).with_state(AppState { tx }); let listener = tokio::net::TcpListener::bind("0.0.0.0:3000").await.unwrap(); // Blindly shuts down listener; ongoing streaming tasks are abandoned instantly. axum::serve(listener, app) .with_graceful_shutdown(async { tokio::signal::ctrl_c().await.unwrap(); }) .await .unwrap(); } ``` #### Production Standard (Correct) > **Standard**: Implements a robust two-phase barrier using `CancellationToken` and `TaskTracker`. When SIGTERM triggers, ingress stops while active streams are informed via cancellation to yield a final shutdown event. The orchestrator bounds the connection drain phase to 15 seconds, allowing existing buffers to flush gracefully without message loss. ```rust use axum::{response::sse::{Event, KeepAlive, Sse}, routing::get, Router}; use futures_util::stream::Stream; use std::{convert::Infallible, sync::Arc, time::Duration}; use tokio::sync::broadcast; use tokio_util::sync::CancellationToken; use tokio_util::task::TaskTracker; #[derive(Clone)] struct AppState { tx: broadcast::Sender<String>, tracker: TaskTracker, token: CancellationToken, } async fn sse_handler( axum::extract::State(state): axum::extract::State<AppState>, ) -> Sse<impl Stream<Item = Result<Event, Infallible>>> { let mut rx = state.tx.subscribe(); let token = state.token.clone(); let tracker = state.tracker.clone(); let stream = async_stream::stream! { // Track active client within the TaskTracker let _guard = tracker.track_future(async {}); loop { tokio::select! { biased; _ = token.cancelled() => { // Send clean EOF signal to client before socket closes yield Ok(Event::default().event("shutdown").data("node_draining")); break; } res = rx.recv() => match res { Ok(msg) => yield Ok(Event::default().data(msg)), Err(broadcast::error::RecvError::Lagged(_)) => continue, Err(broadcast::error::RecvError::Closed) => break, } } } }; Sse::new(stream).keep_alive(KeepAlive::default()) } pub async fn run_server() { let (tx, _) = broadcast::channel(1024); let tracker = TaskTracker::new(); let token = CancellationToken::new(); let state = AppState { tx, tracker: tracker.clone(), token: token.clone() }; let app = Router::new().route("/events", get(sse_handler)).with_state(state); let listener = tokio::net::TcpListener::bind("0.0.0.0:3000").await.unwrap(); let server_token = token.clone(); let server = axum::serve(listener, app).with_graceful_shutdown(async move { tokio::signal::ctrl_c().await.ok(); server_token.cancel(); }); // Phase 1: Wait for HTTP server to finish ingress shutdown if let Err(e) = server.await { eprintln!("Server error: {e}"); } // Phase 2: Close tracker to new tasks and await active streams with bounded timeout tracker.close(); let drain_barrier = tokio::time::timeout(Duration::from_secs(15), tracker.wait()); if drain_barrier.await.is_err() { eprintln!("Drain timeout reached; forcefully aborting remaining connections"); } } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: Axum 0.8 Tokio Broadcast Channel Graceful Teardown Barrier

Axum 0.8 Tokio Broadcast Channel Graceful Teardown Barrier defines an industrial-grade behavioral standard for backend engineering. Enforces a two-phase connection draining and broadcast channel shutdown barrier for long-lived streaming connections (SSE and WebSockets) in Axum 0.8. Prevents message loss, abrupt socket resets (RST), and half-open deadlocks during zero-downtime rolling container deployments by synchronizing HTTP server termination with consumer stream exhaustion via cancellation tokens and task trackers. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Abrupt Runtime Termination of Axum SSE/WebSocket Streams on SIGTERM Causing Message Drops and Client Socket Resets Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Abrupt Runtime Termination of Axum SSE/WebSocket Streams on SIGTERM Causing Message Drops and Client Socket Resets	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS couple Axum 0.8's `axum

Engineers MUST ALWAYS couple Axum 0.8's `axum::serve::with_graceful_shutdown` with a centralized `tokio_util::sync::CancellationToken` and a `tokio_util::task::TaskTracker`.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.