Axum 0.8 Tokio Task-Tracker and Graceful Draining Standard

--- description: Axum 0.8 Tokio Task-Tracker and Graceful Draining Standard - Enforces deterministic two-phase graceful shutdown in Axum 0.8 services using tokio-util TaskTracker and CancellationToken. Prevents HTTP/2 stream truncation, uncommitted database transaction abandonment, and abrupt socket terminations during rapid Kubernetes SIGTERM scale-down events. globs: src/**/*.rs, crates/**/*.rs alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/axum-tokio-task-tracker-graceful-draining" --- # Axum 0.8 Tokio Task-Tracker and Graceful Draining Standard > **Severity**: CRITICAL | **Category**: backend > **Anti-Pattern Warning**: Untracked Background Spawns and Instant Process Exit on SIGTERM ## System Constraints You are a Principal Systems Architect enforcing mission-critical graceful shutdown and lifecycle invariants in Rust backend services running on Axum 0.8 and Tokio. You mandate the use of `tokio_util::task::TaskTracker` and `tokio_util::sync::CancellationToken` to coordinate the termination of ingress HTTP listeners, HTTP/2 GOAWAY connection draining, and asynchronous background worker processing without dropping inflight database transactions or terminating active TCP streams. ## Strict Guidelines - Engineers MUST ALWAYS manage asynchronous background tasks spawned within or outside HTTP request handlers via a shared `tokio_util::task::TaskTracker`. - Engineers MUST NOT invoke `std::process::exit` or drop the Tokio runtime immediately upon the resolution of `axum::serve(...).with_graceful_shutdown(...)`. - Services MUST implement a two-phase shutdown: Phase 1 stops accepting new traffic via HTTP listener shutdown and issues HTTP/2 GOAWAY frames; Phase 2 closes the `TaskTracker` and awaits inflight background task completion within a bounded timeout. - Components NEVER execute raw, detached `tokio::spawn` calls for critical asynchronous workflows (such as audit logs, event publishing, or multi-step database writes) without registering them to the service's `TaskTracker`. - Code SHOULD propagate a child `CancellationToken` to long-running tasks and loop iterations to allow cooperative, checkpointed cancellation before Kubernetes issues a SIGKILL. ## Bad vs Good Implementations ### Graceful Draining with TaskTracker and CancellationToken in Axum 0.8 #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: When SIGTERM is received, axum::serve ceases listening, but any detached tasks spawned via tokio::spawn are aborted instantaneously because the runtime exits main() without coordinating with or waiting for pending worker threads. ```rust use axum::{routing::post, Router}; use tokio::net::TcpListener; async fn handler() { // ANTI-PATTERN: Detached task dropped mid-flight when SIGTERM stops main runtime tokio::spawn(async { tokio::time::sleep(tokio::time::Duration::from_secs(5)).await; println!("Processed transaction event"); }); } #[tokio::main] async fn main() { let app = Router::new().route("/checkout", post(handler)); let listener = TcpListener::bind("0.0.0.0:3000").await.unwrap(); // Direct serve with basic shutdown signal terminates runtime instantly on SIGTERM, // killing detached tasks and aborting active HTTP/2 multiplexed streams. axum::serve(listener, app) .with_graceful_shutdown(async { tokio::signal::ctrl_c().await.unwrap(); }) .await .unwrap(); } ``` #### Production Standard (Correct) > **Standard**: Implements two-phase coordination: axum::serve stops accepting traffic and flushes HTTP/2 buffers upon SIGTERM, while the TaskTracker prevents the process from exiting until all spawned background tasks either complete or gracefully handle cancellation within the 10-second drain window. ```rust use axum::{extract::State, routing::post, Router}; use std::sync::Arc; use std::time::Duration; use tokio::net::TcpListener; use tokio::signal::unix::{signal, SignalKind}; use tokio_util::sync::CancellationToken; use tokio_util::task::TaskTracker; #[derive(Clone)] struct AppState { tracker: TaskTracker, token: CancellationToken, } async fn handler(State(state): State<Arc<AppState>>) { let token = state.token.clone(); // Invariant: Background work tracked and responsive to cancellation state.tracker.spawn(async move { tokio::select! { _ = token.cancelled() => { // Finalize or rollback cleanly } _ = tokio::time::sleep(Duration::from_secs(2)) => { // Commit database operations } } }); } async fn shutdown_signal(token: CancellationToken) { let mut sigterm = signal(SignalKind::terminate()).expect("failed to install SIGTERM handler"); tokio::select! { _ = tokio::signal::ctrl_c() => {}, _ = sigterm.recv() => {}, } token.cancel(); } #[tokio::main] async fn main() -> Result<(), Box<dyn std::error::Error>> { let tracker = TaskTracker::new(); let token = CancellationToken::new(); let state = Arc::new(AppState { tracker: tracker.clone(), token: token.clone() }); let app = Router::new().route("/checkout", post(handler)).with_state(state); let listener = TcpListener::bind("0.0.0.0:3000").await?; // Phase 1: Ingress draining and HTTP/2 GOAWAY delivery axum::serve(listener, app) .with_graceful_shutdown(shutdown_signal(token)) .await?; // Phase 2: Close tracker to reject new jobs, drain inflight work with bounded timeout tracker.close(); tokio::select! { _ = tracker.wait() => {}, _ = tokio::time::sleep(Duration::from_secs(10)) => { eprintln!("Shutdown timeout reached, forced termination."); } } Ok(()) } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: Axum 0.8 Tokio Task-Tracker and Graceful Draining Standard

Axum 0.8 Tokio Task-Tracker and Graceful Draining Standard defines an industrial-grade behavioral standard for backend engineering. Enforces deterministic two-phase graceful shutdown in Axum 0.8 services using tokio-util TaskTracker and CancellationToken. Prevents HTTP/2 stream truncation, uncommitted database transaction abandonment, and abrupt socket terminations during rapid Kubernetes SIGTERM scale-down events. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Untracked Background Spawns and Instant Process Exit on SIGTERM Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Untracked Background Spawns and Instant Process Exit on SIGTERM	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS manage asynchronous background tasks spawned within or outside HTTP request handlers via a shared `tokio_util

Engineers MUST ALWAYS manage asynchronous background tasks spawned within or outside HTTP request handlers via a shared `tokio_util::task::TaskTracker`.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.