Axum 0.8 Tokio Cooperative Task Cancellation & Drain Standard

--- description: Axum 0.8 Tokio Cooperative Task Cancellation & Drain Standard - Enforces structured task lifecycles and cooperative cancellation trees using Tokio and Axum 0.8 during rolling deployments. Guarantees inflight HTTP request completion, orderly background task draining, and prevention of database pool exhaustion caused by abruptly dropped futures. globs: src/**/*.rs alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/axum-tokio-task-cancellation-drain-standard" --- # Axum 0.8 Tokio Cooperative Task Cancellation & Drain Standard > **Severity**: CRITICAL | **Category**: backend > **Anti-Pattern Warning**: Unbounded detached task spawning and abrupt signal termination causing orphaned transactions and pool starvation ## System Constraints You are a Principal Rust Concurrency Engineer enforcing the Axum 0.8 Tokio Cooperative Task Cancellation & Drain Standard. You strictly prohibit unmonitored `tokio::spawn` invocations, require top-level `tokio_util::sync::CancellationToken` trees paired with `tokio_util::task::TaskTracker`, and mandate explicit drain timeouts prior to closing persistent resources like database pools. ## Strict Guidelines - Engineers MUST ALWAYS couple `axum::serve(...).with_graceful_shutdown(...)` with a root `tokio_util::sync::CancellationToken` and a `tokio_util::task::TaskTracker`. - Engineers MUST NOT invoke detached `tokio::spawn` in request handlers for async side-effects; all background operations MUST be spawned through a tracked `TaskTracker` or bound to a child cancellation token. - Services MUST implement a two-phase drain sequence on shutdown: first, stop accepting new HTTP connections via Axum's graceful shutdown; second, invoke `tracker.close()` and await `tracker.wait()` bounded by a strict `tokio::time::timeout`. - Handlers executing long-running streams or operations MUST select over `cancellation_token.cancelled()` to guarantee early exit and avoid blocking the deploy pipeline. - Components NEVER drop or forcefully close the `sqlx::Pool` or equivalent connection pool before the `TaskTracker` has completely drained inflight tasks. ## Bad vs Good Implementations ### Axum 0.8 Server Shutdown and Background Task Management #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Spawning detached futures via `tokio::spawn` inside handlers strips them of lifecycle tracking. When Axum finishes its graceful shutdown on SIGTERM, `main` exits immediately, terminating the runtime and dropping active transactions mid-flight. This results in connection pool exhaustion, database connection deadlocks, and corrupted business workflows during deployments. ```rust use axum::{routing::post, Router}; use sqlx::PgPool; use std::net::SocketAddr; async fn handler(pool: axum::extract::Extension<PgPool>) -> &'static str { // ANTI-PATTERN: Detached task spawned without tracker or cancellation handle tokio::spawn(async move { let mut tx = pool.begin().await.unwrap(); // Long-running write; will be severed mid-flight on SIGTERM tokio::time::sleep(std::time::Duration::from_secs(10)).await; sqlx::query("UPDATE accounts SET active = true").execute(&mut *tx).await.unwrap(); tx.commit().await.unwrap(); }); "Accepted" } #[tokio::main] async fn main() { let pool = PgPool::connect("postgres://...").await.unwrap(); let app = Router::new().route("/process", post(handler)).layer(axum::extract::Extension(pool)); let listener = tokio::net::TcpListener::bind("0.0.0.0:3000").await.unwrap(); // ANTI-PATTERN: Server drops immediately on Ctrl+C, orphaning the background task and corrupting pool state axum::serve(listener, app) .with_graceful_shutdown(async { tokio::signal::ctrl_c().await.unwrap(); }) .await .unwrap(); } ``` #### Production Standard (Correct) > **Standard**: The production implementation uses `tokio_util::task::TaskTracker` alongside hierarchical `CancellationToken` primitives. Axum completes its graceful drain of incoming connections first, after which `tracker.close()` prevents new tasks while inflight operations are given up to 30 seconds to finish or cooperatively abort. Only after all tasks complete is the SQL pool closed. ```rust use axum::{extract::State, routing::post, Router}; use sqlx::PgPool; use std::{sync::Arc, time::Duration}; use tokio::net::TcpListener; use tokio_util::sync::CancellationToken; use tokio_util::task::TaskTracker; #[derive(Clone)] struct AppState { pool: PgPool, tracker: TaskTracker, token: CancellationToken, } async fn handler(State(state): State<AppState>) -> &'static str { let token = state.token.child_token(); let pool = state.pool.clone(); // PRODUCTION FIX: Task is registered with TaskTracker and responds to cooperative cancellation state.tracker.spawn(async move { tokio::select! { _ = token.cancelled() => { tracing::warn!("Background task aborted cooperatively due to shutdown drain"); } res = async move { let mut tx = pool.begin().await?; sqlx::query("UPDATE accounts SET active = true").execute(&mut *tx).await?; tx.commit().await?; Ok::<(), sqlx::Error>(()) } => { if let Err(err) = res { tracing::error!(%err, "Background transaction failed"); } } } }); "Accepted" } #[tokio::main] async fn main() -> Result<(), Box<dyn std::error::Error>> { let pool = PgPool::connect("postgres://...").await?; let tracker = TaskTracker::new(); let token = CancellationToken::new(); let state = AppState { pool: pool.clone(), tracker: tracker.clone(), token: token.clone() }; let app = Router::new().route("/process", post(handler)).with_state(state); let listener = TcpListener::bind("0.0.0.0:3000").await?; let server_token = token.clone(); let server = axum::serve(listener, app) .with_graceful_shutdown(async move { let _ = tokio::signal::ctrl_c().await; tracing::info!("SIGTERM received, initiating cooperative drain"); server_token.cancel(); }); server.await?; // PRODUCTION FIX: Orderly two-phase drain with bounded timeout before resource destruction tracker.close(); if tokio::time::timeout(Duration::from_secs(30), tracker.wait()).await.is_err() { tracing::error!("Drain timeout exceeded; force terminating remaining background workers"); } pool.close().await; tracing::info!("Database pool closed cleanly; zero-downtime shutdown complete"); Ok(()) } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: Axum 0.8 Tokio Cooperative Task Cancellation & Drain Standard

Axum 0.8 Tokio Cooperative Task Cancellation & Drain Standard defines an industrial-grade behavioral standard for backend engineering. Enforces structured task lifecycles and cooperative cancellation trees using Tokio and Axum 0.8 during rolling deployments. Guarantees inflight HTTP request completion, orderly background task draining, and prevention of database pool exhaustion caused by abruptly dropped futures. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Unbounded detached task spawning and abrupt signal termination causing orphaned transactions and pool starvation Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Unbounded detached task spawning and abrupt signal termination causing orphaned transactions and pool starvation	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS couple `axum

Engineers MUST ALWAYS couple `axum::serve(...).with_graceful_shutdown(...)` with a root `tokio_util::sync::CancellationToken` and a `tokio_util::task::TaskTracker`.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.