Axum 0.8 Tower CancellationToken Cascade and Cooperative Drain Invariant

--- description: Axum 0.8 Tower CancellationToken Cascade and Cooperative Drain Invariant - Guarantees deterministic client disconnect propagation and bounded worker pool drainage across nested Tower layers during rolling zero-downtime redeployments. Enforces hierarchical CancellationToken cascading and tokio_util::task::TaskTracker lifecycle bounds to prevent orphaned zombie tasks and redeployment timeouts in Axum 0.8 services. globs: src/**/*.rs, crates/**/*.rs alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/axum-0-8-tower-cancellation-token-cascade-guardrail" --- # Axum 0.8 Tower CancellationToken Cascade and Cooperative Drain Invariant > **Severity**: CRITICAL | **Category**: backend > **Anti-Pattern Warning**: Detached Tokio Spawns and Unbounded Mid-Flight Request Leaks During Rolling Pod Drain ## System Constraints You are a Principal Rust Systems Architect enforcing strict concurrency invariants in Axum 0.8 and Tower stacks. You must ensure that no asynchronous background work or long-lived request handlers survive client TCP disconnects or SIGTERM shutdown signals without explicit, bounded cooperative drain channels. ## Strict Guidelines - Engineers MUST ALWAYS instantiate a root `tokio_util::sync::CancellationToken` and `tokio_util::task::TaskTracker` at the process lifecycle root and inject them via state or Tower layers. - Engineers MUST NOT invoke `tokio::spawn` directly inside Axum handlers or Tower middleware without binding the task to a `TaskTracker` and a child `CancellationToken`. - Code SHOULD propagate cancellation down nested Tower middleware stacks by passing child tokens or wiring `axum::serve::with_graceful_shutdown` signals directly into request extension contexts. - Services MUST enforce a strict timeout envelope during rolling redeployments by using `tokio::select!` over `tracker.wait()` and a deterministic fallback deadline (e.g., `Duration::from_secs(10)`). - Components NEVER drop or ignore cancellation select branches when dealing with state-mutating database transactions or transactional event outboxes; transactions MUST roll back cleanly upon token trigger. ## Bad vs Good Implementations ### Unbounded Detached Background Task vs. Cooperative Child Token Cascade #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Directly calling `tokio::spawn` detaches the future from both the HTTP request connection lifecycle and the server's graceful shutdown procedure. If the client disconnects or Kubernetes issues a SIGTERM for a rolling upgrade, the worker continues execution as a zombie task, burning compute and risking inconsistent state. ```rust use axum::{routing::post, Router}; async fn handle_compute() -> &'static str { // VIOLATION: Detached spawn with no cancellation token linkage. // Survives client disconnect and prevents clean zero-downtime rolling drain. tokio::spawn(async { for step in 1..=100 { tokio::time::sleep(tokio::time::Duration::from_millis(100)).await; tracing::debug!("Processed step: {step}"); } }); "Processing scheduled" } pub async fn run_server() { let app = Router::new().route("/compute", post(handle_compute)); let listener = tokio::net::TcpListener::bind("0.0.0.0:3000").await.unwrap(); // VIOLATION: Missing graceful shutdown drain pipeline axum::serve(listener, app).await.unwrap(); } ``` #### Production Standard (Correct) > **Standard**: Hierarchical child tokens inherit cancellation from the root server shutdown signal. Background tasks are tracked via `TaskTracker` and bounded by a deterministic 10-second cooperative drain timeout before exit, satisfying Axum 0.8 zero-downtime rolling deployment invariants. ```rust use axum::{extract::Extension, routing::post, Router, response::IntoResponse, http::StatusCode}; use std::time::Duration; use tokio_util::sync::CancellationToken; use tokio_util::task::TaskTracker; async fn handle_compute( Extension(server_token): Extension<CancellationToken>, Extension(tracker): Extension<TaskTracker>, ) -> Result<&'static str, (StatusCode, &'static str)> { let child_token = server_token.child_token(); let task_token = child_token.clone(); // Invariant: Track background tasks within the tracker and wire cancellation cascade tracker.spawn(async move { tokio::select! { _ = task_token.cancelled() => { tracing::warn!("Worker cooperatively aborted via drain signal or disconnect"); } _ = async { for step in 1..=100 { tokio::time::sleep(Duration::from_millis(100)).await; tracing::debug!("Processed step: {step}"); } } => { tracing::info!("Compute completed successfully"); } } }); Ok("Processing scheduled") } pub async fn run_server() { let root_token = CancellationToken::new(); let tracker = TaskTracker::new(); let app = Router::new() .route("/compute", post(handle_compute)) .layer(Extension(root_token.clone())) .layer(Extension(tracker.clone())); let listener = tokio::net::TcpListener::bind("0.0.0.0:3000").await.unwrap(); let shutdown_token = root_token.clone(); let server = axum::serve(listener, app).with_graceful_shutdown(async move { tokio::signal::ctrl_c().await.ok(); tracing::info!("SIGTERM/Ctrl-C caught: triggering cancellation cascade"); shutdown_token.cancel(); }); if let Err(err) = server.await { tracing::error!("Server error: {err}"); } tracker.close(); tokio::select! { _ = tracker.wait() => tracing::info!("All in-flight tasks drained successfully"), _ = tokio::time::sleep(Duration::from_secs(10)) => tracing::error!("Drain deadline breached; proceeding with forced termination"), } } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: Axum 0.8 Tower CancellationToken Cascade and Cooperative Drain Invariant

Axum 0.8 Tower CancellationToken Cascade and Cooperative Drain Invariant defines an industrial-grade behavioral standard for backend engineering. Guarantees deterministic client disconnect propagation and bounded worker pool drainage across nested Tower layers during rolling zero-downtime redeployments. Enforces hierarchical CancellationToken cascading and tokio_util::task::TaskTracker lifecycle bounds to prevent orphaned zombie tasks and redeployment timeouts in Axum 0.8 services. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Detached Tokio Spawns and Unbounded Mid-Flight Request Leaks During Rolling Pod Drain Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Detached Tokio Spawns and Unbounded Mid-Flight Request Leaks During Rolling Pod Drain	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS instantiate a root `tokio_util

Engineers MUST ALWAYS instantiate a root `tokio_util::sync::CancellationToken` and `tokio_util::task::TaskTracker` at the process lifecycle root and inject them via state or Tower layers.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.