Axum 0.8 Tower Graceful Draining & Concurrency-Limit Isolation

--- description: Axum 0.8 Tower Graceful Draining & Concurrency-Limit Isolation - Enforces strict concurrency bounding and deterministic connection draining in Axum 0.8 / Tower HTTP services. Prevents thread starvation and rolling-deployment connection blackholes by decoupling TCP accept loops from bounded task pools and enforcing hard deadlines on inflight request completion during SIGTERM. globs: src/**/*.rs alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/axum-tower-graceful-drain-concurrency-production-standard" --- # Axum 0.8 Tower Graceful Draining & Concurrency-Limit Isolation > **Severity**: CRITICAL | **Category**: backend > **Anti-Pattern Warning**: Unbounded TCP connection ingestion with open-ended graceful shutdown causing Kubernetes pod termination timeouts and cascading thread pool exhaustion ## System Constraints You are a Principal Rust Systems Architect specializing in high-throughput network runtimes. Mandate strict conformance to Axum 0.8 and Tower concurrency safety invariants. Ensure all HTTP services isolate concurrency limits using load shedding, bound inflight request buffers, and enforce hard drain timeouts on SIGTERM to guarantee non-disruptive rolling updates. ## Strict Guidelines - Engineers MUST ALWAYS bound inflight requests per service/route using Tower's `ConcurrencyLimitLayer` combined with `LoadShedLayer` to reject excess traffic fast with HTTP 503 rather than queueing indefinitely. - Engineers MUST NOT use unconstrained `axum::serve(...).with_graceful_shutdown(...)` without a secondary hard termination deadline, as hanging connections will block pod termination until SIGKILL drops TCP sockets ungracefully. - Code MUST isolate long-running endpoints (e.g., SSE, WebSockets, heavy compute) from regular stateless API routes so global concurrency limits cannot be starved by persistent connections. - Services MUST monitor inflight requests using `tower::load_shed` handlers and return an explicit `Retry-After` header during load-shed shedding states. - Components NEVER rely on default OS-level TCP socket teardown during deployments; shutdown sequences must actively signal listeners to stop accepting new sockets while draining existing pipelines within Kubernetes `terminationGracePeriodSeconds`. ## Bad vs Good Implementations ### Graceful Draining and Load-Shed Concurrency Bounding #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: This allows unbounded concurrent requests to saturate Tokio worker threads, leading to memory exhaustion under traffic spikes. Furthermore, `with_graceful_shutdown` without a deadline blocks indefinitely if a client leaks an HTTP/1.1 keep-alive connection or streams slowly, causing Kubernetes to issue a hard SIGKILL and abruptly drop client connections. ```rust use axum::{routing::get, Router}; use tokio::net::TcpListener; #[tokio::main] async fn main() { let app = Router::new().route("/api", get(|| async { "ok" })); let listener = TcpListener::bind("0.0.0.0:3000").await.unwrap(); // ANTI-PATTERN: No concurrency bounds, no load-shedding, and graceful // shutdown waits indefinitely on hanging keep-alive or slow clients. axum::serve(listener, app) .with_graceful_shutdown(async { tokio::signal::ctrl_c().await.unwrap(); }) .await .unwrap(); } ``` #### Production Standard (Correct) > **Standard**: Protects Tokio executors with a bounded `ConcurrencyLimitLayer` combined with `LoadShedLayer` that fails fast with HTTP 503 when overloaded. Enforces dual SIGTERM/SIGINT listeners and brackets the graceful shutdown sequence with a strict 25-second deadline safely below standard K8s 30-second termination grace periods. ```rust use axum::{ error_handling::HandleErrorLayer, http::StatusCode, response::IntoResponse, routing::get, BoxError, Router, }; use std::time::Duration; use tokio::net::TcpListener; use tokio::signal::unix::{signal, SignalKind}; use tower::{load_shed::LoadShedLayer, limit::ConcurrencyLimitLayer, ServiceBuilder}; async fn handle_rejection(err: BoxError) -> impl IntoResponse { if err.is::<tower::load_shed::error::Overloaded>() { (StatusCode::SERVICE_UNAVAILABLE, "Server overloaded").into_response() } else { (StatusCode::INTERNAL_SERVER_ERROR, "Internal server error").into_response() } } async fn shutdown_signal() { let mut sigterm = signal(SignalKind::terminate()).expect("failed to install SIGTERM handler"); tokio::select! { _ = tokio::signal::ctrl_c() => {}, _ = sigterm.recv() => {}, } } #[tokio::main] async fn main() { let middleware = ServiceBuilder::new() .layer(HandleErrorLayer::new(handle_rejection)) .layer(LoadShedLayer::new()) .layer(ConcurrencyLimitLayer::new(1024)); let app = Router::new() .route("/api", get(|| async { "ok" })) .layer(middleware); let listener = TcpListener::bind("0.0.0.0:3000").await.unwrap(); let (drain_tx, drain_rx) = tokio::sync::watch::channel(()); let server = axum::serve(listener, app) .with_graceful_shutdown(async move { shutdown_signal().await; drop(drain_tx); }); tokio::select! { res = server => { res.unwrap(); }, _ = async { let mut rx = drain_rx; let _ = rx.changed().await; tokio::time::sleep(Duration::from_secs(25)).await; } => { eprintln!("Drain timeout reached; forcing socket shutdown"); } } } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: Axum 0.8 Tower Graceful Draining & Concurrency-Limit Isolation

Axum 0.8 Tower Graceful Draining & Concurrency-Limit Isolation defines an industrial-grade behavioral standard for backend engineering. Enforces strict concurrency bounding and deterministic connection draining in Axum 0.8 / Tower HTTP services. Prevents thread starvation and rolling-deployment connection blackholes by decoupling TCP accept loops from bounded task pools and enforcing hard deadlines on inflight request completion during SIGTERM. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Unbounded TCP connection ingestion with open-ended graceful shutdown causing Kubernetes pod termination timeouts and cascading thread pool exhaustion Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Unbounded TCP connection ingestion with open-ended graceful shutdown causing Kubernetes pod termination timeouts and cascading thread pool exhaustion	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS bound inflight requests per service/route using Tower's `ConcurrencyLimitLayer` combined with `LoadShedLayer` to reject excess traffic fast with HTTP 503 rather than queueing indefinitely.

Engineers MUST ALWAYS bound inflight requests per service/route using Tower's `ConcurrencyLimitLayer` combined with `LoadShedLayer` to reject excess traffic fast with HTTP 503 rather than queueing indefinitely.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.