Axum 0.8 Adaptive Load Shedding and Tower Cancellation Propagation

--- description: Axum 0.8 Adaptive Load Shedding and Tower Cancellation Propagation - Enforces proactive concurrency limits and cancellation propagation across Axum 0.8 HTTP pipelines and Tower middleware layers. Prevents thread pool saturation, executor starvation, and orphaned distributed RPC calls when upstream clients terminate connections under extreme load spikes. globs: src/**/*.rs alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/axum-0-8-load-shedding-cancellation-propagation" --- # Axum 0.8 Adaptive Load Shedding and Tower Cancellation Propagation > **Severity**: CRITICAL | **Category**: backend > **Anti-Pattern Warning**: Unbounded Request Queuing and Disconnected Background Tasks Without Drop-Driven Cancellation ## System Constraints You are a Principal Rust Backend Architect specializing in high-throughput network services with Axum 0.8 and Tokio. Mandate strict invariants regarding Tower load-shedding middleware, CancellationToken propagation, bounded concurrency layers, and RAII-based future cancellation. Ensure no background compute or downstream network I/O outlives disconnected client sockets. ## Strict Guidelines - Engineers MUST ALWAYS wrap Axum routers with `tower::load_shed::LoadShedLayer` combined with `tower::limit::GlobalConcurrencyLimitLayer` or `ConcurrencyLimitLayer` to fail fast with HTTP 503 Service Unavailable when saturated. - Engineers MUST NOT spawn unlinked detached Tokio tasks (`tokio::spawn`) inside route handlers for critical execution paths without wiring `tokio_util::sync::CancellationToken` or `tokio::select!` cancellation safety. - Handlers invoking downstream gRPC, HTTP, or database queries MUST bind operations directly to the request future lifecycle or pass a child `CancellationToken` derived from Axum's connection drop signal. - Services MUST implement custom error-handling middleware (`axum::error_handling::HandleErrorLayer`) directly above Tower's `LoadShedLayer` to convert `tower::load_shed::error::Overloaded` into a structured, backoff-compliant HTTP 503 response. - Components NEVER execute long-running compute loops or synchronous blocking blocks (`tokio::task::spawn_blocking`) without regular cooperative cancellation checks (`tokio::task::yield_now` or `token.is_cancelled()`). ## Bad vs Good Implementations ### Adaptive Concurrency Limiting and Cancellation-Aware Downstream Dispatch #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: The router lacks a load-shedding layer, allowing incoming spikes to queue infinitely until the OS runs out of file descriptors or memory. Furthermore, `tokio::spawn` detaches downstream processing; if an upstream client aborts the connection, the orphaned task continues consuming downstream RPC capacity and executor cycles. ```rust use axum::{routing::post, Router, Json}; use serde::Deserialize; #[derive(Deserialize)] struct Payload { task_id: String } // ANTI-PATTERN: Unbounded concurrency, handler spawns an unlinked task // that continues running even if the client disconnects or the server is overloaded. async fn bad_handler(Json(payload): Json<Payload>) -> &'static str { tokio::spawn(async move { // If client terminates connection, this expensive downstream call still proceeds call_expensive_downstream_service(&payload.task_id).await; }); "Accepted" } pub fn app() -> Router { Router::new().route("/process", post(bad_handler)) } async fn call_expensive_downstream_service(_id: &str) {} ``` #### Production Standard (Correct) > **Standard**: Combines `GlobalConcurrencyLimitLayer` and `LoadShedLayer` wrapped in a `HandleErrorLayer` to reject excess traffic instantly with HTTP 503 instead of queueing unbounded work. Handles downstream operations inline with RAII cancellation tokens so that if the client connection drops, the future is dropped and downstream I/O stops instantly. ```rust use axum::{ error_handling::HandleErrorLayer, http::StatusCode, routing::post, BoxError, Json, Router, }; use std::time::Duration; use tokio_util::sync::CancellationToken; use tower::limit::GlobalConcurrencyLimitLayer; use tower::load_shed::LoadShedLayer; use tower::ServiceBuilder; use serde::Deserialize; #[derive(Deserialize)] struct Payload { task_id: String } async fn handle_shed_error(err: BoxError) -> (StatusCode, &'static str) { if err.is::<tower::load_shed::error::Overloaded>() { (StatusCode::SERVICE_UNAVAILABLE, "Server overloaded; retry later") } else { (StatusCode::INTERNAL_SERVER_ERROR, "Unhandled middleware error") } } // INVARIANT: Request future drives execution. Dropping connection cancels downstream work immediately. async fn resilient_handler(Json(payload): Json<Payload>) -> Result<StatusCode, StatusCode> { let cancel_token = CancellationToken::new(); let child_token = cancel_token.child_token(); // Guard against orphaned execution using select! with token cancellation tokio::select! { _ = cancel_token.cancelled() => { Err(StatusCode::REQUEST_TIMEOUT) } res = call_expensive_downstream_service(&payload.task_id, child_token) => { res.map(|_| StatusCode::OK).map_err(|_| StatusCode::BAD_GATEWAY) } } } pub fn app() -> Router { let middleware_stack = ServiceBuilder::new() .layer(HandleErrorLayer::new(handle_shed_error)) .layer(LoadShedLayer::new()) .layer(GlobalConcurrencyLimitLayer::new(1024)) .timeout(Duration::from_secs(5)); Router::new() .route("/process", post(resilient_handler)) .layer(middleware_stack) } async fn call_expensive_downstream_service(_id: &str, _token: CancellationToken) -> Result<(), ()> { Ok(()) } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: Axum 0.8 Adaptive Load Shedding and Tower Cancellation Propagation

Axum 0.8 Adaptive Load Shedding and Tower Cancellation Propagation defines an industrial-grade behavioral standard for backend engineering. Enforces proactive concurrency limits and cancellation propagation across Axum 0.8 HTTP pipelines and Tower middleware layers. Prevents thread pool saturation, executor starvation, and orphaned distributed RPC calls when upstream clients terminate connections under extreme load spikes. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Unbounded Request Queuing and Disconnected Background Tasks Without Drop-Driven Cancellation Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Unbounded Request Queuing and Disconnected Background Tasks Without Drop-Driven Cancellation	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS wrap Axum routers with `tower

Engineers MUST ALWAYS wrap Axum routers with `tower::load_shed::LoadShedLayer` combined with `tower::limit::GlobalConcurrencyLimitLayer` or `ConcurrencyLimitLayer` to fail fast with HTTP 503 Service Unavailable when saturated.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.