Axum 0.8 Tower Middleware Adaptive Backpressure and Load-Shedding Standard

--- description: Axum 0.8 Tower Middleware Adaptive Backpressure and Load-Shedding Standard - Enforces deterministic p99 latency boundaries and prevents Tokio async executor threadpool starvation in Axum 0.8 microservices. Mandates explicit Tower load-shedding, bounded concurrency limits, and fail-fast admission controls returning HTTP 503 over unbounded task queuing. globs: src/**/*.rs, crates/**/*.rs alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/axum-tower-adaptive-backpressure-load-shedding" --- # Axum 0.8 Tower Middleware Adaptive Backpressure and Load-Shedding Standard > **Severity**: CRITICAL | **Category**: backend > **Anti-Pattern Warning**: Unbounded Request Buffering Causing Tokio Executor Starvation, P99 Latency Spikes, and OOM Crashes ## System Constraints You are a Principal Software Architect enforcing the Axum 0.8 Tower Middleware Adaptive Backpressure and Load-Shedding Standard. Your job is to strictly eliminate unbounded task execution, unbounded queues, and unmetered async handlers in Rust Axum services. Mandate fail-fast rejection under congestion with HTTP 503/429 status codes, deterministic timeout budgets, and strict bounded concurrency pipelines using Tower layers. ## Strict Guidelines - Engineers MUST ALWAYS order `tower::load_shed::LoadShedLayer` upstream of `tower::limit::ConcurrencyLimitLayer` combined with `tower::buffer::BufferLayer` or `HandleErrorLayer` to fail-fast shed excess requests before executor saturation. - Engineers MUST NOT spawn detached, unmetered futures via raw `tokio::spawn` inside Axum request handlers without a bounded permit or semaphore. - Services MUST return HTTP 503 `Service Unavailable` or HTTP 429 `Too Many Requests` with a deterministic `Retry-After` header when shedding traffic under high load. - Components NEVER use `tokio::sync::mpsc::unbounded_channel` for request buffering, work ingestion, or inter-task messaging in HTTP pipelines. - Engineers MUST configure end-to-end request timeout layers (e.g., `tower_http::timeout::TimeoutLayer`) around downstream services to prevent slow connections from holding concurrency slots indefinitely. ## Bad vs Good Implementations ### Unbounded Task Spawning and Missing Concurrency Bounds vs. Bounded Tower Load-Shedding Pipeline #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Under traffic spikes, `tokio::spawn` allocates an unbounded number of tasks into the runtime run queue. This exhausts memory, causes context-switching thrashing, degrades p99 latency to multi-second intervals, and risks OOM termination because the service has no mechanism to push back or shed load. ```rust use axum::{routing::post, Router, Json}; use serde::Deserialize; #[derive(Deserialize)] struct WorkPayload { data: String } // ANTI-PATTERN: No concurrency bounds, unbounded background spawn, // and no fail-fast mechanism when the Tokio worker pool is saturated. async fn handle_work(Json(payload): Json<WorkPayload>) -> &'static str { tokio::spawn(async move { // Heavy async processing spawned without bounding permits process_heavy_task(payload.data).await; }); "accepted" } async fn process_heavy_task(_data: String) { tokio::time::sleep(tokio::time::Duration::from_millis(500)).await; } pub fn app() -> Router { Router::new().route("/work", post(handle_work)) } ``` #### Production Standard (Correct) > **Standard**: The pipeline wraps routes in a `ConcurrencyLimitLayer` bounded to 1024 concurrent requests, immediately shedding incoming excess with `LoadShedLayer`. Saturated requests bypass handler execution and fail-fast with HTTP 503 and `Retry-After`, preserving runtime stability and ensuring bounded p99 latency for admitted requests. ```rust use axum::{ error_handling::HandleErrorLayer, http::StatusCode, response::{IntoResponse, Response}, routing::post, BoxError, Json, Router, }; use serde::Deserialize; use std::time::Duration; use tower::{ limit::ConcurrencyLimitLayer, load_shed::error::Overloaded, load_shed::LoadShedLayer, ServiceBuilder, }; use tower_http::timeout::TimeoutLayer; #[derive(Deserialize)] struct WorkPayload { data: String } async fn handle_work(Json(payload): Json<WorkPayload>) -> Result<&'static str, StatusCode> { process_heavy_task(payload.data).await; Ok("processed") } async fn process_heavy_task(_data: String) { tokio::time::sleep(Duration::from_millis(50)).await; } async fn handle_middleware_error(err: BoxError) -> Response { if err.is::<Overloaded>() { return ( StatusCode::SERVICE_UNAVAILABLE, [("Retry-After", "1")], "Load shed: server overloaded, please retry", ).into_response(); } (StatusCode::INTERNAL_SERVER_ERROR, "Internal server error").into_response() } pub fn app() -> Router { // LoadShedLayer must sit outside ConcurrencyLimitLayer so saturated requests // fail-fast with Overloaded error rather than queueing indefinitely. let middleware_stack = ServiceBuilder::new() .layer(HandleErrorLayer::new(handle_middleware_error)) .layer(LoadShedLayer::new()) .layer(ConcurrencyLimitLayer::new(1024)) .layer(TimeoutLayer::new(Duration::from_millis(500))); Router::new() .route("/work", post(handle_work)) .layer(middleware_stack) } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: Axum 0.8 Tower Middleware Adaptive Backpressure and Load-Shedding Standard

Axum 0.8 Tower Middleware Adaptive Backpressure and Load-Shedding Standard defines an industrial-grade behavioral standard for backend engineering. Enforces deterministic p99 latency boundaries and prevents Tokio async executor threadpool starvation in Axum 0.8 microservices. Mandates explicit Tower load-shedding, bounded concurrency limits, and fail-fast admission controls returning HTTP 503 over unbounded task queuing. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Unbounded Request Buffering Causing Tokio Executor Starvation, P99 Latency Spikes, and OOM Crashes Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Unbounded Request Buffering Causing Tokio Executor Starvation, P99 Latency Spikes, and OOM Crashes	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS order `tower

Engineers MUST ALWAYS order `tower::load_shed::LoadShedLayer` upstream of `tower::limit::ConcurrencyLimitLayer` combined with `tower::buffer::BufferLayer` or `HandleErrorLayer` to fail-fast shed excess requests before executor saturation.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.