Axum 0.8 Deterministic Concurrency Limits & Transport Load Shedding

--- description: Axum 0.8 Deterministic Concurrency Limits & Transport Load Shedding - Enforces deterministic bounded concurrency and proactive HTTP 503 load shedding in Axum 0.8 applications using Tower middleware. Guarantees that overload conditions trigger explicit transport-level shedding with Retry-After headers rather than thread pool exhaustion or generic 500 server errors. globs: src/**/*.rs alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/axum-tower-concurrency-limits-loadshedding" --- # Axum 0.8 Deterministic Concurrency Limits & Transport Load Shedding > **Severity**: CRITICAL | **Category**: backend > **Anti-Pattern Warning**: Unbounded In-Flight Requests and Unhandled Tower BoxError Overloads Causing Cascading Failures ## System Constraints You are a Principal Rust Architect enforcing deterministic concurrency limits and proactive load shedding for Axum 0.8 and Tower services. You must reject any Axum service topology that allows unbounded concurrency or that handles Tower load-shedding errors as opaque 500 Internal Server Errors. ## Strict Guidelines - Engineers MUST ALWAYS enclose `tower::load_shed::LoadShedLayer` and `tower::limit::ConcurrencyLimitLayer` within an outer `axum::error_handling::HandleErrorLayer`. - Engineers MUST downcast `tower::BoxError` inside `HandleErrorLayer` to detect `tower::load_shed::error::Overloaded` and return HTTP 503 (`StatusCode::SERVICE_UNAVAILABLE`) with a non-zero `Retry-After` header. - Services MUST NOT place liveness or readiness probes (`/livez`, `/readyz`, `/healthz`) behind the saturated `ConcurrencyLimitLayer` or `LoadShedLayer` pipeline to prevent orchestration death spirals during traffic spikes. - Engineers MUST calibrate the `ConcurrencyLimitLayer` bound to the maximum concurrency supported by downstream resource limits (such as `sqlx::Pool` max connections or CPU core limits). - Services MUST order `HandleErrorLayer` as the outermost wrapper relative to `LoadShedLayer` and `ConcurrencyLimitLayer` because Tower evaluates layers from outermost to innermost for request interception. ## Bad vs Good Implementations ### Naive Axum Service Without Handled Load Shedding vs. Resilient Bounded Architecture #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Omitting LoadShedLayer causes pending requests to queue indefinitely until client timeouts occur. Furthermore, health endpoints are trapped behind the application bottleneck, and any Tower limit failures will cause unhandled runtime panics or generic 500 errors. ```rust use axum::{routing::get, Router}; use tower::limit::ConcurrencyLimitLayer; pub fn app() -> Router { // ANTI-PATTERN: Indiscriminate concurrency limit with no load shedding or error mapping. // Requests exceeding limit will hang in backlog until socket timeouts occur, and health // check endpoints share the same bounded pipe, inducing false-positive pod restarts. Router::new() .route("/healthz", get(|| async { "OK" })) .route("/api/compute", get(|| async { "Processed" })) .layer(ConcurrencyLimitLayer::new(100)) } ``` #### Production Standard (Correct) > **Standard**: Properly positions HandleErrorLayer as the outermost wrapper, proactively sheds requests with HTTP 503 and a Retry-After header, bounds concurrent work to 64 in-flight units, and decouples the /healthz probe from the overloaded work queue. ```rust use axum::{ error_handling::HandleErrorLayer, http::{header, HeaderValue, StatusCode}, response::{IntoResponse, Response}, routing::get, BoxError, Router, }; use std::time::Duration; use tower::{limit::ConcurrencyLimitLayer, load_shed::LoadShedLayer, ServiceBuilder}; async fn handle_tower_error(err: BoxError) -> Response { if err.is::<tower::load_shed::error::Overloaded>() { ( StatusCode::SERVICE_UNAVAILABLE, [(header::RETRY_AFTER, HeaderValue::from_static("5"))], "Service saturated: request shed proactively.", ) .into_response() } else { ( StatusCode::INTERNAL_SERVER_ERROR, format!("Unhandled middleware error: {err}"), ) .into_response() } } pub fn app() -> Router { let protected_api = Router::new() .route("/compute", get(|| async { "Processed" })) .layer( ServiceBuilder::new() .layer(HandleErrorLayer::new(handle_tower_error)) .layer(LoadShedLayer::new()) .layer(ConcurrencyLimitLayer::new(64)), ); // Probes are explicitly decoupled from shedding and concurrency bounds Router::new() .route("/healthz", get(|| async { StatusCode::OK })) .nest("/api", protected_api) } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: Axum 0.8 Deterministic Concurrency Limits & Transport Load Shedding

Axum 0.8 Deterministic Concurrency Limits & Transport Load Shedding defines an industrial-grade behavioral standard for backend engineering. Enforces deterministic bounded concurrency and proactive HTTP 503 load shedding in Axum 0.8 applications using Tower middleware. Guarantees that overload conditions trigger explicit transport-level shedding with Retry-After headers rather than thread pool exhaustion or generic 500 server errors. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Unbounded In-Flight Requests and Unhandled Tower BoxError Overloads Causing Cascading Failures Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Unbounded In-Flight Requests and Unhandled Tower BoxError Overloads Causing Cascading Failures	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS enclose `tower

Engineers MUST ALWAYS enclose `tower::load_shed::LoadShedLayer` and `tower::limit::ConcurrencyLimitLayer` within an outer `axum::error_handling::HandleErrorLayer`.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.