Axum 0.8 Tower Adaptive Concurrency & Load-Shedding Invariant

--- description: Axum 0.8 Tower Adaptive Concurrency & Load-Shedding Invariant - Mandates deterministic token-bucket or semaphore-based load-shedding and concurrency limiting at the outer boundary of Axum 0.8 Tower service pipelines. Prevents unbounded inflight heap exhaustion, Tokio runtime worker starvation, and cascading socket stall cascades under flash crowds by shedding excess traffic before payload consumption or asynchronous task spawning. globs: src/**/*.rs, crates/**/*.rs alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/axum-tower-adaptive-concurrency-loadshedding" --- # Axum 0.8 Tower Adaptive Concurrency & Load-Shedding Invariant > **Severity**: CRITICAL | **Category**: backend > **Anti-Pattern Warning**: Unbounded Task Spawning and Late Payload Buffering Without Outer Tower Load-Shedding Layers ## System Constraints You are a Principal Systems Architect enforcing the Axum 0.8 Tower Service Adaptive Concurrency Load-Shedding Invariant. Axum 0.8 applications must guarantee strict backpressure invariants: backpressure must be applied at the outer Tower layer before request buffering, allocation, or database connection acquisition. You must enforce explicit error handling for shed traffic (e.g., HTTP 503 with Retry-After headers) and prohibit unbounded concurrency or post-routing resource consumption during flash traffic surges. ## Strict Guidelines - Engineers MUST ALWAYS place Tower `LoadShedLayer` and concurrency limiters (`ConcurrencyLimitLayer` or adaptive token-bucket limiters) outside memory-buffering layers and routing extractors. - Engineers MUST ALWAYS pair `LoadShedLayer` with a `HandleErrorLayer` that captures `tower::load_shed::error::Overloaded` and maps it immediately to an HTTP 503 `StatusCode::SERVICE_UNAVAILABLE` or 429 `StatusCode::TOO_MANY_REQUESTS` with an appropriate `Retry-After` header. - Engineers MUST NOT rely on TCP listen backlogs or OS socket queues as primary traffic throttling mechanisms. - Code SHOULD enforce inflight request bounds using deterministic tokens or semaphores scaled to the host Tokio runtime's worker pool capacity and database connection pool size. - Services MUST NEVER allocate unbounded heap buffers (such as full body aggregation via `axum::body::to_bytes`) before the request passes the outer load-shedding gate. - Components NEVER spawn unmetered `tokio::spawn` tasks inside Axum handler paths under the assumption that HTTP client timeouts will safely prune inflight allocations. ## Bad vs Good Implementations ### Unbounded Request Ingestion Without Tower Load-Shedding (Memory Leak Under Flash Crowds) #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: The router extracts and buffers the full JSON payload into heap memory before any concurrency guard evaluates system health. Flash crowds flood the process with unconstrained allocations, and unbounded calls to tokio::spawn exhaust system memory and starve the Tokio threadpool without returning 503 backpressure. ```rust use axum::{routing::post, Router, extract::Json}; use serde::Deserialize; #[derive(Deserialize)] struct WorkPayload { data: Vec<u8>, } async fn handle_work(Json(payload): Json<WorkPayload>) -> &'static str { // ANTI-PATTERN: Ingests full body and spawns unmetered Tokio task. // Under 50,000 req/sec flash crowd, memory explodes before handling backpressure, // exhausting memory (OOM) and starving the Tokio reactor. tokio::spawn(async move { process_expensive_task(payload).await; }); "Accepted" } pub fn app() -> Router { Router::new().route("/work", post(handle_work)) } ``` #### Production Standard (Correct) > **Standard**: ConcurrencyLimitLayer and LoadShedLayer wrap the service boundary. Inflight requests exceeding 1024 are rejected instantaneously with Overloaded errors handled by HandleErrorLayer, generating HTTP 503 responses before body memory is allocated or Tokio tasks are spawned. ```rust use axum::{ error_handling::HandleErrorLayer, http::StatusCode, response::IntoResponse, routing::post, BoxError, Json, Router, }; use serde::Deserialize; use std::time::Duration; use tower::{ limit::ConcurrencyLimitLayer, load_shed::LoadShedLayer, ServiceBuilder, }; #[derive(Deserialize)] struct WorkPayload { data: Vec<u8>, } async fn handle_work(Json(payload): Json<WorkPayload>) -> impl IntoResponse { process_inline(payload).await; StatusCode::OK } async fn handle_rejection(err: BoxError) -> (StatusCode, [(&'static str, &'static str); 1], &'static str) { if err.is::<tower::load_shed::error::Overloaded>() { ( StatusCode::SERVICE_UNAVAILABLE, [("Retry-After", "5")], "Service Overloaded: Request shed by concurrency limiter", ) } else { ( StatusCode::INTERNAL_SERVER_ERROR, [("Retry-After", "0")], "Unhandled Service Fault", ) } } pub fn app() -> Router { // Load-shedding applied at the boundary BEFORE memory allocation and execution let middleware_stack = ServiceBuilder::new() .layer(HandleErrorLayer::new(handle_rejection)) .layer(LoadShedLayer::new()) .layer(ConcurrencyLimitLayer::new(1024)); Router::new() .route("/work", post(handle_work)) .layer(middleware_stack) } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: Axum 0.8 Tower Adaptive Concurrency & Load-Shedding Invariant

Axum 0.8 Tower Adaptive Concurrency & Load-Shedding Invariant defines an industrial-grade behavioral standard for backend engineering. Mandates deterministic token-bucket or semaphore-based load-shedding and concurrency limiting at the outer boundary of Axum 0.8 Tower service pipelines. Prevents unbounded inflight heap exhaustion, Tokio runtime worker starvation, and cascading socket stall cascades under flash crowds by shedding excess traffic before payload consumption or asynchronous task spawning. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Unbounded Task Spawning and Late Payload Buffering Without Outer Tower Load-Shedding Layers Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Unbounded Task Spawning and Late Payload Buffering Without Outer Tower Load-Shedding Layers	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS place Tower `LoadShedLayer` and concurrency limiters (`ConcurrencyLimitLayer` or adaptive token-bucket limiters) outside memory-buffering layers and routing extractors.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.