Axum 0.8 Tower Service Graceful Shedding & Payload Streaming Memory Cap Standard

--- description: Axum 0.8 Tower Service Graceful Shedding & Payload Streaming Memory Cap Standard - Enforces strict heap-bound backpressure, explicit payload streaming limits, and graceful load shedding in Axum 0.8 applications. Prevents Tokio worker pool starvation and Out-Of-Memory (OOM) cascading failures by combining Tower LoadShedLayer with deterministic byte-stream caps and fail-fast backpressure mechanisms. globs: src/**/*.rs, crates/**/*.rs alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/axum-0-8-tower-graceful-shedding-payload-streaming-memory-cap" --- # Axum 0.8 Tower Service Graceful Shedding & Payload Streaming Memory Cap Standard > **Severity**: CRITICAL | **Category**: backend > **Anti-Pattern Warning**: Unbounded Payload Buffering and Missing Load Shedding Leading to Tokio Worker Pool Starvation ## System Constraints You are a Principal Rust Architect enforcing the Axum 0.8 Tower Service Graceful Shedding & Payload Streaming Memory Cap Standard. You must reject any unbounded byte buffering, unconstrained chunk streaming, or routers lacking deterministic load shedding layers. Mandate explicit size guards, Tower LoadShedLayer with HandleErrorLayer, and constant-space streaming consumers. ## Strict Guidelines - Engineers MUST ALWAYS declare an explicit `DefaultBodyLimit::max(N)` or route-level `RequestBodyLimitLayer` to prevent unbounded heap allocations on inbound requests. - Engineers MUST NOT collect raw `axum::body::Body` or `http_body_util::BodyExt::collect` streams into memory without checking running totals against a strict heap budget. - Routers MUST wrap high-throughput or resource-intensive routes in a `tower::ServiceBuilder` containing `tower::load_shed::LoadShedLayer` paired with a `tower::buffer::BufferLayer` and `axum::error_handling::HandleErrorLayer` that converts `tower::load_shed::error::Overloaded` into HTTP 503 Service Unavailable. - Code SHOULD prefer streaming chunk validation via bounded channels or bounded async sinks (e.g., persisting directly to disk/object storage via fixed-size memory buffers) rather than loading entire multi-part or streaming payloads into memory. - Handlers MUST NEVER block the asynchronous Tokio runtime; CPU-heavy decompression or cryptographic parsing of streamed frames must be dispatched to `tokio::task::spawn_blocking`. - Components consuming streams MUST handle client disconnects and premature stream terminations gracefully without leaking allocated buffers or orphaned tasks. ## Bad vs Good Implementations ### Bounded Streaming and Load Shedding in Axum 0.8 #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: The handler consumes the entire request body via `BodyExt::collect()` into an unbounded `Bytes` buffer. It lacks `DefaultBodyLimit`, lacks Tower `LoadShedLayer`, and can trigger an OOM panic or starvation on Tokio workers under high concurrency or malicious slowloris/payload attacks. ```rust use axum::{routing::post, Router, body::Body, extract::Request, response::IntoResponse}; use http_body_util::BodyExt; // ANTI-PATTERN: Unbounded body collection bypasses memory safety guarantees. // If thousands of clients stream slow 50MB payloads simultaneously, // the Tokio threadpool runs out of memory, crashing the worker process. async fn ingest_data(req: Request<Body>) -> impl IntoResponse { let entire_body = req.into_body().collect().await.unwrap().to_bytes(); // Allocating arbitrary heap bytes without limits format!("Received {} bytes", entire_body.len()) } pub fn app() -> Router { Router::new().route("/ingest", post(ingest_data)) } ``` #### Production Standard (Correct) > **Standard**: Combines Axum 0.8 `DefaultBodyLimit`, Tower's `BufferLayer` and `LoadShedLayer` wrapped in `HandleErrorLayer`, and continuous incremental byte accounting in the streaming loop. Requests beyond capacity are gracefully shed with HTTP 503 instead of exhausting the Tokio heap. ```rust use axum::{ body::Body, error_handling::HandleErrorLayer, extract::{DefaultBodyLimit, Request}, http::StatusCode, response::IntoResponse, routing::post, BoxError, Router, }; use futures_util::StreamExt; use std::time::Duration; use tower::{load_shed::LoadShedLayer, buffer::BufferLayer, ServiceBuilder}; const MAX_PAYLOAD_BYTES: usize = 10 * 1024 * 1024; // 10 MiB const BUFFER_CAPACITY: usize = 1024; async fn handle_overload_or_timeout(err: BoxError) -> (StatusCode, &'static str) { if err.is::<tower::load_shed::error::Overloaded>() { (StatusCode::SERVICE_UNAVAILABLE, "Server overloaded: request shed") } else { (StatusCode::INTERNAL_SERVER_ERROR, "Internal service error") } } async fn ingest_data_stream(req: Request<Body>) -> Result<impl IntoResponse, (StatusCode, &'static str)> { let mut stream = req.into_body().into_data_stream(); let mut total_bytes: usize = 0; while let Some(chunk_result) = stream.next().await { let chunk = chunk_result.map_err(|_| (StatusCode::BAD_REQUEST, "Stream read error"))?; total_bytes += chunk.len(); if total_bytes > MAX_PAYLOAD_BYTES { return Err((StatusCode::PAYLOAD_TOO_LARGE, "Exceeded maximum payload size")); } // Process chunk incrementally with constant memory overhead } Ok((StatusCode::OK, format!("Processed {} bytes safely", total_bytes))) } pub fn app() -> Router { let shed_protection = ServiceBuilder::new() .layer(HandleErrorLayer::new(handle_overload_or_timeout)) .layer(BufferLayer::new(BUFFER_CAPACITY)) .layer(LoadShedLayer::new()) .timeout(Duration::from_secs(30)); Router::new() .route("/ingest", post(ingest_data_stream)) .layer(DefaultBodyLimit::max(MAX_PAYLOAD_BYTES)) .layer(shed_protection) } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: Axum 0.8 Tower Service Graceful Shedding & Payload Streaming Memory Cap Standard

Axum 0.8 Tower Service Graceful Shedding & Payload Streaming Memory Cap Standard defines an industrial-grade behavioral standard for backend engineering. Enforces strict heap-bound backpressure, explicit payload streaming limits, and graceful load shedding in Axum 0.8 applications. Prevents Tokio worker pool starvation and Out-Of-Memory (OOM) cascading failures by combining Tower LoadShedLayer with deterministic byte-stream caps and fail-fast backpressure mechanisms. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Unbounded Payload Buffering and Missing Load Shedding Leading to Tokio Worker Pool Starvation Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Unbounded Payload Buffering and Missing Load Shedding Leading to Tokio Worker Pool Starvation	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS declare an explicit `DefaultBodyLimit

Engineers MUST ALWAYS declare an explicit `DefaultBodyLimit::max(N)` or route-level `RequestBodyLimitLayer` to prevent unbounded heap allocations on inbound requests.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.