Axum 0.8 Async Streaming Multipart Backpressure & OOM Invariant

--- description: Axum 0.8 Async Streaming Multipart Backpressure & OOM Invariant - Enforces strict end-to-end backpressure during high-throughput multipart file uploads in Axum 0.8. Mandates direct bounded pipeline streaming from client socket to storage sinks without intermediate heap accumulation, preventing unbounded memory consumption and Tokio worker thread starvation. globs: src/**/*.rs, crates/**/*.rs alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/axum-streaming-multipart-backpressure-invariant" --- # Axum 0.8 Async Streaming Multipart Backpressure & OOM Invariant > **Severity**: CRITICAL | **Category**: backend > **Anti-Pattern Warning**: Unbounded buffering of multipart chunks into in-memory Vec/Bytes or using unthrottled channels, causing OOM heap blowouts during file ingestion. ## System Constraints You are a Principal Systems Engineer enforcing strict backpressure invariants in Axum 0.8 and Tokio. You MUST reject any upload handling pattern that buffers incoming HTTP multipart byte streams into memory before writing to storage sinks, or that decouples socket reads from sink writes using unbounded channels or unthrottled background tasks. ## Strict Guidelines - Engineers MUST ALWAYS stream multipart fields directly from `axum::extract::Multipart` to the downstream sink using `tokio::io::copy` with `tokio_util::io::StreamReader` or explicit `await` loops on field chunks. - Engineers MUST NOT collect multipart chunks into `Vec<u8>`, `BytesMut`, or any unbounded heap structure unless explicitly gated by a strict `Content-Length` hard limit lower than 1MB. - Services MUST apply `tokio::io::AsyncWriteExt::flush` or leverage bounded channels with a capacity equal to or less than 2 chunks when streaming across Tokio task boundaries. - Code SHOULD propagate chunk read errors and sink write errors immediately to abort upload processing and trigger TCP RST/connection close to save server bandwidth. - Handlers NEVER spawn detached tasks (`tokio::spawn`) to process multipart streams without bidirectional backpressure synchronization and a join handle barrier. ## Bad vs Good Implementations ### Axum 0.8 Multipart Stream to File with Proper Backpressure #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Buffering incoming chunks into an unbounded Vec<u8> disables TCP backpressure. A fast client uploading a large file will allocate gigabytes of RAM in the Tokio worker heap, causing OOM panics and starving sibling tasks. ```rust use axum::{extract::Multipart, http::StatusCode, response::IntoResponse}; use tokio::fs::File; use tokio::io::AsyncWriteExt; pub async fn upload_handler(mut multipart: Multipart) -> Result<impl IntoResponse, (StatusCode, String)> { while let Ok(Some(mut field)) = multipart.next_field().await { // ANTI-PATTERN: Collecting entire stream into RAM before writing to disk let mut in_memory_buffer = Vec::new(); while let Ok(Some(chunk)) = field.chunk().await { in_memory_buffer.extend_from_slice(&chunk); // Heap allocation explosion } let mut file = File::create("/tmp/upload.bin").await.map_err(|e| { (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()) })?; file.write_all(&in_memory_buffer).await.map_err(|e| { (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()) })?; } Ok(StatusCode::OK) } ``` #### Production Standard (Correct) > **Standard**: Using StreamReader combined with tokio::io::copy bounds heap allocation to a small internal 8KB buffer. Upstream socket reads are naturally throttled whenever the local storage sink is busy flushing, guaranteeing zero-heap exhaustion. ```rust use axum::{extract::Multipart, http::StatusCode, response::IntoResponse}; use futures_util::TryStreamExt; use std::io; use tokio::fs::File; use tokio_util::io::StreamReader; pub async fn upload_handler(mut multipart: Multipart) -> Result<impl IntoResponse, (StatusCode, String)> { while let Ok(Some(field)) = multipart.next_field().await { let mut file = File::create("/tmp/upload.bin").await.map_err(|e| { (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()) })?; // INVARIANT: Map multipart stream to AsyncRead and copy boundedly to sink let stream = field.map_err(|err| io::Error::new(io::ErrorKind::Other, err)); let mut reader = StreamReader::new(stream); // tokio::io::copy reads a chunk, writes to sink, and flushes before requesting next chunk tokio::io::copy(&mut reader, &mut file).await.map_err(|e| { (StatusCode::INTERNAL_SERVER_ERROR, e.to_string()) })?; } Ok(StatusCode::OK) } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: Axum 0.8 Async Streaming Multipart Backpressure & OOM Invariant

Axum 0.8 Async Streaming Multipart Backpressure & OOM Invariant defines an industrial-grade behavioral standard for backend engineering. Enforces strict end-to-end backpressure during high-throughput multipart file uploads in Axum 0.8. Mandates direct bounded pipeline streaming from client socket to storage sinks without intermediate heap accumulation, preventing unbounded memory consumption and Tokio worker thread starvation. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Unbounded buffering of multipart chunks into in-memory Vec/Bytes or using unthrottled channels, causing OOM heap blowouts during file ingestion. Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Unbounded buffering of multipart chunks into in-memory Vec/Bytes or using unthrottled channels, causing OOM heap blowouts during file ingestion.	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS stream multipart fields directly from `axum

Engineers MUST ALWAYS stream multipart fields directly from `axum::extract::Multipart` to the downstream sink using `tokio::io::copy` with `tokio_util::io::StreamReader` or explicit `await` loops on field chunks.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.