Axum Multipart Stream Backpressure & Kernel Buffer Protection Protocol

--- description: Axum Multipart Stream Backpressure & Kernel Buffer Protection Protocol - Enforces strict demand-driven consumption of Axum multipart streaming bodies to eliminate Tokio executor task starvation and heap exhaustion. Prohibits in-memory payload aggregation across Tokio runtimes by mandating synchronous pull-based writes directly into bounded sinks to maintain TCP window zero-window advertisements. globs: src/**/*.rs alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/axum-zero-copy-multipart-backpressure-protocol" --- # Axum Multipart Stream Backpressure & Kernel Buffer Protection Protocol > **Severity**: CRITICAL | **Category**: backend > **Anti-Pattern Warning**: Buffering Multi-Gigabyte Stream Chunks into Unbounded Memory or Spawning Detached Channel Writers without Socket Backpressure ## System Constraints You are a Principal Systems Architect enforcing the Axum Zero-Copy Multipart Stream Backpressure Protocol. You must reject any code that consumes Axum `Field` payloads using full-buffer extractors (`field.bytes().await`, `field.text().await`, or `Vec::new()`), as well as detached producer-consumer loops missing explicit backpressure signals. Mandate bounded chunk-by-chunk iteration directly into `tokio::io::AsyncWrite` sinks. ## Strict Guidelines - Engineers MUST ALWAYS iterate over `Field::chunk().await` sequentially, writing each chunk directly to the underlying `AsyncWrite` sink before pulling subsequent frames. - Engineers MUST NOT call `field.bytes().await`, `to_bytes()`, or collect stream chunks into intermediary collections like `Vec<u8>` or `BytesMut` for file or payload uploads. - Code SHOULD leverage `tokio::io::copy` with `StreamReader` adapter or explicit `tokio::io::AsyncWriteExt::write_all` loops to yield task control to the Tokio reactor during I/O stalls. - Services MUST configure Axum request size limits using `DefaultBodyLimit::disable()` accompanied by an explicit, bounded per-field stream length validator. - Components NEVER spawn unbounded Tokio channels (`mpsc::unbounded_channel`) to offload incoming multipart network bytes to secondary tasks. ## Bad vs Good Implementations ### Multipart Stream Consumption with Immediate Backpressure #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Calling `field.bytes().await` buffers the complete multi-gigabyte network payload into the application process heap before initiating disk writes. Under high concurrency, this triggers rapid memory bloat, kernel OOM panics, and disables TCP socket flow control. ```rust use axum::extract::Multipart; use axum::http::StatusCode; use axum::response::IntoResponse; pub async fn upload_bad(mut multipart: Multipart) -> Result<impl IntoResponse, StatusCode> { while let Some(field) = multipart.next_field().await.map_err(|_| StatusCode::BAD_REQUEST)? { // CRITICAL FLAW: Loads entire multi-gigabyte file into heap memory let data = field.bytes().await.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?; tokio::fs::write("/tmp/uploaded_file.bin", data) .await .map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?; } Ok(StatusCode::OK) } ``` #### Production Standard (Correct) > **Standard**: Chunk-by-chunk processing awaits each `write_all` completion before requesting subsequent network chunks via `field.chunk().await`. If disk write throughput drops, the Tokio reactor halts socket reads, filling TCP socket buffers and applying native TCP receive-window backpressure to the client. ```rust use axum::extract::Multipart; use axum::http::StatusCode; use axum::response::IntoResponse; use tokio::fs::File; use tokio::io::AsyncWriteExt; pub async fn upload_good(mut multipart: Multipart) -> Result<impl IntoResponse, StatusCode> { while let Some(mut field) = multipart.next_field().await.map_err(|_| StatusCode::BAD_REQUEST)? { let mut destination = File::create("/tmp/uploaded_file.bin") .await .map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?; // INVARIANT FIX: Sequential stream polling guarantees that downstream // I/O blocks upstream TCP chunk ingestion, throttling the client via TCP window while let Some(chunk) = field.chunk().await.map_err(|_| StatusCode::BAD_REQUEST)? { destination .write_all(&chunk) .await .map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?; } destination.flush().await.map_err(|_| StatusCode::INTERNAL_SERVER_ERROR)?; } Ok(StatusCode::OK) } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: Axum Multipart Stream Backpressure & Kernel Buffer Protection Protocol

Axum Multipart Stream Backpressure & Kernel Buffer Protection Protocol defines an industrial-grade behavioral standard for backend engineering. Enforces strict demand-driven consumption of Axum multipart streaming bodies to eliminate Tokio executor task starvation and heap exhaustion. Prohibits in-memory payload aggregation across Tokio runtimes by mandating synchronous pull-based writes directly into bounded sinks to maintain TCP window zero-window advertisements. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Buffering Multi-Gigabyte Stream Chunks into Unbounded Memory or Spawning Detached Channel Writers without Socket Backpressure Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Buffering Multi-Gigabyte Stream Chunks into Unbounded Memory or Spawning Detached Channel Writers without Socket Backpressure	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS iterate over `Field

Engineers MUST ALWAYS iterate over `Field::chunk().await` sequentially, writing each chunk directly to the underlying `AsyncWrite` sink before pulling subsequent frames.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.