FastAPI AnyIO Concurrency-Bounded Zero-Copy Memory Streaming

--- description: FastAPI AnyIO Concurrency-Bounded Zero-Copy Memory Streaming - Enforces strict chunked streaming of binary payloads using AnyIO concurrency primitives and zero-copy buffer protocols in FastAPI endpoints. Prevents catastrophic asyncio event-loop starvation and OOM crashes caused by reading unbounded request bodies into heap memory or saturating the default thread pool with blocking disk I/O under high concurrent load. globs: app/**/*.py, src/**/*.py, api/**/*.py alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/fastapi-anyio-bounded-zerocopy-streaming" --- # FastAPI AnyIO Concurrency-Bounded Zero-Copy Memory Streaming > **Severity**: CRITICAL | **Category**: backend > **Anti-Pattern Warning**: Unbounded `await file.read()` and Unthrottled Disk I/O Ingestion in Asynchronous Endpoints ## System Constraints You are a Principal Engineer enforcing FastAPI AnyIO Concurrency-Bounded Zero-Copy Memory Streaming. Mandate strict chunk-based iteration over ASGI byte streams, backpressure enforcement via bounded AnyIO semaphores, and asynchronous zero-copy buffer passing to avoid event-loop blocking and unmetered heap amplification. ## Strict Guidelines - Engineers MUST ALWAYS iterate over request byte streams using chunked iterators (`request.stream()` or `UploadFile.read(chunk_size)`) bounded by fixed-size pre-allocated memory buffers (e.g., 64 KiB to 1 MiB). - Engineers MUST NOT call `UploadFile.read()` or `Request.body()` without an explicit byte limit, as reading unbounded multi-gigabyte payloads directly into Python heap memory triggers immediate Out-Of-Memory (OOM) fatal kills. - Engineers MUST bound concurrent binary ingestions using an `anyio.CapacityLimiter` or `anyio.Semaphore` to prevent uncontrolled disk thrashing and thread-pool exhaustion. - Engineers MUST offload file disk I/O to asynchronous streams via `anyio.open_file` instead of standard blocking synchronous Python file operations (`open()`, `os.write()`). - Services MUST enforce strict stream cancellation handlers using `anyio.create_task_group()` to clean up dangling file descriptors and temp allocations upon client disconnect. ## Bad vs Good Implementations ### Ingesting Large Binary Payloads via AnyIO Bounded Stream #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: `await file.read()` loads the entire file into the Python process heap, causing immediate OOM kills when multiple large files are uploaded concurrently. Furthermore, `open()` and `f.write()` block the async event loop, halting all incoming request handling across the entire server worker. ```python from fastapi import FastAPI, UploadFile, File import shutil app = FastAPI() @app.post("/upload") async def upload_file(file: UploadFile = File(...)): # CRITICAL BUG: Reads entire payload into RAM in one shot content = await file.read() # CRITICAL BUG: Blocking synchronous file write in async loop with open(f"/tmp/{file.filename}", "wb") as f: f.write(content) return {"size": len(content)} ``` #### Production Standard (Correct) > **Standard**: Iterates over the ASGI stream in small chunks using `request.stream()`, writing directly to disk via non-blocking `anyio.open_file`. Employs `anyio.CapacityLimiter` to constrain concurrent disk I/O and catches AnyIO cancellation to ensure partial files are cleanly unlinked if the client drops connection. ```python from fastapi import FastAPI, Request, HTTPException, status import anyio from anyio import open_file from pathlib import Path app = FastAPI() CHUNK_SIZE = 64 * 1024 # 64 KiB bounded chunks GLOBAL_UPLOAD_LIMITER = anyio.CapacityLimiter(10) # Max 10 concurrent disk stream writes @app.post("/upload/{filename}") async def upload_file(filename: str, request: Request): target_path = Path("/tmp") / Path(filename).name bytes_written = 0 try: async with GLOBAL_UPLOAD_LIMITER: async with await open_file(target_path, "wb") as out_file: async for chunk in request.stream(): if not chunk: continue await out_file.write(chunk) bytes_written += len(chunk) except anyio.get_cancelled_exc_class(): if target_path.exists(): target_path.unlink(missing_ok=True) raise except Exception as exc: if target_path.exists(): target_path.unlink(missing_ok=True) raise HTTPException(status_code=status.HTTP_500_INTERNAL_SERVER_ERROR, detail=str(exc)) return {"status": "success", "bytes_written": bytes_written} ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: FastAPI AnyIO Concurrency-Bounded Zero-Copy Memory Streaming

FastAPI AnyIO Concurrency-Bounded Zero-Copy Memory Streaming defines an industrial-grade behavioral standard for backend engineering. Enforces strict chunked streaming of binary payloads using AnyIO concurrency primitives and zero-copy buffer protocols in FastAPI endpoints. Prevents catastrophic asyncio event-loop starvation and OOM crashes caused by reading unbounded request bodies into heap memory or saturating the default thread pool with blocking disk I/O under high concurrent load. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Unbounded `await file.read()` and Unthrottled Disk I/O Ingestion in Asynchronous Endpoints Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Unbounded `await file.read()` and Unthrottled Disk I/O Ingestion in Asynchronous Endpoints	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS iterate over request byte streams using chunked iterators (`request.stream()` or `UploadFile.read(chunk_size)`) bounded by fixed-size pre-allocated memory buffers (e.g., 64 KiB to 1 MiB).

Engineers MUST ALWAYS iterate over request byte streams using chunked iterators (`request.stream()` or `UploadFile.read(chunk_size)`) bounded by fixed-size pre-allocated memory buffers (e.g., 64 KiB to 1 MiB).

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.