FastAPI & Pydantic-Core Zero-Copy Bounded Stream Invariant

Q: How does adopting this rule prevent AI hallucinations?

This rule explicitly declares what the AI must avoid (Buffering Unbounded HTTP Streams into Monolithic In-Memory Pydantic Deserializers) and prescribes verified positive examples. Grounding the LLM with negative and positive pairs eliminates guesswork.

Q: Which AI coding environments are supported?

This specification is multi-platform compatible: Cursor (.cursor/rules/*.mdc & .cursorrules), Anthropic Claude Code (CLAUDE.md), Windsurf (.windsurfrules), Codex / Cline (AGENTS.md), and GitHub Copilot (.github/copilot-instructions.md).

Q: Can I combine this rule with other technology stacks?

Yes. Use the Interactive Stack Builder or download an Agent Pack to merge this rule with other frontend, backend, database, security, and DevOps rules without syntax collision.

FastAPI & Pydantic-Core Zero-Copy Bounded Stream Invariant

--- description: FastAPI & Pydantic-Core Zero-Copy Bounded Stream Invariant - Defends asynchronous FastAPI endpoints against memory-inflation and recursion exhaustion Denial of Service (DoS) attacks. Mandates bounded chunked stream consumption with hard byte caps prior to model deserialization and configures pydantic-core depth limits. Prevents monolithic in-memory buffering and stack overflow crashes from malicious deeply nested JSON payloads. globs: app/**/*.py, src/**/*.py, api/**/*.py alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/fastapi-pydantic-bounded-stream-invariant" --- # FastAPI & Pydantic-Core Zero-Copy Bounded Stream Invariant > **Severity**: CRITICAL | **Category**: backend > **Anti-Pattern Warning**: Buffering Unbounded HTTP Streams into Monolithic In-Memory Pydantic Deserializers ## System Constraints You are a Principal Software Architect enforcing zero-copy bounded streaming and strict pydantic-core parsing invariants in FastAPI. You must never allow unbounded `await request.body()` reads or unconstrained recursive parsing of arbitrary JSON streams. ## Strict Guidelines - Engineers MUST ALWAYS enforce strict payload byte limits via a bounded streaming byte-counting generator before buffering chunks into memory. - Engineers MUST NOT rely exclusively on the HTTP `Content-Length` header for size validation, as chunked transfer encodings omit or spoof this value. - Services MUST configure `pydantic-core` deserializers with explicit recursion depth limits (`validate_depth_limit` <= 64) when processing dynamic nested payloads. - Engineers MUST consume large or streaming uploads using `request.stream()` with fixed-size buffer chunks (e.g., 64KB) to maintain bounded memory footprint. - Endpoints accepting batch or stream data MUST process elements via zero-copy iterative parsers or chunked validations rather than deserializing multi-megabyte payloads in a single frame. ## Bad vs Good Implementations ### Bounded Chunked Stream Consumption with Pydantic-Core Depth Limitation #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Calling `await request.body()` buffers the entire incoming payload directly into system memory, allowing malicious clients to exhaust process memory with slow or infinite transfer streams. Additionally, using standard deserialization on untrusted JSON without explicit recursion depth limits leaves the worker vulnerable to C-level stack overflow or high CPU lockup. ```python from fastapi import FastAPI, Request, HTTPException from pydantic import BaseModel app = FastAPI() class DeeplyNestedPayload(BaseModel): data: dict @app.post("/upload") async def upload_payload(request: Request): # CRITICAL: Vulnerable to memory-inflation and stack exhaustion DoS. # 1. Loads entire request body into RAM without size upper bounds. # 2. Uses default Pydantic validation vulnerable to deeply nested recursive JSON bomb. raw_bytes = await request.body() return DeeplyNestedPayload.model_validate_json(raw_bytes) ``` #### Production Standard (Correct) > **Standard**: The bounded streaming generator enforces a strict runtime byte limit during iteration, aborting immediately with HTTP 413 if the boundary is breached. It uses a custom `pydantic-core.SchemaValidator` configured with `validate_depth_limit: 32` to neutralize nested recursive JSON bomb attacks. ```python from fastapi import FastAPI, Request, HTTPException, status from pydantic_core import SchemaValidator, core_schema import json app = FastAPI() MAX_STREAM_BYTES = 10 * 1024 * 1024 # 10 MB Hard Ceiling CHUNK_SIZE = 64 * 1024 # 64 KB Buffer # Configure schema validator with strict recursion depth cap validator = SchemaValidator( core_schema.chain_schema([ core_schema.dict_schema() ]), config={"validate_depth_limit": 32} ) async def bounded_byte_stream(request: Request, max_bytes: int): total_bytes = 0 buffer = bytearray() async for chunk in request.stream(): total_bytes += len(chunk) if total_bytes > max_bytes: raise HTTPException( status_code=status.HTTP_413_REQUEST_ENTITY_TOO_LARGE, detail="Payload exceeds maximum streaming memory limit." ) buffer.extend(chunk) return bytes(buffer) @app.post("/upload") async def upload_payload(request: Request): safe_bytes = await bounded_byte_stream(request, MAX_STREAM_BYTES) try: # Validate using bounded recursion depth and zero-copy JSON parsing return validator.validate_json(safe_bytes) except ValueError as exc: raise HTTPException(status_code=status.HTTP_422_UNPROCESSABLE_ENTITY, detail=str(exc)) ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: FastAPI & Pydantic-Core Zero-Copy Bounded Stream Invariant

FastAPI & Pydantic-Core Zero-Copy Bounded Stream Invariant defines an industrial-grade behavioral standard for backend engineering. Defends asynchronous FastAPI endpoints against memory-inflation and recursion exhaustion Denial of Service (DoS) attacks. Mandates bounded chunked stream consumption with hard byte caps prior to model deserialization and configures pydantic-core depth limits. Prevents monolithic in-memory buffering and stack overflow crashes from malicious deeply nested JSON payloads. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Buffering Unbounded HTTP Streams into Monolithic In-Memory Pydantic Deserializers Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Buffering Unbounded HTTP Streams into Monolithic In-Memory Pydantic Deserializers	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS enforce strict payload byte limits via a bounded streaming byte-counting generator before buffering chunks into memory.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.