FastAPI AnyIO Worker Pool Capacity & Thread Starvation Guardrail

--- description: FastAPI AnyIO Worker Pool Capacity & Thread Starvation Guardrail - Enforces explicit thread pool capacity configuration and isolated AnyIO CapacityLimiter instances for blocking synchronous dependencies in FastAPI. Prevents default worker pool saturation from starving the event loop, deadlocking sync dependency resolution, and triggering health check failures under high concurrency. globs: **/*.py alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/fastapi-anyio-thread-starvation-guardrail" --- # FastAPI AnyIO Worker Pool Capacity & Thread Starvation Guardrail > **Severity**: CRITICAL | **Category**: backend > **Anti-Pattern Warning**: Exhausting Default AnyIO Thread Limiter with Unbounded Blocking I/O and Sync Dependencies ## System Constraints You are a Principal Backend Engineer enforcing concurrency safety and thread-pool isolation in FastAPI and AnyIO architectures. Mandate explicit worker pool bounds, dedicated capacity limiters for blocking dependencies, and strict segregation between async event-loop operations and blocking synchronous tasks. ## Strict Guidelines - Engineers MUST ALWAYS tune `anyio.to_thread.current_default_thread_limiter().total_tokens` inside the FastAPI lifespan context manager to match infrastructure compute envelopes. - Engineers MUST NOT execute blocking synchronous I/O (e.g., `time.sleep`, `requests.get`, synchronous SQLAlchemy sessions) directly inside `async def` endpoints. - Services MUST isolate distinct blocking domains (e.g., legacy sync database calls, CPU-bound image processing, PDF generation) into dedicated `anyio.CapacityLimiter` instances to prevent pool contention. - Engineers MUST execute blocking third-party functions via `anyio.to_thread.run_sync(func, *args, limiter=specific_limiter)` instead of letting them inherit the global default thread limiter. - Critical operational endpoints (e.g., `/healthz`, `/livez`, `/metrics`) MUST be implemented as non-blocking `async def` endpoints with zero synchronous dependencies to guarantee availability during thread pool saturation. - Code SHOULD prefer native asynchronous drivers (e.g., `httpx.AsyncClient`, `asyncpg`) over offloading synchronous libraries to thread pools. ## Bad vs Good Implementations ### Blocking I/O and Unbounded Thread Pool Saturation vs Isolated AnyIO Capacity Limiter #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Executing blocking I/O directly in an async def function freezes the asyncio event loop entirely, preventing all concurrent requests from processing. Relying on default `def` routes without tuning AnyIO default tokens causes rapid exhaustion of the default 40-token thread limiter under moderate load, causing all subsequent synchronous endpoints and dependencies to queue indefinitely. ```python import time import requests from fastapi import FastAPI app = FastAPI() # Anti-pattern 1: Direct blocking synchronous call inside async endpoint # Anti-pattern 2: Uses standard def without pool tuning, exhausting default 40 AnyIO tokens @app.get("/data") async def fetch_legacy_data(item_id: str): # Direct blocking HTTP call blocks the entire asyncio event loop thread response = requests.get(f"https://legacy-api.internal/items/{item_id}", timeout=10) return response.json() @app.get("/report") def generate_sync_report(): # Consumes default thread pool tokens; under 40+ concurrent requests, # all sync dependencies and endpoints deadlock time.sleep(5) return {"status": "completed"} ``` #### Production Standard (Correct) > **Standard**: Configures the AnyIO default thread limiter during lifespan startup. Uses non-blocking `httpx.AsyncClient` instead of blocking `requests`. Isolates unavoidable synchronous work using `anyio.to_thread.run_sync` constrained by a dedicated `anyio.CapacityLimiter(10)`, guaranteeing that legacy operations cannot consume the global worker pool or degrade the responsiveness of `/healthz`. ```python from contextlib import asynccontextmanager import anyio import anyio.to_thread from fastapi import FastAPI, Depends import httpx # Dedicated isolated limiter for heavy legacy blocking operations LEGACY_SERVICE_LIMITER = anyio.CapacityLimiter(10) def blocking_legacy_computation(data: dict) -> dict: # Dedicated synchronous task import time time.sleep(2) return {"processed": True, "payload": data} @asynccontextmanager async def lifespan(app: FastAPI): # Explicitly configure default AnyIO worker pool tokens to match capacity limiter = anyio.to_thread.current_default_thread_limiter() limiter.total_tokens = 100 yield app = FastAPI(lifespan=lifespan) @app.get("/data") async def fetch_legacy_data(item_id: str): # Use non-blocking async HTTP client for external network I/O async with httpx.AsyncClient(timeout=10.0) as client: response = await client.get(f"https://legacy-api.internal/items/{item_id}") response.raise_for_status() return response.json() @app.post("/report") async def generate_sync_report(payload: dict): # Offload blocking task to an isolated thread pool with dedicated limiter result = await anyio.to_thread.run_sync( blocking_legacy_computation, payload, limiter=LEGACY_SERVICE_LIMITER ) return result @app.get("/healthz") async def health_check(): # Pure async endpoint with zero sync dependencies: never starves return {"status": "healthy"} ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: FastAPI AnyIO Worker Pool Capacity & Thread Starvation Guardrail

FastAPI AnyIO Worker Pool Capacity & Thread Starvation Guardrail defines an industrial-grade behavioral standard for backend engineering. Enforces explicit thread pool capacity configuration and isolated AnyIO CapacityLimiter instances for blocking synchronous dependencies in FastAPI. Prevents default worker pool saturation from starving the event loop, deadlocking sync dependency resolution, and triggering health check failures under high concurrency. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Exhausting Default AnyIO Thread Limiter with Unbounded Blocking I/O and Sync Dependencies Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Exhausting Default AnyIO Thread Limiter with Unbounded Blocking I/O and Sync Dependencies	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS tune `anyio.to_thread.current_default_thread_limiter().total_tokens` inside the FastAPI lifespan context manager to match infrastructure compute envelopes.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.