FastAPI Async Generator Client Disconnect Cancellation Interlock

Enforces deterministic cancellation and resource deallocation for FastAPI async generators and SSE streams upon client socket termination. Without an explicit ASGI disconnect polling loop or cancellation interlock, orphaned async generators continue streaming tokens from upstream LLM inference providers, triggering thread-pool exhaustion, runaway token bills, and memory leaks. Architects must wire Request.is_disconnected() polling or Task cancellation directly into async generator pipelines.

--- description: FastAPI Async Generator Client Disconnect Cancellation Interlock - Enforces deterministic cancellation and resource deallocation for FastAPI async generators and SSE streams upon client socket termination. Without an explicit ASGI disconnect polling loop or cancellation interlock, orphaned async generators continue streaming tokens from upstream LLM inference providers, triggering thread-pool exhaustion, runaway token bills, and memory leaks. Architects must wire Request.is_disconnected() polling or Task cancellation directly into async generator pipelines. globs: app/**/*.py, src/**/*.py, routers/**/*.py, api/**/*.py alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/fastapi-async-generator-disconnect-cancellation" --- # FastAPI Async Generator Client Disconnect Cancellation Interlock > **Severity**: CRITICAL | **Category**: backend > **Anti-Pattern Warning**: Unchecked Async Generators Consuming Upstream LLM Streams Post-Client Disconnect ## System Constraints You are a Principal Backend Architect enforcing strict ASGI socket lifecycle interlocks in FastAPI and Starlette. Mandate that all streaming responses, SSE generators, and upstream LLM pipelines continuously monitor client connection states and guarantee instantaneous cancellation of upstream network I/O upon client disconnect. ## Strict Guidelines - Engineers MUST ALWAYS pass the Starlette/FastAPI `Request` instance into streaming generator functions. - Engineers MUST periodically check `await request.is_disconnected()` inside long-running token loops or execute generation within an `asyncio.TaskGroup` / `asyncio.wait(..., return_when=FIRST_COMPLETED)` interlock against a disconnect sentinel task. - Code SHOULD wrap upstream LLM streaming responses in a `try...finally` block to invoke `await upstream_stream.close()` or explicitly cancel the upstream client task. - Services MUST NOT execute unmonitored blocking or threadpool-based generator loops (`run_in_threadpool`) for streaming I/O, as Starlette cannot propagate ASGI cancellation across non-cooperative thread boundaries. - Components NEVER yield chunk data after `await request.is_disconnected()` evaluates to True; they must break immediately and log an intentional client abort at INFO or DEBUG severity. ## Bad vs Good Implementations ### FastAPI SSE Generator with Disconnect Monitor Interlock #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Starlette's default StreamingResponse only detects client disconnection when attempting to send a chunk down a dead ASGI socket. If the upstream provider has network latency or buffers responses, the loop continues fetching costly LLM tokens indefinitely without releasing resources. ```python from fastapi import FastAPI from fastapi.responses import StreamingResponse from openai import AsyncOpenAI app = FastAPI() client = AsyncOpenAI() async def bad_llm_stream(prompt: str): # ANTI-PATTERN: Completely unmonitored upstream stream. # When client closes browser tab, this loop continues fetching # tokens from OpenAI until completion, draining quota and bandwidth. stream = await client.chat.completions.create( model="gpt-4o", messages=[{"role": "user", "content": prompt}], stream=True, ) async for chunk in stream: yield chunk.choices[0].delta.content or "" @app.get("/stream") async def stream_endpoint(prompt: str): return StreamingResponse(bad_llm_stream(prompt), media_type="text/event-stream") ``` #### Production Standard (Correct) > **Standard**: The generator receives the ASGI Request object, monitors request.is_disconnected() before processing subsequent tokens, properly handles asyncio.CancelledError, and executes upstream cleanup in a guaranteed finally block. ```python import asyncio from fastapi import FastAPI, Request from fastapi.responses import StreamingResponse from openai import AsyncOpenAI app = FastAPI() client = AsyncOpenAI() async def resilient_llm_stream(request: Request, prompt: str): stream = await client.chat.completions.create( model="gpt-4o", messages=[{"role": "user", "content": prompt}], stream=True, ) try: async for chunk in stream: # Invariant: Terminate immediately if client closed socket if await request.is_disconnected(): break content = chunk.choices[0].delta.content if content: yield content except asyncio.CancelledError: # Clean exit when ASGI worker cancels task raise finally: # Guaranteed upstream cleanup on client abort await stream.close() @app.get("/stream") async def stream_endpoint(request: Request, prompt: str): return StreamingResponse( resilient_llm_stream(request, prompt), media_type="text/event-stream", ) ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: FastAPI Async Generator Client Disconnect Cancellation Interlock

FastAPI Async Generator Client Disconnect Cancellation Interlock defines an industrial-grade behavioral standard for backend engineering. Enforces deterministic cancellation and resource deallocation for FastAPI async generators and SSE streams upon client socket termination. Without an explicit ASGI disconnect polling loop or cancellation interlock, orphaned async generators continue streaming tokens from upstream LLM inference providers, triggering thread-pool exhaustion, runaway token bills, and memory leaks. Architects must wire Request.is_disconnected() polling or Task cancellation directly into async generator pipelines. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Unchecked Async Generators Consuming Upstream LLM Streams Post-Client Disconnect Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Unchecked Async Generators Consuming Upstream LLM Streams Post-Client Disconnect	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS pass the Starlette/FastAPI `Request` instance into streaming generator functions.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.