Bun 1.2 Native S3 Streaming Backpressure Protocol & Zero-Heap Invariant

--- description: Bun 1.2 Native S3 Streaming Backpressure Protocol & Zero-Heap Invariant - Enforces process-safe object ingestion by piping raw HTTP incoming ReadableStreams directly into Bun 1.2's native S3 client without heap allocation. Bypassing native backpressure by materializing payloads via ArrayBuffer, Blob, or unthrottled AsyncIterables causes deterministic kernel OOM kills under concurrent multipart workloads. All S3 egress and ingress pipelines must bind native TCP socket backpressure to S3 upload sinks. globs: src/**/*.{ts,tsx,js,jsx} alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/bun-v12-s3-streaming-backpressure-protocol" --- # Bun 1.2 Native S3 Streaming Backpressure Protocol & Zero-Heap Invariant > **Severity**: CRITICAL | **Category**: backend > **Anti-Pattern Warning**: Buffering In-Flight HTTP Request Payloads in V8/JSC Heap Prior to S3 Transmission ## System Constraints You are a Principal Systems Architect enforcing strict memory and I/O efficiency standards in Bun 1.2 runtimes. You must prevent memory exhaustion (SIGKILL/137) caused by payload buffering. Mandate the use of Bun 1.2's native S3 API (Bun.s3 / S3Client) and direct ReadableStream passthrough so that kernel TCP socket backpressure is preserved throughout multi-gigabyte uploads. ## Strict Guidelines - Engineers MUST ALWAYS pass the raw incoming `req.body` ReadableStream directly into `s3.file(path).write()` or `s3.write(path, req.body)` without intermediate buffering. - Engineers MUST NOT invoke `.arrayBuffer()`, `.text()`, `.bytes()`, or `.json()` on incoming request bodies destined for S3 object persistence. - Engineers MUST NOT collect stream chunks manually via `for await (const chunk of req.body)` into an in-memory accumulator array. - Services MUST declare expected `Content-Length` and `Content-Type` headers directly in `S3FileOptions` to allow native chunk optimization without requiring stream inspection. - Services MUST handle `AbortSignal` downstream cancellation to immediately release socket descriptors and terminate the pending S3 upload pipeline. - Components NEVER use legacy AWS SDK (@aws-sdk/client-s3 or @aws-sdk/lib-storage) when running on Bun >= 1.2 unless specific unsupported AWS extensions are strictly required. ## Bad vs Good Implementations ### Catastrophic In-Memory Buffering vs. Bun 1.2 Native S3 Backpressure Stream #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Reading the entire incoming body into an ArrayBuffer forces the runtime to allocate userland heap for the full object size. If multiple large uploads arrive concurrently, heap memory spikes abruptly, triggering process-level OOM crashes (SIGKILL 137). TCP backpressure to the client is completely severed. ```typescript import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3"; const client = new S3Client({ region: "us-east-1" }); export async function handleUpload(req: Request): Promise<Response> { // CRITICAL DEFECT: Buffers entire stream into JavaScript heap. // A 2GB upload concurrently executed 5 times will exhaust a 4GB container RAM limit. const payload = await req.arrayBuffer(); const key = `uploads/${crypto.randomUUID()}`; await client.send(new PutObjectCommand({ Bucket: process.env.AWS_BUCKET_NAME!, Key: key, Body: new Uint8Array(payload), ContentLength: payload.byteLength, ContentType: req.headers.get("content-type") ?? "application/octet-stream", })); return new Response(JSON.stringify({ key, size: payload.byteLength }), { status: 201, headers: { "content-type": "application/json" }, }); } ``` #### Production Standard (Correct) > **Standard**: Passing the raw `req.body` directly to `destination.write()` delegates upload streaming to Bun's native C++/Zig engine. If the remote S3 endpoint slows down, TCP read windows contract, propagating backpressure directly to the client socket without consuming userland heap. ```typescript import { s3 } from "bun"; export async function handleUpload(req: Request): Promise<Response> { if (!req.body) { return new Response("Missing payload", { status: 400 }); } const key = `uploads/${crypto.randomUUID()}`; const contentLength = req.headers.get("content-length"); const contentType = req.headers.get("content-type") ?? "application/octet-stream"; // PRODUCTION INVARIANT: Native Bun.s3 stream write binds TCP socket backpressure directly. // Zero userland memory buffering occurs; chunks transfer via internal native ring buffers. const destination = s3.file(key, { bucket: process.env.AWS_BUCKET_NAME!, type: contentType, }); const bytesWritten = await destination.write(req.body, { signal: req.signal, size: contentLength ? Number.parseInt(contentLength, 10) : undefined, }); return new Response(JSON.stringify({ key, bytesWritten }), { status: 201, headers: { "content-type": "application/json" }, }); } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: Bun 1.2 Native S3 Streaming Backpressure Protocol & Zero-Heap Invariant

Bun 1.2 Native S3 Streaming Backpressure Protocol & Zero-Heap Invariant defines an industrial-grade behavioral standard for backend engineering. Enforces process-safe object ingestion by piping raw HTTP incoming ReadableStreams directly into Bun 1.2's native S3 client without heap allocation. Bypassing native backpressure by materializing payloads via ArrayBuffer, Blob, or unthrottled AsyncIterables causes deterministic kernel OOM kills under concurrent multipart workloads. All S3 egress and ingress pipelines must bind native TCP socket backpressure to S3 upload sinks. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Buffering In-Flight HTTP Request Payloads in V8/JSC Heap Prior to S3 Transmission Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Buffering In-Flight HTTP Request Payloads in V8/JSC Heap Prior to S3 Transmission	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS pass the raw incoming `req.body` ReadableStream directly into `s3.file(path).write()` or `s3.write(path, req.body)` without intermediate buffering.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.