Bun 1.2 Native S3 Streaming Backpressure and Zero-Allocation Upload Protocol

--- description: Bun 1.2 Native S3 Streaming Backpressure and Zero-Allocation Upload Protocol - Mandates streaming inbound HTTP request bodies directly into object storage via Bun 1.2 native S3 APIs without buffering payload chunks into the JavaScript runtime heap. Enforces strict upstream backpressure signaling, native chunk draining, and transactional multipart abort handling to prevent memory exhaustion under concurrent multi-gigabyte transfers. globs: src/**/*.{ts,tsx}, server/**/*.{ts,tsx}, api/**/*.{ts,tsx} alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/bun-s3-streaming-backpressure-multipart" --- # Bun 1.2 Native S3 Streaming Backpressure and Zero-Allocation Upload Protocol > **Severity**: CRITICAL | **Category**: backend > **Anti-Pattern Warning**: Buffering Inbound Multi-Gigabyte Payloads into JSC Heap Buffers Before S3 Dispatch ## System Constraints You are a Principal Software Architect enforcing Bun 1.2 Native S3 Multipart Upload Streaming Backpressure Protocol. You must reject any implementation that buffers entire files, uses req.arrayBuffer(), req.blob(), or accumulates chunks in JS arrays prior to S3 persistence. Enforce zero-allocation streaming pipes via Bun.s3 native writers with deterministic backpressure and abort signaling. ## Strict Guidelines - Engineers MUST ALWAYS stream inbound HTTP request bodies (req.body) directly to Bun.s3 writer streams or pipe through native Web Streams without materializing buffers in JavaScriptCore heap. - Engineers MUST NOT invoke req.arrayBuffer(), req.blob(), req.bytes(), or Buffer.concat() on unbounded file transfer endpoints. - Code MUST respect stream backpressure by awaiting writer.drain() or using native pipeTo() semantics when feeding native S3 sinks. - Services MUST register AbortSignal handlers to cancel pending uploads and issue AbortMultipartUpload or writer.abort() if the client disconnects prematurely. - Components NEVER store multipart upload part chunks in memory arrays while awaiting concurrent part dispatch; chunking must be bounded by a fixed-size queue with explicit highWaterMark limits. ## Bad vs Good Implementations ### Inbound Stream to Bun.s3 Native Writer with Backpressure #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Calling `await req.arrayBuffer()` loads the entire file into JavaScript engine memory. At high concurrency or with multi-gigabyte files, this exhausts host memory and triggers process termination without any backpressure applied to the client. ```typescript import { S3Client, PutObjectCommand } from '@aws-sdk/client-s3'; const s3 = new S3Client({ region: 'us-east-1' }); Bun.serve({ port: 3000, async fetch(req) { if (req.method === 'POST' && req.url.endsWith('/upload')) { // FATAL: Buffers full payload (e.g., 2GB) into JSC heap. // Concurrent uploads trigger immediate OOM killer crash. const buffer = await req.arrayBuffer(); await s3.send(new PutObjectCommand({ Bucket: 'production-assets', Key: 'uploads/file.bin', Body: Buffer.from(buffer), })); return new Response('Uploaded', { status: 200 }); } return new Response('Not Found', { status: 404 }); }, }); ``` #### Production Standard (Correct) > **Standard**: Leverages Bun 1.2's native `Bun.s3` zero-copy stream writer. Piping `req.body` directly to the file writer applies backpressure down the TCP connection to the client and aborts object creation immediately if the client disconnects, maintaining constant memory overhead (O(1)). ```typescript import { s3 } from 'bun'; const bucket = s3.file('production-assets', { region: process.env.AWS_REGION, accessKeyId: process.env.AWS_ACCESS_KEY_ID, secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY, }); Bun.serve({ port: 3000, async fetch(req) { if (req.method === 'POST' && req.url.endsWith('/upload')) { if (!req.body) { return new Response('Empty body', { status: 400 }); } const destination = bucket.file(`uploads/${crypto.randomUUID()}.bin`); const writer = destination.writer({ highWaterMark: 1024 * 1024 * 8 }); // 8MB native buffer try { // Direct native streaming pipe with automatic backpressure await req.body.pipeTo(writer, { signal: req.signal, preventClose: false, }); return new Response(JSON.stringify({ key: destination.name }), { status: 201, headers: { 'Content-Type': 'application/json' }, }); } catch (err: unknown) { await writer.abort('Client aborted or connection dropped'); return new Response('Stream aborted', { status: 499 }); } } return new Response('Not Found', { status: 404 }); }, }); ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: Bun 1.2 Native S3 Streaming Backpressure and Zero-Allocation Upload Protocol

Bun 1.2 Native S3 Streaming Backpressure and Zero-Allocation Upload Protocol defines an industrial-grade behavioral standard for backend engineering. Mandates streaming inbound HTTP request bodies directly into object storage via Bun 1.2 native S3 APIs without buffering payload chunks into the JavaScript runtime heap. Enforces strict upstream backpressure signaling, native chunk draining, and transactional multipart abort handling to prevent memory exhaustion under concurrent multi-gigabyte transfers. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Buffering Inbound Multi-Gigabyte Payloads into JSC Heap Buffers Before S3 Dispatch Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Buffering Inbound Multi-Gigabyte Payloads into JSC Heap Buffers Before S3 Dispatch	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS stream inbound HTTP request bodies (req.body) directly to Bun.s3 writer streams or pipe through native Web Streams without materializing buffers in JavaScriptCore heap.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.