Bun 1.2 Embedded SQLite WAL Concurrency & Native S3 Zero-Copy Streaming Standard

--- description: Bun 1.2 Embedded SQLite WAL Concurrency & Native S3 Zero-Copy Streaming Standard - Enforces strict production invariants for Bun 1.2 runtime storage primitives: SQLite WAL mode with non-blocking concurrent readers, zero-copy query execution via bun:sqlite prepared statements, and direct multipart streaming transfers using Bun.s3 native client without intermediate memory buffering. globs: src/**/*.{ts,tsx}, lib/**/*.{ts,tsx} alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/bun-12-sqlite-wal-s3-native-storage-standard" --- # Bun 1.2 Embedded SQLite WAL Concurrency & Native S3 Zero-Copy Streaming Standard > **Severity**: CRITICAL | **Category**: backend > **Anti-Pattern Warning**: Memory-Buffered S3 Transfers, Unprepared Statements, and Exclusive Locking SQLite Mode in Bun 1.2 ## System Constraints You are a Principal Engineer enforcing Bun 1.2 native storage architectural standards. Mandate zero-copy query pipelines with bun:sqlite, enforce WAL (Write-Ahead Logging) with explicit busy timeouts and defensive transactions, and require direct streaming for Bun.s3 native uploads/downloads without buffering large blobs into JavaScript V8/JSC heap. ## Strict Guidelines - Engineers MUST configure SQLite databases with `journal_mode = WAL`, `synchronous = NORMAL`, and a non-zero `busy_timeout` (e.g. 5000ms) immediately upon instantiation to prevent deadlocks and blocking concurrent readers. - Engineers MUST use pre-compiled statements (`db.query()`) and native type bindings (`get()`, `all()`, `values()`, `as()`) to leverage Bun's zero-copy C++ fast path instead of raw string interpolation or dynamically repeated statement compilation. - Engineers MUST NOT read full files or network responses into memory (e.g., using `await file.arrayBuffer()` or `await response.bytes()`) before sending them to S3; always pass `ReadableStream`, `Blob`, or Bun `FileRef` directly to `Bun.s3.write()` or `s3Client.write()`. - Engineers MUST wrap all multi-statement mutations in `db.transaction()` callbacks, relying on Bun's automatic `IMMEDIATE` transaction handling to guarantee isolation under high concurrent WAL contention. - Engineers MUST ensure S3 download streams via `Bun.s3.file().stream()` are piped directly to consumer response bodies or disk destinations using `Bun.write(destination, stream)` without intermediate memory concatenation. ## Bad vs Good Implementations ### SQLite WAL Concurrency and Zero-Copy Prepared Queries #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Default journal mode lacks WAL concurrency, leading to reader starvation during writes. Re-instantiating the Database per call and re-parsing queries destroys performance, while string interpolation creates severe SQL injection risks. ```typescript import { Database } from "bun:sqlite"; // ANTI-PATTERN: Default DELETE journal mode causes lock escalation on writes, // raw queries cause continuous re-parsing, and string interpolation introduces SQL injection. export function queryUser(userId: string) { const db = new Database("app.db"); const rawQuery = `SELECT id, email, metadata FROM users WHERE id = '${userId}'`; const results = db.query(rawQuery).all(); db.close(); return results[0]; } ``` #### Production Standard (Correct) > **Standard**: Configures WAL mode and 5000ms busy timeout to prevent concurrency blocks, and pre-compiles the query with strict typed parameters leveraging Bun's native zero-copy C++ execution engine. ```typescript import { Database } from "bun:sqlite"; // Initialize persistent singleton database instance with production pragmas export const db = new Database("app.db", { create: true, strict: true }); // Mandatory WAL and concurrency invariants db.run("PRAGMA journal_mode = WAL;"); db.run("PRAGMA synchronous = NORMAL;"); db.run("PRAGMA busy_timeout = 5000;"); interface UserRow { id: string; email: string; metadata: string; } // Pre-compiled prepared statement utilizing Bun's fast zero-copy path const findUserById = db.query<UserRow, [string]>( "SELECT id, email, metadata FROM users WHERE id = ?1" ); export function queryUser(userId: string): UserRow | null { return findUserById.get(userId); } ``` ### Direct Zero-Memory S3 Multipart Streaming #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Awaiting `request.arrayBuffer()` loads the entire payload into the runtime heap, causing high GC pauses, excessive memory usage, and Out-Of-Memory (OOM) fatal crashes on large multipart uploads. ```typescript import { S3Client } from "bun"; const client = new S3Client({ accessKeyId: process.env.AWS_ACCESS_KEY_ID!, secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!, bucket: process.env.S3_BUCKET!, }); // ANTI-PATTERN: Buffering entire stream into memory array before uploading to S3 export async function uploadArtifact(key: string, request: Request) { const bytes = await request.arrayBuffer(); // Causes OOM under high payload sizes await client.write(key, bytes); return new Response("Uploaded"); } ``` #### Production Standard (Correct) > **Standard**: Directly passes `request.body` (a `ReadableStream`) to `s3.write()`, delegating chunked multipart upload processing to Bun's native C++ transport layer with zero intermediate JavaScript heap allocations. ```typescript import { S3Client } from "bun"; const s3 = new S3Client({ accessKeyId: process.env.AWS_ACCESS_KEY_ID!, secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY!, bucket: process.env.S3_BUCKET!, }); // Stream directly from HTTP Request into Bun's native S3 client export async function uploadArtifact(key: string, request: Request): Promise<Response> { if (!request.body) { return new Response("Missing request body", { status: 400 }); } // Zero-copy direct streaming upload handled at C++ native layer await s3.write(key, request.body, { type: request.headers.get("content-type") ?? "application/octet-stream", }); return new Response(JSON.stringify({ key, uploaded: true }), { status: 201, headers: { "content-type": "application/json" }, }); } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: Bun 1.2 Embedded SQLite WAL Concurrency & Native S3 Zero-Copy Streaming Standard

Bun 1.2 Embedded SQLite WAL Concurrency & Native S3 Zero-Copy Streaming Standard defines an industrial-grade behavioral standard for backend engineering. Enforces strict production invariants for Bun 1.2 runtime storage primitives: SQLite WAL mode with non-blocking concurrent readers, zero-copy query execution via bun:sqlite prepared statements, and direct multipart streaming transfers using Bun.s3 native client without intermediate memory buffering. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Memory-Buffered S3 Transfers, Unprepared Statements, and Exclusive Locking SQLite Mode in Bun 1.2 Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Memory-Buffered S3 Transfers, Unprepared Statements, and Exclusive Locking SQLite Mode in Bun 1.2	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST configure SQLite databases with `journal_mode = WAL`, `synchronous = NORMAL`, and a non-zero `busy_timeout` (e.g. 5000ms) immediately upon instantiation to prevent deadlocks and blocking concurrent readers.

Engineers MUST configure SQLite databases with `journal_mode = WAL`, `synchronous = NORMAL`, and a non-zero `busy_timeout` (e.g. 5000ms) immediately upon instantiation to prevent deadlocks and blocking concurrent readers.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.