Bun 1.2 Embedded SQLite WAL & S3 Native Zero-Copy Streaming Invariants

--- description: Bun 1.2 Embedded SQLite WAL & S3 Native Zero-Copy Streaming Invariants - Enforces high-throughput architectural patterns using Bun 1.2 native bun:sqlite and Bun.s3. Mandates strict WAL mode configuration with synchronous NORMAL pragmas, zero-copy buffer queries, and streaming multipart uploads directly via Bun.s3 client to prevent event loop starvation and heap bloat. globs: src/**/*.{ts,js}, lib/**/*.{ts,js} alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/bun-sqlite-wal-s3-native-standard" --- # Bun 1.2 Embedded SQLite WAL & S3 Native Zero-Copy Streaming Invariants > **Severity**: CRITICAL | **Category**: backend > **Anti-Pattern Warning**: Blocking disk I/O via default DELETE journal mode and buffering S3 payloads entirely in RAM ## System Constraints You are a Principal Engineer enforcing Bun 1.2 Embedded SQLite WAL & S3 Native Storage standards. Mandate zero-copy queries via bun:sqlite, mandatory Write-Ahead Logging (WAL) concurrency pragmas, and true native backpressure streaming with Bun.s3. Forbid intermediate full-buffer memory accumulation and blocking transaction models. ## Strict Guidelines - Engineers MUST ALWAYS configure SQLite instances with `PRAGMA journal_mode = WAL;` and `PRAGMA synchronous = NORMAL;` immediately upon opening a connection. - Engineers MUST ALWAYS use native Bun 1.2 `bun:sqlite` statement caching via `db.query(sql)` rather than recompiling ad-hoc SQL strings on hot paths. - Code MUST use typed array parameter bindings and avoid unnecessary serialization/deserialization cycles when persisting binary blobs or passing byte slices. - Engineers MUST NOT read entire payloads into `Buffer` or `ArrayBuffer` prior to writing to S3; all writes over 5MB MUST use native `Bun.s3.file(...).write(stream)` or `Bun.write(Bun.s3.file(...), stream)`. - Transactions involving multiple sequential writes MUST use `db.transaction()` wrapper functions to yield native C-level batching instead of manual multi-statement `BEGIN`/`COMMIT` SQL execution strings. - Components NEVER execute long-running asynchronous promises within an exclusive SQLite transaction, as it locks the database write pipe and causes SQLITE_BUSY errors in concurrent workers. ## Bad vs Good Implementations ### Embedded SQLite WAL Initialization and Zero-Copy Execution #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Default SQLite settings lock the entire database file during writes, stalling readers. String interpolation introduces SQL injection, and executing raw 'BEGIN' via db.run() bypasses Bun's native V8-to-C++ optimized transaction wrapper. ```typescript import { Database } from "bun:sqlite"; // ANTI-PATTERN: Default DELETE journal mode blocks concurrent reads during writes // Missing synchronous = NORMAL pragma risks high I/O latency const db = new Database("app.db"); export function recordEvent(userId: string, payload: unknown) { // Ad-hoc statement compilation on every execution // Manual BEGIN/COMMIT without C-level transaction lifecycle db.run("BEGIN TRANSACTION"); try { db.run(`INSERT INTO events (user_id, data) VALUES ('${userId}', '${JSON.stringify(payload)}')`); db.run("COMMIT"); } catch (e) { db.run("ROLLBACK"); throw e; } } ``` #### Production Standard (Correct) > **Standard**: Configures WAL mode for lock-free reader concurrency, applies synchronous=NORMAL for optimized disk durability, pre-compiles prepared statements, and employs `db.transaction()` for C-level atomic batching. ```typescript import { Database } from "bun:sqlite"; const db = new Database("app.db", { create: true }); // Mandatory WAL and synchronous pragmas for concurrent high-throughput workloads db.exec("PRAGMA journal_mode = WAL;"); db.exec("PRAGMA synchronous = NORMAL;"); db.exec("PRAGMA busy_timeout = 5000;"); // Pre-compile statement once outside hot-paths const insertEventStmt = db.query< { id: number }, { $userId: string; $data: string } >("INSERT INTO events (user_id, data) VALUES ($userId, $data) RETURNING id;"); // Native fast-path transaction const insertTx = db.transaction((userId: string, data: string) => { return insertEventStmt.get({ $userId: userId, $data: data }); }); export function recordEvent(userId: string, payload: unknown): number { const serialized = JSON.stringify(payload); const result = insertTx(userId, serialized); if (!result) throw new Error("Failed to record event"); return result.id; } ``` ### Streaming Request Body Direct to S3 Native Storage #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Buffers the entire request body into the Bun heap via `req.arrayBuffer()`, causing massive memory pressure and potential OOM errors during concurrent large-file uploads. ```typescript import { S3Client, PutObjectCommand } from "@aws-sdk/client-s3"; const s3 = new S3Client({ region: "us-east-1" }); export async function handleUpload(req: Request) { // ANTI-PATTERN: Buffers entire stream into RAM before sending // Out of memory crash under heavy concurrent traffic const arrayBuffer = await req.arrayBuffer(); const buffer = Buffer.from(arrayBuffer); await s3.send(new PutObjectCommand({ Bucket: "production-artifacts", Key: "uploads/data.bin", Body: buffer, })); return new Response("Uploaded"); } ``` #### Production Standard (Correct) > **Standard**: Uses Bun's native `s3.file().write(stream)` to pipe incoming request stream chunks directly to S3 via native multipart upload without ever materializing the entire payload in the JavaScript heap. ```typescript import { s3 } from "bun"; export async function handleUpload(req: Request): Promise<Response> { if (!req.body) { return new Response("Missing body", { status: 400 }); } const targetFile = s3.file("uploads/data.bin", { bucket: "production-artifacts", }); // Native Bun 1.2 zero-copy stream direct pipe to S3 multipart storage await targetFile.write(req.body); return new Response(JSON.stringify({ success: true, key: targetFile.name }), { status: 201, headers: { "Content-Type": "application/json" }, }); } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: Bun 1.2 Embedded SQLite WAL & S3 Native Zero-Copy Streaming Invariants

Bun 1.2 Embedded SQLite WAL & S3 Native Zero-Copy Streaming Invariants defines an industrial-grade behavioral standard for backend engineering. Enforces high-throughput architectural patterns using Bun 1.2 native bun:sqlite and Bun.s3. Mandates strict WAL mode configuration with synchronous NORMAL pragmas, zero-copy buffer queries, and streaming multipart uploads directly via Bun.s3 client to prevent event loop starvation and heap bloat. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Blocking disk I/O via default DELETE journal mode and buffering S3 payloads entirely in RAM Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Blocking disk I/O via default DELETE journal mode and buffering S3 payloads entirely in RAM	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS configure SQLite instances with `PRAGMA journal_mode = WAL

Engineers MUST ALWAYS configure SQLite instances with `PRAGMA journal_mode = WAL;` and `PRAGMA synchronous = NORMAL;` immediately upon opening a connection.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.