Bun 1.2 SQLite WAL Concurrency & S3 Zero-Copy Streaming Standard

--- description: Bun 1.2 SQLite WAL Concurrency & S3 Zero-Copy Streaming Standard - Mandates production configuration for Bun 1.2's native bun:sqlite engine using Write-Ahead Logging (WAL), busy timeout resiliency, and zero-copy iteration. Enforces direct kernel-level streaming for S3 native storage via Bun.s3 without intermediate heap memory buffering. globs: src/**/*.ts, src/db/**/*.ts, src/storage/**/*.ts alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/bun-sqlite-wal-s3-streaming-standard" --- # Bun 1.2 SQLite WAL Concurrency & S3 Zero-Copy Streaming Standard > **Severity**: CRITICAL | **Category**: backend > **Anti-Pattern Warning**: Blocking In-Memory SQLite Contention & Heap-Exhausting S3 Buffering ## System Constraints You are a Principal Software Architect enforcing Bun 1.2 embedded SQLite WAL concurrency and S3 native storage invariants. Mandate strict PRAGMA configurations, zero-copy query execution, and streaming S3 I/O. ## Strict Guidelines - Engineers MUST ALWAYS initialize `bun:sqlite` connections with `PRAGMA journal_mode = WAL;`, `PRAGMA synchronous = NORMAL;`, and `PRAGMA busy_timeout = 5000;` immediately upon opening. - Engineers MUST NOT read or buffer entire S3 object payloads into memory via `await response.arrayBuffer()` or `Buffer.concat()`; always stream directly via Bun.s3 or standard `ReadableStream` pipelines. - Code SHOULD use `query.values()` or `query.iterate()` instead of `query.all()` when executing high-throughput batch reads to eliminate redundant V8/JavaScript object key allocation overhead. - Services MUST execute multi-statement database mutations inside `db.transaction()` closures to guarantee atomicity and optimize WAL write batches. - Components NEVER instantiate S3 SDK clients (e.g., `@aws-sdk/client-s3`) when the native high-performance Bun 1.2 `S3Client` from `bun` is available. ## Bad vs Good Implementations ### Configuring bun:sqlite with WAL and Zero-Copy Iteration #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Default SQLite settings create a rollback journal that blocks readers during writes, leading to SQLITE_BUSY crashes under load. Calling .all() allocates JavaScript object wrappers with key names for every row, drastically increasing GC pressure. ```typescript import { Database } from 'bun:sqlite'; // ANTI-PATTERN: Default rollback journal causes lock contention under concurrency. // Unbounded query.all() allocates full object shapes for millions of rows. const db = new Database('production.db'); export function fetchRecentEvents(limit: number) { const query = db.query('SELECT id, payload, created_at FROM events ORDER BY id DESC LIMIT ?'); return query.all(limit); // Massive object allocation overhead } ``` #### Production Standard (Correct) > **Standard**: Enabling WAL mode alongside synchronous=NORMAL permits non-blocking concurrent reads while writes are ongoing. Setting a busy_timeout allows write queues to drain cleanly. Using query.values() returns lean positional tuples, bypassing object key instantiation. ```typescript import { Database } from 'bun:sqlite'; // PRODUCTION INVARIANT: Enforce WAL, busy_timeout, and normal sync for concurrency. const db = new Database('production.db'); db.run('PRAGMA journal_mode = WAL;'); db.run('PRAGMA synchronous = NORMAL;'); db.run('PRAGMA busy_timeout = 5000;'); const recentEventsStmt = db.query< [number, string, number], [number] >('SELECT id, payload, created_at FROM events ORDER BY id DESC LIMIT ?'); export function fetchRecentEvents(limit: number): [number, string, number][] { // Zero-copy array tuples eliminate object property allocation return recentEventsStmt.values(limit); } ``` ### Native Bun.s3 Multipart Upload Streaming #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Awaiting arrayBuffer() loads the entire file into Bun's heap. If multiple uploads execute simultaneously or files exceed memory constraints, the process crashes via Out-Of-Memory (OOM). ```typescript import { S3Client } from 'bun'; const client = new S3Client(); // ANTI-PATTERN: Ingesting whole stream into memory buffer before uploading to S3 export async function backupDatabaseToS3(req: Request) { const buffer = await req.arrayBuffer(); // Memory spike risking OOM on large payloads const file = client.file('backups/daily.db'); await file.write(buffer); return new Response('Uploaded'); } ``` #### Production Standard (Correct) > **Standard**: Leverages Bun.s3 file.writer() and streams the incoming request body directly to S3 via pipeTo. Backpressure is automatically maintained, and heap usage stays strictly flat regardless of file size. ```typescript import { S3Client } from 'bun'; const client = new S3Client(); // PRODUCTION INVARIANT: Stream input directly to S3 writer without in-memory buffering export async function backupDatabaseToS3(req: Request): Promise<Response> { if (!req.body) { return new Response('Missing request body', { status: 400 }); } const s3File = client.file(`backups/${crypto.randomUUID()}.db`); const writer = s3File.writer({ retry: 3, queueSize: 4 }); // Stream chunks directly from request to S3 without heap retention await req.body.pipeTo(writer); return new Response(JSON.stringify({ key: s3File.name }), { status: 201, headers: { 'Content-Type': 'application/json' } }); } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: Bun 1.2 SQLite WAL Concurrency & S3 Zero-Copy Streaming Standard

Bun 1.2 SQLite WAL Concurrency & S3 Zero-Copy Streaming Standard defines an industrial-grade behavioral standard for backend engineering. Mandates production configuration for Bun 1.2's native bun:sqlite engine using Write-Ahead Logging (WAL), busy timeout resiliency, and zero-copy iteration. Enforces direct kernel-level streaming for S3 native storage via Bun.s3 without intermediate heap memory buffering. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Blocking In-Memory SQLite Contention & Heap-Exhausting S3 Buffering Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Blocking In-Memory SQLite Contention & Heap-Exhausting S3 Buffering	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS initialize `bun

Engineers MUST ALWAYS initialize `bun:sqlite` connections with `PRAGMA journal_mode = WAL;`, `PRAGMA synchronous = NORMAL;`, and `PRAGMA busy_timeout = 5000;` immediately upon opening.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.