Bun 1.2 Embedded SQLite WAL Concurrency & S3 Multipart Stream Guardrails

--- description: Bun 1.2 Embedded SQLite WAL Concurrency & S3 Multipart Stream Guardrails - Enforces high-throughput, low-latency architecture using Bun 1.2's native `bun:sqlite` engine and native `Bun.s3` client. Guarantees non-blocking WAL-mode concurrency, zero-copy buffer queries, robust checkpointing, and unbuffered backpressure-governed multipart stream uploads directly between storage tiers. globs: src/**/*.ts, services/**/*.ts, lib/**/*.ts alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/bun-1-2-sqlite-wal-s3-native-storage" --- # Bun 1.2 Embedded SQLite WAL Concurrency & S3 Multipart Stream Guardrails > **Severity**: CRITICAL | **Category**: backend > **Anti-Pattern Warning**: Blocking disk I/O with standard Node wrappers, unbuffered in-memory S3 uploads, and default SQLite rollback journals causing concurrency deadlocks. ## System Constraints You are a Principal Engineer enforcing Bun 1.2 Embedded SQLite WAL & S3 Native Storage invariants. Mandate zero-copy binary queries via bun:sqlite, explicit WAL mode with proper busy timeouts, and direct non-buffering stream piping to native Bun.s3 storage. Prevent Node.js legacy shims (such as better-sqlite3 or AWS SDK v3) when native Bun primitives exist. ## Strict Guidelines - Engineers MUST ALWAYS initialize `bun:sqlite` with `PRAGMA journal_mode = WAL;`, `PRAGMA synchronous = NORMAL;`, and explicit `PRAGMA busy_timeout = 5000;` to avoid database locked errors under concurrency. - Engineers MUST ALWAYS use native `Bun.s3.file()` and `Bun.write()` or `Bun.s3.write()` with `ReadableStream` pipelines to stream data, avoiding intermediate `Buffer.concat()` in memory. - Engineers MUST NOT import Node.js legacy SQLite drivers (`better-sqlite3`, `sqlite3`) or heavyweight AWS SDK clients (`@aws-sdk/client-s3`) when native `bun:sqlite` and native `Bun.s3` client are available in Bun 1.2. - Code SHOULD use `query.as(type)` and zero-copy binary reads via `query.values()` or TypedArrays when reading SQLite blob data directly for S3 streaming. - Services MUST execute routine non-blocking SQLite checkpoints using `PRAGMA wal_checkpoint(PASSIVE);` or manage auto-checkpoint thresholds to prevent unbounded WAL file growth. ## Bad vs Good Implementations ### Native SQLite WAL Query and S3 Streaming Pipeline #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Uses external Node.js dependencies instead of Bun 1.2 native primitives, runs SQLite in standard journal mode without concurrency pragmas, and loads all rows/blobs into high-heap memory instead of native streaming. ```typescript import Database from 'better-sqlite3'; import { S3Client, PutObjectCommand } from '@aws-sdk/client-s3'; // Bad: Node library shims, default journal mode, in-memory buffering const db = new Database('data.db'); const s3 = new S3Client({ region: 'us-east-1' }); export async function exportBlobs() { const rows = db.prepare('SELECT id, payload FROM export_queue').all() as { id: string; payload: Buffer }[]; for (const row of rows) { // In-memory array allocation and high memory consumption await s3.send(new PutObjectCommand({ Bucket: 'app-exports', Key: `exports/${row.id}.bin`, Body: row.payload })); } } ``` #### Production Standard (Correct) > **Standard**: Leverages Bun 1.2 native `bun:sqlite` with WAL mode and native `s3` binding. Avoids external dependencies, streams binary payloads without unnecessary allocation, and executes passive checkpoints safely. ```typescript import { Database } from 'bun:sqlite'; import { s3 } from 'bun'; // Good: Native Bun SQLite with WAL concurrency & busy timeouts const db = new Database('data.db', { create: true }); db.run('PRAGMA journal_mode = WAL;'); db.run('PRAGMA synchronous = NORMAL;'); db.run('PRAGMA busy_timeout = 5000;'); const query = db.query<[string, Uint8Array], [string]>( 'SELECT id, payload FROM export_queue WHERE processed = 0 LIMIT ?' ); export async function exportBlobs(batchSize: number = 100): Promise<void> { const rows = query.values(batchSize); for (const [id, payload] of rows) { // Native zero-copy S3 streaming write via Bun 1.2 S3 API const s3File = s3.file(`exports/${id}.bin`, { bucket: 'app-exports', acl: 'private' }); await Bun.write(s3File, payload); } // Non-blocking passive checkpoint to maintain WAL size db.run('PRAGMA wal_checkpoint(PASSIVE);'); } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: Bun 1.2 Embedded SQLite WAL Concurrency & S3 Multipart Stream Guardrails

Bun 1.2 Embedded SQLite WAL Concurrency & S3 Multipart Stream Guardrails defines an industrial-grade behavioral standard for backend engineering. Enforces high-throughput, low-latency architecture using Bun 1.2's native `bun:sqlite` engine and native `Bun.s3` client. Guarantees non-blocking WAL-mode concurrency, zero-copy buffer queries, robust checkpointing, and unbuffered backpressure-governed multipart stream uploads directly between storage tiers. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Blocking disk I/O with standard Node wrappers, unbuffered in-memory S3 uploads, and default SQLite rollback journals causing concurrency deadlocks. Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Blocking disk I/O with standard Node wrappers, unbuffered in-memory S3 uploads, and default SQLite rollback journals causing concurrency deadlocks.	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS initialize `bun

Engineers MUST ALWAYS initialize `bun:sqlite` with `PRAGMA journal_mode = WAL;`, `PRAGMA synchronous = NORMAL;`, and explicit `PRAGMA busy_timeout = 5000;` to avoid database locked errors under concurrency.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.