pgvector Halfvec Quantization Memory-Budget & HNSW Convergence Bound

--- description: pgvector Halfvec Quantization Memory-Budget & HNSW Convergence Bound - Enforces strict off-heap memory budgets and bounded graph convergence for pgvector workloads using halfvec (16-bit floating point) quantization under write saturation. Prevents database worker OOM terminations, index bloat, and recall degradation by standardizing memory limits, staging ingest pipelines, and isolating construction parameters. globs: src/repositories/**/*.ts, src/database/**/*.ts, src/services/**/*.ts, migrations/**/*.sql alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/pgvector-halfvec-hnsw-memory-convergence-bound" --- # pgvector Halfvec Quantization Memory-Budget & HNSW Convergence Bound > **Severity**: CRITICAL | **Category**: backend > **Anti-Pattern Warning**: Unquantized In-Place HNSW Writes Under High Ingestion Inducing Out-Of-Memory and Graph Fragmentation ## System Constraints You are a Principal Software Architect enforcing pgvector Halfvec Quantization Memory-Budget & HNSW Convergence Bounds. You must mandate FP16 halfvec data types for all high-scale vector stores, isolate dynamic writes from real-time indexing using staged unindexed partitions or controlled batch upserts, and explicitly configure transaction-scoped memory parameters (`maintenance_work_mem`, `hnsw.ef_search`, and `max_parallel_maintenance_workers`) to guarantee deterministic memory ceilings and bounded recall. ## Strict Guidelines - Engineers MUST ALWAYS declare vector columns as `halfvec(dimensions)` instead of single-precision `vector(dimensions)` for embedding indices exceeding 100,000 vectors. - Engineers MUST NOT execute continuous high-frequency singleton inserts directly into tables with active HNSW indices during write saturation; ingest into an unindexed staging table or buffer partition before merge. - Code SHOULD explicitly set transaction-level `SET LOCAL hnsw.ef_search = <bound>` tailored to target recall vs. latency bounds rather than relying on system defaults. - Services MUST enforce strict off-heap maintenance limits (`SET LOCAL maintenance_work_mem = '<budget>MB'`) before triggering index creation or `REINDEX CONCURRENTLY` to prevent Linux OOM-killer invocations. - Components NEVER construct HNSW indexes with `m > 32` or `ef_construction > 128` on write-saturated tables without validating memory sizing: ensure `(m * 2 * 4 bytes + dimension * 2 bytes) * row_count` fits within the allocated shared buffer working set. ## Bad vs Good Implementations ### HNSW Vector Ingestion and Query Execution Guardrail #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Using unquantized 32-bit `vector` doubles working-set RAM. Performing direct singleton writes into an active HNSW index under write saturation fragments the graph, degrades recall accuracy, and risks process termination. Omitting `hnsw.ef_search` bounding provides no convergence control under load. ```typescript import { Pool } from 'pg'; export class VectorRepository { constructor(private readonly pool: Pool) {} // BAD: Direct singleton insert into FP32 vector with active HNSW index // Causes rapid graph node thrashing, uncontrolled WAL amplification, and memory bloat. async insertEmbedding(id: string, embedding: number[]): Promise<void> { await this.pool.query( 'INSERT INTO documents (id, embedding) VALUES ($1, $2::vector)', [id, JSON.stringify(embedding)] ); } // BAD: Uses default unbounded ef_search and unquantized cosine distance async findSimilar(queryVector: number[], limit: number) { const result = await this.pool.query( 'SELECT id, embedding <=> $1::vector AS distance FROM documents ORDER BY distance ASC LIMIT $2', [JSON.stringify(queryVector), limit] ); return result.rows; } } ``` #### Production Standard (Correct) > **Standard**: The good implementation uses `halfvec` to halve working RAM requirements, validates dimensionality, stages writes to decouple ingestion bursts from real-time HNSW index locks, and bounds the HNSW convergence via explicit session-scoped `SET LOCAL hnsw.ef_search`. ```typescript import { Pool, PoolClient } from 'pg'; export interface SearchOptions { efSearch?: number; limit: number; } export class VectorRepository { // Invariant: Vector dimension 1536 quantized to FP16 halfvec private static readonly DIMENSIONS = 1536; private static readonly DEFAULT_EF_SEARCH = 40; constructor(private readonly pool: Pool) {} // GOOD: Batch inserts into an unindexed staging partition, casting explicitly to halfvec async insertStagedBatch(records: Array<{ id: string; embedding: number[] }>): Promise<void> { if (records.length === 0) return; const client: PoolClient = await this.pool.connect(); try { await client.query('BEGIN'); // Enforce batch write into unindexed staging table to prevent HNSW lock contention const ids = records.map(r => r.id); const vectors = records.map(r => { if (r.embedding.length !== VectorRepository.DIMENSIONS) { throw new Error(`Embedding size mismatch: expected ${VectorRepository.DIMENSIONS}`); } return `[${r.embedding.join(',')}]`; }); await client.query( `INSERT INTO documents_staging (id, embedding) SELECT id, emb::halfvec(${VectorRepository.DIMENSIONS}) FROM UNNEST($1::text[], $2::text[]) AS t(id, emb) ON CONFLICT (id) DO UPDATE SET embedding = EXCLUDED.embedding`, [ids, vectors] ); await client.query('COMMIT'); } catch (error) { await client.query('ROLLBACK'); throw error; } finally { client.release(); } } // GOOD: Bounded hnsw.ef_search scoped per transaction, querying halfvec column with fixed memory budget async findSimilar(queryVector: number[], options: SearchOptions) { const client: PoolClient = await this.pool.connect(); const efSearch = options.efSearch ?? VectorRepository.DEFAULT_EF_SEARCH; const vectorStr = `[${queryVector.join(',')}]`; try { await client.query('BEGIN READ ONLY'); // Deterministic convergence bound per transaction session await client.query(`SET LOCAL hnsw.ef_search = ${efSearch}`); const result = await client.query( `SELECT id, embedding <-> $1::halfvec(${VectorRepository.DIMENSIONS}) AS distance FROM documents ORDER BY embedding <-> $1::halfvec(${VectorRepository.DIMENSIONS}) LIMIT $2`, [vectorStr, options.limit] ); await client.query('COMMIT'); return result.rows; } catch (error) { await client.query('ROLLBACK'); throw error; } finally { client.release(); } } } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: pgvector Halfvec Quantization Memory-Budget & HNSW Convergence Bound

pgvector Halfvec Quantization Memory-Budget & HNSW Convergence Bound defines an industrial-grade behavioral standard for backend engineering. Enforces strict off-heap memory budgets and bounded graph convergence for pgvector workloads using halfvec (16-bit floating point) quantization under write saturation. Prevents database worker OOM terminations, index bloat, and recall degradation by standardizing memory limits, staging ingest pipelines, and isolating construction parameters. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Unquantized In-Place HNSW Writes Under High Ingestion Inducing Out-Of-Memory and Graph Fragmentation Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Unquantized In-Place HNSW Writes Under High Ingestion Inducing Out-Of-Memory and Graph Fragmentation	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS declare vector columns as `halfvec(dimensions)` instead of single-precision `vector(dimensions)` for embedding indices exceeding 100,000 vectors.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.