pgvector 0.8+ Halfvec Quantized HNSW Index Memory Footprint Guardrail

Q: How does adopting this rule prevent AI hallucinations?

This rule explicitly declares what the AI must avoid (Building Standard 32-bit Float HNSW Indexes at Scale Causing Memory Exhaustion and Node Crashes) and prescribes verified positive examples. Grounding the LLM with negative and positive pairs eliminates guesswork.

Q: Which AI coding environments are supported?

This specification is multi-platform compatible: Cursor (.cursor/rules/*.mdc & .cursorrules), Anthropic Claude Code (CLAUDE.md), Windsurf (.windsurfrules), Codex / Cline (AGENTS.md), and GitHub Copilot (.github/copilot-instructions.md).

Q: Can I combine this rule with other technology stacks?

Yes. Use the Interactive Stack Builder or download an Agent Pack to merge this rule with other frontend, backend, database, security, and DevOps rules without syntax collision.

pgvector 0.8+ Halfvec Quantized HNSW Index Memory Footprint Guardrail

--- description: pgvector 0.8+ Halfvec Quantized HNSW Index Memory Footprint Guardrail - Enforces 16-bit scalar quantization (halfvec) on PostgreSQL pgvector 0.8+ HNSW indexes to reduce index working set memory footprint by 50% and prevent out-of-memory (OOM) kernel panics at hyper-scale. Mandates maintenance_work_mem budget bounds during index construction, iterative re-ranking pipelines against full-precision vectors when required by recall SLAs, and strict validation of operator classes. globs: src/db/**/*.ts, src/repositories/**/*.ts, migrations/**/*.sql, prisma/schema.prisma alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/pgvector-halfvec-hnsw-memory-guardrail" --- # pgvector 0.8+ Halfvec Quantized HNSW Index Memory Footprint Guardrail > **Severity**: CRITICAL | **Category**: backend > **Anti-Pattern Warning**: Building Standard 32-bit Float HNSW Indexes at Scale Causing Memory Exhaustion and Node Crashes ## System Constraints You are a Principal Database Infrastructure Engineer enforcing strict pgvector 0.8+ halfvec quantization and memory guardrails. Large-scale vector indexes using default float32 vectors allocate immense RAM inside PostgreSQL shared buffers and process memory, inevitably causing Linux OOM killer termination under concurrent query load. You must enforce halfvec quantization on HNSW indexes for dimensions >= 768, explicitly parameterize index builds with m and ef_construction, and architect two-phase retrieval (halfvec approximate nearest neighbor index scan followed by precise float32 cosine re-ranking) to maintain sub-millisecond latencies and >= 99% recall SLAs. ## Strict Guidelines - Engineers MUST ALWAYS cast full-precision vector columns to `halfvec` when creating HNSW indexes on embeddings with dimension >= 768 (e.g., `CREATE INDEX ... ON table USING hnsw ((embedding::halfvec(dim)) halfvec_cosine_ops)`). - Engineers MUST NOT create default float32 `hnsw (embedding vector_cosine_ops)` indexes on production tables exceeding 100,000 embeddings without documented hardware sizing sign-off. - Database migrations MUST explicitly set `maintenance_work_mem` within transactional migration scripts to prevent OOM panics during HNSW index builds. - Query services SHOULD implement a two-stage retrieval pipeline: retrieve candidates using `halfvec_cosine_ops` index lookup, then re-rank top-K candidates using exact float32 `<=>` distance calculations against original column data. - Index definitions MUST explicitly specify `m` and `ef_construction` parameters tuned for RAM limits, ensuring `m` <= 24 and `ef_construction` <= 128 for high-dimensional halfvec indexes unless custom performance benchmarks require higher values. ## Bad vs Good Implementations ### SQL Migration: Unbounded Float32 HNSW vs Memory-Bounded Halfvec HNSW Index #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Creating an unquantized HNSW index on 1536-dimensional float32 vectors consumes enormous working memory, causing severe buffer cache thrashing and OOM kernel panics under concurrent write or query load. ```sql -- Anti-pattern: High-dimensional full 32-bit float vector index -- Will allocate ~3.5GB+ RAM for 1M 1536-dim vectors, risking PostgreSQL OOM crash CREATE INDEX idx_documents_embedding ON documents USING hnsw (embedding vector_cosine_ops); ``` #### Production Standard (Correct) > **Standard**: Casting the vector column to `halfvec(1536)` with `halfvec_cosine_ops` halves RAM requirements for the HNSW graph while explicit `m` and `ef_construction` control graph complexity and build memory. ```sql -- Guarded pattern: Explicit memory allocation and halfvec quantization SET LOCAL maintenance_work_mem = '2GB'; -- Create HNSW index using 16-bit halfvec expression to reduce index RAM footprint by 50% CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_documents_embedding_halfvec ON documents USING hnsw ( (embedding::halfvec(1536)) halfvec_cosine_ops ) WITH (m = 16, ef_construction = 64); ``` ### TypeScript Repository: Single-Phase Float Query vs Two-Phase Quantized Retrieval & Re-ranking #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Queries against float32 vectors directly bypass halfvec indexes, triggering sequential scans or traversing uncompressed indexes that overload buffer memory. ```typescript import { sql } from 'kysely'; import type { Database } from '../db/types'; export async function searchDocuments(db: Database, queryEmbedding: number[], limit = 10) { // Inefficient: Forces float32 index scan or seq scan without quantization const embeddingStr = `[${queryEmbedding.join(',')}]`; return await sql` SELECT id, content, (embedding <=> ${embeddingStr}::vector) AS distance FROM documents ORDER BY embedding <=> ${embeddingStr}::vector LIMIT ${limit}; `.execute(db); } ``` #### Production Standard (Correct) > **Standard**: Implements two-phase retrieval: first utilizes the halfvec HNSW index to fetch top candidates with sub-millisecond quantized traversal, then computes exact float32 cosine distance on the small candidate pool to prevent recall loss. ```typescript import { sql } from 'kysely'; import type { Database } from '../db/types'; export async function searchDocumentsWithRerank( db: Database, queryEmbedding: number[], limit = 10, candidateMultiplier = 4 ) { const embeddingStr = `[${queryEmbedding.join(',')}]`; const candidateLimit = limit * candidateMultiplier; // Phase 1: Rapid index scan via halfvec quantized HNSW index // Phase 2: Precise float32 re-ranking on candidate subset to preserve recall SLA return await sql` WITH candidates AS ( SELECT id, content, embedding FROM documents ORDER BY (embedding::halfvec(1536)) <=> (${embeddingStr}::halfvec(1536)) LIMIT ${candidateLimit} ) SELECT id, content, (embedding <=> ${embeddingStr}::vector) AS exact_distance FROM candidates ORDER BY exact_distance ASC LIMIT ${limit}; `.execute(db); } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: pgvector 0.8+ Halfvec Quantized HNSW Index Memory Footprint Guardrail

pgvector 0.8+ Halfvec Quantized HNSW Index Memory Footprint Guardrail defines an industrial-grade behavioral standard for backend engineering. Enforces 16-bit scalar quantization (halfvec) on PostgreSQL pgvector 0.8+ HNSW indexes to reduce index working set memory footprint by 50% and prevent out-of-memory (OOM) kernel panics at hyper-scale. Mandates maintenance_work_mem budget bounds during index construction, iterative re-ranking pipelines against full-precision vectors when required by recall SLAs, and strict validation of operator classes. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Building Standard 32-bit Float HNSW Indexes at Scale Causing Memory Exhaustion and Node Crashes Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Building Standard 32-bit Float HNSW Indexes at Scale Causing Memory Exhaustion and Node Crashes	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS cast full-precision vector columns to `halfvec` when creating HNSW indexes on embeddings with dimension >= 768 (e.g., `CREATE INDEX ... ON table USING hnsw ((embedding

Engineers MUST ALWAYS cast full-precision vector columns to `halfvec` when creating HNSW indexes on embeddings with dimension >= 768 (e.g., `CREATE INDEX ... ON table USING hnsw ((embedding::halfvec(dim)) halfvec_cosine_ops)`).

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.