AgentRules.org
RulesStacksClientsAuditorPrunerSelf-HealRFCsBenchmarkCLICI/CDAuthorsSubmit
This specification is maintained by the AgentRules Working Group. Have enhancements or want to claim maintainership?Submit Specification →
backendcritical PriorityID: pgvector-halfvec-hnsw-memory-guardrail-pgvector-0-8-hal-939

pgvector 0.8 Half-Vector HNSW Index Build Memory Guardrail

Enforces deterministic maintenance_work_mem boundaries and max_parallel_maintenance_workers constraints during pgvector 0.8 halfvec (fp16) HNSW index builds. Prevents Linux Out-Of-Memory (OOM) killer panics caused by unconstrained per-worker memory allocation multiplying across concurrent vector indexing pipelines.

Prohibited Anti-Pattern: Global Unchecked maintenance_work_mem and Vector Casting During Parallel HNSW Indexing

Export Ready-to-Drop Configuration

Select format below to copy code or download file directly.

---
description: pgvector 0.8 Half-Vector HNSW Index Build Memory Guardrail - Enforces deterministic maintenance_work_mem boundaries and max_parallel_maintenance_workers constraints during pgvector 0.8 halfvec (fp16) HNSW index builds. Prevents Linux Out-Of-Memory (OOM) killer panics caused by unconstrained per-worker memory allocation multiplying across concurrent vector indexing pipelines.
globs: src/database/migrations/**/*.sql, src/database/migrations/**/*.ts, src/services/vector/**/*.ts, migrations/**/*.sql, prisma/migrations/**/*.sql
alwaysApply: true
verified_by: "agentrules.org"
license: "CC-BY-4.0"
source: "https://agentrules.org/rules/pgvector-halfvec-hnsw-memory-guardrail-pgvector-0-8-hal-939"
---

# pgvector 0.8 Half-Vector HNSW Index Build Memory Guardrail

> **Severity**: CRITICAL | **Category**: backend
> **Anti-Pattern Warning**: Global Unchecked maintenance_work_mem and Vector Casting During Parallel HNSW Indexing

## System Constraints
You are a Principal Database Architect and Infrastructure Reliability Engineer enforcing pgvector 0.8 half-vector (halfvec) HNSW index build memory invariants. Mandate strict transaction-local or session-scoped memory caps, explicit dimension quantization via halfvec(N), and deterministic max_parallel_maintenance_workers limits to eliminate kernel OOM terminations during migrations.

## Strict Guidelines
- Engineers MUST ALWAYS set session-scoped 'maintenance_work_mem' and 'max_parallel_maintenance_workers' immediately before executing 'CREATE INDEX ... USING hnsw'.
- Engineers MUST NOT rely on instance-wide maintenance_work_mem defaults when building HNSW indexes over large vector tables (>100k rows).
- Calculations for maintenance_work_mem MUST follow: RAM allocation <= (Available_Free_RAM * 0.70) / (max_parallel_maintenance_workers + 1).
- Engineers MUST cast raw float32 embeddings to halfvec(dim) during index creation or migrate vector columns to halfvec data types to achieve 50% memory footprint reduction.
- Migrations MUST NEVER build HNSW vector indexes within implicit long-running transactional migrations without isolated autocommit execution flags.
- Index builds MUST tune m and ef_construction parameters explicitly to balance graph precision against heap work memory.

## Bad vs Good Implementations
### Unsafe Parallel HNSW Build vs. Scoped Memory Guarded Halfvec Index Creation

#### Bad Antipattern (AI Common Hallucination)
> **Why it fails**: Executing an unconstrained full-precision (vector/float32) HNSW index build across unthrottled parallel maintenance workers causes cumulative memory consumption to exceed Linux cgroup limits, triggering an unrecoverable SIGKILL from the OOM killer.
```typescript
import { Client } from 'pg';

export async function runMigration(client: Client): Promise<void> {
  // DANGEROUS: Uses cluster-wide maintenance_work_mem multiplied by default parallel workers.
  // Building full-precision float32 hnsw index without quantization leads to Linux OOM killer panic.
  await client.query(`
    CREATE INDEX idx_documents_embedding 
    ON documents 
    USING hnsw (embedding vector_cosine_ops);
  `);
}
```

#### Production Standard (Correct)
> **Standard**: Scopes maintenance_work_mem and worker limits to safe, isolated boundaries and quantizes 1536-dim vectors into pgvector 0.8 halfvec (fp16), reducing working memory by 50% and protecting database containers from kernel OOM termination.
```typescript
import { Client } from 'pg';

export async function runMigration(client: Client): Promise<void> {
  // Production Invariant: Explicitly calculate allocation budget:
  // Target 4GB budget across 2 workers -> 4096 / (2 + 1) ~= 1365MB per worker.
  // Use halfvec (pgvector 0.8+ fp16) to halve the graph construction memory footprint.
  await client.query('SET LOCAL max_parallel_maintenance_workers = 2;');
  await client.query("SET LOCAL maintenance_work_mem = '1300MB';");

  await client.query(`
    CREATE INDEX CONCURRENTLY IF NOT EXISTS idx_documents_embedding_halfvec 
    ON documents 
    USING hnsw ((embedding::halfvec(1536)) halfvec_cosine_ops)
    WITH (m = 16, ef_construction = 64);
  `);

  await client.query('RESET maintenance_work_mem;');
  await client.query('RESET max_parallel_maintenance_workers;');
}
```
Recommended MCP Servers (mcp-serverlist.com)

Equip agent with system tools to verify code for this stack.

PostgreSQL MCP Server
Executes query plans and introspects PostgreSQL session memory and pgvector index configurations.
Recommended Agent Skills (askills.dev)

Ready-to-use executable skill functions tailored to this stack.

Postgres Vector Index Auditor
Calculates memory budgets and verifies halfvec quantization parameters against database hardware capacity.
Architectural Specification & Invariant Rationale

Architectural Invariant Design: pgvector 0.8 Half-Vector HNSW Index Build Memory Guardrail

pgvector 0.8 Half-Vector HNSW Index Build Memory Guardrail defines an industrial-grade behavioral standard for backend engineering. Enforces deterministic maintenance_work_mem boundaries and max_parallel_maintenance_workers constraints during pgvector 0.8 halfvec (fp16) HNSW index builds. Prevents Linux Out-Of-Memory (OOM) killer panics caused by unconstrained per-worker memory allocation multiplying across concurrent vector indexing pipelines. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Global Unchecked maintenance_work_mem and Vector Casting During Parallel HNSW Indexing Without an explicit system instruction file, developers spend hours debugging silent failures.
Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure ModeAI Symptom (The Anti-Pattern)SeverityProduction Impact & Remedy
AI Hallucination & Deprecation DriftGlobal Unchecked maintenance_work_mem and Vector Casting During Parallel HNSW IndexingCritical
Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production.
Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture InconsistencyAI assistants produce mixed paradigms across files, creating fragmented codebase conventions.Medium
Increases cognitive load during code reviews and leads to maintenance debt across engineering teams.
Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.
Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

1

Engineers MUST ALWAYS set session-scoped 'maintenance_work_mem' and 'max_parallel_maintenance_workers' immediately before executing 'CREATE INDEX ... USING hnsw'.

Engineers MUST ALWAYS set session-scoped 'maintenance_work_mem' and 'max_parallel_maintenance_workers' immediately before executing 'CREATE INDEX ... USING hnsw'.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.
Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.
2

Engineers MUST NOT rely on instance-wide maintenance_work_mem defaults when building HNSW indexes over large vector tables (>100k rows).

Engineers MUST NOT rely on instance-wide maintenance_work_mem defaults when building HNSW indexes over large vector tables (>100k rows).

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.
Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.
3

Calculations for maintenance_work_mem MUST follow

Calculations for maintenance_work_mem MUST follow: RAM allocation <= (Available_Free_RAM * 0.70) / (max_parallel_maintenance_workers + 1).

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.
Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.
4

Engineers MUST cast raw float32 embeddings to halfvec(dim) during index creation or migrate vector columns to halfvec data types to achieve 50% memory footprint reduction.

Engineers MUST cast raw float32 embeddings to halfvec(dim) during index creation or migrate vector columns to halfvec data types to achieve 50% memory footprint reduction.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.
Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.
5

Migrations MUST NEVER build HNSW vector indexes within implicit long-running transactional migrations without isolated autocommit execution flags.

Migrations MUST NEVER build HNSW vector indexes within implicit long-running transactional migrations without isolated autocommit execution flags.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.
Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.
6

Index builds MUST tune m and ef_construction parameters explicitly to balance graph precision against heap work memory.

Index builds MUST tune m and ef_construction parameters explicitly to balance graph precision against heap work memory.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.
Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.
Technical Questions & Answers

Frequently Asked Technical Questions

Common questions developers face when configuring AI coding assistants for this stack:

Q:Why is pgvector 0.8 Half-Vector HNSW Index Build Memory Guardrail critical for AI coding assistants?

Enforces deterministic maintenance_work_mem boundaries and max_parallel_maintenance_workers constraints during pgvector 0.8 halfvec (fp16) HNSW index builds. Prevents Linux Out-Of-Memory (OOM) killer panics caused by unconstrained per-worker memory allocation multiplying across concurrent vector indexing pipelines. Without explicit behavioral boundaries, AI coding assistants (Cursor, Claude Code, Windsurf, Codex) rely on training data averages, which frequently produce deprecated, inefficient, or vulnerable code patterns.

Q:How does adopting this rule prevent AI hallucinations?

This rule explicitly declares what the AI must avoid (Global Unchecked maintenance_work_mem and Vector Casting During Parallel HNSW Indexing) and prescribes verified positive examples. Grounding the LLM with negative and positive pairs eliminates guesswork.

Q:Which AI coding environments are supported?

This specification is multi-platform compatible: Cursor (.cursor/rules/*.mdc & .cursorrules), Anthropic Claude Code (CLAUDE.md), Windsurf (.windsurfrules), Codex / Cline (AGENTS.md), and GitHub Copilot (.github/copilot-instructions.md).

Q:Can I combine this rule with other technology stacks?

Yes. Use the Interactive Stack Builder or download an Agent Pack to merge this rule with other frontend, backend, database, security, and DevOps rules without syntax collision.

Open Specification · CC BY 4.0·Free for Personal, Team & Commercial In-Repo Use

Citation & Open Source Attribution Guidelines

This specification is licensed under Creative Commons Attribution 4.0 International (CC BY 4.0). You are completely free to adapt and share this rule in your workspace. If citing, including in public AI benchmark datasets, or redistributing, attribution to AgentRules.org is required.

Related BACKEND Rules

View all rules →
backend
42

Fastify 5 Schema Compilers & Encapsulation Production Standard

Fastify relies on lexical encapsulation contexts to sandbox decorators, hooks, and compiled validation/serialization schemas without memory leaks or cross-tenant bleed. In Fastify 5, misuse of global compilers or failing to encapsulate plugin boundaries causes compiled `fast-json-stringify` serializers and Ajv validators to either crash on duplicate IDs, leak private data across sibling scopes, or bypass JIT compilation optimizations. Production services must enforce strict encapsulation hierarchies and type-safe schema compilation across every routing subtree.

backend
68

Bun Embedded SQLite WAL Mode & Transaction Synchronization Standard

Production database standard for bun:sqlite. Mandates enabling Write-Ahead Logging (PRAGMA journal_mode = WAL), busy timeouts, and using db.transaction() for atomic operations without lock starvation.

backend
68

Drizzle ORM Batch API & Interactive Transaction Isolation Standard

Production database standard for Drizzle ORM (PostgreSQL & MySQL). Mandates using db.batch() over HTTP drivers to prevent round-trip waterfalls, explicit db.transaction() with serializable/read-committed isolation, and strictly avoids unawaited query promises.

1 Rule in Stack

pgvector