AgentRules.org
RulesStacksClientsAuditorPrunerSelf-HealRFCsBenchmarkCLICI/CDAuthorsSubmit
This specification is maintained by the AgentRules Working Group. Have enhancements or want to claim maintainership?Submit Specification →
backendcritical PriorityID: pg17-iterative-hnsw-hybrid-optimizer-standard

PostgreSQL 17 Iterative Index Scan Guardrail for Vector-HNSW Hybrid Search

Enforces deterministic bounded-memory iterative index scan execution paths on PostgreSQL 17 when combining pgvector HNSW similarity queries with restrictive relational predicates or BM25 text rankers. Prevents catastrophic query planner regression to unindexed full sequential scans when filtering selectivity drops below standard cost thresholds. Requires strict session- or transaction-level tuning of hnsw.iterative_scan and work_mem to eliminate OOM kills and high-latency table sweeps.

Prohibited Anti-Pattern: Unchecked Predicate Selectivity Forcing HNSW Planner Fallback to Full Table Sequential Scans

Export Ready-to-Drop Configuration

Select format below to copy code or download file directly.

---
description: PostgreSQL 17 Iterative Index Scan Guardrail for Vector-HNSW Hybrid Search - Enforces deterministic bounded-memory iterative index scan execution paths on PostgreSQL 17 when combining pgvector HNSW similarity queries with restrictive relational predicates or BM25 text rankers. Prevents catastrophic query planner regression to unindexed full sequential scans when filtering selectivity drops below standard cost thresholds. Requires strict session- or transaction-level tuning of hnsw.iterative_scan and work_mem to eliminate OOM kills and high-latency table sweeps.
globs: src/repositories/**/*.ts, src/database/**/*.ts, src/services/search/**/*.ts, migrations/**/*.sql
alwaysApply: true
verified_by: "agentrules.org"
license: "CC-BY-4.0"
source: "https://agentrules.org/rules/pg17-iterative-hnsw-hybrid-optimizer-standard"
---

# PostgreSQL 17 Iterative Index Scan Guardrail for Vector-HNSW Hybrid Search

> **Severity**: CRITICAL | **Category**: backend
> **Anti-Pattern Warning**: Unchecked Predicate Selectivity Forcing HNSW Planner Fallback to Full Table Sequential Scans

## System Constraints
You are a Principal Database Architect enforcing PostgreSQL 17 Iterative Index Scan Vector-HNSW Hybrid Query Optimizer Standards. Mandate deterministic index paths for hybrid vector/lexical retrieval, ensuring relational predicates do not destabilize the planner into linear relation scans.

## Strict Guidelines
- Engineers MUST ALWAYS enable SET LOCAL hnsw.iterative_scan = 'relaxed' or 'strict' within the active transaction when querying HNSW indexes coupled with restrictive WHERE filters.
- Engineers MUST NOT issue unbounded multi-stage hybrid vector queries without explicitly bounding candidate generation via Reciprocal Rank Fusion (RRF) CTEs or explicit LIMIT subqueries.
- Code SHOULD enforce explicit hnsw.ef_search sizing (e.g., matching or exceeding the top-k multiplier) alongside hnsw.max_scan_tuples to prevent infinite index iteration loops under low-selectivity filters.
- Services MUST execute EXPLAIN (ANALYZE, BUFFERS) validation in integration tests to verify the execution plan reports 'Index Scan using ... on ...' and never 'Seq Scan' on vector relations.
- Components NEVER execute raw cosine/L2 distance operators (<=>, <->) against pre-filtered subqueries without asserting index availability and planner cost-model verification.

## Bad vs Good Implementations
### PostgreSQL 17 Bounded Iterative Scan vs Unbounded Planner Fallback in TypeScript Repository

#### Bad Antipattern (AI Common Hallucination)
> **Why it fails**: Omitting PostgreSQL 17 iterative scan parameters causes the query optimizer to abandon HNSW index traversal when org_id and category predicates filter out the majority of index-matched candidate nodes, resulting in a full table sequential scan.
```typescript
import { sql } from 'kysely';
import type { Kysely } from 'kysely';
import type { Database } from '../types/db';

export async function searchHybridBad(
  db: Kysely<Database>,
  orgId: string,
  category: string,
  queryVector: number[],
  limit: number
) {
  // FAILS: Under restrictive tenant/category filters, Postgres planner drops
  // the HNSW index scan and executes a Seq Scan on all vector embeddings,
  // spiking I/O and latency from 4ms to 8500ms.
  const vectorLiteral = `[${queryVector.join(',')}]`;
  return await sql`
    SELECT id, document_id, content,
           (embedding <=> ${vectorLiteral}::vector) AS distance
    FROM document_chunks
    WHERE org_id = ${orgId}
      AND category = ${category}
    ORDER BY distance ASC
    LIMIT ${limit};
  `.execute(db);
}
```

#### Production Standard (Correct)
> **Standard**: Configures hnsw.iterative_scan to 'relaxed', caps hnsw.max_scan_tuples, and adjusts hnsw.ef_search locally within the transaction. This forces the PG17 optimizer to iteratively walk the HNSW graph until enough filtered tuples pass predicates, strictly avoiding full table sequential scans.
```typescript
import { sql } from 'kysely';
import type { Kysely } from 'kysely';
import type { Database } from '../types/db';

export async function searchHybridGood(
  db: Kysely<Database>,
  orgId: string,
  category: string,
  queryVector: number[],
  limit: number
) {
  const vectorLiteral = `[${queryVector.join(',')}]`;
  const boundedScanTuples = limit * 50;
  const targetEfSearch = Math.max(limit * 2, 40);

  // ENFORCES: Postgres 17 iterative index scan with deterministic scan limits
  // within an isolated transaction to prevent unbounded index walks or table fallback.
  return await db.transaction().execute(async (trx) => {
    await sql`SET LOCAL hnsw.iterative_scan = 'relaxed'`.execute(trx);
    await sql`SET LOCAL hnsw.max_scan_tuples = ${boundedScanTuples}`.execute(trx);
    await sql`SET LOCAL hnsw.ef_search = ${targetEfSearch}`.execute(trx);

    return await sql`
      WITH candidate_vectors AS (
        SELECT id, document_id, content,
               (embedding <=> ${vectorLiteral}::vector) AS vector_distance
        FROM document_chunks
        WHERE org_id = ${orgId}
          AND category = ${category}
        ORDER BY embedding <=> ${vectorLiteral}::vector ASC
        LIMIT ${limit}
      )
      SELECT id, document_id, content, vector_distance
      FROM candidate_vectors
      ORDER BY vector_distance ASC;
    `.execute(trx);
  });
}
```
Recommended MCP Servers (mcp-serverlist.com)

Equip agent with system tools to verify code for this stack.

PostgreSQL MCP Inspector
Validates live query plans and ensures HNSW index usage under restrictive predicate combinations.
Recommended Agent Skills (askills.dev)

Ready-to-use executable skill functions tailored to this stack.

Postgres Query Plan Verifier
CLI automation utility to verify EXPLAIN ANALYZE results avoid Sequential Scans on vector columns.
Architectural Specification & Invariant Rationale

Architectural Invariant Design: PostgreSQL 17 Iterative Index Scan Guardrail for Vector-HNSW Hybrid Search

PostgreSQL 17 Iterative Index Scan Guardrail for Vector-HNSW Hybrid Search defines an industrial-grade behavioral standard for backend engineering. Enforces deterministic bounded-memory iterative index scan execution paths on PostgreSQL 17 when combining pgvector HNSW similarity queries with restrictive relational predicates or BM25 text rankers. Prevents catastrophic query planner regression to unindexed full sequential scans when filtering selectivity drops below standard cost thresholds. Requires strict session- or transaction-level tuning of hnsw.iterative_scan and work_mem to eliminate OOM kills and high-latency table sweeps. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Unchecked Predicate Selectivity Forcing HNSW Planner Fallback to Full Table Sequential Scans Without an explicit system instruction file, developers spend hours debugging silent failures.
Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure ModeAI Symptom (The Anti-Pattern)SeverityProduction Impact & Remedy
AI Hallucination & Deprecation DriftUnchecked Predicate Selectivity Forcing HNSW Planner Fallback to Full Table Sequential ScansCritical
Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production.
Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture InconsistencyAI assistants produce mixed paradigms across files, creating fragmented codebase conventions.Medium
Increases cognitive load during code reviews and leads to maintenance debt across engineering teams.
Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.
Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

1

Engineers MUST ALWAYS enable SET LOCAL hnsw.iterative_scan = 'relaxed' or 'strict' within the active transaction when querying HNSW indexes coupled with restrictive WHERE filters.

Engineers MUST ALWAYS enable SET LOCAL hnsw.iterative_scan = 'relaxed' or 'strict' within the active transaction when querying HNSW indexes coupled with restrictive WHERE filters.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.
Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.
2

Engineers MUST NOT issue unbounded multi-stage hybrid vector queries without explicitly bounding candidate generation via Reciprocal Rank Fusion (RRF) CTEs or explicit LIMIT subqueries.

Engineers MUST NOT issue unbounded multi-stage hybrid vector queries without explicitly bounding candidate generation via Reciprocal Rank Fusion (RRF) CTEs or explicit LIMIT subqueries.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.
Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.
3

Code SHOULD enforce explicit hnsw.ef_search sizing (e.g., matching or exceeding the top-k multiplier) alongside hnsw.max_scan_tuples to prevent infinite index iteration loops under low-selectivity filters.

Code SHOULD enforce explicit hnsw.ef_search sizing (e.g., matching or exceeding the top-k multiplier) alongside hnsw.max_scan_tuples to prevent infinite index iteration loops under low-selectivity filters.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.
Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.
4

Services MUST execute EXPLAIN (ANALYZE, BUFFERS) validation in integration tests to verify the execution plan reports 'Index Scan using ... on ...' and never 'Seq Scan' on vector relations.

Services MUST execute EXPLAIN (ANALYZE, BUFFERS) validation in integration tests to verify the execution plan reports 'Index Scan using ... on ...' and never 'Seq Scan' on vector relations.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.
Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.
5

Components NEVER execute raw cosine/L2 distance operators (<=>, <->) against pre-filtered subqueries without asserting index availability and planner cost-model verification.

Components NEVER execute raw cosine/L2 distance operators (<=>, <->) against pre-filtered subqueries without asserting index availability and planner cost-model verification.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.
Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.
Technical Questions & Answers

Frequently Asked Technical Questions

Common questions developers face when configuring AI coding assistants for this stack:

Q:Why is PostgreSQL 17 Iterative Index Scan Guardrail for Vector-HNSW Hybrid Search critical for AI coding assistants?

Enforces deterministic bounded-memory iterative index scan execution paths on PostgreSQL 17 when combining pgvector HNSW similarity queries with restrictive relational predicates or BM25 text rankers. Prevents catastrophic query planner regression to unindexed full sequential scans when filtering selectivity drops below standard cost thresholds. Requires strict session- or transaction-level tuning of hnsw.iterative_scan and work_mem to eliminate OOM kills and high-latency table sweeps. Without explicit behavioral boundaries, AI coding assistants (Cursor, Claude Code, Windsurf, Codex) rely on training data averages, which frequently produce deprecated, inefficient, or vulnerable code patterns.

Q:How does adopting this rule prevent AI hallucinations?

This rule explicitly declares what the AI must avoid (Unchecked Predicate Selectivity Forcing HNSW Planner Fallback to Full Table Sequential Scans) and prescribes verified positive examples. Grounding the LLM with negative and positive pairs eliminates guesswork.

Q:Which AI coding environments are supported?

This specification is multi-platform compatible: Cursor (.cursor/rules/*.mdc & .cursorrules), Anthropic Claude Code (CLAUDE.md), Windsurf (.windsurfrules), Codex / Cline (AGENTS.md), and GitHub Copilot (.github/copilot-instructions.md).

Q:Can I combine this rule with other technology stacks?

Yes. Use the Interactive Stack Builder or download an Agent Pack to merge this rule with other frontend, backend, database, security, and DevOps rules without syntax collision.

Open Specification · CC BY 4.0·Free for Personal, Team & Commercial In-Repo Use

Citation & Open Source Attribution Guidelines

This specification is licensed under Creative Commons Attribution 4.0 International (CC BY 4.0). You are completely free to adapt and share this rule in your workspace. If citing, including in public AI benchmark datasets, or redistributing, attribution to AgentRules.org is required.

Related BACKEND Rules

View all rules →
backend
42

Fastify 5 Schema Compilers & Encapsulation Production Standard

Fastify relies on lexical encapsulation contexts to sandbox decorators, hooks, and compiled validation/serialization schemas without memory leaks or cross-tenant bleed. In Fastify 5, misuse of global compilers or failing to encapsulate plugin boundaries causes compiled `fast-json-stringify` serializers and Ajv validators to either crash on duplicate IDs, leak private data across sibling scopes, or bypass JIT compilation optimizations. Production services must enforce strict encapsulation hierarchies and type-safe schema compilation across every routing subtree.

backend
68

Bun Embedded SQLite WAL Mode & Transaction Synchronization Standard

Production database standard for bun:sqlite. Mandates enabling Write-Ahead Logging (PRAGMA journal_mode = WAL), busy timeouts, and using db.transaction() for atomic operations without lock starvation.

backend
68

Drizzle ORM Batch API & Interactive Transaction Isolation Standard

Production database standard for Drizzle ORM (PostgreSQL & MySQL). Mandates using db.batch() over HTTP drivers to prevent round-trip waterfalls, explicit db.transaction() with serializable/read-committed isolation, and strictly avoids unawaited query promises.

1 Rule in Stack

PostgreSQL