AgentRules.org
RulesStacksClientsAuditorPrunerSelf-HealRFCsBenchmarkCLICI/CDAuthorsSubmit
This specification is maintained by the AgentRules Working Group. Have enhancements or want to claim maintainership?Submit Specification →
devopscritical PriorityID: otel-collector-cgroupv2-memory-limiter-backpressure

OpenTelemetry Collector cgroup v2 Dynamic Memory Limiter & Upstream Backpressure Invariant

Enforces dynamic memory governance for OpenTelemetry Collector instances running under Linux cgroup v2 container runtimes. Guarantees that collector memory boundaries adapt dynamically to cgroup memory.max and memory.current telemetry, immediately exerting gRPC/HTTP backpressure to upstream telemetry sources before triggering kernel OOM-killer evictions.

Prohibited Anti-Pattern: Static memory limit allocation and missing cgroup v2 pressure hooks causing ungraceful container OOMKills

Export Ready-to-Drop Configuration

Select format below to copy code or download file directly.

---
description: OpenTelemetry Collector cgroup v2 Dynamic Memory Limiter & Upstream Backpressure Invariant - Enforces dynamic memory governance for OpenTelemetry Collector instances running under Linux cgroup v2 container runtimes. Guarantees that collector memory boundaries adapt dynamically to cgroup memory.max and memory.current telemetry, immediately exerting gRPC/HTTP backpressure to upstream telemetry sources before triggering kernel OOM-killer evictions.
globs: **/otel-collector*.{yaml,yml}, **/collector-config*.{yaml,yml}, **/k8s/**/*.{yaml,yml}, **/helm/**/values*.{yaml,yml}
alwaysApply: true
verified_by: "agentrules.org"
license: "CC-BY-4.0"
source: "https://agentrules.org/rules/otel-collector-cgroupv2-memory-limiter-backpressure"
---

# OpenTelemetry Collector cgroup v2 Dynamic Memory Limiter & Upstream Backpressure Invariant

> **Severity**: CRITICAL | **Category**: devops
> **Anti-Pattern Warning**: Static memory limit allocation and missing cgroup v2 pressure hooks causing ungraceful container OOMKills

## System Constraints
You are a Principal DevOps and Observability Architect specializing in high-throughput OpenTelemetry Collector deployments under Linux cgroup v2. Your task is to enforce strict memory bounds and upstream backpressure invariants. Never permit hardcoded absolute memory allocations without runtime container limit awareness, and mandate memory_limiter as the primary processor in all data pipelines.

## Strict Guidelines
- Engineers MUST ALWAYS configure the `memory_limiter` processor as the very first processor in every telemetry pipeline (traces, metrics, logs).
- Engineers MUST set `limit_percentage` (recommended 75-80%) and `spike_limit_percentage` (recommended 20%) instead of static `limit_mib` when running in dynamic containerized environments.
- Engineers MUST configure `check_interval` to 1s or less (e.g., 500ms) to ensure cgroup v2 memory growth bursts are intercepted before kernel OOM-killer invocation.
- Engineers MUST align the runtime `GOMEMLIMIT` environment variable directly to 80-85% of the cgroup v2 `memory.max` constraint to trigger Go runtime GC cycles before backpressure rejection.
- Receivers MUST be configured to convert memory_limiter rejections into standard upstream backpressure signals (gRPC Status Code 14 `UNAVAILABLE` or HTTP `429 Too Many Requests`) to force upstream SDK client retry-backs.

## Bad vs Good Implementations
### Anti-Pattern: Static Memory Allocation with Missing Pipeline Limiter

#### Bad Antipattern (AI Common Hallucination)
> **Why it fails**: Using static limit_mib breaks pod portability across differing node/cgroup specs. A 5-second check interval is too slow to stop microbursts before the kernel OOM killer terminates the pod. Furthermore, placing batch before memory_limiter buffers data in heap prior to pressure evaluation.
```yaml
# Anti-Pattern: Hardcoded absolute memory limits bypassing cgroup v2 dynamic sizing
processors:
  batch:
    send_batch_size: 10000
    timeout: 10s
  memory_limiter:
    # FLUID HAZARD: Static limit_mib risks OOMKill if pod memory limit is changed
    limit_mib: 4096
    spike_limit_mib: 1024
    check_interval: 5s # Too slow for bursty telemetry in cgroup v2

service:
  pipelines:
    traces:
      receivers: [otlp]
      # FATAL: memory_limiter is placed AFTER batch processor
      processors: [batch, memory_limiter]
      exporters: [otlp]
```

#### Production Standard (Correct)
> **Standard**: Configuring percentage-based limits allows the OpenTelemetry Collector runtime to inspect cgroup v2 limits directly (/sys/fs/cgroup/memory.max). Placing memory_limiter as the first processor drops or rejects requests at the ingress edge with gRPC UNAVAILABLE/HTTP 429, shielding collector heap integrity.
```yaml
# Production Invariant: cgroup v2-aware dynamic percentage limits with immediate backpressure
processors:
  memory_limiter:
    check_interval: 500ms
    limit_percentage: 80
    spike_limit_percentage: 20
  batch:
    send_batch_max_size: 8192
    timeout: 1s

service:
  pipelines:
    traces:
      receivers: [otlp]
      # MANDATE: memory_limiter MUST precede batch and transforms to shed load instantly
      processors: [memory_limiter, batch]
      exporters: [otlp]
    metrics:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlp]
    logs:
      receivers: [otlp]
      processors: [memory_limiter, batch]
      exporters: [otlp]
```
Recommended MCP Servers (mcp-serverlist.com)

Equip agent with system tools to verify code for this stack.

Kubernetes Resource Inspector MCP
Validates live Kubernetes Pod container specs, cgroup mount paths, and resource limits.
Recommended Agent Skills (askills.dev)

Ready-to-use executable skill functions tailored to this stack.

OpenTelemetry Collector Config Validator
CLI utility to validate collector configuration schema, component order, and processor sequences.
Architectural Specification & Invariant Rationale

Architectural Invariant Design: OpenTelemetry Collector cgroup v2 Dynamic Memory Limiter & Upstream Backpressure Invariant

OpenTelemetry Collector cgroup v2 Dynamic Memory Limiter & Upstream Backpressure Invariant defines an industrial-grade behavioral standard for devops engineering. Enforces dynamic memory governance for OpenTelemetry Collector instances running under Linux cgroup v2 container runtimes. Guarantees that collector memory boundaries adapt dynamically to cgroup memory.max and memory.current telemetry, immediately exerting gRPC/HTTP backpressure to upstream telemetry sources before triggering kernel OOM-killer evictions. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Static memory limit allocation and missing cgroup v2 pressure hooks causing ungraceful container OOMKills Without an explicit system instruction file, developers spend hours debugging silent failures.
Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure ModeAI Symptom (The Anti-Pattern)SeverityProduction Impact & Remedy
AI Hallucination & Deprecation DriftStatic memory limit allocation and missing cgroup v2 pressure hooks causing ungraceful container OOMKillsCritical
Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production.
Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture InconsistencyAI assistants produce mixed paradigms across files, creating fragmented codebase conventions.Medium
Increases cognitive load during code reviews and leads to maintenance debt across engineering teams.
Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.
Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

1

Engineers MUST ALWAYS configure the `memory_limiter` processor as the very first processor in every telemetry pipeline (traces, metrics, logs).

Engineers MUST ALWAYS configure the `memory_limiter` processor as the very first processor in every telemetry pipeline (traces, metrics, logs).

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.
Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.
2

Engineers MUST set `limit_percentage` (recommended 75-80%) and `spike_limit_percentage` (recommended 20%) instead of static `limit_mib` when running in dynamic containerized environments.

Engineers MUST set `limit_percentage` (recommended 75-80%) and `spike_limit_percentage` (recommended 20%) instead of static `limit_mib` when running in dynamic containerized environments.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.
Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.
3

Engineers MUST configure `check_interval` to 1s or less (e.g., 500ms) to ensure cgroup v2 memory growth bursts are intercepted before kernel OOM-killer invocation.

Engineers MUST configure `check_interval` to 1s or less (e.g., 500ms) to ensure cgroup v2 memory growth bursts are intercepted before kernel OOM-killer invocation.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.
Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.
4

Engineers MUST align the runtime `GOMEMLIMIT` environment variable directly to 80-85% of the cgroup v2 `memory.max` constraint to trigger Go runtime GC cycles before backpressure rejection.

Engineers MUST align the runtime `GOMEMLIMIT` environment variable directly to 80-85% of the cgroup v2 `memory.max` constraint to trigger Go runtime GC cycles before backpressure rejection.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.
Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.
5

Receivers MUST be configured to convert memory_limiter rejections into standard upstream backpressure signals (gRPC Status Code 14 `UNAVAILABLE` or HTTP `429 Too Many Requests`) to force upstream SDK client retry-backs.

Receivers MUST be configured to convert memory_limiter rejections into standard upstream backpressure signals (gRPC Status Code 14 `UNAVAILABLE` or HTTP `429 Too Many Requests`) to force upstream SDK client retry-backs.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.
Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.
Technical Questions & Answers

Frequently Asked Technical Questions

Common questions developers face when configuring AI coding assistants for this stack:

Q:Why is OpenTelemetry Collector cgroup v2 Dynamic Memory Limiter & Upstream Backpressure Invariant critical for AI coding assistants?

Enforces dynamic memory governance for OpenTelemetry Collector instances running under Linux cgroup v2 container runtimes. Guarantees that collector memory boundaries adapt dynamically to cgroup memory.max and memory.current telemetry, immediately exerting gRPC/HTTP backpressure to upstream telemetry sources before triggering kernel OOM-killer evictions. Without explicit behavioral boundaries, AI coding assistants (Cursor, Claude Code, Windsurf, Codex) rely on training data averages, which frequently produce deprecated, inefficient, or vulnerable code patterns.

Q:How does adopting this rule prevent AI hallucinations?

This rule explicitly declares what the AI must avoid (Static memory limit allocation and missing cgroup v2 pressure hooks causing ungraceful container OOMKills) and prescribes verified positive examples. Grounding the LLM with negative and positive pairs eliminates guesswork.

Q:Which AI coding environments are supported?

This specification is multi-platform compatible: Cursor (.cursor/rules/*.mdc & .cursorrules), Anthropic Claude Code (CLAUDE.md), Windsurf (.windsurfrules), Codex / Cline (AGENTS.md), and GitHub Copilot (.github/copilot-instructions.md).

Q:Can I combine this rule with other technology stacks?

Yes. Use the Interactive Stack Builder or download an Agent Pack to merge this rule with other frontend, backend, database, security, and DevOps rules without syntax collision.

Open Specification · CC BY 4.0·Free for Personal, Team & Commercial In-Repo Use

Citation & Open Source Attribution Guidelines

This specification is licensed under Creative Commons Attribution 4.0 International (CC BY 4.0). You are completely free to adapt and share this rule in your workspace. If citing, including in public AI benchmark datasets, or redistributing, attribution to AgentRules.org is required.

Related DEVOPS Rules

View all rules →
devops
86

Kubernetes Gateway API v1 HTTPRoute Canary and ReferenceGrant Invariants

Enforces strict conformance to Kubernetes Gateway API v1 specifications during migration from legacy Ingress controllers. Mandates explicit ReferenceGrant resources for cross-namespace routing, strict relative integer weighting for HTTPRoute canary releases, and explicit backendRef port declarations.

devops
118

ArgoCD ApplicationSet Matrix Generator Invariants & Cascade Deletion Guards

Enforces strict structural constraints on ArgoCD ApplicationSet Matrix generators combining cluster selectors and Git directories. Prevents catastrophic multi-cluster cascade pruning, accidental secret exposure through parameter interpolation, and runaway Application CR explosion via explicit sync policies, preservation finalizers, and parameter validation.

devops
40

Karpenter Resilient Topology, Interruption Queue, and Consolidation Budget Standard

Enforces deterministic disruption budgets, multi-instance-type diversification, and event-driven SQS interruption queue handling across Karpenter NodePools and EC2NodeClasses. Eliminates cascading eviction loops, instance starvation, and availability downtime during AWS Spot rebalances and underutilized node consolidation.

1 Rule in Stack

OpenTelemetry