KEDA Kubernetes Event-Driven Autoscaler & Queue Metric Governance

--- description: KEDA Kubernetes Event-Driven Autoscaler & Queue Metric Governance - Production autoscaling standard for Kubernetes worker workloads using KEDA. Replaces generic CPU/memory HPA with event-driven triggers (Kafka lag, SQS queue depth, Redis list length) and enforces scale-to-zero governance. globs: **/*keda*.yaml, **/*scaledobject*.yaml, k8s/**/*.yaml alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/keda-event-driven-autoscaling" --- # KEDA Kubernetes Event-Driven Autoscaler & Queue Metric Governance > **Severity**: CRITICAL | **Category**: devops > **Anti-Pattern Warning**: AI configures standard CPU HPA for async queue workers, failing to scale out during massive message backlogs. ## System Constraints You are a Kubernetes platform and site reliability engineer. For event-driven background workers (Kafka consumers, SQS processors), NEVER rely solely on CPU/RAM HorizontalPodAutoscaler. Always define KEDA ScaledObject manifests with target queue depth metrics, cooldown periods, and authentication triggers. ## Strict Guidelines - Define KEDA ScaledObject with explicit message queue triggers (e.g. aws-sqs, apache-kafka, redis) to scale directly on queue lag. - Configure cooldownPeriod (default 300s) to prevent rapid pod thrashing (flapping) during erratic message bursts. - Specify minReplicaCount: 0 for cost-effective scale-to-zero background tasks, or minReplicaCount: 1 for latency-sensitive consumers. - Use TriggerAuthentication with Kubernetes ServiceAccount IRSA or HashiCorp Vault to authenticate against cloud message brokers. ## Bad vs Good Implementations ### KEDA ScaledObject on SQS Queue Depth vs Unresponsive CPU HPA #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: IO-bound asynchronous workers rarely exceed CPU thresholds, causing critical message processing backlogs. ```yaml # ANTI-PATTERN: Standard HPA cannot see SQS queue depth! apiVersion: autoscaling/v2 kind: HorizontalPodAutoscaler metadata: name: order-worker-hpa spec: scaleTargetRef: { apiVersion: apps/v1, kind: Deployment, name: order-worker } minReplicas: 2 maxReplicas: 20 metrics: - type: Resource resource: name: cpu # Worker is IO-bound waiting on network; CPU stays at 10% while 100k messages accumulate! target: { type: Utilization, averageUtilization: 70 } ``` #### Production Standard (Correct) > **Standard**: Immediately scales worker pods up proportionally to actual pending message volume and scales back down when the queue drains. ```yaml # PRODUCTION STANDARD: KEDA ScaledObject Triggered on SQS Queue Lag apiVersion: keda.sh/v1alpha1 kind: ScaledObject metadata: name: order-worker-scaledobject namespace: production spec: scaleTargetRef: apiVersion: apps/v1 kind: Deployment name: order-worker minReplicaCount: 1 maxReplicaCount: 30 cooldownPeriod: 180 pollingInterval: 15 triggers: - type: aws-sqs metadata: queueURL: https://sqs.us-east-1.amazonaws.com/123456789/order-processing-queue queueLength: "20" # Scale out 1 additional pod for every 20 pending messages awsRegion: "us-east-1" authenticationRef: name: keda-aws-credentials ``` ### TriggerAuthentication with AWS IRSA #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Exposes permanent credentials in git history. ```yaml # ANTI-PATTERN: Hardcoding AWS access keys inside manifest metadata: awsAccessKeyId: AKIAIOSFODNN7EXAMPLE ``` #### Production Standard (Correct) > **Standard**: Uses short-lived OIDC tokens for zero-secret credential rotation. ```yaml # PRODUCTION STANDARD: Pod identity with IAM Roles for Service Accounts (IRSA) apiVersion: keda.sh/v1alpha1 kind: TriggerAuthentication metadata: name: keda-aws-credentials spec: podIdentity: provider: aws-eks ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: KEDA Kubernetes Event-Driven Autoscaler & Queue Metric Governance

KEDA Kubernetes Event-Driven Autoscaler & Queue Metric Governance defines an industrial-grade behavioral standard for devops engineering. Production autoscaling standard for Kubernetes worker workloads using KEDA. Replaces generic CPU/memory HPA with event-driven triggers (Kafka lag, SQS queue depth, Redis list length) and enforces scale-to-zero governance. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: AI configures standard CPU HPA for async queue workers, failing to scale out during massive message backlogs. Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	AI configures standard CPU HPA for async queue workers, failing to scale out during massive message backlogs.	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Define KEDA ScaledObject with explicit message queue triggers (e.g. aws-sqs, apache-kafka, redis) to scale directly on queue lag.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.

Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.

Configure cooldownPeriod (default 300s) to prevent rapid pod thrashing (flapping) during erratic message bursts.

Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.

Specify minReplicaCount

Specify minReplicaCount: 0 for cost-effective scale-to-zero background tasks, or minReplicaCount: 1 for latency-sensitive consumers.

Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.

Use TriggerAuthentication with Kubernetes ServiceAccount IRSA or HashiCorp Vault to authenticate against cloud message brokers.

Enforcement:Integrated into the Agent Pack rule file. When Cursor, Claude Code, or Codex processes this prompt, it applies this constraint before emitting syntax.

Technical Questions & Answers

Frequently Asked Technical Questions

Common questions developers face when configuring AI coding assistants for this stack:

Q:Why is KEDA Kubernetes Event-Driven Autoscaler & Queue Metric Governance critical for AI coding assistants?

Production autoscaling standard for Kubernetes worker workloads using KEDA. Replaces generic CPU/memory HPA with event-driven triggers (Kafka lag, SQS queue depth, Redis list length) and enforces scale-to-zero governance. Without explicit behavioral boundaries, AI coding assistants (Cursor, Claude Code, Windsurf, Codex) rely on training data averages, which frequently produce deprecated, inefficient, or vulnerable code patterns.

Q:How does adopting this rule prevent AI hallucinations?

This rule explicitly declares what the AI must avoid (AI configures standard CPU HPA for async queue workers, failing to scale out during massive message backlogs.) and prescribes verified positive examples. Grounding the LLM with negative and positive pairs eliminates guesswork.

Q:Which AI coding environments are supported?

This specification is multi-platform compatible: Cursor (.cursor/rules/*.mdc & .cursorrules), Anthropic Claude Code (CLAUDE.md), Windsurf (.windsurfrules), Codex / Cline (AGENTS.md), and GitHub Copilot (.github/copilot-instructions.md).

Q:Can I combine this rule with other technology stacks?

Yes. Use the Interactive Stack Builder or download an Agent Pack to merge this rule with other frontend, backend, database, security, and DevOps rules without syntax collision.