Tokio Cooperative Task Budgeting & epoll Starvation Guardrail

--- description: Tokio Cooperative Task Budgeting & epoll Starvation Guardrail - Enforces cooperative task budgeting and mandatory yield points within compute-intensive async futures to prevent worker thread monopolization. Preserves deterministic epoll event loop cadences in high-throughput network edge proxies, eliminating tail-latency amplification (p99/p99.9) on multiplexed I/O connections. globs: src/**/*.rs, crates/**/*.rs, proxy/**/*.rs alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/tokio-cooperative-budgeting-epoll-starvation-guardrail" --- # Tokio Cooperative Task Budgeting & epoll Starvation Guardrail > **Severity**: CRITICAL | **Category**: devops > **Anti-Pattern Warning**: Monopolizing Tokio Worker Threads with Unbounded In-Band Compute Bypassing epoll Poll Cadence ## System Constraints You are a Principal Systems & SRE Architect enforcing Tokio Cooperative Task Budgeting and epoll starvation guardrails in high-throughput Rust edge services. Mandate that async tasks processing payload validation, cryptographic operations, decompressions, or regex evaluations never monopolize a Tokio worker thread beyond the runtime cooperative budget. Enforce offloading to `spawn_blocking` or explicit cooperative yield boundaries via `tokio::task::yield_now()` to preserve regular driver polling (`epoll_wait`). Reject any code introducing synchronous I/O or unbounded iteration inside network event loops. ## Strict Guidelines - Engineers MUST ALWAYS offload CPU-bound or blocking computations exceeding 10–50 microseconds to `tokio::task::spawn_blocking` or a dedicated thread pool (e.g., Rayon). - Engineers MUST NOT execute synchronous file I/O, heavy cryptographic hashing, unbounded deserialization, or intensive regex processing directly within an async task context. - Code processing large streams or chunked collections in async loops MUST invoke `tokio::task::yield_now().await` or rely on `tokio::task::consume_budget().await` at deterministic batch intervals. - Services MUST configure Tokio runtime I/O drivers with deterministic `event_interval` and `global_queue_interval` thresholds tuned for target proxy p99 SLA limits. - Components NEVER execute raw stdlib synchronization primitives (`std::sync::Mutex`, `std::sync::RwLock`) across `.await` points or within hot network packet paths. ## Bad vs Good Implementations ### Unbounded Payload Parsing Starving epoll vs. Cooperative Budget Yielding #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: The synchronous CPU-intensive loop runs uninterrupted on the Tokio worker thread for tens of milliseconds. This prevents Tokio from processing its run queue, checking the epoll file descriptor, and dispatching other ready network connections, triggering massive tail-latency spikes across all concurrent streams multiplexed on that core. ```rust use tokio::net::TcpStream; use tokio::io::AsyncReadExt; async fn process_proxy_stream(mut stream: TcpStream) -> Result<(), Box<dyn std::error::Error>> { let mut buf = vec![0u8; 1024 * 1024]; let n = stream.read(&mut buf).await?; // ANTI-PATTERN: Heavy cryptographic parsing running synchronously on the Tokio worker thread. // Consumes milliseconds of execution time, starving the mio/epoll driver from driving pending socket readiness. let mut state: u64 = 0; for chunk in buf[..n].chunks(32) { for byte in chunk { state = state.rotate_left(5) ^ (*byte as u64); // Complex cryptographic / validation transformation simulation for i in 0..10_000 { state = state.wrapping_mul(31).wrapping_add(i); } } } Ok(()) } ``` #### Production Standard (Correct) > **Standard**: Moving the CPU-bound operation to `tokio::task::spawn_blocking` frees the primary worker thread to immediately return to Tokio's runtime loop, process `epoll_wait` events via Mio, and handle active connections without queue degradation. ```rust use tokio::net::TcpStream; use tokio::io::AsyncReadExt; async fn process_proxy_stream(mut stream: TcpStream) -> Result<(), Box<dyn std::error::Error>> { let mut buf = vec![0u8; 1024 * 1024]; let n = stream.read(&mut buf).await?; buf.truncate(n); // PRODUCTION FIX: Offload heavy compute to the blocking thread pool // or yield cooperatively back to the scheduler to allow epoll event processing. let state = tokio::task::spawn_blocking(move || { let mut state: u64 = 0; for chunk in buf.chunks(32) { for byte in chunk { state = state.rotate_left(5) ^ (*byte as u64); for i in 0..10_000 { state = state.wrapping_mul(31).wrapping_add(i); } } } state }).await?; tracing::debug!(state, "Payload successfully processed without epoll starvation"); Ok(()) } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: Tokio Cooperative Task Budgeting & epoll Starvation Guardrail

Tokio Cooperative Task Budgeting & epoll Starvation Guardrail defines an industrial-grade behavioral standard for devops engineering. Enforces cooperative task budgeting and mandatory yield points within compute-intensive async futures to prevent worker thread monopolization. Preserves deterministic epoll event loop cadences in high-throughput network edge proxies, eliminating tail-latency amplification (p99/p99.9) on multiplexed I/O connections. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Monopolizing Tokio Worker Threads with Unbounded In-Band Compute Bypassing epoll Poll Cadence Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Monopolizing Tokio Worker Threads with Unbounded In-Band Compute Bypassing epoll Poll Cadence	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS offload CPU-bound or blocking computations exceeding 10–50 microseconds to `tokio

Engineers MUST ALWAYS offload CPU-bound or blocking computations exceeding 10–50 microseconds to `tokio::task::spawn_blocking` or a dedicated thread pool (e.g., Rayon).

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.