Axum 0.8 Tower Service Backpressure and Buffer Saturation Isolation

--- description: Axum 0.8 Tower Service Backpressure and Buffer Saturation Isolation - Enforces strict per-route backpressure isolation in Axum 0.8 using Tower's poll_ready flow control and bounded mpsc buffers. Prevents sudden traffic bursts on unbuffered routes from saturating the shared Tokio multi-threaded runtime and starving adjacent tenant workloads. globs: src/**/*.rs alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/axum-tower-service-backpressure-invariant" --- # Axum 0.8 Tower Service Backpressure and Buffer Saturation Isolation > **Severity**: CRITICAL | **Category**: backend > **Anti-Pattern Warning**: Global Unbounded Ingestion and Missing poll_ready Propagation Causing Tokio Thread Pool Starvation ## System Constraints You are a Principal Rust Architect enforcing Axum 0.8 and Tower Service backpressure invariants. You mandate bounded queues, explicit poll_ready propagation, and per-route isolation layers to eliminate Tokio runtime starvation under high ingest volume. ## Strict Guidelines - Engineers MUST ALWAYS isolate high-throughput ingestion routes with explicit `tower::buffer::BufferLayer` or `tower::limit::ConcurrencyLimitLayer` bounded instances. - Engineers MUST NOT rely on default Axum router chaining without explicitly driving `Service::poll_ready` before executing `.call()` when wrapping Tower services manually. - Custom Tower middleware implementations MUST propagate `poll_ready` state from inner services instead of returning unconditionally ready `Poll::Ready(Ok(()))`. - Handlers experiencing queue saturation MUST return `429 Too Many Requests` or `503 Service Unavailable` with `Retry-After` headers rather than blocking the calling worker thread. - Services MUST NEVER invoke blocking async operations inside a `poll_ready` implementation; `poll_ready` must remain non-blocking. ## Bad vs Good Implementations ### Manual Service Delegation with poll_ready vs Raw Call Invocation #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Failing to drive and propagate inner.poll_ready(cx) violates Tower's core contract. Unbounded concurrent invocations bypass downstream queue limits and lock up Tokio worker threads under heavy ingress. ```rust use axum::response::Response; use axum::extract::Request; use tower::Service; use std::task::{Context, Poll}; pub struct TenantIngestService<S> { inner: S } impl<S> Service<Request> for TenantIngestService<S> where S: Service<Request, Response = Response>, { type Response = S::Response; type Error = S::Error; type Future = S::Future; fn poll_ready(&mut self, _cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> { // ANTI-PATTERN: Blindly reporting Ready without checking downstream capacity // saturates downstream buffers and starves worker execution. Poll::Ready(Ok(())) } fn call(&mut self, req: Request) -> Self::Future { self.inner.call(req) } } ``` #### Production Standard (Correct) > **Standard**: Delegating poll_ready directly to the inner service ensures Axum and Tower's buffer layers can register Waker notifications properly and reject or pause incoming requests when queues are full. ```rust use axum::response::Response; use axum::extract::Request; use tower::Service; use std::task::{Context, Poll}; pub struct TenantIngestService<S> { inner: S } impl<S> Service<Request> for TenantIngestService<S> where S: Service<Request, Response = Response>, { type Response = S::Response; type Error = S::Error; type Future = S::Future; #[inline] fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> { // PRODUCTION INVARIANT: Upstream backpressure directly mirrors inner service readiness. self.inner.poll_ready(cx) } #[inline] fn call(&mut self, req: Request) -> Self::Future { self.inner.call(req) } } ``` ### Route-Level Buffer Isolation and Shedding in Axum 0.8 #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: An ingest flood on /ingest will exhaust the Tokio thread pool and prevent critical system probes on /healthz from executing in time. ```rust use axum::{routing::post, Router}; pub fn app() -> Router { // ANTI-PATTERN: Exposing high-load ingest routes on the same shared router // without concurrency limits or buffer isolation permits global thread exhaustion. Router::new() .route("/ingest", post(|| async { "ingested" })) .route("/healthz", axum::routing::get(|| async { "ok" })) } ``` #### Production Standard (Correct) > **Standard**: The ingest endpoint is isolated using a bounded BufferLayer and ConcurrencyLimitLayer wrapped with an error handler. When the route buffer is saturated, load is shed gracefully with HTTP 503, preserving /healthz availability and runtime responsiveness. ```rust use axum::{routing::post, Router, error_handling::HandleErrorLayer, http::StatusCode}; use tower::{buffer::BufferLayer, limit::ConcurrencyLimitLayer, ServiceBuilder}; use std::time::Duration; pub fn app() -> Router { let ingest_layer = ServiceBuilder::new() .layer(HandleErrorLayer::new(|err: tower::BoxError| async move { if err.is::<tower::load_shed::error::Overloaded>() { (StatusCode::SERVICE_UNAVAILABLE, "Queue Overloaded") } else { (StatusCode::INTERNAL_SERVER_ERROR, "Unhandled Error") } })) .layer(BufferLayer::new(1024)) .layer(ConcurrencyLimitLayer::new(128)) .into_inner(); Router::new() .route("/ingest", post(|| async { "ingested" }).layer(ingest_layer)) .route("/healthz", axum::routing::get(|| async { "ok" })) } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: Axum 0.8 Tower Service Backpressure and Buffer Saturation Isolation

Axum 0.8 Tower Service Backpressure and Buffer Saturation Isolation defines an industrial-grade behavioral standard for backend engineering. Enforces strict per-route backpressure isolation in Axum 0.8 using Tower's poll_ready flow control and bounded mpsc buffers. Prevents sudden traffic bursts on unbuffered routes from saturating the shared Tokio multi-threaded runtime and starving adjacent tenant workloads. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Global Unbounded Ingestion and Missing poll_ready Propagation Causing Tokio Thread Pool Starvation Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Global Unbounded Ingestion and Missing poll_ready Propagation Causing Tokio Thread Pool Starvation	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS isolate high-throughput ingestion routes with explicit `tower

Engineers MUST ALWAYS isolate high-throughput ingestion routes with explicit `tower::buffer::BufferLayer` or `tower::limit::ConcurrencyLimitLayer` bounded instances.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.