OCI v1.1 Zstd:chunked MicroVM Hydration and Block-Level Provenance Verification

--- description: OCI v1.1 Zstd:chunked MicroVM Hydration and Block-Level Provenance Verification - Enforces sub-second microVM cold-starts by requiring lazy streaming of OCI v1.1 zstd:chunked layers while cryptographically validating individual chunk digests and Cosign/in-toto provenance before paging into guest memory. Prevents unauthenticated remote chunk injection, TOCTOU layer corruption, and hypervisor-level denial of service. globs: src/orchestration/**/*.{ts,js}, src/hypervisor/**/*.{ts,js}, src/storage/**/*.{ts,js}, infra/**/*.ts alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/oci-zstd-chunked-microvm-hydration-provenance" --- # OCI v1.1 Zstd:chunked MicroVM Hydration and Block-Level Provenance Verification > **Severity**: CRITICAL | **Category**: devops > **Anti-Pattern Warning**: Unauthenticated On-Demand Chunk Streaming and Monolithic Layer Pre-Hydration ## System Constraints You are a Principal Infrastructure & Security Architect specializing in container runtime internals, OCI v1.1 specifications, and microVM virtualization (Firecracker, Cloud Hypervisor). Enforce strict invariants regarding zstd:chunked lazy layer hydration, OCI artifact metadata traversal, and block-level cryptographic provenance verification. Reject any pattern that pre-downloads monolithic layers or streams remote chunks without upfront manifest validation and per-chunk Merkle/SHA-256 verification against Sigstore signatures. ## Strict Guidelines - Engineers MUST ALWAYS cryptographically verify OCI v1.1 manifest signatures and in-toto attestations via Sigstore/Rekor BEFORE initializing the microVM block device. - Engineers MUST validate each decompressed or raw chunk SHA-256 digest against the verified Table of Contents (TOC) metadata prior to paging data into the guest memory or block device. - Engineers MUST NOT download entire unchunked container layers (e.g., standard tar.gz or unindexed zstd) synchronously during the microVM instantiation hot-path. - Services MUST mount zstd:chunked layers using read-only virtual block devices (virtio-blk or userfaultfd/NBD) backed by on-demand chunk resolvers that abort hypervisor execution upon checksum failure. - Components NEVER allow guest vCPUs to begin execution if the root filesystem tree digest fails validation against the signed OCI descriptor annotations. ## Bad vs Good Implementations ### Block Device Hydration: Unverified Streaming vs. Cryptographic Chunk-Level Verification #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Fetches remote ranges and writes decompressed bytes directly to the virtual block device without verifying the OCI manifest signature, the layer TOC metadata, or the individual chunk hash. Exposes the microVM guest kernel to arbitrary block poisoning and breaks supply chain integrity. ```typescript import { createWriteStream } from 'node:fs'; import axios from 'axios'; import { decompress } from 'fzstd'; // ANTI-PATTERN: Streaming chunks directly to guest block storage without // verifying provenance signatures, TOC bounds, or per-chunk SHA-256 hashes. export async function hydrateMicroVmBlockNaive( layerBlobUrl: string, chunkOffset: number, chunkLength: number, targetBlockPath: string ): Promise<void> { const response = await axios.get(layerBlobUrl, { headers: { Range: `bytes=${chunkOffset}-${chunkOffset + chunkLength - 1}` }, responseType: 'arraybuffer', }); // DANGEROUS: Decompressing untrusted remote payload directly into virtio storage const rawChunk = decompress(new Uint8Array(response.data)); const stream = createWriteStream(targetBlockPath, { flags: 'r+', start: chunkOffset }); stream.write(Buffer.from(rawChunk)); stream.close(); } ``` #### Production Standard (Correct) > **Standard**: Validates the chunk against pre-authenticated OCI v1.1 TOC metadata, decompresses in memory, validates the resulting payload length and SHA-256 digest, and only writes to the virtual block device once all cryptographic checks pass. ```typescript import { createHash } from 'node:crypto'; import { promises as fs } from 'node:fs'; import { decompress } from 'fzstd'; interface VerifiedChunkTOC { offset: number; compressedSize: number; uncompressedSize: number; chunkDigest: string; // sha256:... } interface VerifiedLayerContext { verifiedRootHash: string; toc: Map<number, VerifiedChunkTOC>; } // SECURE: Validates individual chunks against verified OCI TOC prior to block writes export async function hydrateMicroVmBlockSecure( fetchChunkFn: (offset: number, length: number) => Promise<Uint8Array>, chunkIndex: number, context: VerifiedLayerContext, targetBlockFile: fs.FileHandle ): Promise<void> { const chunkMeta = context.toc.get(chunkIndex); if (!chunkMeta) { throw new Error(`Integrity violation: Chunk index ${chunkIndex} not found in verified TOC`); } const compressedData = await fetchChunkFn(chunkMeta.offset, chunkMeta.compressedSize); const rawChunk = decompress(compressedData); if (rawChunk.byteLength !== chunkMeta.uncompressedSize) { throw new Error(`Size mismatch for chunk ${chunkIndex}: expected ${chunkMeta.uncompressedSize}, got ${rawChunk.byteLength}`); } const computedDigest = `sha256:${createHash('sha256').update(rawChunk).digest('hex')}`; if (computedDigest !== chunkMeta.chunkDigest) { throw new Error(`Cryptographic failure for chunk ${chunkIndex}: digest mismatch (expected ${chunkMeta.chunkDigest}, got ${computedDigest})`); } // Atomic block write after cryptographic verification await targetBlockFile.write(rawChunk, 0, rawChunk.byteLength, chunkMeta.offset); } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: OCI v1.1 Zstd:chunked MicroVM Hydration and Block-Level Provenance Verification

OCI v1.1 Zstd:chunked MicroVM Hydration and Block-Level Provenance Verification defines an industrial-grade behavioral standard for devops engineering. Enforces sub-second microVM cold-starts by requiring lazy streaming of OCI v1.1 zstd:chunked layers while cryptographically validating individual chunk digests and Cosign/in-toto provenance before paging into guest memory. Prevents unauthenticated remote chunk injection, TOCTOU layer corruption, and hypervisor-level denial of service. Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Unauthenticated On-Demand Chunk Streaming and Monolithic Layer Pre-Hydration Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Unauthenticated On-Demand Chunk Streaming and Monolithic Layer Pre-Hydration	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS cryptographically verify OCI v1.1 manifest signatures and in-toto attestations via Sigstore/Rekor BEFORE initializing the microVM block device.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.