Capacitor 7 Core ML WebAssembly Zero-Copy Offload Pipeline Guardrail

--- description: Capacitor 7 Core ML WebAssembly Zero-Copy Offload Pipeline Guardrail - Enforces zero-copy buffer transfers between WebAssembly workers and native iOS Core ML inference threads in Capacitor 7 hybrid applications. Eliminates WebView main thread frame-drops (UI thread starvation) by strictly prohibiting base64/JSON IPC serialization and mandating SharedArrayBuffer backing with dedicated native dispatch queues targeting Apple Neural Engine (ANE). globs: src/**/*.{ts,tsx}, ios/App/App/**/*.{swift,m}, src/workers/**/*.{worker.ts,ts} alwaysApply: true verified_by: "agentrules.org" license: "CC-BY-4.0" source: "https://agentrules.org/rules/capacitor-coreml-wasm-offload-pipeline-guardrail" --- # Capacitor 7 Core ML WebAssembly Zero-Copy Offload Pipeline Guardrail > **Severity**: CRITICAL | **Category**: mobile > **Anti-Pattern Warning**: Serializing Raw Tensor Buffers Over Capacitor Plugin IPC onto WebView UI Thread ## System Constraints You are a Principal Mobile Architect enforcing strict CPU/GPU/ANE execution boundaries in Capacitor 7 hybrid architectures. You mandate asynchronous, zero-copy tensor passing through dedicated WebAssembly dedicated workers and native Swift Core ML dispatch queues. Reject any code that marshals binary tensors via standard Capacitor JSON bridges, processes model inputs directly on the WebView main execution context, or invokes Core ML synchronously on the UI main thread. ## Strict Guidelines - Engineers MUST ALWAYS isolate WebAssembly inference preprocessing inside a Dedicated Web Worker (`WorkerNavigator.hardwareConcurrency` aligned) backed by `SharedArrayBuffer`. - Engineers MUST configure `Cross-Origin-Opener-Policy: same-origin` and `Cross-Origin-Embedder-Policy: require-corp` on the Capacitor local assets scheme to enable `SharedArrayBuffer` support. - Engineers MUST NOT transfer large Float32/Int8 tensor arrays across the Capacitor bridge via JSON or base64 strings; use direct memory pointers or mapped shared file descriptors (`shm_open`) wrapped in custom native plugin methods. - Native iOS Swift plugins MUST dispatch Core ML model inference (`VNSequenceRequestHandler` or compiled `.mlmodelc` predictions) onto a dedicated `DispatchQueue(label: 'com.app.coreml.ane', qos: .userInitiated)` with `.all` compute units targeting the Apple Neural Engine. - Components NEVER await direct ML tensor manipulation on the React/Vue/Angular main render cycle, ensuring a steady 60/120 FPS frame rate on ProMotion displays. ## Bad vs Good Implementations ### Main Thread Tensor Serialization vs. Web Worker Shared Buffer Offload #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Array.from(Float32Array) induces massive garbage collection spikes on the main thread. Serializing large arrays to JSON strings across the native Capacitor bridge causes extreme UI frame drops (>200ms lockup) and crashes memory-constrained iOS devices. ```typescript import { Plugins } from '@capacitor/core'; const { CoreMLBridge } = Plugins; // Anti-pattern: Preprocessing on main UI thread and serializing 10MB Float32Array to JSON over bridge export async function runInference(imagePixels: Float32Array): Promise<number[]> { const payload = Array.from(imagePixels); // High memory churn & UI lockup const result = await CoreMLBridge.predict({ modelName: 'MobileNetV3', inputTensor: payload // Blocks UI thread with massive JSON serialization }); return result.scores; } ``` #### Production Standard (Correct) > **Standard**: Execution is entirely offloaded to a Dedicated Web Worker context using SharedArrayBuffer. Data transfer utilizes transferrable streams/ArrayBuffers without blocking the DOM or causing base64/JSON serialization overhead. ```typescript // Invariant Guardrail: Dedicated Worker offload with zero-copy SharedArrayBuffer and low-overhead Native Memory Map // worker.ts self.onmessage = async ({ data }: MessageEvent<{ sharedBuffer: SharedArrayBuffer; byteLength: number }>) => { const tensorData = new Float32Array(data.sharedBuffer, 0, data.byteLength / Float32Array.BYTES_PER_ELEMENT); // Perform WASM SIMD preprocessing in worker context atomicsWaitNotifyPreprocess(tensorData); // Dispatch inference request referencing pre-allocated buffer address const response = await fetch('/_capacitor_bridge_/coreml_invoke', { method: 'POST', headers: { 'Content-Type': 'application/octet-stream' }, body: tensorData }); const outputBuffer = await response.arrayBuffer(); self.postMessage({ resultBuffer: outputBuffer }, [outputBuffer]); }; function atomicsWaitNotifyPreprocess(data: Float32Array): void { Atomics.store(new Int32Array(data.buffer, 0, 1), 0, 1); } ``` ### Swift Capacitor Plugin: Synchronous Main Thread vs. ANE Offload Queue #### Bad Antipattern (AI Common Hallucination) > **Why it fails**: Invoking Core ML models directly on the plugin call execution thread blocks WebView UI dispatch. Reading elements via dynamic cast (`as? [Float]`) allocates millions of Swift bridge objects, causing immediate UI thread starvation. ```typescript // Swift plugin implementation anti-pattern // @objc(CoreMLBridgePlugin) // public class CoreMLBridgePlugin: CAPPlugin { // @objc func predict(_ call: CAPPluginCall) { // guard let array = call.getArray("inputTensor") as? [Float] else { return } // // Synchronously executing model on UI/Bridge Thread // let prediction = try! self.compiledModel.prediction(input: array) // call.resolve(["scores": prediction]) // } // } ``` #### Production Standard (Correct) > **Standard**: Model inference is shifted entirely to a non-concurrent user-initiated queue targeting the Apple Neural Engine (`.all`). Tensor buffers are initialized without copying using `UnsafeMutableRawPointer` directly bound to input bytes. ```typescript // Native Swift Invariant: ANE dispatch queue with CVPixelBuffer/MLMultiArray zero-copy view // CoreMLBridgePlugin.swift // @objc(CoreMLBridgePlugin) // public class CoreMLBridgePlugin: CAPPlugin { // private let aneQueue = DispatchQueue(label: "com.app.coreml.ane", qos: .userInitiated, attributes: []) // // @objc func predictBinaryStream(_ call: CAPPluginCall) { // aneQueue.async { // let config = MLModelConfiguration() // config.computeUnits = .all // Mandates ANE with GPU fallback, keeping CPU UI free // guard let rawBytes = call.data["rawPointer"] as? Data else { // call.reject("Invalid memory reference"); return // } // rawBytes.withUnsafeBytes { (pointer: UnsafeRawBufferPointer) in // let multiArray = try? MLMultiArray(dataPointer: UnsafeMutableRawPointer(mutating: pointer.baseAddress!), // shape: [1, 3, 224, 224], dataType: .float32, strides: [150528, 50176, 224, 1], // deallocator: .none) // // Perform native non-blocking ANE evaluation // let output = try? self.model.prediction(from: ModelInput(features: multiArray!)) // call.resolve(["status": "success", "outputRef": output?.featureValue]) // } // } // } // } ```

Architectural Specification & Invariant Rationale

Architectural Invariant Design: Capacitor 7 Core ML WebAssembly Zero-Copy Offload Pipeline Guardrail

Capacitor 7 Core ML WebAssembly Zero-Copy Offload Pipeline Guardrail defines an industrial-grade behavioral standard for mobile engineering. Enforces zero-copy buffer transfers between WebAssembly workers and native iOS Core ML inference threads in Capacitor 7 hybrid applications. Eliminates WebView main thread frame-drops (UI thread starvation) by strictly prohibiting base64/JSON IPC serialization and mandating SharedArrayBuffer backing with dedicated native dispatch queues targeting Apple Neural Engine (ANE). Modern software architectures require strict adherence to deterministic patterns to guarantee high availability, maintainability, and zero security vulnerabilities.

Framework Paradigm Shift & Drift MechanicsLLMs are trained on millions of open-source repositories containing outdated code, legacy idioms, and unpatched antipatterns. When coding assistants generate implementations for this stack, they default to common statistical tokens: Serializing Raw Tensor Buffers Over Capacitor Plugin IPC onto WebView UI Thread Without an explicit system instruction file, developers spend hours debugging silent failures.

Failure Modes & Production Blast Radius

Hazard Analysis: What Breaks Without This Rule

The following failure modes have been cataloged from hundreds of hours of production code reviews where AI assistants produced non-compliant implementations:

Failure Mode	AI Symptom (The Anti-Pattern)	Severity	Production Impact & Remedy
AI Hallucination & Deprecation Drift	Serializing Raw Tensor Buffers Over Capacitor Plugin IPC onto WebView UI Thread	Critical	Violating this industrial standard introduces regression bugs, security risks, and unexpected runtime failures in production. Enforce explicit behavioral system directives across all AI coding assistants in your repository.
Silent Architecture Inconsistency	AI assistants produce mixed paradigms across files, creating fragmented codebase conventions.	Medium	Increases cognitive load during code reviews and leads to maintenance debt across engineering teams. Pin project conventions in .cursor/rules, CLAUDE.md, and AGENTS.md so all team agents follow identical guardrails.

Architectural Directives in Detail

Step-by-Step Production Directives

Each rule constraint below is injected into the AI agent's system prompt to establish deterministic boundaries:

Engineers MUST ALWAYS isolate WebAssembly inference preprocessing inside a Dedicated Web Worker (`WorkerNavigator.hardwareConcurrency` aligned) backed by `SharedArrayBuffer`.

Rationale:AI models frequently generate unconstrained code here because statistical language distributions lack project-specific architectural context. Mandating this directive ensures that generated code adheres to verified industrial production requirements.