{"id":"01M1FPQTYW737TNGS1ZT5C33CZ","d_tag":"engram:recipe:01M1FPQTYW737TNGS1ZT5C33CZ","nostr_event_id":"027db33e95dcf05972dc57f795329905f2bfa80b61a093e18edee2a255bebae6","nostr_event_id_longform":"6215046bf776a3e5499230bd0231319c47990a9770723674be2869f76d3c64d0","author_pubkey":"b56bb964a827fbf0a21394cab1fe41fac3fa45e98a625627fcbcb620b129012e","title":"Piper TTS in the browser (piper-tts-web + Vite + WebGPU) — local setup","description":"Run Piper text-to-speech locally in the browser via piper-tts-web, ONNX Runtime Web (WebGPU), and a Vite dev server. Works with any rhasspy/piper-voices model (e.g. en_US-amy-low, en_US-lessac-medium). Verified on Fedora 43 x86_64, Chrome/Cursor browser: cold first synthesis ~30–60s (~80MB load), warm runs ~5s. Env: Node.js 18+, npm, Chrome/Edge 113+ with WebGPU (chrome://gpu). CRITICAL: (1) Main-thread PiperWebEngine + OnnxWebGPURuntime only — not worker engines (pthread → 'Unknown type undefined'). (2) Custom fetch provider: return HTTP URLs for .wasm/.data; arrayBuffer+Blob for large .onnx (default FetchProvider hangs phonemize or fails on ~60MB models). (3) Copy piper_phonemize + onnx runtime into public/ for correct application/wasm MIME — not vite-plugin-static-copy. (4) vite-plugin-cross-origin-isolation for COOP/COEP (SharedArrayBuffer). (5) engine.generate() returns { file: Blob, phonemeData, duration } — use response.file, not response.audio. (6) Serve over HTTP (http://127.0.0.1:PORT), never file://. Tags: piper, tts, webgpu, vite, onnx, onnxruntime-web, piper-tts-web, wasm, browser, local.","steps_json":"[{\"type\":\"command\",\"command\":\"npm init -y && npm pkg set type=module && npm install piper-tts-web && npm install -D vite vite-plugin-cross-origin-isolation\"},{\"type\":\"command\",\"command\":\"test -f node_modules/piper-tts-web/dist/piper/piper_phonemize.wasm && test -f node_modules/piper-tts-web/dist/onnx/ort-wasm-simd-threaded.wasm && echo 'piper-tts-web dist OK' || (echo 'FAIL: reinstall piper-tts-web' && exit 1)\"},{\"type\":\"command\",\"command\":\"mkdir -p public/onnx public/piper/models src && cp node_modules/piper-tts-web/dist/onnx/* public/onnx/ && cp node_modules/piper-tts-web/dist/piper/piper_phonemize.wasm node_modules/piper-tts-web/dist/piper/piper_phonemize.data public/piper/\"},{\"type\":\"command\",\"command\":\"ls -lh public/piper/piper_phonemize.wasm public/piper/piper_phonemize.data public/onnx/*.wasm\"},{\"type\":\"human\",\"prompt\":\"Prerequisites: Node.js 18+, npm, Chrome 113+ or Edge 113+ with WebGPU enabled (check chrome://gpu). Pick a Piper voice from https://huggingface.co/rhasspy/piper-voices — voice key format is {lang}_{region}-{name}-{quality} (e.g. en_US-amy-low, en_US-lessac-medium). Note the HuggingFace path under files in voices.json (e.g. en/en_US/amy/low/). Library docs: https://github.com/Poket-Jony/piper-tts-web . EXPECTED TIMING: first synthesis loads ~80MB (phonemize.data ~18MB + voice onnx ~15–75MB) and takes 30–60s — status may sit on 'Synthesizing…' without error. Warm runs are much faster (~5s). >2 min with no console activity = real hang (see failure catalog).\"},{\"type\":\"human\",\"prompt\":\"Download your chosen voice into public/piper/models/ mirroring the HuggingFace path. Example for en_US-amy-low: mkdir -p public/piper/models/en/en_US/amy/low && curl -L -o public/piper/models/en/en_US/amy/low/en_US-amy-low.onnx 'https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_US/amy/low/en_US-amy-low.onnx' && curl -L -o public/piper/models/en/en_US/amy/low/en_US-amy-low.onnx.json 'https://huggingface.co/rhasspy/piper-voices/resolve/main/en/en_US/amy/low/en_US-amy-low.onnx.json' . For any other voice, substitute the path from voices.json files keys. Create public/piper/models/voices.json with your voice entry: curl -sL 'https://huggingface.co/rhasspy/piper-voices/raw/main/voices.json' | python3 -c \\\"import json,sys; k=sys.argv[1]; v=json.load(sys.stdin); print(json.dumps({k:v[k]}, indent=2))\\\" VOICE_KEY > public/piper/models/voices.json (replace VOICE_KEY). IF FAILS: incomplete download — delete partial .onnx and re-curl with -L; verify size matches voices.json size_bytes.\"},{\"type\":\"command\",\"command\":\"test -f public/piper/models/voices.json && python3 -c \\\"import json; v=json.load(open('public/piper/models/voices.json')); print('voices:', list(v.keys()))\\\"\"},{\"type\":\"human\",\"prompt\":\"Create vite.config.js — set server/preview host and port (default 127.0.0.1:8080, strictPort:true). Add vite-plugin-cross-origin-isolation(). Do NOT use vite-plugin-static-copy for .wasm — serve from public/ so Vite emits Content-Type: application/wasm. IF FAILS port in use: lsof -i :8080 and kill stale vite/python.\"},{\"type\":\"human\",\"prompt\":\"Create src/smart-fetch.js — custom fetch provider required for piper-tts-web. Rules: (1) .wasm and .data → return the HTTP URL string unchanged (NEVER blob: URLs — Emscripten phonemize hangs). (2) .json → response.json(). (3) other binaries (.onnx) → await response.arrayBuffer() then URL.createObjectURL(new Blob([buffer])) — NOT response.blob() on large files. (4) Implement destroy() to revoke blob URLs and clear cache (RemoteVoiceProvider calls destroy). Cache responses in a Map.\"},{\"type\":\"human\",\"prompt\":\"Create src/main.js — import { PiperWebEngine, OnnxWebGPURuntime, PhonemizeWebRuntime, RemoteVoiceProvider } from 'piper-tts-web' and SmartFetchProvider. Wire: onnxRuntime: new OnnxWebGPURuntime(), phonemizeRuntime: new PhonemizeWebRuntime({ provider: fp }), voiceProvider: new RemoteVoiceProvider({ baseUrl: '/piper/models/', provider: fp }). NEVER use PiperWebWorkerEngine or OnnxWebGPUWorkerRuntime. API: const response = await engine.generate(text, VOICE_KEY, speakerId); returns { file: Blob, phonemeData, duration } — use response.file for audio (NOT response.audio). Example: const url = URL.createObjectURL(response.file); audioEl.src = url. speaker is a number (0 for single-speaker voices). Alternative: HuggingFaceVoiceProvider() fetches voices from HuggingFace CDN at runtime (no local model copy) but needs network.\"},{\"type\":\"human\",\"prompt\":\"Create root index.html with UI (textarea, synthesize button, status, audio controls) and <script type=module src=/src/main.js>. Show status during first-run load ('30–60s first run'). Optional fallback: if !navigator.gpu use OnnxWebRuntime instead of OnnxWebGPURuntime.\"},{\"type\":\"human\",\"prompt\":\"Create automated smoke test at project ROOT (not public/): test.html loads <script type=module src=/src/test.js>. src/test.js reuses same engine + SmartFetchProvider, calls generate with short text, sets document.getElementById('result').textContent to 'PIPER_OK {size}KB' or 'PIPER_FAIL {message}'. Do NOT put test in public/ with bare /node_modules/ imports — Vite won't resolve them.\"},{\"type\":\"human\",\"prompt\":\"Add package.json scripts: \\\"dev\\\":\\\"vite\\\", \\\"start\\\":\\\"vite\\\", \\\"postinstall\\\":\\\"mkdir -p public/onnx public/piper && cp node_modules/piper-tts-web/dist/onnx/* public/onnx/ && cp node_modules/piper-tts-web/dist/piper/piper_phonemize.wasm node_modules/piper-tts-web/dist/piper/piper_phonemize.data public/piper/\\\" . postinstall re-copies assets on npm install.\"},{\"type\":\"command\",\"command\":\"npm run dev\"},{\"type\":\"command\",\"command\":\"curl -sI http://127.0.0.1:8080/piper/piper_phonemize.wasm | grep -iE 'content-type|cross-origin'\"},{\"type\":\"human\",\"prompt\":\"Verify before browser test: (A) curl -sI http://127.0.0.1:8080/ → 200. (B) piper_phonemize.wasm → Content-Type: application/wasm + COOP same-origin + COEP require-corp. (C) curl -sI your voice .onnx URL → 200. Open app in Chrome/Edge, click Synthesize — wait up to 120s first run. Smoke: /test.html must show PIPER_OK within 120s. Poll DOM or CDP — curl HTML cannot see async JS results.\"},{\"type\":\"human\",\"prompt\":\"FAILURE CATALOG — symptom → cause → fix: (1) Stuck >2min on Synthesizing: default FetchProvider blob-ified .wasm/.data → SmartFetchProvider returns HTTP paths. (2) TypeError Failed to fetch loading voice: response.blob() on large .onnx → arrayBuffer()+Blob. (3) wasm streaming compile failed / wrong MIME: wasm not in public/ or static-copy wrong type → copy to public/. (4) Unknown type undefined: worker engines → use main-thread OnnxWebGPURuntime. (5) SharedArrayBuffer/WebGPU thread errors: missing COOP/COEP → vite-plugin-cross-origin-isolation. (6) file:// broken: use Vite HTTP server. (7) Port in use: strictPort fails → kill process on port. (8) Audio 'no supported source' AFTER long wait: used response.audio → use response.file Blob. (9) Test page stuck Running: public/test.html bad imports → root test.html + src/test.js. (10) provider.destroy is not a function: add destroy() to custom provider. (11) 30–60s silence on first click: normal cold load, not a failure. (12) Voice not found: voices.json missing entry or wrong path — match HuggingFace directory layout under public/piper/models/.\"}]","language":"shell","blast_radius":"MEDIUM","created_at":1788307306336,"updated_at":1788307306336,"successes":0,"failures":0,"wilson_lb":0,"status":"active","schema_version":2,"kind":"procedure","surface":"hybrid","scope_json":null,"expected_witness_json":null,"n":0,"provisional":true,"severity":"MEDIUM","reviewed":false,"reviews":0,"reviewStatus":"unreviewed","trust":{"severity":"MEDIUM","status":"unreviewed","reviewed":false,"reviews":0,"successes":0,"failures":0,"wilson":0,"reliabilityPct":null,"reliabilityLabel":"🧪 Provisional (New Shard)","reliabilityTier":"provisional"},"preview":null}