Getting pixels out: WebCodecs export, NDI, and a Rust LAN caster

A visualiser that can't leave the preview panel is a screensaver. This week VidViz finished growing its full set of exits: file export rewritten on WebCodecs, WebRTC streaming to any browser on the network, NDI for the pro-video crowd, and — the fun one — a Rust LAN caster with hardware HEVC and its own little receiver app. This post is the tour, including the corpses.

v1: FFmpeg over a WebSocket, and the classic deadlock

The first export pipeline (week one) was honest plumbing: render each frame to an offscreen canvas at time = frame / fps, read the RGBA pixels, stream raw buffers over a WebSocket to the Python backend, which pipes them into FFmpeg's stdin. Flow control by acking every tenth frame. It worked, then it hung on longer exports, and the cause is one every subprocess user eventually meets: FFmpeg chats progress onto stderr, the code only read stderr after proc.wait(), the OS pipe buffer filled, FFmpeg blocked mid-write, and everyone waited for everyone forever. The fix is a daemon thread draining stderr from the moment of spawn. If you take one line from this post: a pipe you don't drain is a deadlock you've scheduled.

v2: WebCodecs, and export as a pure function

This week's rewrite removes the WebSocket, the Python hop, and FFmpeg from the encode path entirely. The renderer encodes with the browser's VideoEncoder (hardware-accelerated H.264 or H.265, level chosen automatically from resolution and framerate), muxes MP4/WebM in the renderer with mp4-muxer/webm-muxer, and streams chunks to disk through small IPC file-stream handlers. Backpressure is the encoder's own queue depth. Audio still uses FFmpeg for one job it's unbeatable at — decoding the source track to raw PCM — muxed in parallel with the video encode.

The part I care most about isn't speed, it's determinism. Export doesn't capture the live preview; it re-executes the graph frame by frame with exporting: true and the seeded RNG reset per frame from a fixed seed. Same project, same seed, same bytes of visual content — render farms and A/B comparisons become meaningful, and "the export looks different from the preview" is a bug class that structurally can't exist. The quality ladder runs from draft (x264 CRF 28) through ultra and master (x265 CRF 5 and 1) to lossless, plus ProRes and VP9 for people with workflows; resolutions to 4K, framerates to 120.

Streaming: three audiences, three transports

WebRTC, for "just look at this". Electron's main process runs a tiny HTTP+WebSocket server; any browser on the LAN opens the viewer page, signaling happens over the socket, and the canvas streams via captureStream. One gotcha worth recording: with preserveDrawingBuffer: false, capturing the WebGL canvas directly yields blank frames — so a proxy 2D canvas drawImages the WebGL output each frame and it gets captured. Wasteful, reliable, shipped.

NDI, for people with video mixers. Rather than a native addon build, the main process talks to libndi.dylib directly through FFI. The bug of the week here: sending frames as RGBA FourCC corrupted downstream compositing, because our alpha channel is full of compositing residue that NDI receivers dutifully honoured. Switching the FourCC to RGBX — same bytes, "ignore the fourth one" — fixed every downstream mixer at once. Sometimes the fix is telling the truth about your data.

LAN Cast, for a projector across the room. This one went native: a Rust child process reads RGBA frames from a shared-memory file the renderer mmaps (an 8-byte frame counter as the world's smallest synchronisation protocol), encodes with VideoToolbox hardware HEVC, and ships RTP, advertising itself over mDNS as _vidviz-cast._tcp. On the other end sits vidviz-view, a separate minimal Electron app wrapping the matching Rust receiver: browse mDNS, connect, VideoToolbox decode, frames to a canvas. Laptop renders, random Mac by the projector displays, nothing but the LAN in between.

The same shared-memory trick currently powers the fullscreen output window — renderer writes frames to an mmap'd file, the fullscreen window reads and draws them via WebGL2. Two Electron windows exchanging pixels through /tmp is the kind of solution that works perfectly while feeling like a note from your future self saying "revisit this".

The scoreboard

Four transports, one source of truth: everything downstream of the graph executor consumes the same RGBA frames, whether they're headed for an encoder queue, a proxy canvas, an FFI call, or a shared-memory file. That discipline — one render path, many exits — is what let three streaming transports land in a single week without touching the engine. The pipeline count says the app has opinions about being seen now. Next problem: making sure what everyone sees lands on the beat.