A node-based visual programming system in one week

VidViz is a week old. The idea: a music visualisation studio where you build the visuals by wiring nodes together — audio analysis in one end, textures flowing through transforms and compositors, video out the other. This week the node system went from "Phase 2" in a commit message to 58 node types across eight categories: 27 visualisations, 15 transforms, 5 analysis nodes, compositors, sources, outputs, and the humble reroute dot.

The engine is worth writing down while it still fits in one head.

What a node is

Every node is one file: a definition object plus a processor factory, self-registering into a registry on import.

{
  type, category, label, description,
  inputs: PortDefinition[],
  outputs: PortDefinition[],
  parameters: ParameterDefinition[],
  createProcessor(),        // → { configure, process, dispose }
  pure?: boolean,           // stateless → output cacheable
  feedback?: boolean,       // temporal cycle-breaker
}

The simplest real node, Reroute, is a processor whose entire job is process(inputs) { return { out: inputs.in } } — and even it earns its pure: true flag, because pure nodes get output-cached and skipped until a parameter or upstream value actually changes.

Ports are typed: Texture, Number, NumberArray, Color, Trigger, Vec2, Waveform, Unit, Any, and a few more. Textures travel between nodes as THREE.js render targets, not raw pixels — the graph is really a compositor pipeline that happens to look like boxes and wires.

The two features that make it an instrument

Parameter modulation. Any parameter can declare a port name, at which point it exists twice: as a slider, and as an input. Wire an LFO into it and the live value overrides the slider — which turns into an animated purple ghost tracking the modulated value, so you can see the LFO breathing through the UI. The implementation threads the node ID into the processor so a live-values store can be polled at frame rate without touching React at all; the sliders animate, the component tree doesn't re-render.

Unit-mapped ports. Modulation sources all emit Unit — 0 to 1 — because an LFO shouldn't need to know that hue is 0–360 while blur radius is 0–50. The mapping to a real range lives on the destination:

const mapMin = node.parameters[`${param.name}_mapMin`] ?? param.portMapMin;
inputs[param.portName] = mapMin + unitVal * (mapMax - mapMin);

and the min/max are themselves editable per node instance, so "wobble the zoom, but only between 1.1× and 1.3×" is two number fields, not a Math node sandwich. This is the single decision that most changed how patching feels — every source speaks the same language and every destination translates locally.

The executor

Each animation frame, the executor runs the graph in topological order, gathering each node's inputs from upstream outputs and handing every processor a context: time, delta, frame index, resolution, a seed. Three design points earn their keep:

Feedback without cycles. Video feedback — the trails-and-tunnels staple — needs a node's output fed back into its input, which a topological sort flatly refuses. Feedback nodes therefore run twice per frame: phase one emits their stored frame from last time, phase two runs normally and captures the new frame. Back-edges into feedback nodes are excluded from cycle detection. The graph stays a DAG in the math while being a loop in the eye.

Freeze protection. Users will wire a 4K fluid simulation into a fractal into six shader passes, and the app must not die. Every node carries a rolling 30-frame timing window; average above 80ms and the node is auto-bypassed, with a total frame budget of 150ms and a recovery cooldown before a bypassed node gets another chance. (All of it disabled during export, where correctness outranks interactivity.) It's the circuit-breaker pattern, applied to art.

Texture pooling. Render targets are acquired from a keyed pool and reused across frames unless their size changed. Allocating GPU textures per frame is how you turn a 60fps patch into a slideshow with extra steps.

Alpha, the eternal humiliation

The Stack compositor does alpha-over blending for six layers entirely in a fragment shader with straight (non-premultiplied) alpha — not via WebGL blend state. And the reason is the classic trap: rendering into an off-screen RGBA target with the default blend function multiplies alpha in twice — once into the colour channels, once into the alpha channel — so the layer comes out ghostly when composited later. The fix is blendFuncSeparate, keeping colour blending and alpha accumulation on separate rules. Every graphics programmer apparently pays this toll once; this was my week to pay it.

The fun ones

Not everything is infrastructure. This week also shipped a GPU fluid simulation node, a Milkdrop-style node with spatial port mapping, and a geometry node that grew fifty shape primitives — stars, gears, hearts, crosses, arrows — in a single 600-line growth spurt. One node with fifty shapes beat fifty nodes with one shape; the palette is for capabilities, not for enum values.

Honest ledger, week one

What doesn't exist yet: the preview loop hardcodes the beat signal to false — audio analysis runs, but proper beat reactivity in the graph is still ahead. Group nodes exist in the type system but are barely more than folded boxes. The per-frame loop lives inside a React component, which is already smelling like a future refactor.

But the core bet looks right. A node graph is more work up front than a settings panel — an executor, a type system, pooling, freeze protection, all before the first pretty pixel. The return is that every feature from here on is compositional: a new analysis node multiplies against every visual that can consume a number, instead of adding one more checkbox. Week one bought the multiplier. Now to feed it.