The sequencer: giving a node graph a sense of song

A node graph makes a moment of visuals: wire it up, press play, the patch reacts. What it doesn't give you is a song — this section gets the calm patch, the drop gets the strobing one, everything crossfades on the downbeat. That's arrangement, and this week VidViz grew a sequencer for it.

The model: layers with envelopes, not clips on tracks

The first version (two days ago) had DAW-style regions on tracks. It lasted about 36 hours before collapsing into something simpler and, for this app, more honest. The sequencer's data model is now three things:

• Section markers — labelled points in time (with a colour, and a skip flag we'll get to), defining the song's structure.
• Layers — each with a name, visibility, solo, and one opacity envelope: points in normalised track time, values 0–1, and a curve per segment (linear, ease-in/out, step).
• A master volume envelope for the audio itself.

The insight behind the simplification: in a visual arranger, a "clip" is fake. The content isn't on the timeline — it's in the node graph. All the timeline needs to say is how much of each layer you see, when. An envelope per layer says exactly that and nothing else. Regions, headers, and clip-dragging were UI cosplay of an audio DAW, deleted with relief.

Crossing the wall between sequencer and graph

The sequencer and the node graph are separate worlds — a Zustand store on one side, a WebGL executor on the other. They meet in a single node: Sequencer Bridge. Each frame it evaluates every layer's envelope at the playhead and emits one Unit output port per layer (ports appear and vanish as layers do). Wire layer1 into a Stack compositor's opacity, layer2 into another, and the timeline is now conducting the graph. A sibling SongPosition node emits normalised playhead position, for visuals that evolve across the whole track.

Nothing about the graph knows the sequencer exists; nothing about the sequencer knows what a texture is. One node is the entire treaty between them, and both sides are simpler for it.

Crossfades fell out of the envelope model rather than being built: a fade-out and the next layer's fade-in overlap the same span, boundaries land exactly on the section marker, and dragging one handle auto-adjusts its partner with ease-out curves on both. What would have been a "crossfade engine" over clips is just four envelope points with manners.

The crackle

The master volume envelope produced the week's best bug: audible crackling during fades. The gain was being driven the officially-blessed way — Web Audio's setTargetAtTime / linearRampToValueAtTime — but re-scheduled every frame, each time cancelling the previous automation. Sixty cancel-and-reschedule operations a second produce micro- discontinuities in the gain curve, worst at low volumes where your ear is most sensitive to steps.

The fix abandons the automation API entirely: seize the gain node once, then set param.value directly each frame with exponential smoothing (τ ≈ 50ms), snapping to target when within a hair. Web Audio automation is built for scheduled-ahead curves; if a JS loop is deciding the value every frame anyway, the scheduler is just an expensive way to add clicks. Also filed under lessons: the mute logic was re-asserting the gain every frame and fighting the fades — now it only acts on actual mute transitions.

Skip sections, and the two clocks

Mark a section as skipped and playback jumps over it — instantly good for "the second verse is just the first verse again". The implementation forced a distinction that feels obvious only afterwards: there are two clocks. Audio time is real file time, used for the audio element and analysis lookups. Display time is audio time with the skipped gaps removed — and that is what feeds the node graph's context, so LFOs, shader time, and every animation stay continuous across the jump instead of visibly lurching. The rulers render skipped ranges as hatched bands with struck-through labels, and the playhead carries its overshoot across the gap so nothing stutters.

Small mechanics that make it feel like a tool

Beat detection moved to Essentia this week, and the beat ruler now aligns to a proper BPM grid — dragging a marker with beat-quantise on snaps to real musical time, with a vertical guide line during the drag. A dedicated section ruler handles structure: double-click to create a marker, drag handles to move it, inline label editing, and the current section's name rides along at the playhead. Transport controls moved into the sequencer's own toolbar, which is an admission of where this app's centre of gravity has moved.

Persistence took the lazy-but-correct route: sequencer state serialises inside the graph document — one file describes both the patch and its arrangement — with undo/redo via a temporal store wrapper (50 steps) and only cosmetic panel state (height, collapsed) kept in localStorage.

The shape of the thing

What the week really established is a division of labour: the graph is the instrument, the sequencer is the score. Keeping the score's vocabulary tiny — markers, envelopes, one bridge node — means the instrument stays free to grow without the two designs colliding. The beat grid the ruler now sits on is doing more load-bearing work every day, though, and I can feel a reckoning coming about who owns musical time in this app. A post for another week.