Giving AI agents eyes and hands inside a running Electron app

Most of VidViz is written with AI coding agents, and for the first two weeks there was a grinding mismatch in the loop: the agent writes renderer code, I run the app, I look at the screen, and then I type a description of what I see back to a system that just wrote three hundred lines of state management it now can't observe. The agent was coding blind, and I was employed as its eyes.

This week that got fixed, twice — once for hands, once for eyes.

Hands first: an MCP server that drives the app

The Model Context Protocol server came first, oddly. It's a first-party server registered in the repo's .mcp.json, so any MCP client — Claude Code included — picks it up automatically. Getting a message from an MCP client into a running Electron renderer takes three hops:

MCP client ─stdio─> server ─ws://127.0.0.1:8766─> Electron main
                                 └─ executeJavaScript ─> renderer bridge

The stdio server forwards each tool call over a local WebSocket to a relay inside Electron's main process, which injects it into the renderer where a bridge function dispatches into the real stores. Around forty tools come out the other end: graph_* (add and remove nodes, wire edges, batch-update parameters, group, bypass, auto-layout), transport_* (play, pause, seek, loop), project_*, sequencer_*, and preset_* families.

The family that earns its keep is diagnose_*: find cycles, list disconnected nodes, dump execution order, check port compatibility. An agent that just assembled a twenty-node graph can immediately ask the app whether the graph makes sense — and fix its own wiring before I ever look. Tools that act need tools that check; giving an agent hands without giving it a mirror just produces confident mess faster. There are also test scripts that hit the relay's WebSocket directly, so "build a beat-reactive chain and press play" is a runnable script rather than a clicking ritual.

Then eyes: the CDP bridge

The MCP tools are curated verbs. The second mechanism is the opposite: raw, total visibility. The app already launches with --remote-debugging-port=9229 in dev, which means Chrome DevTools Protocol is sitting right there. Two globals get installed at renderer startup:

window.__debug — snapshot methods per store, returning JSON-safe summaries: graph(), project(), sequencer(), timeline(), theme(), analysis(), or state() for everything at once.
window.__stores — the raw Zustand stores, actions included.

A 60-line Node script completes the loop: it asks CDP for the renderer target, opens the debugger WebSocket, and sends one Runtime.evaluate with awaitPromise and returnByValue. App state becomes a shell command:

node scripts/debug-eval.mjs '__debug.sequencer()'
node scripts/debug-eval.mjs '__stores.graph.getState().graph.name'

# actions work too — wrap side effects in an IIFE
node scripts/debug-eval.mjs \
  '(() => { __stores.sequencer.getState().togglePanelCollapsed(); return {done:true}; })()'

That last one matters more than it looks: the bridge isn't read-only, so a failing interaction can be reproduced from the shell, not just observed.

Why two mechanisms and not one

They're different tools for different trust levels. The MCP server is a typed, curated surface — the verbs I'm happy for any agent to use, with sensible semantics, discoverable through the protocol. The CDP bridge is an escape hatch — eval into the live renderer, capable of anything, documented in the repo's CLAUDE.md precisely so that agents reach for __debug.state() before theorising about what the code "should" be doing. Curated hands, unrestricted eyes. The reverse — unrestricted hands, curated eyes — would be exactly wrong.

Both are dev-only. The stores land on window only in development builds, and the debugging port is a dev-launcher flag. (Landing alongside all this, and surfaced through the bridge: a demo-project lock, so neither an enthusiastic agent nor a demo user can rename or delete the bundled Demo project unless dev mode is explicitly on. Ask me how that rule earned its place.)

What the loop feels like now

The payoff shows up in the texture of a working session. "Did the crossfade refactor keep the envelope points?" is no longer a question I answer by squinting at a panel — the agent runs __stores.sequencer.getState().layers and compares counts itself. A bug report can be a state snapshot instead of a paragraph. And the end-to-end agent workflow — build a graph via MCP, diagnose_* it, press play, then read __debug.timeline() to confirm the playhead is actually moving — runs without a human in the loop at all.

The general principle, for anyone pairing agents with a GUI app: the quality of AI-assisted work on stateful software is capped by the agent's access to that state. Logs are the state you predicted you'd need; a live query surface is the state you actually need, queryable after the fact. It cost about a day to build — one file exposing snapshots, one script speaking CDP, one relay for MCP — and it's the highest-leverage day this project has had.