PROJECT 08 / AUTOMATION / 2026

Hermod.

Phone access to GenericAgent and its Windows desktop.

Architecture sketch / Mobile gateway for an existing desktop agent
01 / THE PROBLEM

Problem.

GenericAgent already ran on my Windows desktop, but checking its work from a phone meant returning to a terminal or fighting a desktop-sized remote interface. I wanted mobile access to the agent already running, with its conversation intact and an explicit access boundary.

02 / THE APPROACH

Implementation.

  1. 01

    Built an authenticated web gateway around upstream GenericAgent, with expiring single-use pairing, rate limits, session revocation, and Origin/CSRF checks. Kept the integration separate from the agent’s existing runtime and configuration.

  2. 02

    Bridged phone requests into the running terminal agent’s event loop and streamed progress with SSE. Added guarded conversation resume and new-agent launch flows, plus clear status and Stop controls.

  3. 03

    Integrated noVNC for a phone-sized desktop view, with trackpad, keyboard, click, drag, and scroll controls. Viewing is the default; switching modes releases held input so a gesture cannot leave a remote button stuck down.

  4. 04

    Reworked verbose agent output into named progress and safe Markdown built with DOM elements. Explicit public-summary boundaries keep native reasoning and raw tool arguments out of the displayed transcript; uncertain output waits instead of guessing.

Paired phone
Web gateway
Agent + desktop
03 / THE RESULT

Result.

Installed the gateway and mobile interface on the existing Windows environment. Verified paired API access, delivered a real desktop framebuffer, and confirmed an idle terminal agent reported phone readiness. Local browser fixtures exercise chat, resume, desktop controls, and transcript rendering.

VERIFICATION

Thirty-one local tests passed, including browser checks against the vendored noVNC client and safe transcript rendering. Live checks verified deployed files, paired access, readiness, and framebuffer delivery. Physical Android testing and a full live phone-driven task remain pending; answering an agent question while it is busy is a known limitation.

04 / THE TAKEAWAY

What I learned.

A remote interface has to respect the state it connects to. Busy agents, saved conversations, held input, and disconnected desktop sessions all need explicit handling. Public progress also needs a reliable boundary in the source stream; formatting alone cannot decide what is private reasoning.

Questions about this project?

Email Binh ↗
NEXT PROJECT

Heimdall