nanobot-voice-interface/README.md

Nanobot Voice Interface

Local-first web UI for Nanobot with:

• WebRTC voice input and streamed audio output
• text chat and agent state/log display
• live cards rendered from HTML templates
• manual tool calls from cards through the Nanobot API socket

This repo is currently designed for a single-user, local workstation setup.

What This Repo Contains

• app.py: FastAPI backend that serves the frontend, card HTML, and bridge endpoints
• frontend/: Preact + TypeScript app built with Vite
• voice_rtc.py: WebRTC voice session handling
• supertonic_gateway.py: bridge between the web app and Nanobot
• start.sh: local startup script that installs Python deps, builds the frontend, and launches uvicorn
• examples/cards/: checked-in snapshots of example card templates and saved card instances

How It Fits Together

The app has three main pieces:

1. Browser UI

   • connects to the backend over HTTPS
   • establishes a WebRTC session for audio + data channel events
   • renders persisted cards and runs their inline template scripts

2. Web backend

   • serves the built frontend from frontend/dist
   • exposes /rtc/offer, /message, /cards, /tools, /tools/call, and /tools/jobs/{id}
   • materializes card templates from the Nanobot workspace card directory

3. Nanobot runtime

   • runs separately in the Nanobot repo
   • exposes the Unix socket API used by this UI
   • provides message.send, command.execute, tool.list, and tool.call

Cards do not talk to Nanobot directly. They call browser helpers such as window.__nanobotCallTool(...), which hit the web backend, which then forwards requests to the Nanobot API socket.

Current Runtime Assumptions

This repo currently assumes:

• single user
• local-only usage
• trusted local card templates
• a separately running Nanobot instance on the same machine

If you expose this service beyond loopback, the trust model changes materially.

Prerequisites

Required

• python3
• uv
• openssl
• bun or npm for the frontend build
• Nanobot checked out locally and runnable

Runtime dependencies outside this repo

• Nanobot gateway/API from the Nanobot repo
• local Nanobot config under ~/.nanobot/config.json
• optional voice env overrides in .env.voice

Quick Start

1. Install frontend dependencies

From frontend/:

bun install

This also installs the versioned git hooks via postinstall.

2. Make sure Nanobot is running

This UI expects Nanobot to be running separately. In the Nanobot repo, the local combined branch setup currently uses:

python -m nanobot gateway

The web app talks to the Nanobot Unix socket under ~/.nanobot/api.sock.

3. Start the web UI

From this repo:

./start.sh

What start.sh does:

• creates .venv/ if missing
• installs requirements.txt
• builds the frontend bundle by default
• sources .env.voice if present
• generates a local TLS certificate if missing
• starts uvicorn

Default URL:

https://localhost:8000

Important Environment Variables

Web server

• UVICORN_HOST
• UVICORN_PORT
• ENABLE_HTTPS
• SSL_CERT_FILE
• SSL_KEY_FILE
• FRONTEND_BUILD_ON_START

Nanobot bridge

• NANOBOT_WORKDIR
• NANOBOT_COMMAND
• NANOBOT_SUPPRESS_NOISY_UI
• NANOBOT_OUTPUT_DEDUP_WINDOW_S

Voice pipeline

Configured through .env.voice and defaults in start.sh, including:

• STT provider/model/device settings
• TTS provider/model/voice settings
• push-to-talk timing
• RTC outbound buffering settings

Read start.sh for the exact supported variables.

Frontend Commands

Run from frontend/:

bun install
bun run dev
bun run check
bun run build
bunx knip

Meaning:

• dev: Vite dev server
• check: Biome check, warning-failing
• build: tsc && vite build
• knip: dead-code scan

Backend Checks

Run from the repo root:

./scripts/check_python_quality.sh

That runs:

• Ruff format check
• Ruff lint
• Deptry
• Vulture

Frontend Checks

Run from the repo root:

./scripts/check_frontend_quality.sh

That runs:

• bun run check
• bun run build

Git Hooks

Versioned hooks live in .githooks/.

They are installed by:

bun install

or manually:

./scripts/install_git_hooks.sh

The pre-commit hook runs:

• backend Python quality checks for staged backend changes
• frontend checks for staged frontend changes

CI

Forgejo Actions config lives at:

• ci.yml

Pushes and pull requests run the same check scripts used locally.

Cards

The running application loads cards from the Nanobot workspace, not from this repo:

• templates: ~/.nanobot/cards/templates/<template-key>/template.html
• template metadata: ~/.nanobot/cards/templates/<template-key>/manifest.json
• instances: ~/.nanobot/cards/instances/<card-id>/card.json
• per-instance state: ~/.nanobot/cards/instances/<card-id>/state.json

The backend materializes card HTML by reading the template from disk and injecting the instance template_state into a hidden JSON script tag.

Example snapshots

Checked-in snapshots live under:

• examples/cards

These are for reference and iteration. They are not the live source of truth used at runtime.

Manual Tool Calls From Cards

Cards can call Nanobot tools through the browser helper layer:

• window.__nanobotListTools()
• window.__nanobotCallTool(name, args)
• window.__nanobotCallToolAsync(name, args, options)

Those map to:

• GET /tools
• POST /tools/call
• GET /tools/jobs/{job_id}

The backend forwards those requests to the Nanobot API socket.

WebRTC Notes

The connection path is optimized for local-only operation:

• no public STUN server by default
• short ICE gather waits on both frontend and backend
• HTTPS is enabled by default so browser media and WebRTC work consistently

If you need external ICE servers later, you can add a frontend STUN URL with:

• VITE_WEBRTC_STUN_URL

Repo Status

This repo is usable for local single-user workflows, but it is still opinionated and local-environment-dependent:

• card data lives outside the repo in ~/.nanobot/cards
• Nanobot is a separate repo/runtime dependency
• some startup behavior still depends on local config and local MCP servers

For now, treat this as a local application repo rather than a portable hosted product.