nanobot-voice-interface/README.md

# Nanobot Voice Interface

Local-first web UI for Nanobot with:

- WebRTC voice input and streamed audio output
- text chat and agent state/log display
- live cards rendered from HTML templates
- manual tool calls from cards through the Nanobot API socket

This repo is currently designed for a single-user, local workstation setup.

## What This Repo Contains

- `app.py`: FastAPI backend that serves the frontend, card HTML, and bridge endpoints
- `frontend/`: Preact + TypeScript app built with Vite
- `voice_rtc.py`: WebRTC voice session handling
- `supertonic_gateway.py`: bridge between the web app and Nanobot
- `start.sh`: local startup script that installs Python deps, builds the frontend, and launches `uvicorn`
- `examples/cards/`: checked-in snapshots of example card templates and saved card instances

## How It Fits Together

The app has three main pieces:

1. Browser UI
   - connects to the backend over HTTPS
   - establishes a WebRTC session for audio + data channel events
   - renders persisted cards and runs their inline template scripts

2. Web backend
   - serves the built frontend from `frontend/dist`
   - exposes `/rtc/offer`, `/message`, `/cards`, `/tools`, `/tools/call`, and `/tools/jobs/{id}`
   - materializes card templates from the Nanobot workspace card directory

3. Nanobot runtime
   - runs separately in the Nanobot repo
   - exposes the Unix socket API used by this UI
   - provides `message.send`, `command.execute`, `tool.list`, and `tool.call`

Cards do not talk to Nanobot directly. They call browser helpers such as `window.__nanobotCallTool(...)`, which hit the web backend, which then forwards requests to the Nanobot API socket.

## Current Runtime Assumptions

This repo currently assumes:

- single user
- local-only usage
- trusted local card templates
- a separately running Nanobot instance on the same machine

If you expose this service beyond loopback, the trust model changes materially.

## Prerequisites

### Required

- `python3`
- `uv`
- `openssl`
- `bun` or `npm` for the frontend build
- Nanobot checked out locally and runnable

### Runtime dependencies outside this repo

- Nanobot gateway/API from the Nanobot repo
- local Nanobot config under `~/.nanobot/config.json`
- optional voice env overrides in `.env.voice`

## Quick Start

### 1. Install frontend dependencies

From `frontend/`:

```bash
bun install
```

This also installs the versioned git hooks via `postinstall`.

### 2. Make sure Nanobot is running

This UI expects Nanobot to be running separately. In the Nanobot repo, the local combined branch setup currently uses:

```bash
python -m nanobot gateway
```

The web app talks to the Nanobot Unix socket under `~/.nanobot/api.sock`.

### 3. Start the web UI

From this repo:

```bash
./start.sh
```

What `start.sh` does:

- creates `.venv/` if missing
- installs `requirements.txt`
- builds the frontend bundle by default
- sources `.env.voice` if present
- generates a local TLS certificate if missing
- starts `uvicorn`

Default URL:

```text
https://localhost:8000
```

## Important Environment Variables

### Web server

- `UVICORN_HOST`
- `UVICORN_PORT`
- `ENABLE_HTTPS`
- `SSL_CERT_FILE`
- `SSL_KEY_FILE`
- `FRONTEND_BUILD_ON_START`

### Nanobot bridge

- `NANOBOT_WORKDIR`
- `NANOBOT_COMMAND`
- `NANOBOT_SUPPRESS_NOISY_UI`
- `NANOBOT_OUTPUT_DEDUP_WINDOW_S`

### Voice pipeline

Configured through `.env.voice` and defaults in `start.sh`, including:

- STT provider/model/device settings
- TTS provider/model/voice settings
- push-to-talk timing
- RTC outbound buffering settings

Read [start.sh](/home/kacper/nanobot-supertonic-wisper-web/start.sh) for the exact supported variables.

## Frontend Commands

Run from `frontend/`:

```bash
bun install
bun run dev
bun run check
bun run build
bunx knip
```

Meaning:

- `dev`: Vite dev server
- `check`: Biome check, warning-failing
- `build`: `tsc && vite build`
- `knip`: dead-code scan

## Backend Checks

Run from the repo root:

```bash
./scripts/check_python_quality.sh
```

That runs:

- Ruff format check
- Ruff lint
- Deptry
- Vulture

## Frontend Checks

Run from the repo root:

```bash
./scripts/check_frontend_quality.sh
```

That runs:

- `bun run check`
- `bun run build`

## Git Hooks

Versioned hooks live in `.githooks/`.

They are installed by:

```bash
bun install
```

or manually:

```bash
./scripts/install_git_hooks.sh
```

The pre-commit hook runs:

- backend Python quality checks for staged backend changes
- frontend checks for staged frontend changes

## CI

Forgejo Actions config lives at:

- [ci.yml](/home/kacper/nanobot-supertonic-wisper-web/.forgejo/workflows/ci.yml)

Pushes and pull requests run the same check scripts used locally.

## Cards

The running application loads cards from the Nanobot workspace, not from this repo:

- templates: `~/.nanobot/cards/templates/<template-key>/template.html`
- template metadata: `~/.nanobot/cards/templates/<template-key>/manifest.json`
- instances: `~/.nanobot/cards/instances/<card-id>/card.json`
- per-instance state: `~/.nanobot/cards/instances/<card-id>/state.json`

The backend materializes card HTML by reading the template from disk and injecting the instance `template_state` into a hidden JSON script tag.

### Example snapshots

Checked-in snapshots live under:

- [examples/cards](/home/kacper/nanobot-supertonic-wisper-web/examples/cards)

These are for reference and iteration. They are not the live source of truth used at runtime.

## Manual Tool Calls From Cards

Cards can call Nanobot tools through the browser helper layer:

- `window.__nanobotListTools()`
- `window.__nanobotCallTool(name, args)`
- `window.__nanobotCallToolAsync(name, args, options)`

Those map to:

- `GET /tools`
- `POST /tools/call`
- `GET /tools/jobs/{job_id}`

The backend forwards those requests to the Nanobot API socket.

## WebRTC Notes

The connection path is optimized for local-only operation:

- no public STUN server by default
- short ICE gather waits on both frontend and backend
- HTTPS is enabled by default so browser media and WebRTC work consistently

If you need external ICE servers later, you can add a frontend STUN URL with:

- `VITE_WEBRTC_STUN_URL`

## Repo Status

This repo is usable for local single-user workflows, but it is still opinionated and local-environment-dependent:

- card data lives outside the repo in `~/.nanobot/cards`
- Nanobot is a separate repo/runtime dependency
- some startup behavior still depends on local config and local MCP servers

For now, treat this as a local application repo rather than a portable hosted product.