274 lines
6.4 KiB
Markdown
274 lines
6.4 KiB
Markdown
# Nanobot Voice Interface
|
|
|
|
Local-first web UI for Nanobot with:
|
|
|
|
- WebRTC voice input and streamed audio output
|
|
- text chat and agent state/log display
|
|
- live cards rendered from HTML templates
|
|
- manual tool calls from cards through the Nanobot API socket
|
|
|
|
This repo is currently designed for a single-user, local workstation setup.
|
|
|
|
## What This Repo Contains
|
|
|
|
- `app.py`: FastAPI backend that serves the frontend, card HTML, and bridge endpoints
|
|
- `frontend/`: Preact + TypeScript app built with Vite
|
|
- `voice_rtc.py`: WebRTC voice session handling
|
|
- `supertonic_gateway.py`: bridge between the web app and Nanobot
|
|
- `start.sh`: local startup script that installs Python deps, builds the frontend, and launches `uvicorn`
|
|
- `examples/cards/`: checked-in snapshots of example card templates and saved card instances
|
|
|
|
## How It Fits Together
|
|
|
|
The app has three main pieces:
|
|
|
|
1. Browser UI
|
|
- connects to the backend over HTTPS
|
|
- establishes a WebRTC session for audio + data channel events
|
|
- renders persisted cards and runs their inline template scripts
|
|
|
|
2. Web backend
|
|
- serves the built frontend from `frontend/dist`
|
|
- exposes `/rtc/offer`, `/message`, `/cards`, `/tools`, `/tools/call`, and `/tools/jobs/{id}`
|
|
- materializes card templates from the Nanobot workspace card directory
|
|
|
|
3. Nanobot runtime
|
|
- runs separately in the Nanobot repo
|
|
- exposes the Unix socket API used by this UI
|
|
- provides `message.send`, `command.execute`, `tool.list`, and `tool.call`
|
|
|
|
Cards do not talk to Nanobot directly. They call browser helpers such as `window.__nanobotCallTool(...)`, which hit the web backend, which then forwards requests to the Nanobot API socket.
|
|
|
|
## Current Runtime Assumptions
|
|
|
|
This repo currently assumes:
|
|
|
|
- single user
|
|
- local-only usage
|
|
- trusted local card templates
|
|
- a separately running Nanobot instance on the same machine
|
|
|
|
If you expose this service beyond loopback, the trust model changes materially.
|
|
|
|
## Prerequisites
|
|
|
|
### Required
|
|
|
|
- `python3`
|
|
- `uv`
|
|
- `openssl`
|
|
- `bun` or `npm` for the frontend build
|
|
- Nanobot checked out locally and runnable
|
|
|
|
### Runtime dependencies outside this repo
|
|
|
|
- Nanobot gateway/API from the Nanobot repo
|
|
- local Nanobot config under `~/.nanobot/config.json`
|
|
- optional voice env overrides in `.env.voice`
|
|
|
|
## Quick Start
|
|
|
|
### 1. Install frontend dependencies
|
|
|
|
From `frontend/`:
|
|
|
|
```bash
|
|
bun install
|
|
```
|
|
|
|
This also installs the versioned git hooks via `postinstall`.
|
|
|
|
### 2. Make sure Nanobot is running
|
|
|
|
This UI expects Nanobot to be running separately. In the Nanobot repo, the local combined branch setup currently uses:
|
|
|
|
```bash
|
|
python -m nanobot gateway
|
|
```
|
|
|
|
The web app talks to the Nanobot Unix socket under `~/.nanobot/api.sock`.
|
|
|
|
### 3. Start the web UI
|
|
|
|
From this repo:
|
|
|
|
```bash
|
|
./start.sh
|
|
```
|
|
|
|
What `start.sh` does:
|
|
|
|
- creates `.venv/` if missing
|
|
- installs `requirements.txt`
|
|
- builds the frontend bundle by default
|
|
- sources `.env.voice` if present
|
|
- generates a local TLS certificate if missing
|
|
- starts `uvicorn`
|
|
|
|
Default URL:
|
|
|
|
```text
|
|
https://localhost:8000
|
|
```
|
|
|
|
## Important Environment Variables
|
|
|
|
### Web server
|
|
|
|
- `UVICORN_HOST`
|
|
- `UVICORN_PORT`
|
|
- `ENABLE_HTTPS`
|
|
- `SSL_CERT_FILE`
|
|
- `SSL_KEY_FILE`
|
|
- `FRONTEND_BUILD_ON_START`
|
|
|
|
### Nanobot bridge
|
|
|
|
- `NANOBOT_WORKDIR`
|
|
- `NANOBOT_COMMAND`
|
|
- `NANOBOT_SUPPRESS_NOISY_UI`
|
|
- `NANOBOT_OUTPUT_DEDUP_WINDOW_S`
|
|
|
|
### Voice pipeline
|
|
|
|
Configured through `.env.voice` and defaults in `start.sh`, including:
|
|
|
|
- STT provider/model/device settings
|
|
- TTS provider/model/voice settings
|
|
- push-to-talk timing
|
|
- RTC outbound buffering settings
|
|
|
|
Read [start.sh](/home/kacper/nanobot-supertonic-wisper-web/start.sh) for the exact supported variables.
|
|
|
|
## Frontend Commands
|
|
|
|
Run from `frontend/`:
|
|
|
|
```bash
|
|
bun install
|
|
bun run dev
|
|
bun run check
|
|
bun run build
|
|
bunx knip
|
|
```
|
|
|
|
Meaning:
|
|
|
|
- `dev`: Vite dev server
|
|
- `check`: Biome check, warning-failing
|
|
- `build`: `tsc && vite build`
|
|
- `knip`: dead-code scan
|
|
|
|
## Backend Checks
|
|
|
|
Run from the repo root:
|
|
|
|
```bash
|
|
./scripts/check_python_quality.sh
|
|
```
|
|
|
|
That runs:
|
|
|
|
- Ruff format check
|
|
- Ruff lint
|
|
- Deptry
|
|
- Vulture
|
|
|
|
## Frontend Checks
|
|
|
|
Run from the repo root:
|
|
|
|
```bash
|
|
./scripts/check_frontend_quality.sh
|
|
```
|
|
|
|
That runs:
|
|
|
|
- `bun run check`
|
|
- `bun run build`
|
|
|
|
## Git Hooks
|
|
|
|
Versioned hooks live in `.githooks/`.
|
|
|
|
They are installed by:
|
|
|
|
```bash
|
|
bun install
|
|
```
|
|
|
|
or manually:
|
|
|
|
```bash
|
|
./scripts/install_git_hooks.sh
|
|
```
|
|
|
|
The pre-commit hook runs:
|
|
|
|
- backend Python quality checks for staged backend changes
|
|
- frontend checks for staged frontend changes
|
|
|
|
## CI
|
|
|
|
Forgejo Actions config lives at:
|
|
|
|
- [ci.yml](/home/kacper/nanobot-supertonic-wisper-web/.forgejo/workflows/ci.yml)
|
|
|
|
Pushes and pull requests run the same check scripts used locally.
|
|
|
|
## Cards
|
|
|
|
The running application loads cards from the Nanobot workspace, not from this repo:
|
|
|
|
- templates: `~/.nanobot/cards/templates/<template-key>/template.html`
|
|
- template metadata: `~/.nanobot/cards/templates/<template-key>/manifest.json`
|
|
- instances: `~/.nanobot/cards/instances/<card-id>/card.json`
|
|
- per-instance state: `~/.nanobot/cards/instances/<card-id>/state.json`
|
|
|
|
The backend materializes card HTML by reading the template from disk and injecting the instance `template_state` into a hidden JSON script tag.
|
|
|
|
### Example snapshots
|
|
|
|
Checked-in snapshots live under:
|
|
|
|
- [examples/cards](/home/kacper/nanobot-supertonic-wisper-web/examples/cards)
|
|
|
|
These are for reference and iteration. They are not the live source of truth used at runtime.
|
|
|
|
## Manual Tool Calls From Cards
|
|
|
|
Cards can call Nanobot tools through the browser helper layer:
|
|
|
|
- `window.__nanobotListTools()`
|
|
- `window.__nanobotCallTool(name, args)`
|
|
- `window.__nanobotCallToolAsync(name, args, options)`
|
|
|
|
Those map to:
|
|
|
|
- `GET /tools`
|
|
- `POST /tools/call`
|
|
- `GET /tools/jobs/{job_id}`
|
|
|
|
The backend forwards those requests to the Nanobot API socket.
|
|
|
|
## WebRTC Notes
|
|
|
|
The connection path is optimized for local-only operation:
|
|
|
|
- no public STUN server by default
|
|
- short ICE gather waits on both frontend and backend
|
|
- HTTPS is enabled by default so browser media and WebRTC work consistently
|
|
|
|
If you need external ICE servers later, you can add a frontend STUN URL with:
|
|
|
|
- `VITE_WEBRTC_STUN_URL`
|
|
|
|
## Repo Status
|
|
|
|
This repo is usable for local single-user workflows, but it is still opinionated and local-environment-dependent:
|
|
|
|
- card data lives outside the repo in `~/.nanobot/cards`
|
|
- Nanobot is a separate repo/runtime dependency
|
|
- some startup behavior still depends on local config and local MCP servers
|
|
|
|
For now, treat this as a local application repo rather than a portable hosted product.
|