# Local speech

Install one Suzo helper on a computer that can reach three services:

| From the helper | Required connection |
| --- | --- |
| Suzo server | Outbound HTTPS/WSS to your configured Suzo server, such as `https://api.suzo.app` |
| TTS endpoint | Voice generation API; local or reachable from the helper |
| STT endpoint | Transcription API; local or reachable from the helper |

The helper does not need direct access to your agent process. Your original agent keeps its context, tools and MCP connection to Suzo. Suzo relays audio through the helper and does not need network access to the TTS/STT endpoints. Models can share the helper computer, run on a PC on your LAN/Tailscale, or use existing inference servers. Endpoint URLs determine the connections; there are no computer-location pickers.

The helper opens the Suzo connection, so it needs no public listening port. Plain HTTP model endpoints are accepted only on loopback (`localhost`, `127.0.0.1`, `[::1]`); other model endpoints require HTTPS, including over LAN or Tailscale. `localhost` always means the helper’s computer. Phone-initiated wake-up of an idle agent is a separate [runtime callback](runtime-webhooks.md) connection; its receiver must be reachable by the Suzo server.

## Install and configure

1. Install [Node.js 22.13 or newer](https://nodejs.org/en/download) on the computer running the helper. The helper supports macOS, Windows and Linux; model runtimes have their own hardware/platform requirements.
2. Download `/setup/client.mjs` from your own trusted Suzo server and save it as `suzo-connect.mjs` on that computer.
3. Open a terminal in that folder and run the command copied from the agent’s **Voice → Local speech → Set up manually** guide:

```sh
node suzo-connect.mjs --speech --agent <contact-id>
```

### Connect an existing agent

The wizard reuses the private agent connection previously saved by Suzo’s setup client. On another computer or after a manual MCP setup, it asks for the path to the existing private `connection.json` file. That file contains `serverURL`, `agentId` and `token`; the token is the existing agent’s Suzo bearer credential. Keep it on the computer, never in chat or on the phone. If the original setup did not create a connection file, create it privately from that contact’s existing MCP configuration. No new agent pairing is needed. Server/contact validation precedes use of the saved credential.

The wizard asks for:

- TTS API base URL, model ID, voice ID and raw PCM sample rate (24 kHz unless changed).
- STT API base URL and model ID. Press Enter to reuse the TTS address if both APIs share a server.
- Optional API keys, entered without terminal echo. A saved key is reused only for its matching endpoint address.

It downloads the authenticated helper bundle, installs dependencies with `npm ci` and saves a private per-contact configuration. Accept **Start now** to run it immediately, or use the printed start command later. Leave the terminal running during calls; OS startup/service installation is separate. Config files are owner-only on POSIX; on Windows use an account-private directory with appropriate NTFS permissions. `SUZO_HOME` can override the private storage directory.

Rerunning the wizard on an existing installation can keep its settings or reconfigure endpoint details. Reconfiguration preserves `speech.json.previous` and replaces the config atomically; it does not overwrite installed helper source. If a helper is already online for the contact, the wizard asks you to stop it before changing or starting another instance. Existing source updates remain a separate, deliberate operation through the authenticated bundle.

**Suzo connection verified** proves authenticated MCP discovery, not that your model endpoints can synthesize or transcribe correctly. Once the helper is online, audition a voice in the app to exercise real synthesis. Use the supplied benchmark for a synthesis/transcription smoke check and measure recognition quality separately in your intended language. Setup never rings the phone automatically or changes the contact’s selected voice/provider.

The native guide has three read-only steps: connections, install and configure. It explains the network and endpoint contracts, copies a contact-specific wizard command and links this computer guide. It contains no location pickers, endpoint forms, token fields or JSON editing. Merely browsing it never starts or saves a waiting/handoff state. Only copying the separate agent setup prompt starts a handoff; **Stop waiting** clears it. A real helper connection makes voice audition available, and **Use [voice]** explicitly activates local speech for that contact.

## Endpoint compatibility

Use an OpenAI-compatible API base, usually ending in `/v1`. The helper appends the paths below. A compatible model name alone is insufficient: the serving software must implement these request and audio formats.

### TTS: voice generation

`POST <base>/audio/speech`, with JSON:

```json
{"model":"your-model-id","input":"Text to speak","voice":"your-voice-id","response_format":"pcm","stream":true}
```

Return streaming raw signed 16-bit little-endian mono PCM, with `Content-Type: audio/pcm`, `audio/raw` or `application/octet-stream`. The configured sample rate must match the returned audio (8–48 kHz); the helper converts it to 24 kHz for the phone. WAV, MP3 and JSON responses are rejected. The private configuration may supply `voices: [{"id":"…","name":"…"}]` to expose several voices in the app.

### STT: transcription

`POST <base>/audio/transcriptions`, accepting multipart form fields `file` (a WAV recording), `model`, and optional `language`. Return JSON:

```json
{"text":"Recognized speech"}
```

The helper sends 16 kHz mono WAV. Optional endpoint keys use `Authorization: Bearer <key>` and stay in private configuration. TTS and STT can use the same API base or separate ones; only the helper needs access. An unsupported native model API needs a runtime/adapter implementing this contract.

## Model and runtime examples

These are unranked examples to investigate, not an allowlist or automatic selection. Choose for your language, hardware, quality and measured latency. Check current upstream installation requirements and test the endpoint contract.

| Software/model | Role | Integration notes |
| --- | --- | --- |
| [Speaches](https://github.com/speaches-ai/speaches) | TTS and STT server | Upstream documents OpenAI-compatible APIs with faster-whisper, Kokoro and Piper. One server may cover both endpoints; verify the configured audio format. |
| [Kokoro-FastAPI](https://github.com/remsky/Kokoro-FastAPI) | TTS server | Upstream documents streaming raw PCM through `/v1/audio/speech`, model/voice IDs and 24 kHz playback. |
| [Chatterbox](https://github.com/resemble-ai/chatterbox) | TTS model family | A model/runtime choice; serve through an adapter implementing the contract above. |
| [Qwen3-TTS](https://github.com/QwenLM/Qwen3-TTS) | TTS model family | A model/runtime choice; serve through an adapter implementing the contract above. |

These endpoint integrations have not all been exercised in a physical Suzo call. The agent setup prompt asks the agent to inspect actual hardware, compare current upstream options (including choices outside this table) and explain measured limits. The supplied hardware diagnostic reports facts without ranking models. Detected GPUs are not automatically used by an inference runtime.

## Automation and entirely manual setup

`--input /private/path/input.json` remains available for agents, scripts and advanced configuration. Supply both `tts` and `stt` explicitly; no model family is selected automatically:

```json
{
  "serverURL":"https://your-suzo-server",
  "agentId":"existing-agent-uuid",
  "speech":true,
  "tts":{"engine":"endpoint","url":"http://127.0.0.1:8000/v1","model":"your-tts-model","voice":"your-voice","sampleRate":24000},
  "stt":{"engine":"endpoint","url":"http://127.0.0.1:8001/v1","model":"your-stt-model"}
}
```

The installer reads `~/.suzo/agents/<agentId>/connection.json` unless a private `connectionFile` path or explicit `token` is supplied. Existing private token inputs remain supported. Secrets never belong in command arguments. The actual worker `speech.json` additionally has `version:1` and the resolved private token. Start it with `npm start -- /private/path/speech.json` from the installed helper folder. `npm run benchmark -- /private/path/speech.json` synthesizes and transcribes a short English sample; it measures latency, not accuracy in every language.

For full manual installation, GET the authenticated `/v1/agent/speech-helper` bundle, save its allowlisted relative files in a private folder, run `npm ci`, and create the worker configuration. To connect MCP for a new agent, run the setup client without `--speech`; this asks for its harness and one-time pairing code. Keep separate scoped credentials for different contacts.

### Optional in-process engines

The helper retains the existing optional CPU runners for explicit advanced configuration:

```json
{"tts":{"engine":"kokoro","precision":"q8"},"stt":{"engine":"whisper","model":"base","language":"en"}}
```

Kokoro accepts `q8`, `q4` or `fp32` and currently exposes English voices. Whisper accepts `tiny`, `base` or `small` with optional `language`. Neither is selected by the endpoint wizard. Both run on the helper computer’s CPU. Their models download on first start and cache under `~/.suzo/models` unless `cacheDirectory` overrides it. Endpoint-only preparation does not load or download these models, although their optional runner dependencies remain packaged.

### Voice auditions

`POST /v1/agents/:id/local-speech/preview` is phone-authenticated and accepts only `{"voiceId":"…"}`. It asks that contact’s connected helper to synthesize a fixed short sample, returning a bounded 24 kHz mono WAV with `Cache-Control: no-store`. It does not ring, create call records or transcripts, wake MCP, or change preferences. Audio is held in memory; the app keeps at most six samples during that screen’s lifetime. Samples stop on navigation, backgrounding or an incoming call.

Updated helpers advertise `preview: true` in their version-1 hello. Older helpers remain call-compatible; the app offers an update prompt when audition is unavailable. Preview jobs time out after 60 seconds, are bounded to 20 seconds of PCM, are cancelled on HTTP/helper disconnect, and reject concurrent preview jobs for that helper. Calls take priority and cancel a pending preview. Active calls block new auditions. The helper uses the same synthesis engine and voice as calls; recognition is not started for a preview.

Suzo supplies the helper, installer and documentation; the agent does not need to build the audio relay from scratch. The setup client downloads the helper from your authenticated Suzo server, installs its dependencies and writes a private per-contact configuration. The first start downloads models only for explicitly selected built-in engines. Endpoint engines use the runtime you install separately. Installing a compatible third-party model runtime is a separate step when using external engines.

## Transport and limits

The worker authenticates an outbound WebSocket at `/voice/local/worker` with its agent credential. The phone receives a short-lived, single-use call join credential. Microphone audio is 16 kHz mono PCM; synthesized audio is 24 kHz mono PCM. Production uses WSS through the HTTPS server, including a reverse proxy that permits WebSocket upgrades. MCP transports text and controls, not audio. The server relays audio; it is not end-to-end encrypted.

The phone uses voice-processing audio for echo cancellation. A bounded energy detector segments microphone input; a pause ends a turn. Speech onset cancels queued playback and stale replies. Inference already running on CPU may finish before it can stop, so interruption does not imply instantaneous model cancellation. A helper handles one call at a time, reconnects after transport loss, and advertises its voices only to its own contact. If the helper is offline, Suzo rejects local calls before ringing. Cloud providers remain available per contact.

Audio configuration changes can stop the iPhone's engine. The native session reinstalls its microphone converter for the current hardware format, restarts playback, and retains bounded unplayed PCM. Completion callbacks from an earlier render schedule cannot consume the rescheduled packets. Audio-thread callbacks explicitly avoid inheriting main-actor isolation.

For a connected development phone, launch a Debug build with `--suzo-check-local-audio` to check the real microphone callback, silent playback completion, and recovery after an engine stop. This briefly uses the microphone but does not retain recordings, contact the speech helper, or ring the phone. Read `Documents/Diagnostics/events.jsonl` in the app container for `local_audio_check_passed` or `local_audio_check_failed`. The check does not establish audible Kokoro speech, CallKit routing, or a completed conversation; those require a real call.

On 22 October 2026, the hardware check passed on a physical iPhone, followed by a completed Kokoro/Whisper call lasting over three minutes. The user confirmed audible speech and interruption; spoken turns reached the original MCP agent, and an in-call chart action returned through MCP. This verifies that development setup, not every audio route or hardware combination. The subsequent caption fix sends sentence PCM boundaries and advances captions from phone playback; see [call motion](call-motion.md) for timing limits.

This helper is not an operating-system service installer. To run it persistently, configure your OS's normal service/login mechanism using the printed start command and private config. Avoid a service account that cannot read the model cache or credentials.