# Chatterbox TTS over Tailscale — candidate provider notes

Use this when testing or using the user's local Chatterbox TTS server for AutoShorts/TrueTraceShorts voiceovers.

## Current role

Chatterbox is a **candidate provider**, not the default TTS. Do not silently replace the standard voice pipeline until the user explicitly approves it.

Do not compare against Kokoro/Edge or render videos unless requested. For pure provider tests, keep the task scoped to Chatterbox connectivity, audio generation, and metadata.

## Server endpoints

Base URL:

```text
http://100.101.173.25:8004
```

Useful endpoints:

```text
GET  /api/ui/initial-data
GET  /v1/audio/voices
POST /v1/audio/speech
POST /tts
```

Health check:

```bash
curl -s -o /tmp/chatterbox_initial_data.json -w '%{http_code}' \
  http://100.101.173.25:8004/api/ui/initial-data
```

Voice list:

```bash
curl -s http://100.101.173.25:8004/v1/audio/voices
```

## Endpoints and parameters

### OpenAI-compatible endpoint

`POST /v1/audio/speech`

Supported request shape:

```json
{
  "model": "tts-1",
  "input": "Text",
  "voice": "Taylor.wav",
  "response_format": "wav",
  "speed": 1.0,
  "seed": 4096,
  "language": "en"
}
```

Use this for **standard voice / standard preset** tests. Avoid speed changes if the user is judging voice quality; speed adjustments made Taylor sound metallic/blechern.

### Custom Chatterbox endpoint

`POST /tts`

Supported request shape:

```json
{
  "text": "Text",
  "voice_mode": "predefined",
  "predefined_voice_id": "Taylor.wav",
  "output_format": "wav",
  "split_text": true,
  "chunk_size": 180,
  "temperature": 0.7,
  "exaggeration": 0.3,
  "cfg_weight": 0.5,
  "seed": 4096,
  "speed_factor": 1.0,
  "language": "en",
  "stream": false
}
```

Use `/tts` when applying named Chatterbox preset parameters.

## Voice preference discovered

For the user's Scam/Red-Flag videos, **Gianna clone premium v1** is now the accepted default voice. See `references/chatterbox-gianna-premium-clone-voice.md` for the approved parameters and A/B-testing workflow. Taylor human pacing v2 was an improvement over earlier voices, but user feedback reported that some sentence-end emphasis was still wrong. Treat Taylor v2 as a fallback/test option, not the default.

```json
{
  "voice_mode": "clone",
  "reference_audio": "Gianna.wav",
  "pacing": "warm conversational chunks with fewer hard sentence endings",
  "pause_ms_between_chunks": 500,
  "speed_factor": 1.0,
  "temperature": 0.80,
  "exaggeration": 0.52,
  "cfg_weight": 0.50
}
```

```json
{
  "voice": "Taylor.wav",
  "pacing": "semantic chunks with fewer hard sentence endings",
  "inter_chunk_pause_ms": 520,
  "temperature": 0.72,
  "exaggeration": 0.42,
  "cfg_weight": 0.48,
  "seed": "stable per chunk",
  "speed_factor": 1.0,
  "language": "en"
}
```

If sentence endings still sound unnatural, test audio-only alternatives before re-rendering video:

1. **Taylor continuity take** — use longer paragraph/beat chunks instead of sentence chunks, 430–480ms pauses, `temperature≈0.70`, `exaggeration≈0.28–0.34`, `cfg_weight≈0.45–0.50`. Goal: fewer artificial final cadences.
2. **Taylor warm narrator take** — keep chunks semantic, add conversational punctuation/contractions, `temperature≈0.76–0.82`, `exaggeration≈0.45–0.60`, `cfg_weight≈0.48–0.52`. Goal: more human warmth without “motivational speech” drama.
3. **Alternate voice sanity take** — Ryan, Michael, or Thomas with the same performance script. Goal: identify whether the issue is Taylor-specific.
4. **Chosen premium female take (user accepted): Gianna reference clone warm narrator** — `voice_mode=clone`, `reference_audio_filename=Gianna.wav`, warm narrator chunks, `temperature=0.80`, `exaggeration=0.52`, `cfg_weight=0.50`, `speed_factor=1.0`, ~500ms pauses. Use for ERF renders when the user asks for the premium Chatterbox female voice.

Do not use speed changes to solve this; keep `speed_factor=1.0`. Fix by performance script, chunking, pauses, voice/preset A/B tests, and the accepted Gianna reference-clone route when appropriate.

Older acceptable baseline was Taylor + Scientific Abstract Reading (`temperature=0.7`, `exaggeration=0.3`, `cfg_weight=0.5`, `speed_factor=1.0`).

User feedback:

- `Taylor.wav` was preferred over Emily/Gianna/Olivia/Elena variants.
- Slowing via `speed`/`speed_factor` made the audio metallic/blechern.
- Keep Chatterbox speed at standard `1.0`.
- If the pacing feels too fast, change the script punctuation and sentence breaks instead of time-stretching audio.

Example pacing-safe text style:

```text
This delivery text is designed to make you panic.

A small fee.
A short deadline.
A link that looks official.

If it makes you hurry... slow down.
```

## Standard presets seen on the server

Useful lower-drama presets:

- `Scientific Abstract Reading`: `temperature=0.7`, `exaggeration=0.3`, `cfg_weight=0.5`, `seed=4096`, `speed_factor=1.0`
- `Children's Story Narrator`: `temperature=0.8`, `exaggeration=0.7`, `cfg_weight=0.5`, `seed=888`, `speed_factor=1.0`
- `Motivational Speech`: `temperature=0.8`, `exaggeration=1.2`, `cfg_weight=0.55`, `seed=2025`, `speed_factor=1.0`

For Scam Red Flags, test in this order:

1. Taylor + Scientific Abstract Reading — calm, clear, low drama.
2. Taylor + Children's Story Narrator — warmer, friendlier.
3. Taylor + Motivational Speech — more social energy.

## Metadata requirements for tests

For each Chatterbox test WAV, save a metadata JSON with:

- `provider: chatterbox_http`
- `base_url`
- `speech_endpoint` or `endpoint`
- `voices_endpoint` when relevant
- `model` or preset name
- `voice`
- `text_sha256`
- `output_path`
- `output_sha256`
- `duration`
- `file_size`
- `request_time`
- `success`
- `error` if present

## Telegram delivery

For voice proposal review, convert WAV to Telegram voice-bubble OGG/Opus and place it under the Hermes media cache before returning `MEDIA:` lines:

```bash
ffmpeg -y -i input.wav -vn -c:a libopus -b:a 96k -ar 48000 -application voip output.ogg
```

If a Telegram voice bubble still sounds metallic, send the original WAV as a regular file to distinguish Chatterbox output quality from Telegram/Opus playback artifacts.

## Video integration pattern

When replacing an existing voiceover with Chatterbox:

1. Generate Chatterbox WAV with approved voice/preset.
2. Probe audio duration.
3. Avoid time-stretching the audio.
4. If necessary, adjust video timing to the new audio duration, not the other way around.
5. Preserve 1080x1920, SAR 1:1, DAR 9:16, yuv420p.
6. Recreate the approval package and hash-bound private-upload command.
7. Deliver MP4 + thumbnail + title + description + approval command.

## Pitfalls

- Do not use `speed`/`speed_factor` to slow Taylor for user-facing review; it made the result blechern.
- Do not mark Chatterbox as the default provider from a successful test alone.
- Do not run Kokoro/Edge comparisons when the user explicitly says Chatterbox-only.
- Do not render video during a pure connectivity/TTS test.
- Do not assume `default` is a valid voice ID; pick the first available voice only when no preferred voice is known.