# Audio/video protocol generation with WhisperX + CodexCLI

Use for local meeting-protocol tools that transcribe audio/video then generate a structured `.docx` protocol. This captures the Autoprotocol pattern without binding the skill to one session.

## Architecture pattern

- UI: Streamlit upload flow for audio/video plus optional agenda/previous-protocol context.
- Transcription: WhisperX on GPU with `ffmpeg` for audio extraction and HuggingFace token for diarization.
- Protocol generation: CodexCLI with ChatGPT auth and `gpt-5.5` by default, not a local Ollama/Qwen model when the user asks for the CodexCLI path.
- Output: `python-docx` to write final `.docx` under an ignored `output/` directory.

## Key implementation details

1. Keep raw media, generated transcripts/protocols, `.env`, `.venv`, and caches ignored.
2. Put the LLM bridge behind a small function that builds a complete prompt from:
   - system prompt from `config/prompts.yaml`;
   - session-type task prompt;
   - participant mapping;
   - optional context;
   - diarized transcript lines.
3. Call CodexCLI non-interactively and read from stdin. For long meeting prompts, write the prompt to a file and redirect it into `-`; also capture the final message separately:
   ```bash
   codex exec --skip-git-repo-check \
     --model "${CODEX_MODEL:-gpt-5.5}" \
     --cd "$PROJECT_DIR" \
     --sandbox workspace-write \
     --ephemeral \
     --output-last-message "$OUTPUT_DIR/codex_last_message.txt" \
     - < "$OUTPUT_DIR/prompt.txt"
   ```
   Use `read-only` when no file creation is needed; use `workspace-write` when the prompt asks CodexCLI to write Markdown/DOCX inputs or other artifacts. Avoid `"$(cat prompt.txt)"` for large prompts: it exposes the whole prompt in the process list and can produce confusing stdin/appended-output behaviour. `codex exec` already reports `approval: never` in this mode; do not pass obsolete flags that the installed CLI rejects.
4. Add `AUTOPROTOCOL_LLM_MOCK=1` or equivalent to smoke-test prompt assembly and DOCX writing without spending model calls.
5. For real E2E, verify prerequisites before starting a long transcription:
   - `ffmpeg` exists;
   - `torch.cuda.is_available()` as expected;
   - `whisperx` imports;
   - `codex doctor --summary` is green;
   - `HF_TOKEN` is configured, without printing it.

## Verification commands

```bash
.venv/bin/python -m py_compile app.py autoprotocol.py llm_processor.py
AUTOPROTOCOL_LLM_MOCK=1 .venv/bin/python - <<'PY'
from pathlib import Path
from llm_processor import generate_protocol_with_llm, save_llm_to_word
text = generate_protocol_with_llm([{'speaker':'SPEAKER_00','text':'Wir beschliessen den Test.'}], 'jour_fixe', 'SPEAKER_00 = Testperson')
path = save_llm_to_word(text, 'smoke.wav')
print(path, Path(path).is_file())
PY
codex exec --model gpt-5.5 --cd "$PWD" --sandbox read-only --ephemeral 'Antworte exakt: CODEX_OK'
```

## Pitfalls

- Do not leave Ollama/Qwen-specific cleanup or imports in the UI if the chosen LLM path is CodexCLI; they create misleading runtime requirements and may consume GPU/VRAM.
- Do not commit token-looking placeholders such as `hf_...`; use `<your-huggingface-token>` in docs/examples so secret scanners and humans do not confuse examples for credentials.
- Test Streamlit startup separately from long WhisperX transcription: a simple `streamlit run ...` + HTTP 200 proves UI import/routing without starting expensive audio processing.