# NVIDIA-hosted keyframe generation for Retention Studio

Use this note when a Shorts/Retention-Studio pipeline needs clean photorealistic keyframes before local I2V (for example Wan2.2). The preferred pattern is:

```text
keyframe request package → NVIDIA hosted image API → normalize/crop/resize keyframe → local I2V → deterministic renderer overlays
```

## Hosted API vs self-hosting

NVIDIA Build pages often include both hosted API examples and deploy/self-host NIM options. For this workflow, prefer the hosted API under:

```text
https://ai.api.nvidia.com/v1/genai/...
```

Self-hosting a NIM container is not required if the hosted endpoint is available for the account. Only consider self-hosting when the hosted endpoint is unavailable, rate-limited, too expensive, or policy requires local infrastructure.

## Secret handling

Load the API key from a runtime secret, not from chat or Git. A useful convention is:

```text
~/.hermes/secrets/nvidia_api_key
NVIDIA_API_KEY=...
chmod 600 ~/.hermes/secrets/nvidia_api_key
```

Never print the key. Reports should record model names/endpoints/status only.

## Model fallback pattern

Build the generator with an ordered fallback chain rather than hard-coding one model. During the AutoShortsBot session, the attempted order was:

```text
qwen-image → stable-diffusion-3.5-large → flux.1-dev
```

In that environment, the Qwen-Image and SD3.5 hosted paths returned 404 while the NVIDIA-hosted FLUX endpoint worked:

```text
https://ai.api.nvidia.com/v1/genai/black-forest-labs/flux.1-dev
```

Treat this as a provider-availability quirk, not a permanent claim that Qwen/SD do not work. Future accounts/endpoints may differ. Keep endpoints configurable.

## Prompting rules for keyframes before I2V

For keyframes that will later be animated:

- Ask for one clear subject, one scene, 9:16 vertical composition.
- Reserve clean negative space for captions in upper/lower thirds.
- Require no readable text, no UI, no watermarks, no license plate text.
- For vehicles/products, explicitly request generic/unbranded surfaces and no manufacturer emblems.
- Still expect models to ignore no-logo/no-emblem instructions sometimes.

## Quality pitfall: logos and emblems

Photorealistic image models often generate brand-like car badges, plate strips, and fake emblems even when forbidden. Do not pass those straight into I2V if brand safety matters. Add a deterministic cleanup or inpainting step before Wan2.2:

```text
generated keyframe → visual inspection → remove/blank logo + plate areas → resize to Wan target → I2V
```

For rough pipeline validation, simple Pillow patching can be acceptable. For production, prefer proper inpainting/logo removal because crude patches can become visible or amplified during I2V.

## Reporting artifacts

Record:

- requested model order
- winning model/endpoint
- HTTP status per attempt, without secrets
- output image path
- original provider size/format
- normalized size/format used by I2V
- known visual issues and whether cleanup was applied

Generated media should stay out of Git unless explicitly intended; commit source code, reports, and review JSONs instead.