---
name: llm-model-operations
description: "Umbrella for model operations: Hugging Face model/data handling, local and server LLM inference, evaluation, experiment tracking, audio/image model utilities, and deployment checks."
version: 1.0.0
author: Hermes Agent
license: MIT
metadata:
  hermes:
    tags: [mlops, llm, inference, serving, evaluation, huggingface, wandb, llama-cpp, vllm, models]
---

# LLM Model Operations

Use for hands-on model operations rather than general software work: downloading/uploading from Hugging Face, running local GGUF inference, serving LLM APIs, benchmarking models, tracking experiments, and operating specialized model utilities.

## Triage

- **Model/dataset hub operations**: Hugging Face CLI/API search, download, upload, repo metadata, tokens.
- **Local/edge inference**: llama.cpp and GGUF workflows for CPU/edge/single-user runs.
- **High-throughput serving**: vLLM/OpenAI-compatible API, tensor parallelism, quantization, monitoring, and load testing.
- **Evaluation**: lm-eval-harness style benchmarks, task selection, reproducible configs, and result logging.
- **Experiment tracking**: Weights & Biases runs, sweeps, model registry, and dashboard hygiene.
- **Specialized model utilities**: Segment Anything for masks; AudioCraft/MusicGen for audio generation.

## Workflow

1. Identify hardware, model format, license, target throughput/latency, and whether network/API credentials are available.
2. Choose the minimal reproducible path: local smoke test before long jobs; tiny model before production model; dry-run before upload/deploy.
3. Capture commands/configs in files so runs can be repeated.
4. Verify with real outputs: generated text/audio/masks, endpoint health checks, benchmark result files, or W&B run URLs.
5. Report resource assumptions and failure modes clearly (VRAM, CUDA/ROCm, quantization compatibility, token limits, credentials).

## Preserved source packages

Copied under `references/absorbed-packages/`: `huggingface-hub`, `llama-cpp`, `serving-llms-vllm`, `evaluating-llms-harness`, `weights-and-biases`, `segment-anything-model`, and `audiocraft-audio-generation`.