# Focused paper fleet repair and watchdog hardening

Use this reference when a CryptoTradingBot multi-strategy paper fleet has accumulated blocked/negative strategies and the user asks to continue with a focused paper run.

## Pattern

1. **Decide the focused fleet from the read-only scorecard**
   - Keep only current positive candidates / useful low-sample research strategies.
   - Pause blocked strategies with negative PnL, low profit factor, or enough losing sample.
   - Do not delete journals, paper state, or reports; those remain analysis evidence.

2. **Use a canonical focused runner outside Git**
   - Store it under runtime state, e.g. `~/.local/state/CryptoTradingBot/run_focused_paper_fleet.sh`.
   - The runner should:
     - define an exact `EXPECTED=(...)` strategy list;
     - stop experiment `AutoTrader.py` processes whose `CTB_STRATEGY_ID` is not expected;
     - start only missing expected strategies via `multi_strategy_launcher.py --strategies ... --launch`;
     - verify every running focused process has `CTB_PAPER_TRADING=true` and `CTB_DRY_RUN=false`;
     - verify exact expected process count before reporting success.

3. **Watchdog should be silent when healthy**
   - Keep the cron script under `~/.hermes/scripts/crypto_paper_fleet_watchdog.sh` with `no_agent=true`.
   - Hourly watchdog should compare exact expected strategy IDs, reject unexpected/unsafe env flags, and call the focused runner only when repair is needed.
   - Non-empty stdout should be concise and Telegram-safe; empty stdout means healthy.

4. **Hyperliquid API reachability probe**
   - `/info` expects a JSON POST, not a bare GET.
   - Use a bounded probe like:

```bash
curl -fsS --max-time 8 \
  -H 'Content-Type: application/json' \
  -d '{"type":"meta"}' \
  https://api.hyperliquid.xyz/info >/dev/null
```

   - If this probe fails, report API/DNS outage and do not restart; this avoids restart thrashing during exchange/network incidents.

5. **Verification checklist**
   - `pgrep`/`/proc/<pid>/environ` confirms exact running strategy IDs.
   - Every focused process is paper-only: `CTB_PAPER_TRADING=true`, `CTB_DRY_RUN=false`.
   - Open exposure count is computed from non-zero `paper_state.positions[*].contracts`, not merely position keys.
   - Manual watchdog run returns exit code 0 and empty stdout when healthy.
   - Relevant scorecard/dashboard tests pass after the operational change.

## Pitfalls

- Do not leave blocked strategies running just because the launcher can start many presets; they consume scan/API capacity and muddle signal.
- Do not use a GET-only health probe for Hyperliquid `/info`; it can falsely report the API down.
- Do not treat a launcher that exits after spawning children as durable supervision. The watchdog must inspect actual child processes and repair the focused fleet.
- Do not claim live safety from process names alone; inspect env flags for every running strategy.