# MVP Market Data, Importer, Git-Safety, Alert Lifecycle and Crypto Precision Patterns

Use this reference for future FinanceManager work after the crypto wallet/ledger core, especially before building Streamlit read pages or real-data dry runs.

## Market data provider pattern

- Add a provider abstraction rather than hard-coding CoinGecko into dashboards.
- Dashboard/read pages should query local DB tables (`crypto_prices`, later `market_prices`/`fx_rates`) and must not call external APIs on page load.
- Store for every fetched crypto price:
  - `asset_id`
  - `coingecko_id`
  - `price_currency` such as CHF, with USD/EUR prepared
  - exact Decimal price as TEXT
  - `provider`
  - `fetched_at`
  - optional `provider_timestamp`
  - `quality_status`: `fresh`, `stale`, `missing`, `error`, `conflict`
  - optional `error_message`
- Missing CoinGecko ID should create a deduplicated warning and skip valuation for that asset.
- Missing/stale/error prices should create a deduplicated market-data alert instead of inventing a value.

## CoinGecko rate-limit and price-refresh command pattern

- CoinGecko integration must be mockable; unit tests should not make real API calls.
- Expose price refresh as a CLI/scheduled-job command, not as dashboard/report render logic. A good command shape is `finance update-crypto-prices` / `python -m jarvis_finance.cli.main update-crypto-prices --currency CHF --max-age-seconds 3600`.
- The command should load active crypto assets from the runtime DB, skip assets without `coingecko_id` with a deduplicated warning, and output only aggregate counts: total, success, cached, skipped, warnings, errors. Never print prices, quantities, or holdings.
- Return a structured result object from the refresh function (e.g. `PriceRefreshResult`) rather than a bare list of inserted IDs; tests should assert `written_price_ids`, `success_count`, `cached_count`, `warning_count`, and `error_count`.
- Treat HTTP 429 as a first-class outcome:
  - pause via configurable sleeper
  - retry with exponential backoff
  - config knobs: `max_retries`, `initial_backoff_seconds`, `max_backoff_seconds`
  - after exhaustion, return a quote with `quality_status='stale'` or another non-fresh status; do not crash dashboard jobs
- Use cache/freshness controls (`max_age_seconds` or scheduled jobs) so repeated dashboard reloads do not hammer providers. Fresh cached local prices should prevent provider calls.
- Runtime verification after a productive price update should remain aggregate-only: assets with fresh CHF price, skipped assets, warnings/errors, dashboard rows using local prices, report rows using local prices, `metadata.live_api_calls == False`, Git-safety OK, and clean Git status.

## CSV importer completion pattern

Importer modules for wallet/holding/transaction/watchlist/cash MVP should consistently provide:

- Dry-run and commit modes.
- An `import_sessions` record even for dry runs and failures.
- Stable row hash duplicate protection.
- Per-row validation errors; no silent skips.
- Existing-row classification (`rows_existing`) separate from duplicates/failures.
- Synthetic fixtures only; never import the real coin/wallet spreadsheet until mapping and dry-run review are explicitly approved.

Crypto holdings initial import:

- Quantity is mandatory and parsed as Decimal from string.
- Wallet must exist unless the import explicitly supports controlled creation.
- Coin/CoinGecko ID must be validated; missing CoinGecko ID is a warning.
- Store legacy snapshot values for provenance/control only; they are not current valuations.
- Use the confirmed initial holding snapshot logic, not fake buys.

Crypto transactions import:

- Classify `buy`, `sell`, `transfer`, and `fee` explicitly.
- Enforce wallet requirements per type.
- Prevent negative wallet balances through the core transaction functions.
- Preserve ledger coupling for fiat impact.
- Ensure audit-log entries are produced.

Watchlist import:

- `reason` is mandatory.
- Validate status and asset class.
- Target price/currency are optional; investment case and bear case should be accepted for later reporting.

Cash balance initial import:

- Create `initial_cash_snapshot` transactions.
- Account must exist.
- Preserve CHF/FX status and do not overwrite later cash truth.

## SQLite Decimal precision pattern for crypto

SQLite has no exact DECIMAL type. For crypto values, avoid REAL/float entirely.

- Store crypto quantities, prices, crypto fees, and crypto valuation-relevant amounts as TEXT Decimal strings.
- Python layer should parse and compute with `Decimal`.
- Write Decimal values with `format(value, 'f')` rather than `str(value)` so `Decimal('0.00000001')` remains `0.00000001`, not `1E-8`.
- CSV importers must validate Decimal strings before commit.
- Tests should assert both exact Decimal equality and SQLite `typeof(...) == 'text'` for high-precision quantities/prices.
- General fiat ledger columns may still be legacy NUMERIC in MVP, but avoid using them for crypto precision guarantees.

## Alert deduplication and lifecycle pattern

Alerts should not multiply on every recalculation.

Recommended fields:

- `status`: `active`, `resolved`, `muted`
- `resolved_at`
- `muted_until`
- `last_seen_at`
- `occurrence_count`
- `fingerprint`
- `dedup_key`

Recommended dedup key:

- `rule_id`
- `entity_type`
- `entity_id`
- `priority`
- `fingerprint`

Behaviour:

- If an alert with the same dedup key exists and is active/resolved/muted, update `last_seen_at`, increment `occurrence_count`, refresh evidence, and do not create a new row.
- A resolved or muted identical alert should not immediately reappear as a new active alert.
- If the underlying error changes, e.g. different asset, missing FX pair/date, provider error, or other fingerprint, create a new alert.
- Tests should cover duplicate missing-FX/missing-CoinGecko alerts, resolved/muted no-repopup, and changed fingerprint creates a new alert.

## Git-safety hardening pattern

Finance repos need a project-local safety scan that blocks:

- top-level runtime/real-data directories such as `data/`, `imports/`, `exports/`, `reports/`, `backups/`, `secrets/`, `logs/`, `runtime/`, `db/`, `databases/`
- DB files (`.db`, `.sqlite`, `.sqlite3`, dumps)
- spreadsheets/PDF/DOCX and raw real-data CSV/JSON outside allowed synthetic locations
- token/secret filename patterns such as `Github_token*`, `*secret*`, `*credential*`
- token/private-key content patterns including GitHub PATs and PEM private keys

Allow CSV/JSON only under synthetic/example/test fixture paths such as `examples/synthetic/`, `tests/fixtures/`, and config examples.

Important pitfall: do not block source package names like `src/.../imports`; only top-level runtime/data directories should be blocked. Also ignore or remove generated caches such as `__pycache__` and `.pytest_cache` before final safety verification.

## Verification recipe

Before commit/push:

```bash
find . -type d \( -name __pycache__ -o -name .pytest_cache \) -exec rm -rf {} +
PYTHONPATH=src python -m compileall src tests
PYTHONPATH=src pytest tests -q
find . -type d \( -name __pycache__ -o -name .pytest_cache \) -exec rm -rf {} +
PYTHONPATH=src python -m jarvis_finance.cli.main git-safety-scan .
git status --short
```