# Provider Integration Sprint v2 — Free/Fallback Market Data

Use this reference when extending the FinanceManager Equity/ETF instrument search, price preview/update, FX update, and provider status flows with multiple external providers while preserving privacy and explicit-action semantics.

## Hard constraints

- Never commit API keys, `.env`, runtime DBs, real provider responses tied to real portfolio data, reports, or real financial values.
- Runtime secrets live outside Git, normally `~/jarvis_runtime/finance-system/secrets/.env`; verify mode `600` when practical, but never print values.
- Dashboard render must not call external providers. Provider calls only on explicit user clicks, CLI commands, or scheduled jobs.
- Do not send holdings, quantities, balances, cost basis, account IDs, wallet IDs, broker docs, private notes, or portfolio rows to providers. Only send explicit search terms, ISINs, tickers/symbols, exchanges, or FX pairs entered/confirmed for lookup.
- EODHD is low-volume fallback only; keep it out of default provider chains and require explicit opt-in such as `--provider eodhd --allow-low-volume`.

## Secret aliases to support

- OpenFIGI: `OPENFIGI_API_KEY`, `JARVIS_OPENFIGI_API_KEY`
- FMP: `FMP_API_KEY`, `JARVIS_FMP_API_KEY`
- Finnhub: `FINNHUB_API_KEY`, `JARVIS_FINNHUB_API_KEY`
- Twelve Data: `TWELVEDATA_API_KEY`, `TWELVE_DATA_API_KEY`, `JARVIS_TWELVEDATA_API_KEY`, `JARVIS_TWELVE_DATA_API_KEY`
- Massive: `MASSIVE_API_KEY`, `JARVIS_MASSIVE_API_KEY`
- EODHD: `EODHD_API_KEY`, `JARVIS_EODHD_API_KEY`
- Alpha Vantage status/optional fallback: `ALPHA_VANTAGE_API_KEY`, `JARVIS_ALPHA_VANTAGE_API_KEY`

`TWELVEDATA_API_KEY` without the extra underscore is mandatory; do not only support `TWELVE_DATA_API_KEY`.

## Provider role pattern

### Instrument search / metadata

Preferred explicit lookup order:

1. Local catalog/cache first.
2. OpenFIGI for ISIN/Ticker mapping evidence.
3. FMP as primary public search/preview source where available.
4. Finnhub for symbol lookup/profile fallback.
5. Twelve Data for symbol search/quote fallback.
6. Massive for US ticker metadata/details fallback.
7. Manual creation with warnings.

Do not include EODHD in default search because of daily call limits.

### Equity/ETF prices

Implement a composite provider for CLI/scheduled jobs, not dashboard render:

1. FMP primary.
2. Twelve Data fallback.
3. Finnhub fallback.
4. Massive fallback.
5. EODHD only explicit low-volume opt-in.

Return sanitized quote objects with `provider`, `quality_status`, `error_message`, timestamp/currency, and `close=None` on failure instead of raising raw HTTP/provider exceptions into the UI.

### FX rates

- Frankfurter can be no-key primary for common FX updates.
- Twelve Data can be an optional keyed provider for pairs such as `USD/CHF` and `EUR/CHF`.
- Store rates locally; dashboard/report contexts read local rates only.
- `CHF/CHF = 1` is local, not a provider call.

## Error classification

Map provider failures into stable categories that are safe for UI/log/test output:

- missing key: `*_api_key_missing` / status `missing_key`
- HTTP 401/403: `auth_failed`
- HTTP 402/paywall/tariff limit: `restricted_endpoint`
- HTTP 429: `rate_limited`
- DNS/timeout/connectivity: `network_error`
- empty successful response: `no_results`
- malformed or provider-side error JSON: `provider_error`

Specifics:

- OpenFIGI uses header `X-OPENFIGI-APIKEY` plus `Content-Type: application/json`; a previous 401 must be retested after the user corrects the key and should not poison future status.
- FMP uses `apikey`; 402/tariff failures should be non-fatal `restricted_endpoint`, especially for endpoints not covered by the current plan.
- Massive may need Authorization-header first, with query-param fallback if the provider/client pattern requires it.

## Status display pattern

Provider status rows should be aggregate and secret-free. Use labels like present/missing/auth_failed/error/restricted/rate_limited, never key prefixes, key lengths, raw tokens, raw URLs containing credentials, or response bodies.

When showing status in Streamlit/User Mode, use a non-probing mode (`probe=False`) so normal render only indicates configured/known/last explicit status. Live probes belong behind an explicit button or CLI smoke-test command.

## Smoke-test pattern

For explicit provider test runs, report only aggregate booleans/counts and categories:

- key found yes/no
- reachable yes/no
- auth OK yes/no
- candidate/price/FX found yes/no
- error category
- rate-limit headers read yes/no when applicable
- no position saved
- no portfolio data used

Typical safe instruments/pairs:

- OpenFIGI mapping ISIN `US9229087690`
- OpenFIGI search query `Vanguard Total Stock`, `exchCode=US`
- OpenFIGI mapping ticker `VTI`, `exchCode=US`
- FMP/Finnhub/Twelve/Massive test ticker `VTI` or `AAPL`
- Twelve Data FX `USD/CHF` or `EUR/CHF`
- Avoid EODHD live smoke unless explicitly necessary; at most one call.

## Tests and verification

- Mock provider HTTP responses for unit tests; do not require real keys in test suite.
- Add tests proving missing keys skip provider calls and sanitize errors.
- Add tests proving dashboard/price preview reads local market prices and does not call APIs on render.
- Add tests for provider aliases, especially `TWELVEDATA_API_KEY`.
- After `pytest`, remove `.pytest_cache` and `__pycache__` before `git-safety-scan`; pytest recreates caches.
- Run: compileall, pytest, Git-safety scan, `git diff --check`, `git status`, commit, push, then fetch/update `origin/main` or otherwise verify the remote tracking state.