Erledigt, Sir. Der **produktive Initial-Snapshot-Import für das sichere Subset** wurde durchgeführt — ohne echte Mengen/Werte im Chat und ohne echte Daten im Git-Repository.

## Import-Status

- Import durchgeführt: **ja**
- Snapshot-Stichtag: **31.12.2025**
- Produktive DB: **Runtime-Pfad außerhalb des Repos**
- Echte XLSX: nur temporär gelesen
- Temporäre CSVs: nur Runtime-/Temp-Kontext, anschließend entfernt
- Legacy-CHF-Werte K–Q: **ignoriert**
- Unsichere Coins: **nicht importiert**
- Ignorierte Coins: **nicht importiert**
- Git-Repository: **clean**
- Git-Safety: **GIT_SAFETY_OK**

## Runtime-DB

- Pfad: `/home/agent/jarvis_runtime/finance-system/data/finance.sqlite3`
- DB liegt außerhalb des Repos: **ja**
- Runtime-Basis liegt außerhalb des Repos: **ja**
- Dashboard kann lokale DB lesen: **ja**

## Importierte Struktur, aggregiert

- Importierte / vorhandene Wallets: **8**
- Safe-Subset Crypto Assets vorhanden: **19**
- Safe-Subset Holdings vorhanden: **25**
- Wallets mit Safe-Subset-Holdings: **5**
- Import-Session für Holdings: **committed**
- Holdings-Import-Fehler: **0**
- Audit-Log erstellt: **ja**
- Crypto-Audit-Einträge vorhanden: **52**
- Data-Quality-Alerts: **0**

Hinweis zur Import-Session-Zählung: Während des ersten produktiven Laufs gab es einen technischen Schema-/Importer-Konflikt nach bereits begonnenem Schreiben. Ich habe danach kontrolliert fortgesetzt und nur fehlende Holdings ergänzt. Ergebnis in der DB ist vollständig für das sichere Subset: **25 Safe-Subset-Holdings vorhanden**, keine doppelten Nachimporte für bereits vorhandene Zeilen. Nicht elegant, aber sauber nachkontrolliert — Finanzsysteme lieben bekanntlich kleine Charaktertests.

## Import-Sessions

- Wallet-Import-Session: **committed**
- Holdings-Import-Session: **committed**
- Letzte Holdings-Session:
  - Status: **committed**
  - Fehler: **0**
  - Bestand nach Nachkontrolle vollständig für das sichere Subset

## Importierte Coin-Labels

Importiert bzw. in der Runtime-DB vorhanden:

- AAVE
- BNB
- Band Protocol BAND
- CAKE Pancake
- Celo CGLD
- Compound COMP
- Enjin ENJ
- FDUSD
- Fetch.ai FET
- Flare FLR
- HBAR
- LL Light Link
- Stellar Lumens XLM
- Swissborg
- Tether USD
- The Graph GRT
- VTHO
- VeChain VET
- Wilder World WILD

Bestätigte Mappings wurden angewendet:

- Swissborg → BORG / `swissborg`
- Tether USD → USDT / `tether`
- Swissborg Wallet → SwissBorg
- Metamask → MetaMask
- Yoloi Wallet → Yoroi Wallet

## Weiterhin ausgelassen

Explizit ignoriert:

- NIGHT
- S Sonic

Manuelle Nacharbeit / später im Dashboard ergänzen:

- ADA Cardano
- Amp
- Avalanche AVAX
- Bitcoin BTC
- Chainlink
- DOGE
- Decentraland MANA
- Ethereum ETH
- HIGH Highstreet
- IOTA IOT
- LUNA Terra
- LUNC Terra Classic
- NETVR
- NuCypher NU
- Ox ZRX
- Polygon MATIC
- Ripple XRP
- SAND The Sandbox
- SAROS
- SKY
- SOL Solana
- Sundae
- USTC Terra Classic

## Git-/Safety-Nachkontrolle

- Git Status: **clean**
- Git-Safety Ergebnis: **GIT_SAFETY_OK**
- Keine echten CSVs/XLSX/Reports/SQLite-Dateien ins Repo geschrieben.
- Keine echten Mengen/Werte ausgegeben.

## Nächster UI-Ausbauschritt vorgemerkt

Für das Dashboard sollte als nächstes ein Modul für manuelle Crypto-Korrekturen gebaut werden:

- Coin hinzufügen mit CoinGecko-Suche, Wallet-Auswahl, Menge, Datum, Notiz, Bestätigung, Audit-Log
- Coin entfernen nur als Transaktion/Korrektur, nicht als stilles Löschen
- Coin bearbeiten nur via `manual_adjustment`, nicht durch Überschreiben
- CoinGecko-Suche mit manueller Auswahl bei Mehrdeutigkeit; Symbol allein nie automatisch eindeutig behandeln

Produktiver Initial-Snapshot für das sichere Subset ist damit abgeschlossen.


[M G] Sehr gut. Der produktive Initial-Snapshot-Import für das sichere Subset ist akzeptiert.

Bevor wir weiterbauen, möchte ich wegen des erwähnten Schema-/Importer-Konflikts eine kontrollierte Verifikationsrunde durchführen.

Bitte jetzt noch keine neuen Features bauen.

Bitte führe zuerst eine Konsistenzprüfung der produktiven Runtime-DB durch.

Prüfpunkte:

1. DB-/Runtime-Safety

* Runtime-DB liegt außerhalb des Git-Repos
* Git Status ist clean
* Git-Safety ist GIT_SAFETY_OK
* keine echten CSV/XLSX/Reports/DB-Dateien im Repo

2. Import-Konsistenz
   Bitte prüfen, aber keine echten Mengen/Werte im Chat ausgeben:

* Anzahl Wallets
* Anzahl Crypto Assets
* Anzahl Holdings
* Anzahl Holdings pro sicherem Coin nur als Count
* keine doppelten Holdings für gleiche Kombination aus asset_id + wallet_id
* keine negativen Holdings
* keine Holdings mit leerer Menge
* keine Holdings ohne Wallet
* keine Holdings ohne Asset
* keine importierten Holdings für ignorierte Coins NIGHT und S Sonic
* keine importierten Holdings für unsichere Coins
* Snapshot-Stichtag überall korrekt als 31.12.2025 markiert
* Legacy-CHF-Werte wurden nicht übernommen
* CoinGecko-IDs für importierte Assets vorhanden

3. Audit- und Import-Session-Konsistenz
   Bitte prüfen:

* Import-Session für Wallets vorhanden
* Import-Session für Holdings vorhanden
* Import-Session-Status committed
* Audit-Log-Einträge vorhanden
* jede importierte Holding hat nachvollziehbare Import-/Audit-Spur
* keine halbfertigen Import-Sessions mit Status started/failed/partial, außer sie sind klar als technische Zwischenfälle dokumentiert

4. Dashboard-Test mit echten Daten
   Bitte lokal prüfen:

* Dashboard kann Runtime-DB lesen
* Crypto-Seite zeigt importierte Wallets/Holdings
* keine Live-API beim Laden
* keine echten Mengen/Werte im Chat ausgeben
* nur aggregiert berichten, ob Anzeige funktioniert

5. Crypto-Bestandsreport mit echten Daten
   Bitte einen echten Crypto-Bestandsreport lokal im Runtime-Reports-Ordner erzeugen.

Wichtig:

* Report darf nicht ins Git-Repo
* keine echten Mengen/Werte im Chat
* Git-Safety danach erneut prüfen
* Bitte nur berichten:

  * Report erzeugt: ja/nein
  * Format: PDF oder HTML-Fallback
  * Speicherort: Runtime-Reports-Ordner
  * Data-Quality-Status
  * Anzahl Coins im Report
  * Anzahl Wallets im Report
  * ob fehlende/ausgelassene Coins im Report nicht enthalten sind

6. Ergebnisbericht
   Bitte danach kompakt berichten:

* Konsistenzprüfung bestanden: ja/nein
* Anzahl Wallets
* Anzahl Assets
* Anzahl Holdings
* Anzahl Probleme/Warnings
* Dashboard liest echte Runtime-DB: ja/nein
* Crypto-Report erzeugt: ja/nein
* Report-Format
* Git-Safety Ergebnis
* Git Status
* offene Risiken
* ob aus deiner Sicht der nächste Schritt „manuelle Crypto-Ergänzungs-UI“ vertretbar ist

Bitte weiterhin keine echten Mengen, Werte oder Wallet-Adressen in die Antwort schreiben.


Explanation:
**Continuing with the task**

I need to keep going and figure out if I should load any relevant skills. It's probably mandatory to scan first and see what’s necessary before I reply. I might have to load the finance skill and check for report references. I should inspect everything to ensure accuracy. It’s crucial to use the tools available and avoid any errors. Also, I need to test the dashboard for any necessary information. Let's explore this step-by-step!


finance-system-architecture

---
name: finance-system-architecture
description: "Design/implement the user's JARVIS Finance System: CHF ledger, portfolio accounting, dashboard, imports, reports, secrecy."
version: 1.0.0
author: Hermes Agent
license: MIT
metadata:
  hermes:
    tags: [finance, portfolio, accounting, dashboard, csv, fx, tax, privacy]
---

# Finance System Architecture

## When to use

Use this skill whenever the user asks about the **Finance System – Architektur & Scripts** topic or work involving the user's personal finance-management system: portfolio imports, transaction accounting, Bank/Broker/CSV processing, dashboards, reports, tax/income views, FX logic, market data jobs, alerts, or later implementation.

Do **not** use it for pure investment-strategy discussion; those belong in the separate finance strategy topic unless the user explicitly asks for technical implementation.

## Core principles

1. **No automated trading.** The system may prepare orders and recommendations, but never executes orders or connects to broker trading APIs for execution.
2. **Accounting first, opinions later.** MVP must prioritize correct transaction ledger, cash ledger, FX handling, income/tax data, and dashboard before complex scores, news, analyst ratings, or LLM analysis.
3. **Rule-based first.** Calculations, scores, alerts, rebalancing and reports must be deterministic and auditable. LLMs may only summarize/explain text or assist extraction with human review.
4. **CHF as base currency.** Store every transaction in original currency and CHF using historical FX at transaction/payment date; current valuations use current FX.
5. **Financial data secrecy.** Never expose real financial values, portfolio files, raw exports, reports with real numbers, DBs, API keys, or credentials in chat, GitHub, logs, or shared docs unless the user explicitly approves scope.
6. **Data ownership.** All data must be exportable: CSV, JSON, Parquet, SQLite/Postgres dump, Markdown/HTML/PDF reports.

## Current v0.3 decisions

See `references/spec-v0.2-decisions.md` for the baseline, `references/spec-v0.3-decisions.md` for the priority update, `references/spec-v0.4-implementation-blueprint.md` for the MVP 1 technical implementation blueprint, `references/spec-v0.4.1-corrections.md` for accepted corrections, `references/spec-v0.5-repo-blueprint.md` for the concrete repo/implementation plan, `references/mvp-safety-gate-implementation.md` for the implementation safety-gate sequence and verification commands, `references/mvp-import-pipeline-tasks-9-13.md` for the synthetic CSV import/FX/transaction pipeline pattern, `references/mvp-ledger-cash-wac-positions.md` for cash/WAC/position ledger calculation rules and tests, `references/mvp-crypto-wallet-ledger.md` for crypto wallet/assets/holdings/transfers/buy-sell/fee MVP rules, `references/crypto-xlsx-initial-snapshot-dry-run.md` for the controlled real-file XLSX crypto initial-snapshot dry-run pattern and mapping decisions, `references/crypto-xlsx-initial-snapshot-production-import.md` for productive safe-subset XLSX import operations, compatibility/partial-write recovery, and dashboard correction follow-up, `references/mvp-market-import-safety-hardening.md` for CoinGecko provider/backoff, CSV importer completion, alert lifecycle, Git-safety and crypto Decimal precision patterns, `references/git-safety-verification-coverage.md` for the verifier-warning workflow and minimum Git-safety test matrix, `references/mvp-streamlit-dashboard.md` for the safe local Streamlit dashboard shell/read-model/testing pattern, and `references/mvp-crypto-report-export.md` for local-only crypto report context/export rules. Key decisions:

- Platform/account views: Raiffeisen, PostFinance, True Wealth, Crypto-Coins, plus Gesamtansicht.
- Existing Google Drive files under `Finanzen/01 Aktueller Stand Portfolios` are **initial-snapshot reality checks**, not full ledgers.
- MVP means the smallest useful first version: correct data foundation, ledger, CHF/FX logic, platform views, crypto inventory, watchlist, reports, workflows and audit log.
- Stock/ETF tax documents are downgraded: optional reference/control module and dividend/distribution transparency, not MVP 1 and not a complete tax dossier replacement.
- Crypto inventory is promoted to MVP 1: wallet/platform-level holdings, Crypto PDF, CoinGecko valuation, stale-price warnings and manual verification.
- Crypto snapshot values are stale; use quantities only and recalculate current values via CoinGecko later.
- MVP dashboard recommendation: Streamlit + SQLite + SQLAlchemy/Pandas/Plotly; later migrateable to FastAPI + React/PostgreSQL.
- MVP inputs: manual dashboard entry, standard CSV import/export, later broker/bank export import. No direct broker integrations initially.
- Telegram crypto buys/sells/transfers should be parsed as pending transactions and only saved after explicit confirmation.
- Every portfolio or crypto change must create a History/Audit entry.

## Required data model areas

At minimum design for:

- `platforms`, `accounts`, `instruments`
- `transactions` with original currency and CHF fields
- `cash_ledger`
- derived `positions_current`
- `dividends_distributions`
- `market_prices`, `fx_rates`
- `watchlist_items`
- `alerts`
- `decision_journal`
- `reports`
- `data_quality_issues`

## CSV/import rules

- Use a canonical CSV format before broker-specific importers.
- Imports must be idempotent via external transaction ID or row hash.
- Importers must separate dry-run and commit modes, collect validation errors explicitly, and always create an `import_sessions` record even for dry runs or failed imports.
- Import results should classify rows as new, existing, duplicate, or failed; do not rely on `INSERT OR IGNORE` as the only duplicate-detection mechanism.
- Row hashes must be stable and deterministic: normalize key order, whitespace, and `None` versus empty strings while leaving domain-specific canonicalization to the importer.
- If a source file has only current holdings and no full history, import it as `initial_position_snapshot` / `initial_cash_snapshot`, not fake historical buys.
- Missing historic FX, fees, taxes, dates, investment case, target weights, or dividend details must be flagged as data-quality gaps.

## FX and return rules

For foreign currency assets, always distinguish:

- price/security performance in original currency
- FX effect to CHF
- realized P&L
- unrealized P&L
- dividends/distributions/income
- fees and taxes
- Total Return in CHF

Every transaction/income event needs:

- original amount and currency
- historical FX rate to CHF, or explicit `fx_status=missing` with a critical data-quality alert
- FX source and timestamp
- CHF equivalent at event date when FX is known

Manual FX overrides require a non-empty note and an audit-log entry.

## Ledger calculation rules

- Cash after system start must be calculated from `transactions`; `cash_balances` is only a snapshot/cache/control view.
- `initial_cash_snapshot` and `initial_position_snapshot` are explicit starting truths, not fabricated historical trades.
- Implement Weighted Average Cost as the MVP internal performance method: buy fees increase cost basis, partial sells remove proportional cost basis, realized P&L is calculated against average cost, full sells zero remaining quantity/cost basis, and dividends do not alter quantity or cost basis.
- WAC is not tax-complete; do not add FIFO/LIFO/tax-lot logic in MVP 1, but keep transaction granularity so it can be added later.
- Position calculations must keep zero-quantity historical positions visible when sale history exists.
- Missing market price or FX must degrade data quality and suppress precise Total Return CHF rather than inventing a value.
- Before productive dashboard use, add alert deduplication per rule/entity/date.
- Market-FX for non-CHF market prices belongs to the later Market/FX provider block.

## Crypto MVP rules

- Crypto inventory is MVP-critical but must remain dummy-first until mapper/dry-run layers are ready; do not touch real coin/wallet files during core implementation. After explicit user approval, real files may be read only from temp/runtime for controlled dry runs using `references/crypto-xlsx-initial-snapshot-dry-run.md`; report only aggregate counts/labels and never quantities or values. Productive safe-subset imports require a second explicit approval and must follow `references/crypto-xlsx-initial-snapshot-production-import.md`: external runtime DB only, no real CSV/XLSX/reports/DBs in Git, Git-safety before/after, import-session + audit-log creation, and aggregate-only reporting.
- Wallet names must be unique; wallet addresses are optional and must not appear in tests/fixtures.
- Validate wallet type against: Hardware Wallet, Software Wallet, Exchange, Bank/Broker, DeFi, Sonstiges.
- CoinGecko ID is strongly recommended but not hard-required; missing ID creates a data-quality warning and blocks reliable valuation.
- Crypto symbols are not globally unique; symbol conflicts require warning/manual selection, never silent merging.
- Initial crypto holdings are confirmed snapshots by wallet+asset. Store legacy values for provenance/control only, never current valuation.
- Crypto holdings should be calculated/controlled from initial snapshots plus crypto transactions, not silently overwritten.
- Crypto transfers require different source/target wallets, quantity > 0, and same asset; coin fees reduce source-wallet balance.
- Negative wallet balances should be blocked in MVP and produce a critical alert unless a future explicit correction workflow is implemented.
- Crypto buys/sells with fiat impact must create/reference general ledger transactions via `source_type='crypto'`/`source_id` so cash remains ledger-consistent.
- Fiat fees should be ledger fees when unambiguous; avoid double counting when a fee is already embedded in sell net proceeds.
- Unknown fee currency creates a data-quality warning instead of invented accounting.
- Crypto Cost Basis/WAC is not required for MVP 1; MVP 1 prioritizes crypto holdings, wallet structure, current valuation, reports and audit.
- Crypto report context/export must be renderer-neutral and local-only: build from SQLite, aggregate by coin and wallet, use local cached prices only, preserve Decimal/Text values, ignore legacy snapshot values for current valuation, include data-quality warnings, and write generated files only to the external runtime reports directory.
- Crypto PDF export should be optional-dependency tolerant: prefer PDF when a renderer is available, otherwise fall back to HTML/Markdown with a clear warning while still recording report metadata and audit.
- Crypto tax logic, tax lots, FIFO/LIFO and detailed cost basis are later modules.
- Keep crypto assets separate from `instruments` for MVP 1, but later evaluate a shared Asset Master if it improves reporting, performance views or watchlists.

## Income/tax rules

For dividends and ETF distributions, separately store:

- gross dividend/distribution
- Swiss withholding tax 35%
- foreign withholding tax
- other tax/withholding if present
- net amount
- original currency
- historic FX to CHF
- gross/net CHF equivalents
- platform/account
- ISIN/ticker
- payment date
- ex-dividend date if available

Plan later ESTV/ICTax support for Swiss year-end values and tax reports, but label outputs as technical reports, not tax advice.

## Market-data rules

- Dashboard reads primarily from local DB, not directly from APIs on page load.
- First Streamlit MVP should be a safe, local, read-first shell over SQLite with explicit synthetic demo DB creation only; page render functions must not call CoinGecko/market APIs, and tests should prove that by monkeypatching provider methods to fail.
- Use scheduled jobs to update prices/FX/news.
- Store provider, timestamp, quality status and freshness for every price/FX point.
- Implement caching, local historical storage, batch requests where possible, provider rate-limit queues, and backoff on HTTP 429.
- CoinGecko integrations must be mockable and unit-tested without real API calls; treat HTTP 429 as non-fatal and return a non-fresh data-quality status plus alert after configured retries.
- Candidate providers: Finnhub, Financial Modeling Prep, Twelve Data, Stooq, Alpha Vantage selectively, Yahoo Finance fallback only; CoinGecko for crypto; later DefiLlama; ESTV/ICTax for Swiss tax values.

## Crypto Decimal precision rules

SQLite has no exact native DECIMAL. In the crypto context, store quantities, prices, crypto fees and valuation-relevant amounts as exact Decimal strings (`TEXT`) and compute with Python `Decimal`. Never convert crypto quantity/price/fee/valuation through float/REAL. Format Decimal writes with `format(value, "f")` so small quantities such as `0.00000001` do not become scientific notation. CSV importers must validate Decimal strings, and tests should assert high-precision preservation plus SQLite text storage.

## Alert lifecycle rules

Alerts must be deduplicated and lifecycle-aware before dashboards run repeated calculations. Use a dedup key from `rule_id`, `entity_type`, `entity_id`, `priority` and an error fingerprint. Alerts should carry `status` (`active`, `resolved`, `muted`), `resolved_at`, optional `muted_until`, `last_seen_at`, `occurrence_count`, `fingerprint` and `dedup_key`. Re-seeing an identical active/resolved/muted alert should update `last_seen_at`/`occurrence_count` rather than create a new active row; changed fingerprints may create a new alert.

## Git/GitHub safety

Code/scripts/docs may be stored in a private Git repo, but never commit financial data. Use the `github-repo-management` skill and its `references/sensitive-code-backup.md` guidance.

Minimum `.gitignore` for finance repos must block `data/`, `reports/`, `exports/`, `backups/`, DB files, spreadsheets, PDFs, DOCX, raw CSV/JSON, `.env`, tokens, credentials and client secrets. Only synthetic examples under `examples/` are acceptable.

## MVP implementation safety gate

Before any real broker/bank/portfolio/crypto data is imported, build and verify the safety foundation described in `references/mvp-safety-gate-implementation.md`: repo skeleton, `.gitignore`, `.env.example`, runtime outside repo, settings validation, SQLite schema, synthetic fixtures, audit log, CSV dry-run, duplicate protection, and Git-safety scan. Run the safety scan before staging, committing, or pushing. If GitHub credentials are needed from Drive, use them only in a temporary location and never echo token values.

## Common pitfalls

- Building a fancy dashboard before ledger correctness is verified.
- Treating a holdings snapshot as a real transaction history.
- Mixing price P&L and FX P&L.
- Letting dashboard refreshes hammer APIs.
- Committing realistic sample data to Git because the repo is private. Private GitHub is not a financial data vault.
- Showing real portfolio values back to the user unnecessarily in summaries.
- Designing Git-safety scans that block source package names like `src/.../imports` or `src/.../reports`; block top-level runtime/data directories instead.
- Running compile/tests before Git-safety and forgetting generated `__pycache__`; remove or ignore generated caches before safety scans. `pytest` may recreate `.pytest_cache`, so remove `__pycache__`/`.pytest_cache` again after test execution and before `git-safety-scan`.
- Productive real-data imports can partially write before surfacing schema/importer compatibility issues. Do not rerun blindly or dump row-level data; inspect aggregate counts and unique identifiers needed for idempotency, patch/complete via missing-only inserts, then verify committed import session, audit log, Git clean and Git-safety OK.
- Letting a dashboard correction UI mutate crypto holdings directly. Add/remove/edit operations must become audited transactions or `manual_adjustment` events with mandatory notes where appropriate; preserve history instead of silently overwriting balances.
- Treating verifier/file-mutation warnings as either automatically true or automatically false. Check the exact commit with `git show --name-status --patch <commit> -- <file>`, then separately verify whether the test matrix actually covers the promised cases.
- Adding broad Git-safety tests that only assert `len(findings) >= N`; prefer explicit assertions for token filename, PAT patterns, private keys, CSV/JSON path policy, runtime/data directories, allowed synthetic paths, `config.example/`, and normal `src/` false-positive protection.

To view linked files, call skill_view(name, file_path) where file_path is e.g. 'references/api.md' or 'assets/config.yaml'


# MVP Safety Gate Implementation Pattern

Use this when implementing the user's Finance System repo/MVP tasks. The key lesson from the first MVP bootstrap is that safety infrastructure must be implemented before any real financial import.

## Required sequence before real data

1. Create repo skeleton and package layout.
2. Add defensive `.gitignore` before adding data-like files.
3. Add `.env.example`; never create/commit a real `.env`.
4. Configure runtime outside the repo, defaulting to `~/jarvis_runtime/finance-system/`.
5. Add settings validation that rejects runtime paths inside the Git repo.
6. Add SQLite connection and schema migration foundation.
7. Add synthetic fixtures only.
8. Add CSV dry-run and row-hash duplicate detection.
9. Add audit-log basis for confirmed changes.
10. Add Git safety scan and run it before `git add`, before commit, and before push.

## Git safety scanner design notes

- Block top-level runtime/data dirs: `data/`, `imports/`, `exports/`, `reports/`, `backups/`, `secrets/`, `logs/`, `runtime/`, `local_runtime/`.
- Do **not** block source package directories named `imports` or `reports` under `src/`; match only top-level runtime/data directories.
- Block `.env`, SQLite/DB dumps, spreadsheets, PDFs/DOCX, and raw CSV/JSON except explicitly synthetic paths such as `tests/fixtures/`, `examples/synthetic/`, and config-example paths.
- Ignore `.git/` and `__pycache__/` during scans; compile/test steps can otherwise create false positives.
- If the scanner itself must contain token marker strings for tests, construct them from concatenated literals or whitelist only the scanner/test files. Never include actual token values.

## Verification commands

From repo root:

```bash
find . -type d -name __pycache__ -prune -exec rm -rf {} +
PYTHONPATH=src python -m compileall src tests
PYTHONPATH=src pytest tests -q
find . -type d -name __pycache__ -prune -exec rm -rf {} +
PYTHONPATH=src python -m jarvis_finance.cli.main git-safety-scan .
PYTHONPATH=src python -m jarvis_finance.cli.main init-db
```

Confirm the generated DB is under `~/jarvis_runtime/finance-system/data/`, not inside the repo.

## GitHub token handling

If the GitHub token is stored in Google Drive, download it only to a temporary directory and never print it. If API/repo access fails, report only the permission class/status, never the token. Prefer a fine-grained PAT scoped to only `Gamexgit/FinanceManager` with minimal `Contents: Read and write` permission. Recommend rotation/removal from Drive after setup.

## Reporting format after each task

For each completed task, report briefly:

- what changed
- tests/checks executed
- Git-safety status
- risks/open points

Keep reports technical and avoid echoing real financial data, token contents, or local DB contents.


# MVP Crypto Wallet Ledger Core (Tasks 17–21)

Use this reference when extending the FinanceManager crypto subsystem before any real coin/wallet file is mapped or imported.

## Safety boundary

- Use only synthetic fixtures until the crypto core, mapper, and dry-run flow are verified.
- Do not import or inspect real wallet/coin files during core implementation.
- Do not commit wallet addresses, exchange exports, portfolio CSVs, PDFs, DBs, runtime data, or credentials.
- Real crypto files are initial-snapshot/mapping inputs only after explicit user approval and a dry-run importer exists.

## Wallets

- Wallet name is unique.
- Wallet address is optional; tests/fixtures must not contain real addresses.
- Valid wallet types: `Hardware Wallet`, `Software Wallet`, `Exchange`, `Bank/Broker`, `DeFi`, `Sonstiges`.
- Wallet create/update/deactivate actions should be auditable.

## Crypto assets

- CoinGecko ID is strongly recommended but not hard-required.
- Missing CoinGecko ID creates a data-quality warning because valuation cannot be reliable.
- Symbol alone is not unique; symbol conflicts should create a warning/manual-selection requirement rather than silently merging assets.

## Initial holdings

- Initial crypto holdings are confirmed snapshots, not fabricated trade history.
- Calculate/control holdings by wallet + asset from initial snapshots plus crypto transactions; do not silently overwrite current holdings.
- Store legacy snapshot values for provenance/control only. Never use legacy values as current valuation.
- Track `verification_status`: `verified`, `stale`, `unverified`, `estimated`.
- Store/consider `last_verified_at`.

## Transfers

- Transfer requires source wallet, target wallet, same asset, and quantity > 0.
- Source and target wallet must differ.
- TxHash is optional.
- Coin fee is optional and reduces the source-wallet coin balance when paid in the transferred coin.
- Negative wallet balance should be blocked, or only allowed as an explicit confirmed correction with critical alert. MVP blocks and raises critical alert.
- Confirmed transfers create audit entries.

## Buy/sell with ledger coupling

- Crypto buy/sell with fiat/CHF impact must create or reference a general ledger transaction.
- Use `source_type='crypto'` and `source_id=<crypto_transaction_id>` to keep cash ledger consistency.
- Buy increases target wallet balance.
- Sell reduces source wallet balance.
- Foreign currency requires FX; if missing, set `fx_status='missing'` and `quality_status='incomplete'`, and create a critical data-quality alert.
- Do not automatically import real exchange data in this block.

## Fees

- Coin fee reduces coin balance.
- Fiat fee is represented in the general ledger as a fee transaction when the fee currency is clear.
- Store fee currency explicitly.
- Avoid double counting: if sell net amount already includes fee, do not create a second fee ledger row for the same economic fee.
- Unclear fee currency creates a data-quality warning rather than inventing treatment.
- Fee history must be traceable through crypto transaction, ledger transaction where applicable, and audit/log entries.

## Tests to preserve

At minimum cover:

- wallet create/read/update/deactivate and duplicate-name rejection
- asset with CoinGecko ID
- asset without CoinGecko ID warning
- symbol conflict warning/manual selection
- initial holdings per wallet and aggregate by asset
- legacy snapshot value not used as current valuation
- transfer, transfer with coin fee, same-wallet rejection, overdraft block/critical alert
- buy/sell create ledger links and update wallet balances
- coin fee reduces balance
- fiat fee creates general ledger fee when unambiguous
- confirmed crypto actions create audit entries
- missing FX marks ledger incomplete


# MVP Streamlit Dashboard Pattern

Use this reference for the first local dashboard block of the JARVIS Finance System.

## Scope

Build a safe, local, read-first Streamlit MVP over a local SQLite database with synthetic data only.

Required pages for the v0.5 shell:

- Command Center
- Portfolio
- Platforms
- Equities/ETFs
- Crypto
- Wallets
- Ledger
- Watchlist
- Reports
- Alerts
- Audit
- Settings/Data Quality

## Safety rules

- Never import real finance files, portfolio exports, wallet files, wallet addresses, secrets, runtime data, DBs, reports, or cache files into the repo.
- Do not call market-data providers from dashboard page render functions.
- CoinGecko/current crypto prices shown in the dashboard must come only from local DB/cache tables.
- If no DB exists, show a clear error and an explicit action/hint for creating a synthetic demo DB; do not automatically create or mutate real data.
- Runtime paths must remain outside the repo; keep a test that blocks in-repo runtime configuration.
- Alert lifecycle mutations (`acknowledge`, `resolve`, `mute`) should remain read-only in the first MVP unless explicit mutation code and tests exist.

## Implementation pattern

- Keep `dashboard/main.py` thin: load settings, validate runtime, connect to an existing SQLite DB, show navigation, dispatch to page modules.
- Put all SQL/derived read models in a separate dashboard data module so they can be unit-tested without Streamlit.
- Use simple page modules with a `PAGE_TITLE` and `render(st, conn)` signature.
- Use a tiny fake Streamlit object in tests for page smoke rendering; avoid requiring Streamlit to be installed for unit tests.
- Use Python `Decimal` for crypto quantities/prices and format via `format(value, "f")`; never format crypto amounts through float.
- Tables are enough for MVP. Prefer correctness, clear status/warnings, and data truth over polished UI.

## Demo data pattern

- Seed demo DB only by explicit user action or test helper.
- Use `tests/fixtures/` and domain APIs for synthetic records if importers are stricter than legacy fixtures.
- Store synthetic crypto prices in local DB/cache with provider metadata such as `synthetic-cache`.
- Add synthetic audit and alert records so Audit/Alerts pages can be exercised.

## Minimum tests

- Dashboard modules import and selected page modules smoke-render.
- Dashboard reader functions return expected synthetic data.
- Crypto page render does not call CoinGecko/API provider methods.
- Decimal crypto quantities preserve exact text formatting and avoid scientific notation.
- Alert lifecycle fields load readably (`status`, `priority`, `occurrence_count`, `last_seen_at`, `fingerprint`, `dedup_key`).
- Runtime path inside repo remains blocked.

## Verification sequence

Run from repo root before committing/pushing:

```bash
find . -type d \( -name __pycache__ -o -name .pytest_cache \) -exec rm -rf {} +
PYTHONPATH=src python -m compileall src tests
PYTHONPATH=src pytest tests -q
find . -type d \( -name __pycache__ -o -name .pytest_cache \) -exec rm -rf {} +
PYTHONPATH=src python -m jarvis_finance.cli.main git-safety-scan .
git status --short
```

Important: `pytest` may recreate `.pytest_cache`; remove it before Git-safety because the finance Git-safety scanner intentionally blocks runtime/cache files.


# MVP Crypto Report Context & Export

Session-derived pattern for Finance System Tasks 28–29: build a safe local crypto inventory report layer without touching real finance data.

## Scope

Use this pattern when implementing or reviewing crypto inventory report context/builders/exporters for the Finance System.

## Required architecture

- Source data only from the local SQLite database; never call CoinGecko or any live market API during report context building or page/report rendering.
- Aggregate holdings from the crypto holdings calculator / initial snapshots + crypto transactions, not from legacy snapshot values.
- Keep crypto quantities, prices, fees and valuations as `Decimal`/text end-to-end. Do not use `float`, SQLite `REAL`, or scientific notation for crypto values.
- Treat CHF as the primary base currency. Prepare metadata for later USD/EUR views, but do not dilute the MVP with multi-currency valuation unless local prices exist.
- Export files only to the configured external runtime reports directory. Validate settings before export so repo-contained runtime/report paths are rejected.
- Never commit generated reports, PDFs, DBs or real portfolio files. Git-safety should block top-level `reports/` and `.pdf` files in the repo.

## Context shape

`build_crypto_report_context(conn, ...)` should return a renderer-neutral dict containing:

- Report metadata: id, type, generated_at, base currency, disclaimer, price_as_of, wallet_verification_as_of.
- Executive summary: total crypto CHF value, coin count, wallet/platform count, data quality status.
- Coin rows: coin name, symbol, coingecko_id, total quantity, local CHF price, total CHF value, portfolio share, price source, price timestamp, warning codes.
- Wallet rows: wallet/platform name, wallet type, provider, coin count, CHF value, last verification, verification status, warning codes.
- Detail rows by coin/wallet: wallet, coin, symbol, quantity, price, value CHF, last verification, notes/status.
- Data-quality warnings with stable codes and entity references.

## Data-quality warnings to include

- `missing_coingecko_id`: asset has no CoinGecko ID, so reliable scheduled valuation is blocked.
- `missing_price_chf`: no local CHF price exists; valuation for that coin/wallet/detail row is unknown.
- `stale_price_chf`: local price timestamp is older than the configured freshness threshold.
- `wallet_not_verified`: wallet verification status or last verification is missing/not verified.
- `negative_holding`: calculated holding quantity is negative; do not hide this.
- `legacy_snapshot_value_ignored`: legacy snapshot CHF/currency values exist but are deliberately ignored for current valuation.

## Export pattern

- Provide Markdown and HTML renderers from the same context.
- Prefer PDF only through an optional renderer dependency (e.g. WeasyPrint). If the renderer is unavailable or fails, fall back gracefully to HTML/Markdown and expose a clear warning in the result.
- Save a row in `reports` with `report_type='crypto_inventory'`, `generated_at`, `file_path`, `format`, `data_quality_status`, `summary_json`.
- Create an audit-log event with `action='report_generated'`, `entity_type='report'`, and the report id.

## Minimum tests

- Synthetic DB context builds without live APIs.
- Total quantity per coin is exact.
- Wallet aggregation is correct.
- Decimal strings remain exact; no float/scientific notation.
- Legacy snapshot values do not affect current valuation.
- Missing price and missing CoinGecko ID create warnings.
- Report file is written under external runtime reports dir, not repo.
- Report metadata row is persisted.
- `report_generated` audit event is created.
- PDF renderer can be mocked; fallback works when unavailable.
- Git-safety blocks PDFs/reports inside the repo.

## Pitfalls

- Do not reuse stale legacy snapshot values as current CHF values merely because they are present in the DB.
- Do not silently merge by symbol only; symbols are not globally unique.
- Do not make report generation mutate holdings/prices.
- Do not put generated demo reports in `tests/fixtures` unless they are tiny, synthetic, and explicitly required; prefer temporary runtime dirs in tests.


# Crypto XLSX Initial Snapshot: Production Import Notes

Use after a controlled dry run of a real crypto holdings XLSX has been accepted and the user explicitly approves productive writing.

## Non-negotiables

- Read the real XLSX only from a temp/runtime path; remove temp files after completion.
- Never write real XLSX/CSV/report/SQLite data into the Git repo.
- Run Git status and `git-safety-scan` before and after the import.
- Use the productive SQLite DB only under the configured external runtime path (for this project normally `~/jarvis_runtime/finance-system/data/finance.sqlite3`).
- Report only aggregate counts and labels; never echo real quantities, CHF values, row-level holdings, raw CSV content, or DB contents.
- Ignore legacy CHF/value columns from the XLSX for current valuation. Current valuation belongs to CoinGecko/market-data refreshes, not the import.
- Import only the user-approved safe subset. Unsure mappings must be skipped/needs_manual_review, not blocking and not silently guessed.

## Workflow

1. Preflight: remove generated caches, check `git status --short`, run `PYTHONPATH=src python -m jarvis_finance.cli.main git-safety-scan .`, confirm runtime/DB paths are outside repo, and scan that no real data files are in repo.
2. Prepare temp/runtime input: download/read the XLSX to a temp directory; normalize wallet names; apply only confirmed mappings; set the explicit snapshot date; validate quantities as Decimal text with `format(Decimal(...), 'f')`.
3. Generate intermediate `crypto_wallets.csv` and `crypto_holdings_initial.csv` only in temp/runtime. Do not persist them in the repo or print them.
4. Productive DB import: create wallets if missing, create crypto assets if missing, create initial holding snapshots, write import-session records, and rely on audit-log calls for every created wallet/asset/holding.
5. Postcheck: aggregate holdings/assets/wallets, latest import-session status, audit-log count, alerts count, dashboard-readability, Git status, and Git-safety. Report only aggregate counts.

## Compatibility and partial-write pitfall

A productive import can fail after some writes if older runtime DBs lack newly added schema columns or if importer assumptions conflict with existing rows. Treat this as an operational recovery, not as permission to expose data:

- Before running current migrations against an older DB, inspect compatibility columns for lifecycle tables such as `alerts`. If a migration creates indexes on columns added in newer schema text, add the missing columns first or run the project's explicit migration path.
- If a failure occurs after partial writes, do **not** restart blindly. Query only aggregate counts and identifiers needed for idempotency, then continue by skipping existing wallet/asset/holding rows and inserting only missing rows.
- Record a final committed import session for the holdings completion, and note in the final report that a controlled completion occurred without exposing values.
- Verify the final invariant: safe-subset asset count, safe-subset holdings count, zero failed rows, import-session committed, audit log created, Git clean, Git-safety OK.

## Dashboard follow-up pattern

Manual crypto corrections should be built as auditable events, not silent edits:

- Add coin: CoinGecko name/symbol search, manual selection on ambiguity, wallet selection, Decimal quantity, snapshot/transaction date, optional note, explicit confirmation, audit log.
- Remove coin: transaction/correction only, mandatory reason/note, audit log, historical data preserved.
- Edit/correct holding: write a `manual_adjustment` event with mandatory note; store old/new values in audit metadata instead of overwriting silently.
- CoinGecko search: symbol alone is never enough for auto-selection; mark ambiguous choices for manual review.


     1|from __future__ import annotations
     2|
     3|import html
     4|import json
     5|import uuid
     6|from datetime import datetime, timezone
     7|from decimal import Decimal, ROUND_HALF_UP
     8|from pathlib import Path
     9|from sqlite3 import Connection
    10|from typing import Any, Callable
    11|
    12|from jarvis_finance.audit.log import record_audit_event
    13|from jarvis_finance.config.settings import Settings, ensure_runtime_dirs, validate_settings
    14|from jarvis_finance.crypto.holdings import calculate_crypto_holdings
    15|from jarvis_finance.imports.common import utc_now
    16|
    17|PDFRenderer = Callable[[str, Path], bool]
    18|
    19|
    20|def _d(value: object, default: str = "0") -> Decimal:
    21|    if value is None or str(value).strip() == "":
    22|        return Decimal(default)
    23|    return Decimal(str(value))
    24|
    25|
    26|def _dec_text(value: Decimal | None, places: str | None = None) -> str | None:
    27|    if value is None:
    28|        return None
    29|    if places is not None:
    30|        value = value.quantize(Decimal(places), rounding=ROUND_HALF_UP)
    31|    return format(value, "f")
    32|
    33|
    34|def _latest_crypto_price(conn: Connection, asset_id: str, currency: str) -> Any | None:
    35|    return conn.execute(
    36|        """
    37|        SELECT * FROM crypto_prices
    38|        WHERE asset_id=? AND price_currency=?
    39|        ORDER BY COALESCE(provider_timestamp, fetched_at, '') DESC, fetched_at DESC
    40|        LIMIT 1
    41|        """,
    42|        (asset_id, currency),
    43|    ).fetchone()
    44|
    45|
    46|def _parse_dt(value: str | None) -> datetime | None:
    47|    if not value:
    48|        return None
    49|    try:
    50|        return datetime.fromisoformat(value.replace("Z", "+00:00"))
    51|    except ValueError:
    52|        return None
    53|
    54|
    55|def _is_stale(value: str | None, *, max_age_days: int = 3) -> bool:
    56|    dt = _parse_dt(value)
    57|    if dt is None:
    58|        return False
    59|    if dt.tzinfo is None:
    60|        dt = dt.replace(tzinfo=timezone.utc)
    61|    return datetime.now(timezone.utc) - dt > __import__("datetime").timedelta(days=max_age_days)
    62|
    63|
    64|def _add_warning(warnings: list[dict[str, str]], *, code: str, message: str, entity_type: str, entity_id: str | None = None) -> None:
    65|    item = {"code": code, "message": message, "entity_type": entity_type}
    66|    if entity_id:
    67|        item["entity_id"] = entity_id
    68|    if item not in warnings:
    69|        warnings.append(item)
    70|
    71|
    72|def build_crypto_report_context(conn: Connection, *, base_currency: str = "CHF") -> dict[str, Any]:
    73|    """Build a renderer-neutral crypto inventory report context from local SQLite data only."""
    74|    holdings = calculate_crypto_holdings(conn)
    75|    warnings: list[dict[str, str]] = []
    76|    generated_at = utc_now()
    77|
    78|    wallet_rows = {row["wallet_id"]: row for row in conn.execute("SELECT * FROM crypto_wallets").fetchall()}
    79|    asset_rows = {row["asset_id"]: row for row in conn.execute("SELECT * FROM crypto_assets").fetchall()}
    80|
    81|    coin_rows: list[dict[str, Any]] = []
    82|    total_value = Decimal("0")
    83|    valued_asset_ids: set[str] = set()
    84|    latest_price_timestamp: str | None = None
    85|    latest_wallet_verification: str | None = None
    86|
    87|    for asset_id, total_holding in sorted(holdings.total_by_asset.items()):
    88|        asset = asset_rows.get(asset_id)
    89|        if asset is None:
    90|            continue
    91|        quantity = total_holding.quantity
    92|        if quantity < 0:
    93|            _add_warning(warnings, code="negative_holding", message="Negativer Crypto-Bestand erkannt.", entity_type="crypto_asset", entity_id=asset_id)
    94|        if not asset["coingecko_id"]:
    95|            _add_warning(warnings, code="missing_coingecko_id", message="CoinGecko-ID fehlt; Bewertung ist eingeschränkt.", entity_type="crypto_asset", entity_id=asset_id)
    96|        price = _latest_crypto_price(conn, asset_id, base_currency)
    97|        price_dec: Decimal | None = None
    98|        price_timestamp: str | None = None
    99|        provider: str | None = None
   100|        if price is None or price["price"] is None:
   101|            _add_warning(warnings, code="missing_price", message="Lokaler Crypto-Preis fehlt.", entity_type="crypto_asset", entity_id=asset_id)
   102|        else:
   103|            price_dec = _d(price["price"])
   104|            price_timestamp = price["provider_timestamp"] or price["fetched_at"]
   105|            provider = price["provider"]
   106|            if _is_stale(price_timestamp):
   107|                _add_warning(warnings, code="stale_price", message="Lokaler Crypto-Preis ist veraltet.", entity_type="crypto_asset", entity_id=asset_id)
   108|            latest_price_timestamp = max(filter(None, [latest_price_timestamp, price_timestamp])) if latest_price_timestamp else price_timestamp
   109|        value = quantity * price_dec if price_dec is not None else None
   110|        if value is not None:
   111|            total_value += value
   112|            valued_asset_ids.add(asset_id)
   113|
   114|        has_legacy = any(
   115|            h.asset_id == asset_id and (h.legacy_snapshot_value_chf is not None or h.legacy_snapshot_value_original is not None)
   116|            for h in holdings.wallet_holdings.values()
   117|        )
   118|        if has_legacy:
   119|            _add_warning(warnings, code="legacy_snapshot_value_ignored", message="Legacy Snapshot Value vorhanden, aber für aktuelle Bewertung ignoriert.", entity_type="crypto_asset", entity_id=asset_id)
   120|
   121|        coin_rows.append(
   122|            {
   123|                "asset_id": asset_id,
   124|                "coin_name": asset["coin_name"],
   125|                "symbol": asset["symbol"],
   126|                "coingecko_id": asset["coingecko_id"],
   127|                "total_quantity": _dec_text(quantity),
   128|                "price_chf": _dec_text(price_dec),
   129|                "value_chf": _dec_text(value, "0.0001"),
   130|                "portfolio_share_pct": None,
   131|                "price_source": provider,
   132|                "price_timestamp": price_timestamp,
   133|                "warning": ", ".join(w["code"] for w in warnings if w.get("entity_id") == asset_id) or None,
   134|            }
   135|        )
   136|
   137|    for row in coin_rows:
   138|        if row["asset_id"] in valued_asset_ids and total_value != 0:
   139|            share = (_d(row["value_chf"]) / total_value * Decimal("100"))
   140|            row["portfolio_share_pct"] = _dec_text(share, "0.01")
   141|
   142|    wallet_agg: dict[str, dict[str, Any]] = {}
   143|    detail_rows: list[dict[str, Any]] = []
   144|    for (wallet_id, asset_id), holding in sorted(holdings.wallet_holdings.items()):
   145|        wallet = wallet_rows.get(wallet_id)
   146|        asset = asset_rows.get(asset_id)
   147|        if wallet is None or asset is None:
   148|            continue
   149|        if wallet["last_verified_at"]:
   150|            latest_wallet_verification = max(filter(None, [latest_wallet_verification, wallet["last_verified_at"]])) if latest_wallet_verification else wallet["last_verified_at"]
   151|        if wallet["last_verified_at"] is None or holding.verification_status != "verified":
   152|            _add_warning(warnings, code="unverified_wallet", message="Wallet ist nicht verifiziert oder Holding-Status ist nicht verified.", entity_type="crypto_wallet", entity_id=wallet_id)
   153|        if holding.quantity < 0:
   154|            _add_warning(warnings, code="negative_holding", message="Negativer Wallet-Bestand erkannt.", entity_type="crypto_wallet", entity_id=wallet_id)
   155|        price = _latest_crypto_price(conn, asset_id, base_currency)
   156|        price_dec = _d(price["price"]) if price and price["price"] is not None else None
   157|        value = holding.quantity * price_dec if price_dec is not None else None
   158|        agg = wallet_agg.setdefault(
   159|            wallet_id,
   160|            {
   161|                "wallet_id": wallet_id,
   162|                "wallet_name": wallet["wallet_name"],
   163|                "wallet_type": wallet["wallet_type"],
   164|                "provider": wallet["platform_provider"],
   165|                "coin_ids": set(),
   166|                "total_value_chf_dec": Decimal("0"),
   167|                "last_verified_at": wallet["last_verified_at"],
   168|                "is_active": bool(wallet["is_active"]),
   169|            },
   170|        )
   171|        agg["coin_ids"].add(asset_id)
   172|        if value is not None:
   173|            agg["total_value_chf_dec"] += value
   174|        detail_rows.append(
   175|            {
   176|                "wallet_id": wallet_id,
   177|                "wallet_name": wallet["wallet_name"],
   178|                "asset_id": asset_id,
   179|                "coin": asset["coin_name"],
   180|                "symbol": asset["symbol"],
   181|                "quantity": _dec_text(holding.quantity),
   182|                "price_chf": _dec_text(price_dec),
   183|                "value_chf": _dec_text(value, "0.0001"),
   184|                "last_verified_at": wallet["last_verified_at"] or holding.last_verified_at,
   185|                "verification_status": holding.verification_status,
   186|                "notes": None,
   187|            }
   188|        )
   189|
   190|    wallet_summary = []
   191|    for row in wallet_agg.values():
   192|        wallet_summary.append(
   193|            {
   194|                "wallet_id": row["wallet_id"],
   195|                "wallet_name": row["wallet_name"],
   196|                "wallet_type": row["wallet_type"],
   197|                "provider": row["provider"],
   198|                "coin_count": len(row["coin_ids"]),
   199|                "total_value_chf": _dec_text(row["total_value_chf_dec"]),
   200|                "last_verified_at": row["last_verified_at"],
   201|                "is_active": row["is_active"],
   202|            }
   203|        )
   204|
   205|    status = "ok"
   206|    if any(w["code"] in {"negative_holding", "missing_price", "missing_coingecko_id"} for w in warnings):
   207|        status = "warning"
   208|    if any(w["code"] == "negative_holding" for w in warnings):
   209|        status = "critical"
   210|
   211|    return {
   212|        "title": "JARVIS Finance System – Crypto-Bestandsübersicht",
   213|        "generated_at": generated_at,
   214|        "base_currency": base_currency,
   215|        "price_data_as_of": latest_price_timestamp,
   216|        "wallet_verification_as_of": latest_wallet_verification,
   217|        "disclaimer": "Bestandsübersicht, keine vollständige Steuererklärung.",
   218|        "summary": {
   219|            "total_value_chf": _dec_text(total_value, "0.0001"),
   220|            "coin_count": len(coin_rows),
   221|            "wallet_count": len(wallet_summary),
   222|            "data_quality_status": status,
   223|        },
   224|        "coins": coin_rows,
   225|        "wallets": sorted(wallet_summary, key=lambda r: r["wallet_name"]),
   226|        "coin_wallet_details": detail_rows,
   227|        "data_quality_warnings": warnings,
   228|        "metadata": {
   229|            "report_type": "crypto_inventory",
   230|            "source": "local_sqlite",
   231|            "live_api_calls": False,
   232|            "currencies_prepared": ["CHF", "USD", "EUR"],
   233|        },
   234|    }
   235|
   236|
   237|def render_crypto_report_markdown(context: dict[str, Any]) -> str:
   238|    lines = [
   239|        f"# {context['title']}",
   240|        "",
   241|        f"Erstellt: {context['generated_at']}",
   242|        f"Basiswährung: {context['base_currency']}",
   243|        f"Datenstand Preise: {context.get('price_data_as_of') or 'nicht verfügbar'}",
   244|        f"Datenstand Wallet-Verifikation: {context.get('wallet_verification_as_of') or 'nicht verfügbar'}",
   245|        "",
   246|        f"Disclaimer: {context['disclaimer']}",
   247|        "",
   248|        "## Executive Summary",
   249|        f"- Gesamtwert Crypto CHF: {context['summary']['total_value_chf']} CHF",
   250|        f"- Anzahl Coins: {context['summary']['coin_count']}",
   251|        f"- Anzahl Wallets/Plattformen: {context['summary']['wallet_count']}",
   252|        f"- Datenqualitätsstatus: {context['summary']['data_quality_status']}",
   253|        "",
   254|        "## Coin-Übersicht",
   255|    ]
   256|    for row in context["coins"]:
   257|        lines.append(f"- {row['coin_name']} ({row['symbol']}): {row['total_quantity']} @ {row['price_chf'] or 'n/a'} CHF = {row['value_chf'] or 'n/a'} CHF; Anteil {row['portfolio_share_pct'] or 'n/a'}%; Quelle {row['price_source'] or 'n/a'}; Preiszeit {row['price_timestamp'] or 'n/a'}; Warnung {row['warning'] or '-'}")
   258|    lines.extend(["", "## Wallet-Übersicht"])
   259|    for row in context["wallets"]:
   260|        lines.append(f"- {row['wallet_name']} ({row['wallet_type']}, {row['provider'] or 'n/a'}): {row['coin_count']} Coins, {row['total_value_chf']} CHF, letzte Verifikation {row['last_verified_at'] or 'n/a'}")
   261|

Use offset=261 to continue reading (showing 1-260 of 358 lines)


     1|from __future__ import annotations
     2|
     3|from jarvis_finance.reports.crypto_inventory import (
     4|    build_crypto_report_context,
     5|    export_crypto_inventory_report,
     6|    render_crypto_report_html,
     7|    render_crypto_report_markdown,
     8|)
     9|
    10|__all__ = [
    11|    "build_crypto_report_context",
    12|    "export_crypto_inventory_report",
    13|    "render_crypto_report_html",
    14|    "render_crypto_report_markdown",
    15|]
    16|


     1|from __future__ import annotations
     2|
     3|import json
     4|from decimal import Decimal, InvalidOperation, ROUND_HALF_UP
     5|from pathlib import Path
     6|from sqlite3 import Connection
     7|
     8|from jarvis_finance.config.settings import Settings
     9|from jarvis_finance.crypto.holdings import calculate_crypto_holdings
    10|from jarvis_finance.ledger.cash import calculate_cash_balances
    11|from jarvis_finance.ledger.positions import calculate_positions
    12|
    13|MONEY = Decimal("0.01")
    14|
    15|
    16|def d(value: object, default: str = "0") -> Decimal:
    17|    if value is None or str(value).strip() == "":
    18|        return Decimal(default)
    19|    try:
    20|        return Decimal(str(value))
    21|    except InvalidOperation:
    22|        return Decimal(default)
    23|
    24|
    25|def dec_text(value: Decimal | int | str | None, places: str | None = None) -> str:
    26|    if value is None:
    27|        return ""
    28|    value = d(value)
    29|    if places:
    30|        value = value.quantize(Decimal(places), rounding=ROUND_HALF_UP)
    31|    return format(value, "f")
    32|
    33|
    34|def money_text(value: Decimal | None, currency: str = "CHF") -> str:
    35|    if value is None:
    36|        return f"n/a {currency}"
    37|    return f"{dec_text(value, '0.01')} {currency}"
    38|
    39|
    40|def rowdicts(rows) -> list[dict[str, str]]:
    41|    return [{k: ("" if row[k] is None else str(row[k])) for k in row.keys()} for row in rows]
    42|
    43|
    44|def db_exists(path: str | Path) -> bool:
    45|    return Path(path).expanduser().exists()
    46|
    47|
    48|def runtime_outside_repo(settings: Settings, repo_root: Path) -> bool:
    49|    try:
    50|        settings.runtime_paths.base_dir.resolve().relative_to(repo_root.resolve())
    51|        return False
    52|    except ValueError:
    53|        return True
    54|
    55|
    56|def latest_crypto_price(conn: Connection, asset_id: str, currency: str = "CHF") -> dict[str, str] | None:
    57|    row = conn.execute(
    58|        """
    59|        SELECT price, price_currency, fetched_at, quality_status, provider
    60|        FROM crypto_prices
    61|        WHERE asset_id=? AND price_currency=?
    62|        ORDER BY fetched_at DESC LIMIT 1
    63|        """,
    64|        (asset_id, currency.upper()),
    65|    ).fetchone()
    66|    return dict(row) if row else None
    67|
    68|
    69|def get_crypto_overview(conn: Connection) -> list[dict[str, str]]:
    70|    holdings = calculate_crypto_holdings(conn)
    71|    rows: list[dict[str, str]] = []
    72|    for asset_id, holding in sorted(holdings.total_by_asset.items()):
    73|        qty = holding.quantity
    74|        asset = conn.execute("SELECT * FROM crypto_assets WHERE asset_id=?", (asset_id,)).fetchone()
    75|        if not asset:
    76|            continue
    77|        price = latest_crypto_price(conn, asset_id, "CHF")
    78|        price_dec = d(price["price"]) if price and price["price"] else None
    79|        value = qty * price_dec if price_dec is not None else None
    80|        rows.append({
    81|            "asset_id": asset_id,
    82|            "coin_name": asset["coin_name"],
    83|            "symbol": asset["symbol"],
    84|            "coingecko_id": asset["coingecko_id"] or "",
    85|            "quantity": dec_text(qty),
    86|            "latest_price_chf": dec_text(price_dec, "0.01") if price_dec is not None else "",
    87|            "market_value_chf": dec_text(value, "0.01") if value is not None else "",
    88|            "price_quality": price["quality_status"] if price else "missing",
    89|            "fetched_at": price["fetched_at"] if price else "",
    90|        })
    91|    return rows
    92|
    93|
    94|def get_crypto_by_wallet(conn: Connection) -> list[dict[str, str]]:
    95|    holdings = calculate_crypto_holdings(conn)
    96|    rows: list[dict[str, str]] = []
    97|    for (wallet_id, asset_id), holding in sorted(holdings.wallet_holdings.items()):
    98|        wallet = conn.execute("SELECT wallet_name, wallet_type, platform_provider FROM crypto_wallets WHERE wallet_id=?", (wallet_id,)).fetchone()
    99|        asset = conn.execute("SELECT coin_name, symbol, coingecko_id FROM crypto_assets WHERE asset_id=?", (asset_id,)).fetchone()
   100|        price = latest_crypto_price(conn, asset_id, "CHF")
   101|        price_dec = d(price["price"]) if price and price["price"] else None
   102|        value = holding.quantity * price_dec if price_dec is not None else None
   103|        rows.append({
   104|            "wallet_id": wallet_id,
   105|            "wallet_name": wallet["wallet_name"] if wallet else wallet_id,
   106|            "wallet_type": wallet["wallet_type"] if wallet else "",
   107|            "provider": wallet["platform_provider"] if wallet else "",
   108|            "coin_name": asset["coin_name"] if asset else asset_id,
   109|            "symbol": asset["symbol"] if asset else "",
   110|            "coingecko_id": asset["coingecko_id"] if asset else "",
   111|            "quantity": dec_text(holding.quantity),
   112|            "market_value_chf": dec_text(value, "0.01") if value is not None else "",
   113|            "verification_status": holding.verification_status,
   114|            "last_verified_at": holding.last_verified_at or "",
   115|        })
   116|    return rows
   117|
   118|
   119|def get_wallets(conn: Connection) -> list[dict[str, str]]:
   120|    return rowdicts(conn.execute(
   121|        """
   122|        SELECT wallet_id, wallet_name, wallet_type, platform_provider AS provider,
   123|               network_chain, last_verified_at, is_active, notes
   124|        FROM crypto_wallets ORDER BY wallet_name
   125|        """
   126|    ).fetchall())
   127|
   128|
   129|def get_latest_crypto_transactions(conn: Connection, limit: int = 20) -> list[dict[str, str]]:
   130|    return rowdicts(conn.execute(
   131|        """
   132|        SELECT ct.transaction_datetime, ct.transaction_type, ca.symbol, ct.quantity, ct.fee_quantity,
   133|               fw.wallet_name AS from_wallet, tw.wallet_name AS to_wallet, ct.currency_original,
   134|               ct.amount_chf, ct.confirmation_status, ct.source
   135|        FROM crypto_transactions ct
   136|        LEFT JOIN crypto_assets ca ON ca.asset_id=ct.asset_id
   137|        LEFT JOIN crypto_wallets fw ON fw.wallet_id=ct.from_wallet_id
   138|        LEFT JOIN crypto_wallets tw ON tw.wallet_id=ct.to_wallet_id
   139|        ORDER BY ct.transaction_datetime DESC, ct.created_at DESC LIMIT ?
   140|        """, (limit,)
   141|    ).fetchall())
   142|
   143|
   144|def get_platform_overview(conn: Connection) -> list[dict[str, str]]:
   145|    return rowdicts(conn.execute(
   146|        """
   147|        SELECT p.name AS platform, p.platform_type, a.account_name, a.account_type, a.currency,
   148|               a.performance_included, a.is_active
   149|        FROM accounts a JOIN platforms p ON p.platform_id=a.platform_id
   150|        ORDER BY p.name, a.account_name
   151|        """
   152|    ).fetchall())
   153|
   154|
   155|def get_cash_overview(conn: Connection) -> list[dict[str, str]]:
   156|    result = calculate_cash_balances(conn)
   157|    rows: list[dict[str, str]] = []
   158|    for (account_id, currency), bal in sorted(result.balances.items()):
   159|        row = conn.execute(
   160|            "SELECT a.account_name, p.name AS platform FROM accounts a JOIN platforms p ON p.platform_id=a.platform_id WHERE a.account_id=?",
   161|            (account_id,),
   162|        ).fetchone()
   163|        rows.append({
   164|            "platform": row["platform"] if row else "",
   165|            "account_name": row["account_name"] if row else account_id,
   166|            "currency": currency,
   167|            "amount_original": dec_text(bal.amount_original, "0.01"),
   168|            "amount_chf": dec_text(bal.amount_chf, "0.01") if bal.amount_chf is not None else "",
   169|            "quality_status": bal.quality_status,
   170|        })
   171|    return rows
   172|
   173|
   174|def get_positions(conn: Connection) -> list[dict[str, str]]:
   175|    result = calculate_positions(conn)
   176|    rows: list[dict[str, str]] = []
   177|    for (account_id, instrument_id), pos in sorted(result.positions.items()):
   178|        row = conn.execute(
   179|            """
   180|            SELECT a.account_name, p.name AS platform, i.name, i.ticker, i.isin, i.asset_class, i.currency
   181|            FROM accounts a JOIN platforms p ON p.platform_id=a.platform_id
   182|            JOIN instruments i ON i.instrument_id=? WHERE a.account_id=?
   183|            """,
   184|            (instrument_id, account_id),
   185|        ).fetchone()
   186|        rows.append({
   187|            "platform": row["platform"] if row else "",
   188|            "account_name": row["account_name"] if row else account_id,
   189|            "asset_class": row["asset_class"] if row else "",
   190|            "name": row["name"] if row else instrument_id,
   191|            "ticker": row["ticker"] if row else "",
   192|            "isin": row["isin"] if row else "",
   193|            "quantity": dec_text(pos.quantity),
   194|            "cost_basis_chf": dec_text(pos.cost_basis_chf, "0.01"),
   195|            "market_value_chf": dec_text(pos.market_value_chf, "0.01") if pos.market_value_chf is not None else "",
   196|            "total_return_chf": dec_text(pos.total_return_chf, "0.01") if pos.total_return_chf is not None else "",
   197|            "data_quality_status": pos.data_quality_status,
   198|        })
   199|    return rows
   200|
   201|
   202|def get_ledger_transactions(conn: Connection, limit: int = 200) -> list[dict[str, str]]:
   203|    return rowdicts(conn.execute(
   204|        """
   205|        SELECT t.trade_date, p.name AS platform, a.account_name, t.transaction_type,
   206|               i.name AS instrument, i.ticker, t.quantity, t.gross_amount_original,
   207|               t.net_amount_original, t.currency_original, t.fx_status, t.quality_status,
   208|               t.source_type, t.source_id
   209|        FROM transactions t
   210|        LEFT JOIN accounts a ON a.account_id=t.account_id
   211|        LEFT JOIN platforms p ON p.platform_id=a.platform_id
   212|        LEFT JOIN instruments i ON i.instrument_id=t.instrument_id
   213|        ORDER BY t.trade_date DESC, t.created_at DESC LIMIT ?
   214|        """, (limit,)
   215|    ).fetchall())
   216|
   217|
   218|def get_watchlist(conn: Connection) -> list[dict[str, str]]:
   219|    return rowdicts(conn.execute("SELECT name, asset_class, reason, status, target_entry_price, target_entry_currency, investment_case, bear_case, next_review_date FROM watchlist ORDER BY name").fetchall())
   220|
   221|

Use offset=221 to continue reading (showing 1-220 of 281 lines)


237|def render_crypto_report_markdown(context: dict[str, Any]) -> str:
   238|    lines = [
   239|        f"# {context['title']}",
   240|        "",
   241|        f"Erstellt: {context['generated_at']}",
   242|        f"Basiswährung: {context['base_currency']}",
   243|        f"Datenstand Preise: {context.get('price_data_as_of') or 'nicht verfügbar'}",
   244|        f"Datenstand Wallet-Verifikation: {context.get('wallet_verification_as_of') or 'nicht verfügbar'}",
   245|        "",
   246|        f"Disclaimer: {context['disclaimer']}",
   247|        "",
   248|        "## Executive Summary",
   249|        f"- Gesamtwert Crypto CHF: {context['summary']['total_value_chf']} CHF",
   250|        f"- Anzahl Coins: {context['summary']['coin_count']}",
   251|        f"- Anzahl Wallets/Plattformen: {context['summary']['wallet_count']}",
   252|        f"- Datenqualitätsstatus: {context['summary']['data_quality_status']}",
   253|        "",
   254|        "## Coin-Übersicht",
   255|    ]
   256|    for row in context["coins"]:
   257|        lines.append(f"- {row['coin_name']} ({row['symbol']}): {row['total_quantity']} @ {row['price_chf'] or 'n/a'} CHF = {row['value_chf'] or 'n/a'} CHF; Anteil {row['portfolio_share_pct'] or 'n/a'}%; Quelle {row['price_source'] or 'n/a'}; Preiszeit {row['price_timestamp'] or 'n/a'}; Warnung {row['warning'] or '-'}")
   258|    lines.extend(["", "## Wallet-Übersicht"])
   259|    for row in context["wallets"]:
   260|        lines.append(f"- {row['wallet_name']} ({row['wallet_type']}, {row['provider'] or 'n/a'}): {row['coin_count']} Coins, {row['total_value_chf']} CHF, letzte Verifikation {row['last_verified_at'] or 'n/a'}")
   261|    lines.extend(["", "## Detail pro Coin nach Wallet"])
   262|    for row in context["coin_wallet_details"]:
   263|        lines.append(f"- {row['wallet_name']} / {row['symbol']}: Menge {row['quantity']}, Kurs {row['price_chf'] or 'n/a'}, Wert {row['value_chf'] or 'n/a'} CHF, Verifikation {row['last_verified_at'] or 'n/a'}, Notizen {row['notes'] or '-'}")
   264|    lines.extend(["", "## Datenqualitätswarnungen"])
   265|    if context["data_quality_warnings"]:
   266|        for warning in context["data_quality_warnings"]:
   267|            lines.append(f"- {warning['code']}: {warning['message']}")
   268|    else:
   269|        lines.append("- Keine Warnungen.")
   270|    lines.append("")
   271|    return "\n".join(lines)
   272|
   273|
   274|def render_crypto_report_html(context: dict[str, Any]) -> str:
   275|    markdown = render_crypto_report_markdown(context)
   276|    body = "\n".join(f"
{html.escape(line)}
" if line else "" for line in markdown.splitlines())
   277|    return "<!doctype html><meta charset='utf-8'><title>Crypto-Bestandsübersicht</title>" + body + ""
   278|
   279|
   280|def _default_pdf_renderer(html_text: str, output_path: Path) -> bool:
   281|    try:
   282|        from weasyprint import HTML  # type: ignore
   283|    except Exception:
   284|        return False
   285|    HTML(string=html_text).write_pdf(str(output_path))
   286|    return True
   287|
   288|
   289|def export_crypto_inventory_report(
   290|    conn: Connection,
   291|    *,
   292|    settings: Settings,
   293|    preferred_format: str = "pdf",
   294|    renderer: PDFRenderer | None = _default_pdf_renderer,
   295|) -> dict[str, Any]:
   296|    validate_settings(settings)
   297|    ensure_runtime_dirs(settings)
   298|    reports_dir = settings.runtime_paths.reports_dir.resolve()
   299|    report_id = str(uuid.uuid4())
   300|    context = build_crypto_report_context(conn, base_currency=settings.base_currency)
   301|    stem = f"crypto_inventory_{context['generated_at'].replace(':', '').replace('+', '_')}_{report_id[:8]}"
   302|    warnings: list[str] = []
   303|    html_text = render_crypto_report_html(context)
   304|    markdown_text = render_crypto_report_markdown(context)
   305|
   306|    fmt = preferred_format.lower()
   307|    if fmt == "pdf" and renderer is not None:
   308|        pdf_path = reports_dir / f"{stem}.pdf"
   309|        if renderer(html_text, pdf_path):
   310|            output_path = pdf_path
   311|            final_format = "pdf"
   312|        else:
   313|            warnings.append("PDF renderer unavailable; wrote HTML fallback.")
   314|            output_path = reports_dir / f"{stem}.html"
   315|            output_path.write_text(html_text, encoding="utf-8")
   316|            final_format = "html"
   317|    elif fmt == "markdown":
   318|        output_path = reports_dir / f"{stem}.md"
   319|        output_path.write_text(markdown_text, encoding="utf-8")
   320|        final_format = "markdown"
   321|    else:
   322|        warnings.append("PDF renderer unavailable; wrote HTML fallback." if preferred_format.lower() == "pdf" else "Wrote HTML report.")
   323|        output_path = reports_dir / f"{stem}.html"
   324|        output_path.write_text(html_text, encoding="utf-8")
   325|        final_format = "html"
   326|
   327|    now = utc_now()
   328|    summary_json = json.dumps(context["summary"], sort_keys=True)
   329|    conn.execute(
   330|        """
   331|        INSERT INTO reports(
   332|            report_id, report_type, title, generated_at, file_path, format,
   333|            data_quality_status, summary_json, created_at
   334|        ) VALUES (?, 'crypto_inventory', ?, ?, ?, ?, ?, ?, ?)
   335|        """,
   336|        (
   337|            report_id,
   338|            context["title"],
   339|            now,
   340|            str(output_path),
   341|            final_format,
   342|            context["summary"]["data_quality_status"],
   343|            summary_json,
   344|            now,
   345|        ),
   346|    )
   347|    record_audit_event(
   348|        conn,
   349|        source="reports",
   350|        action="report_generated",
   351|        entity_type="report",
   352|        entity_id=report_id,
   353|        new_values={"report_type": "crypto_inventory", "file_path": str(output_path), "format": final_format},
   354|        confirmed=True,
   355|        created_by="system",
   356|    )
   357|    conn.commit()
   358|    return {"report_id": report_id, "file_path": str(output_path), "format": final_format, "warnings": warnings, "context": context}
   359|


Explanation:
**Verifying report details**

I think I should use a todo list to keep track of everything. I need to verify all the statements to avoid any potential issues, especially since the report contains real quantities in runtime reports. The user is asking me to generate a report without outputting values. Also, I'll need to check the dashboard test since it may not function in a browser. I should ensure there are no live API calls by possibly monkeypatching and searching through the tests.

todo

Runtime-/Git-Safety und Repo-Freiheit prüfen

Produktive Runtime-DB auf Import-, Holding-, Snapshot-, Legacy- und Mapping-Konsistenz prüfen

Audit- und Import-Session-Konsistenz prüfen

Dashboard-Readmodel/Crypto-Seite lokal gegen echte Runtime-DB ohne Live-API testen

Crypto-Bestandsreport im Runtime-Reports-Ordner erzeugen und prüfen

Aggregierten Bericht ohne echte Mengen/Werte liefern


terminal

set -euo pipefail
find . -type d \( -name __pycache__ -o -name .pytest_cache \) -prune -exec rm -rf {} +
PRE_GIT_STATUS=$(git status --short)
PRE_SAFETY=$(PYTHONPATH=src python -m jarvis_finance.cli.main git-safety-scan .)
PYTHONPATH=src python - <<'PY'
from pathlib import Path
import json
from jarvis_finance.config.settings import load_settings, find_repo_root
s=load_settings(); repo=find_repo_root().resolve(); runtime=s.runtime_paths.base_dir.resolve(); db=s.db_path.resolve(); reports=s.runtime_paths.reports_dir.resolve()
real_exts={'.csv','.xlsx','.xls','.pdf','.sqlite','.sqlite3','.db'}
# Search repo filesystem for blocked real-data extensions excluding allowed synthetic/example test paths and git internals.
suspect=[]
for p in repo.rglob('*'):
    if not p.is_file(): continue
    rel=p.relative_to(repo).as_posix()
    if rel.startswith('.git/') or '/__pycache__/' in rel or rel.startswith('examples/synthetic/') or rel.startswith('tests/fixtures/'):
        continue
    if p.suffix.lower() in real_exts:
        suspect.append(rel)
print(json.dumps({
  'repo': str(repo),
  'runtime_base': str(runtime),
  'db_path': str(db),
  'reports_dir': str(reports),
  'runtime_outside_repo': not str(runtime).startswith(str(repo)),
  'db_outside_repo': not str(db).startswith(str(repo)),
  'reports_outside_repo': not str(reports).startswith(str(repo)),
  'suspect_real_data_files_in_repo_count': len(suspect),
}, ensure_ascii=False, indent=2))
PY
printf '\n---PRE_GIT_STATUS---\n%s\n' "$PRE_GIT_STATUS"
printf '\n---PRE_SAFETY---\n%s\n' "$PRE_SAFETY"


Output:
{
  "repo": "/home/agent/.hermes/repos/FinanceManager",
  "runtime_base": "/home/agent/jarvis_runtime/finance-system",
  "db_path": "/home/agent/jarvis_runtime/finance-system/data/finance.sqlite3",
  "reports_dir": "/home/agent/jarvis_runtime/finance-system/reports",
  "runtime_outside_repo": true,
  "db_outside_repo": true,
  "reports_outside_repo": true,
  "suspect_real_data_files_in_repo_count": 0
}

---PRE_GIT_STATUS---

---PRE_SAFETY---
GIT_SAFETY_OK


Explanation:
**Ensuring database consistency**

I need to focus on database consistency, especially with the tricky aspect of snapshot dates, which might be stored as legacy_snapshot_date or last_verified_at. I should check the schema and the holdings function to confirm this. It’s important that I output counts instead of values to ensure unsafe assets from Coingecko aren't present in the holdings. I might also need to create an audit trace per holding using the create_initial_holding_snapshot entity for crypto_holding with the correct entity_id. I'll read the create function to clarify.


     1|from __future__ import annotations
     2|
     3|from dataclasses import dataclass, field
     4|from decimal import Decimal
     5|from sqlite3 import Connection
     6|
     7|from jarvis_finance.audit.log import record_audit_event
     8|from jarvis_finance.imports.common import stable_id, utc_now
     9|from jarvis_finance.quality.alerts import create_alert
    10|
    11|VALID_VERIFICATION_STATUS = {"verified", "stale", "unverified", "estimated"}
    12|
    13|
    14|@dataclass
    15|class CryptoHolding:
    16|    wallet_id: str | None
    17|    asset_id: str
    18|    quantity: Decimal = Decimal("0")
    19|    verification_status: str = "unverified"
    20|    last_verified_at: str | None = None
    21|    legacy_snapshot_value_original: Decimal | None = None
    22|    legacy_snapshot_value_chf: Decimal | None = None
    23|    current_value_chf: Decimal | None = None
    24|    quality_warnings: list[str] = field(default_factory=list)
    25|
    26|
    27|@dataclass
    28|class CryptoHoldingsResult:
    29|    wallet_holdings: dict[tuple[str, str], CryptoHolding] = field(default_factory=dict)
    30|    total_by_asset: dict[str, CryptoHolding] = field(default_factory=dict)
    31|
    32|
    33|def _d(value: object, default: str = "0") -> Decimal:
    34|    if value is None or str(value).strip() == "":
    35|        return Decimal(default)
    36|    return Decimal(str(value))
    37|
    38|
    39|def _ds(value: Decimal | None) -> str | None:
    40|    return format(value, "f") if value is not None else None
    41|
    42|
    43|def _ensure_holding(result: CryptoHoldingsResult, wallet_id: str, asset_id: str) -> CryptoHolding:
    44|    return result.wallet_holdings.setdefault((wallet_id, asset_id), CryptoHolding(wallet_id=wallet_id, asset_id=asset_id))
    45|
    46|
    47|def _ensure_total(result: CryptoHoldingsResult, asset_id: str) -> CryptoHolding:
    48|    return result.total_by_asset.setdefault(asset_id, CryptoHolding(wallet_id=None, asset_id=asset_id))
    49|
    50|
    51|def create_initial_holding_snapshot(
    52|    conn: Connection,
    53|    *,
    54|    asset_id: str,
    55|    wallet_id: str,
    56|    quantity: Decimal,
    57|    verification_status: str,
    58|    note: str,
    59|    last_verified_at: str | None = None,
    60|    legacy_snapshot_value_original: Decimal | None = None,
    61|    legacy_snapshot_value_chf: Decimal | None = None,
    62|    legacy_snapshot_currency: str | None = None,
    63|    legacy_snapshot_date: str | None = None,
    64|) -> str:
    65|    if quantity < 0:
    66|        raise ValueError("initial holding quantity must be >= 0")
    67|    if verification_status not in VALID_VERIFICATION_STATUS:
    68|        raise ValueError("invalid verification_status")
    69|    if not note.strip():
    70|        raise ValueError("initial holding snapshot requires a note")
    71|    now = utc_now()
    72|    holding_id = stable_id("cryptoholding", asset_id, wallet_id)
    73|    conn.execute(
    74|        """
    75|        INSERT OR REPLACE INTO crypto_holdings(
    76|            crypto_holding_id, asset_id, wallet_id, quantity, acquisition_source,
    77|            last_verified_at, verification_status, legacy_snapshot_value_original,
    78|            legacy_snapshot_value_chf, legacy_snapshot_currency, legacy_snapshot_date,
    79|            notes, created_at, updated_at
    80|        ) VALUES (?, ?, ?, ?, 'initial_snapshot', ?, ?, ?, ?, ?, ?, ?,
    81|                  COALESCE((SELECT created_at FROM crypto_holdings WHERE crypto_holding_id=?), ?), ?)
    82|        """,
    83|        (
    84|            holding_id, asset_id, wallet_id, _ds(quantity), last_verified_at, verification_status,
    85|            _ds(legacy_snapshot_value_original),
    86|            _ds(legacy_snapshot_value_chf),
    87|            legacy_snapshot_currency, legacy_snapshot_date, note, holding_id, now, now,
    88|        ),
    89|    )
    90|    record_audit_event(
    91|        conn,
    92|        source="crypto_holdings",
    93|        action="initial_crypto_holding_snapshot",
    94|        entity_type="crypto_holding",
    95|        entity_id=holding_id,
    96|        new_values={"asset_id": asset_id, "wallet_id": wallet_id, "quantity": _ds(quantity), "verification_status": verification_status},
    97|        user_text_note=note,
    98|        confirmed=True,
    99|        created_by="system",
   100|    )
   101|    conn.commit()
   102|    return holding_id
   103|
   104|
   105|def _apply_delta(result: CryptoHoldingsResult, wallet_id: str, asset_id: str, delta: Decimal) -> None:
   106|    holding = _ensure_holding(result, wallet_id, asset_id)
   107|    holding.quantity += delta
   108|    total = _ensure_total(result, asset_id)
   109|    total.quantity += delta
   110|
   111|
   112|def calculate_crypto_holdings(conn: Connection) -> CryptoHoldingsResult:
   113|    result = CryptoHoldingsResult()
   114|    for row in conn.execute("SELECT * FROM crypto_holdings ORDER BY wallet_id, asset_id").fetchall():
   115|        holding = _ensure_holding(result, row["wallet_id"], row["asset_id"])
   116|        holding.quantity += _d(row["quantity"])
   117|        holding.verification_status = row["verification_status"]
   118|        holding.last_verified_at = row["last_verified_at"]
   119|        holding.legacy_snapshot_value_original = _d(row["legacy_snapshot_value_original"]) if row["legacy_snapshot_value_original"] is not None else None
   120|        holding.legacy_snapshot_value_chf = _d(row["legacy_snapshot_value_chf"]) if row["legacy_snapshot_value_chf"] is not None else None
   121|        total = _ensure_total(result, row["asset_id"])
   122|        total.quantity += _d(row["quantity"])
   123|    rows = conn.execute("SELECT * FROM crypto_transactions ORDER BY transaction_datetime, created_at, crypto_transaction_id").fetchall()
   124|    for row in rows:
   125|        asset_id = row["asset_id"]
   126|        qty = _d(row["quantity"])
   127|        fee_qty = _d(row["fee_quantity"])
   128|        t = row["transaction_type"]
   129|        if t == "transfer":
   130|            _apply_delta(result, row["from_wallet_id"], asset_id, -(qty + fee_qty))
   131|            _apply_delta(result, row["to_wallet_id"], asset_id, qty)
   132|        elif t == "buy":
   133|            _apply_delta(result, row["to_wallet_id"], asset_id, qty - fee_qty)
   134|        elif t == "sell":
   135|            _apply_delta(result, row["from_wallet_id"], asset_id, -(qty + fee_qty))
   136|        elif t == "fee":
   137|            _apply_delta(result, row["from_wallet_id"], asset_id, -fee_qty)
   138|    for asset in conn.execute("SELECT asset_id, symbol, coingecko_id FROM crypto_assets").fetchall():
   139|        if not asset["coingecko_id"]:
   140|            msg = "Crypto asset has no CoinGecko ID; current valuation is unavailable."
   141|            for holding in [h for h in result.wallet_holdings.values() if h.asset_id == asset["asset_id"]]:
   142|                holding.quality_warnings.append("missing_coingecko_id")
   143|            create_alert(
   144|                conn,
   145|                priority="warnung",
   146|                category="crypto",
   147|                entity_type="crypto_asset",
   148|                entity_id=asset["asset_id"],
   149|                rule_id="missing_coingecko_id",
   150|                message=msg,
   151|                evidence={"symbol": asset["symbol"]},
   152|                fingerprint="missing_coingecko_id",
   153|            )
   154|    conn.commit()
   155|    return result
   156|


Explanation:
**Checking wallet session details**

I need to explore queries for wallet sessions, particularly focusing on previous sessions that might not have failed. It seems like there was a crash before, preventing any attempts to import holdings. I should check for imported sessions and query counts. The user specifically asked for counts per safe coin, so I’ll list the counts without sensitive values. I'll ensure the output is compact and clear, including only names and counts as requested.
**Reviewing holdings and audits**

I need to ensure there are no negative quantities in my holdings output. I can use a numeric cast for decimal strings in SQLite, but I won't output any values if they are empty. I should check legacy snapshot dates and verify that the CoinGecko IDs are valid. It appears there are no imported holdings for certain labels or coins, so I’ll query them by symbols and names. Also, every imported holding must have an audit trace, so I’ll count any missing audits along the way.


     1|from __future__ import annotations
     2|
     3|from jarvis_finance.dashboard.components.tables import show_table
     4|from jarvis_finance.dashboard import data
     5|
     6|PAGE_TITLE = "Crypto"
     7|
     8|def render(st, conn) -> None:
     9|    st.title("Crypto")
    10|    st.caption("Preise werden nur aus der lokalen DB/Cache gelesen; kein CoinGecko-Aufruf beim Rendern.")
    11|    st.subheader("Coins aggregiert")
    12|    show_table(st, data.get_crypto_overview(conn))
    13|    st.subheader("Coins nach Wallet")
    14|    show_table(st, data.get_crypto_by_wallet(conn))
    15|    st.subheader("Wallet-Gesamtübersicht")
    16|    show_table(st, data.get_wallets(conn))
    17|    st.subheader("Fehlende/veraltete Preis- und CoinGecko-Hinweise")
    18|    quality_rows = [r for r in data.get_crypto_overview(conn) if not r["coingecko_id"] or r["price_quality"] != "fresh"]
    19|    show_table(st, quality_rows)
    20|    st.subheader("Letzte Crypto-Transaktionen")
    21|    show_table(st, data.get_latest_crypto_transactions(conn))
    22|


     1|from __future__ import annotations
     2|
     3|
     4|def show_table(st, rows: list[dict[str, str]], *, empty_message: str = "Keine Daten vorhanden.") -> None:
     5|    if rows:
     6|        st.dataframe(rows, use_container_width=True, hide_index=True)
     7|    else:
     8|        st.info(empty_message)
     9|


Explanation:
**Generating report context**

I'm focused on generating a report that will provide a file path, which could include the filename, but it doesn’t need to. The storage location is referenced as Runtime-Reports-Ordner, without specifying the exact file. I’m considering that the report should only include aggregated data without sensitive content. Also, if there are missing crypto prices, that creates warnings in the report about data quality, which is acceptable. I’ll also ensure the final report has appropriate safety checks and won't print sensitive file paths in the terminal output.
**Creating crypto report safely**

I need to be cautious while rendering the crypto report in Markdown, as it includes real quantities, but I won't print them directly. The export should deliver context with values without printing anything. There’s a potential issue with generating a PDF with WeasyPrint, so I need to decide on the output format. Additionally, I might run a safety check in Python. I can also test by monkeypatching the market provider since the latest crypto price data is local. If I choose to go this route, wrapping `requests` could be useful, ensuring we don’t rely on network access.
**Preparing dashboard rendering**

I need to set up the dashboard render process carefully. I'm using a fake collection to gather counts from the data, but I won't be printing the actual rows. Instead, I’ll implement a fake DataFrame to increment the count without displaying the values. Also, I need to ensure that the "Crypto-Seite zeigt importierte Wallets/Holdings" section reflects counts from data rows greater than zero. Once I have that set up, I can run the script.

set -euo pipefail
PYTHONPATH=src python - <<'PY'
import json, socket
from pathlib import Path
from decimal import Decimal
from jarvis_finance.config.settings import load_settings, find_repo_root, validate_settings
from jarvis_finance.storage.database import connect
from jarvis_finance.dashboard import data as dash_data
from jarvis_finance.dashboard.pages import page_04_crypto
from jarvis_finance.reports.crypto_inventory import export_crypto_inventory_report

safe = {
 'AAVE': ('AAVE','aave'), 'BNB': ('BNB','binancecoin'), 'Band Protocol BAND': ('BAND','band-protocol'),
 'CAKE Pancake': ('CAKE','pancakeswap-token'), 'Celo CGLD': ('CELO','celo'), 'Compound COMP': ('COMP','compound-governance-token'),
 'Enjin ENJ': ('ENJ','enjincoin'), 'FDUSD': ('FDUSD','first-digital-usd'), 'Fetch.ai FET': ('FET','fetch-ai'),
 'Flare FLR': ('FLR','flare-networks'), 'HBAR': ('HBAR','hedera-hashgraph'), 'LL Light Link': ('LL','lightlink'),
 'Stellar Lumens XLM': ('XLM','stellar'), 'Swissborg': ('BORG','swissborg'), 'Tether USD': ('USDT','tether'),
 'The Graph GRT': ('GRT','the-graph'), 'VTHO': ('VTHO','vethor-token'), 'VeChain VET': ('VET','vechain'),
 'Wilder World WILD': ('WILD','wilder-world')
}
ignored_terms=['NIGHT','S Sonic','Sonic']
unsafe_terms=['ADA','Cardano','Amp','AVAX','Avalanche','Bitcoin','BTC','Chainlink','DOGE','Decentraland','MANA','Ethereum','ETH','HIGH','Highstreet','IOTA','LUNA','LUNC','NETVR','NuCypher','ZRX','Polygon','MATIC','POL','Ripple','XRP','SAND','Sandbox','SAROS','SKY','SOL','Solana','Sundae','USTC']
settings=load_settings(); validate_settings(settings)
repo=find_repo_root().resolve(); db=settings.db_path.resolve(); reports_dir=settings.runtime_paths.reports_dir.resolve(); runtime=settings.runtime_paths.base_dir.resolve()
conn=connect(db)
ph=','.join('?' for _ in [v[1] for v in safe.values()])
safe_cg=[v[1] for v in safe.values()]
# Core counts
wallets=conn.execute('SELECT COUNT(*) AS n FROM crypto_wallets').fetchone()['n']
assets=conn.execute('SELECT COUNT(*) AS n FROM crypto_assets').fetchone()['n']
holdings=conn.execute('SELECT COUNT(*) AS n FROM crypto_holdings').fetchone()['n']
safe_assets=conn.execute(f'SELECT COUNT(DISTINCT coingecko_id) AS n FROM crypto_assets WHERE coingecko_id IN ({ph})', safe_cg).fetchone()['n']
safe_holdings=conn.execute(f'SELECT COUNT(*) AS n FROM crypto_holdings h JOIN crypto_assets a ON a.asset_id=h.asset_id WHERE a.coingecko_id IN ({ph})', safe_cg).fetchone()['n']
# per coin count only
per_coin=[]
for label,(sym,cg) in safe.items():
    n=conn.execute('SELECT COUNT(*) AS n FROM crypto_holdings h JOIN crypto_assets a ON a.asset_id=h.asset_id WHERE a.coingecko_id=?',(cg,)).fetchone()['n']
    per_coin.append({'label':label,'holding_count':n})
# Invariants
problems={}
problems['duplicate_asset_wallet_holdings']=conn.execute('SELECT COUNT(*) AS n FROM (SELECT asset_id,wallet_id,COUNT(*) c FROM crypto_holdings GROUP BY asset_id,wallet_id HAVING c>1)').fetchone()['n']
problems['negative_holdings']=conn.execute("SELECT COUNT(*) AS n FROM crypto_holdings WHERE quantity IS NOT NULL AND trim(quantity)<>'' AND CAST(quantity AS NUMERIC)<0").fetchone()['n']
problems['empty_quantity_holdings']=conn.execute("SELECT COUNT(*) AS n FROM crypto_holdings WHERE quantity IS NULL OR trim(quantity)='' ").fetchone()['n']
problems['holdings_without_wallet']=conn.execute('SELECT COUNT(*) AS n FROM crypto_holdings h LEFT JOIN crypto_wallets w ON w.wallet_id=h.wallet_id WHERE w.wallet_id IS NULL').fetchone()['n']
problems['holdings_without_asset']=conn.execute('SELECT COUNT(*) AS n FROM crypto_holdings h LEFT JOIN crypto_assets a ON a.asset_id=h.asset_id WHERE a.asset_id IS NULL').fetchone()['n']
problems['holdings_missing_coingecko_id']=conn.execute("SELECT COUNT(*) AS n FROM crypto_holdings h JOIN crypto_assets a ON a.asset_id=h.asset_id WHERE a.coingecko_id IS NULL OR trim(a.coingecko_id)='' ").fetchone()['n']
problems['snapshot_date_mismatch']=conn.execute("SELECT COUNT(*) AS n FROM crypto_holdings WHERE acquisition_source='initial_snapshot' AND (legacy_snapshot_date<>? OR last_verified_at<>?)",('2025-12-31','2025-12-31')).fetchone()['n']
problems['legacy_values_carried']=conn.execute('SELECT COUNT(*) AS n FROM crypto_holdings WHERE legacy_snapshot_value_original IS NOT NULL OR legacy_snapshot_value_chf IS NOT NULL OR legacy_snapshot_currency IS NOT NULL').fetchone()['n']
# ignored/unsafe imported (by asset symbol/name/cg not in safe)
unsafe_like_count=0
for term in ignored_terms+unsafe_terms:
    unsafe_like_count += conn.execute("SELECT COUNT(*) AS n FROM crypto_holdings h JOIN crypto_assets a ON a.asset_id=h.asset_id WHERE lower(a.coin_name) LIKE lower(?) OR lower(a.symbol)=lower(?) OR lower(a.coingecko_id) LIKE lower(?)", (f'%{term}%', term, f'%{term}%')).fetchone()['n']
problems['ignored_or_manual_review_holdings_detected_by_label_scan']=unsafe_like_count
problems['non_safe_subset_holdings']=conn.execute(f'SELECT COUNT(*) AS n FROM crypto_holdings h JOIN crypto_assets a ON a.asset_id=h.asset_id WHERE a.coingecko_id NOT IN ({ph})', safe_cg).fetchone()['n']
# Sessions and audit
wallet_sessions=conn.execute("SELECT COUNT(*) AS n FROM import_sessions WHERE import_type='crypto_wallets' AND status='committed'").fetchone()['n']
holding_sessions=conn.execute("SELECT COUNT(*) AS n FROM import_sessions WHERE import_type='crypto_holdings' AND status='committed'").fetchone()['n']
bad_sessions=conn.execute("SELECT COUNT(*) AS n FROM import_sessions WHERE status IN ('started','failed','partial')").fetchone()['n']
audit_total=conn.execute('SELECT COUNT(*) AS n FROM audit_log').fetchone()['n']
holding_audit_missing=conn.execute("SELECT COUNT(*) AS n FROM crypto_holdings h LEFT JOIN audit_log a ON a.entity_type='crypto_holding' AND a.entity_id=h.crypto_holding_id AND a.action='initial_crypto_holding_snapshot' WHERE a.audit_id IS NULL").fetchone()['n']
# Dashboard local-read test with network blocked
class FakeSt:
    def __init__(self): self.calls=[]; self.dataframe_rows=[]; self.info_count=0
    def title(self,*a,**k): self.calls.append('title')
    def caption(self,*a,**k): self.calls.append('caption')
    def subheader(self,*a,**k): self.calls.append('subheader')
    def dataframe(self, rows, **kwargs): self.calls.append('dataframe'); self.dataframe_rows.append(len(rows))
    def info(self,*a,**k): self.calls.append('info'); self.info_count += 1
fake=FakeSt()
orig_socket=socket.socket
def blocked_socket(*args, **kwargs):
    raise RuntimeError('network disabled during dashboard render')
socket.socket=blocked_socket
try:
    crypto_overview=dash_data.get_crypto_overview(conn)
    crypto_by_wallet=dash_data.get_crypto_by_wallet(conn)
    wallet_rows=dash_data.get_wallets(conn)
    page_04_crypto.render(fake, conn)
    dashboard_ok=len(crypto_overview)>0 and len(crypto_by_wallet)>0 and len(wallet_rows)>0 and 'dataframe' in fake.calls
    dashboard_live_api=False
finally:
    socket.socket=orig_socket
# Report generation under runtime reports dir only. Do not print file path filename/content.
pre_reports=conn.execute("SELECT COUNT(*) AS n FROM reports WHERE report_type='crypto_inventory'").fetchone()['n']
report_result=export_crypto_inventory_report(conn, settings=settings, preferred_format='pdf')
post_reports=conn.execute("SELECT COUNT(*) AS n FROM reports WHERE report_type='crypto_inventory'").fetchone()['n']
report_path=Path(report_result['file_path']).resolve()
context=report_result['context']
report_coin_labels={row['symbol'] for row in context['coins']} | {row['coin_name'] for row in context['coins']} | {str(row.get('coingecko_id','')) for row in context['coins']}
excluded_in_report=[]
for term in ignored_terms+unsafe_terms:
    for item in report_coin_labels:
        if term.lower() in item.lower():
            excluded_in_report.append(term); break
summary={
 'runtime_db_path': str(db),
 'runtime_db_outside_repo': not str(db).startswith(str(repo)),
 'reports_dir': str(reports_dir),
 'reports_dir_outside_repo': not str(reports_dir).startswith(str(repo)),
 'wallets': wallets,
 'assets': assets,
 'holdings': holdings,
 'safe_assets': safe_assets,
 'safe_holdings': safe_holdings,
 'holdings_per_safe_coin_count': per_coin,
 'problems': problems,
 'problems_total': sum(problems.values()) + bad_sessions + holding_audit_missing,
 'wallet_import_session_present': wallet_sessions>0,
 'holdings_import_session_present': holding_sessions>0,
 'committed_wallet_sessions': wallet_sessions,
 'committed_holding_sessions': holding_sessions,
 'bad_import_sessions_count': bad_sessions,
 'audit_log_entries_present': audit_total>0,
 'holding_audit_missing_count': holding_audit_missing,
 'dashboard_reads_runtime_db': dashboard_ok,
 'dashboard_live_api_on_render': dashboard_live_api,
 'dashboard_crypto_rows': {'coin_rows': len(crypto_overview), 'wallet_holding_rows': len(crypto_by_wallet), 'wallet_rows': len(wallet_rows)},
 'report_generated': report_path.exists() and post_reports==pre_reports+1,
 'report_format': report_result['format'],
 'report_in_runtime_reports_dir': str(report_path).startswith(str(reports_dir)),
 'report_data_quality_status': context['summary']['data_quality_status'],
 'report_coin_count': context['summary']['coin_count'],
 'report_wallet_count': context['summary']['wallet_count'],
 'report_excluded_or_manual_coins_present_count': len(set(excluded_in_report)),
 'report_warning_count': len(context['data_quality_warnings']) + len(report_result.get('warnings', [])),
}
conn.close()
print(json.dumps(summary, ensure_ascii=False, indent=2))
PY
find . -type d \( -name __pycache__ -o -name .pytest_cache \) -prune -exec rm -rf {} +
POST_SAFETY=$(PYTHONPATH=src python -m jarvis_finance.cli.main git-safety-scan .)
POST_GIT_STATUS=$(git status --short)
printf '\n---POST_GIT_STATUS---\n%s\n' "$POST_GIT_STATUS"
printf '\n---POST_SAFETY---\n%s\n' "$POST_SAFETY"