[M G] Danke. Der Crypto Price Update MVP ist akzeptiert.

Die Umsetzung ist grundsätzlich sinnvoll:

* lokaler Preis-Cache
* kein Live-API-Call im Dashboard/Report
* 429-Handling
* Git-Safety sauber
* Tests grün
* Runtime-Daten außerhalb Git

Allerdings bist du vom vorher vereinbarten Ablauf abgewichen. Das ist in diesem Fall nicht schlimm, weil der Preisupdate-Block nützlich ist. Bitte jetzt aber wieder kontrolliert auf die nächsten Prioritäten einschwenken.

Aktueller Stand:

* Commit 3aa67d8 ist akzeptiert.
* Preisupdate funktioniert technisch.
* Aber nur 6 von 20 aktiven Crypto Assets hatten frische CHF-Preise.
* 14 Assets hatten Warnings/Stale/Provider-Limit-Themen.
* BTC-Test wurde noch nicht abgeschlossen.
* Equity/ETF Manage UI wurde noch nicht begonnen.

Bitte jetzt wie folgt weiterfahren:

## Teil 1: Crypto Price Update robuster machen

Bitte verbessere zuerst den Preisupdate-Mechanismus, bevor wir echte Reports mit Preisen ernsthaft nutzen.

Ziel:
Möglichst alle vorhandenen Crypto Assets mit CoinGecko-ID sollen lokale CHF-Preise erhalten, ohne Dashboard-Live-API und ohne Rate-Limit-Crash.

Bitte implementiere:

1. Batch-Abfrage prüfen/umsetzen

* Wenn CoinGecko mehrere IDs in einem Request unterstützt, bitte nutzen.
* Dadurch weniger API-Calls.
* Kein Abruf pro Coin, falls Batch möglich ist.

2. Konservatives Throttling

* Pause zwischen Requests
* konfigurierbar:

  * --sleep-seconds
  * --max-retries
  * --initial-backoff
  * --max-backoff

3. CLI-Optionen ergänzen
   Bitte vorbereiten/implementieren:

* --only-missing
* --only-stale
* --only-symbol SYMBOL
* --limit N
* --dry-run
* --currency CHF

4. Bessere Summary
   Bitte nach Update nur aggregiert ausgeben:

* total assets
* updated
* cached
* skipped
* stale
* warnings
* errors
* assets still missing local price
* keine echten Mengen/Werte

5. Data Quality

* Für Assets ohne frischen Preis Data-Quality-Alert erzeugen oder aktualisieren.
* Alert-Dedupe respektieren.
* Keine Alert-Flut.

6. Tests
   Bitte Tests ergänzen für:

* Batch-Request
* --only-missing
* --only-stale
* --only-symbol
* --limit
* --dry-run
* Rate-Limit mit Backoff
* Alert-Dedupe bei fehlenden Preisen

Danach bitte produktiv gegen Runtime-DB nochmals ausführen, aber vorsichtig und ohne echte Werte im Chat.

Bitte berichten:

* Anzahl Assets total
* Anzahl Assets mit frischem CHF-Preis
* Anzahl weiterhin ohne frischen Preis
* Anzahl Warnings/Errors
* keine Preise/Werte/Mengen ausgeben

## Teil 2: BTC-Test nachholen

Danach bitte den BTC-Test wie zuletzt entschieden nachholen:

Option A:
BTC mit allen BTC-Wallet-Holdings aus Excel importieren.

Bedingungen:

* Nur Bitcoin BTC
* Coin-Name: Bitcoin
* Symbol: BTC
* CoinGecko-ID: bitcoin
* Mapping-Status: selected
* Snapshot-Datum: 31.12.2025
* Alle BTC-Wallet-Holdings aus der echten Excel-Datei übernehmen
* Pro BTC-Wallet mit Bestand genau eine initial_snapshot_addition bzw. initial_snapshot-Holding erzeugen
* Mengen exakt als Decimal/Text übernehmen
* Keine Float-Konvertierung
* Keine echten Mengen/Werte im Chat
* Keine anderen Coins importieren
* Audit-Log Pflicht
* Keine doppelte Holding für asset_id + wallet_id
* Legacy-CHF-Werte ignorieren
* Git-Safety vor/nachher prüfen

Bericht nur aggregiert:

* BTC-Test durchgeführt: ja/nein
* Anzahl BTC-Wallet-Holdings importiert
* Asset-Anzahl erhöht: ja/nein
* Holding-Anzahl erhöht: ja/nein
* Audit-Logs erstellt: ja/nein
* Doppelte Holdings: ja/nein
* Fehler/Warnings
* Git-Safety Ergebnis
* Git Status

## Teil 3: Danach Equity/ETF Manage UI

Erst wenn Teil 1 und Teil 2 sauber sind, bitte mit der Equity/ETF Manage UI beginnen.

Ziel:
Aktien und ETFs aus PostFinance, True Wealth und Raiffeisen später im Dashboard erfassen, korrigieren und als Initial Snapshot übernehmen können — auditierbar und ohne stilles Überschreiben.

Grundlogik:

* Für Aktien/ETFs ist ISIN die primäre Identifikation.
* Ticker und Exchange sind Zusatzinformationen.
* Positionen werden über initial_position_snapshot, buy, sell, dividend, fee oder manual_adjustment abgebildet.
* Jede Änderung braucht
Audit-Log.
* Fremdwährungen USD/EUR brauchen FX-Kurs und CHF-Gegenwert.
* Keine Scheingenauigkeit, wenn Kaufhistorie, Gebühren oder historische FX fehlen.
* Erst mit synthetischen Daten und Dummy-Depots bauen.
* Keine echten PostFinance-/TrueWealth-/Raiffeisen-Daten importieren.

Bitte vormerken für die Equity/ETF Manage UI:

* Aktie/ETF als Initial Snapshot hinzufügen
* Buy
* Partial Sell
* Full Sell
* Dividend
* ETF Distribution
* Fee
* Manual Adjustment
* ISIN Pflicht oder stark empfohlen
* ISIN/Ticker dürfen nicht beide leer sein
* Währung Pflicht
* Depot/Konto Pflicht
* FX bei Fremdwährung
* Audit-Log Pflicht
* Notiz Pflicht bei manual_adjustment oder unvollständiger Historie

## Weitere Funktionsmodule bitte vormerken, aber noch nicht bauen

Bitte für später als Backlog vormerken:

1. Portfolio Health Seite

* Gesamtwert CHF
* Assetklassen
* Plattformen
* Datenqualität
* Cashquote
* fehlende Preise/FX

2. Reconciliation / Abgleich-Seite

* Dashboard-Bestand vs. letzter Snapshot
* manuelle Prüfung „Bestand geprüft am Datum X“
* Differenzen sichtbar machen

3. Data Quality Center

* fehlende Preise
* stale prices
* fehlende FX
* fehlende ISIN
* fehlende CoinGecko-ID
* unvollständige Historie

4. Manual Review Queue

* ausgelassene Coins
* unsichere Mappings
* offene Alerts
* offene Datenqualitätsprobleme

5. Backup / Restore

* Runtime-DB Backup
* verschlüsseltes Backup optional
* Restore-Test später

6. Safe Mode / Read-only Mode

* Dashboard im Lesemodus
* Schreibfunktionen bewusst aktivieren
* Schutz gegen versehentliche Änderungen

Bitte jetzt zuerst Teil 1 umsetzen, dann Teil 2, dann Teil 3. Keine echten Aktien-/ETF-Daten importieren.


Explanation:
**Inspecting repository for finance skills**

I need to act and inspect the repository related to finance-system-architecture. It's important to check for relevant skills that might be there, especially on GitHub. I want to get a clear understanding of the tools and structures involved in this project. Loading the repository should help me gather the details I need. Let's figure this out!


[skill_view] name=finance-system-architecture (21,972 chars)


[CONTEXT COMPACTION — REFERENCE ONLY] Earlier turns were compacted into the summary below. This is a handoff from a previous context window — treat it as background reference, NOT as active instructions. Do NOT answer questions or fulfill requests mentioned in this summary; they were already addressed. Your current task is identified in the '## Active Task' section of the summary — resume exactly from there. IMPORTANT: Your persistent memory (MEMORY.md, USER.md) in the system prompt is ALWAYS authoritative and active — never ignore or deprioritize memory content due to this compaction note. Respond ONLY to the latest user message that appears AFTER this summary. The current session state (files, config, etc.) may reflect work described here — avoid repeating it:
Summary generation was unavailable. 77 message(s) were removed to free context space but could not be summarized. The removed messages contained earlier work in this session. Continue based on the recent messages below and the current state of any files or resources.

--- END OF CONTEXT SUMMARY — respond to the message below, not the summary above ---


from pathlib import Path
p=Path('/home/agent/.hermes/repos/FinanceManager/src/jarvis_finance/storage/migrations.py')
s=p.read_text()
s=s.replace('MIGRATION_VERSION = 4\nMIGRATION_NAME = "004_import_wi...[truncated]

Explanation:
**Processing synthetic files**

I'm thinking about how to parse synthetic files, specifically as simple zip formats like docx XML and xlsx zip. I need to create synthetic files using zip XML, and I'll implement a parser using zipfile and ElementTree. Also, there are migrations needed to add a new table, version 5, and I'll need data functions. It might be easier to code with execute_code to streamline the process. Let’s see how this unfolds!


Output: migration v5 patched


from __future__ import annotations

import re
import zipfile
from collections import Counter
from dataclasses import dataclass, field
from pathlib import Path
from typing import Iterable
from xml.etre...[truncated]


from pathlib import Path
p=Path('/home/agent/.hermes/repos/FinanceManager/src/jarvis_finance/imports/broker_mapping.py')
s=p.read_text()
# add helpers before get_manual_review_queue
insert=r'''

def _...[truncated]


Output: broker_mapping extended


from pathlib import Path
p=Path('/home/agent/.hermes/repos/FinanceManager/src/jarvis_finance/dashboard/data.py')
s=p.read_text()
# insert safe status funcs before command_center maybe
insert='''\n\nde...[truncated]


Output: dashboard data b2 patched


from __future__ import annotations

from jarvis_finance.dashboard import data
from jarvis_finance.dashboard.components.tables import show_table
from jarvis_finance.imports.broker_mapping import assert...[truncated]


[write_file] wrote to /home/agent/.hermes/repos/FinanceManager/src/jarvis_finance/dashboard/pages/page_14_import_wizard.py (41 lines)


from __future__ import annotations

from jarvis_finance.dashboard import data
from jarvis_finance.dashboard.components.tables import show_table

PAGE_TITLE = "Manual Review Queue"

def render(st, con...[truncated]


[write_file] wrote to /home/agent/.hermes/repos/FinanceManager/src/jarvis_finance/dashboard/pages/page_15_manual_review_queue.py (27 lines)


from __future__ import annotations

from jarvis_finance.dashboard import data
from jarvis_finance.dashboard.components.tables import show_table

PAGE_TITLE = "Data Quality Center"

def render(st, con...[truncated]


[write_file] wrote to /home/agent/.hermes/repos/FinanceManager/src/jarvis_finance/dashboard/pages/page_16_data_quality_center.py (22 lines)


from pathlib import Path
# command center add safe mode display
p=Path('/home/agent/.hermes/repos/FinanceManager/src/jarvis_finance/dashboard/pages/page_00_command_center.py')
s=p.read_text()
s=s.repl...[truncated]


Output: safe mode global patched


Explanation:
**Developing synthetic file builders**

I need to create synthetic file builders for testing purposes and also work on writing the tests. It's important to have a schema test version. I’m considering inspecting existing tests to see if I can reuse any fake data, but that seems okay for now. It feels like a good plan to ensure coverage in my tests! This should help streamline the testing process.


[patch] replace in /home/agent/.hermes/repos/FinanceManager/tests/unit/test_schema.py (832 chars result)


patch

assert {"source_platform", "source_file_type", "source_filename_hash", "detected_snapshot_date", "snapshot_date_status", "candidate_positions", "candidate_cash_rows", "mapped_positions", "blocked_...[truncated]


[patch] replace in /home/agent/.hermes/repos/FinanceManager/tests/unit/test_schema.py (1,523 chars result)


from __future__ import annotations

import zipfile
from pathlib import Path

from jarvis_finance.dashboard import data as dashboard_data
from jarvis_finance.imports.broker_mapping import assert_write_...[truncated]


terminal

set -euo pipefail
cd /home/agent/.hermes/repos/FinanceManager
PYTHONPATH=src pytest tests/unit/test_broker_parser_dry_run_pipeline.py tests/unit/test_broker_mapping_import_wizard.py tests/unit/test_schema.py -q


Output:
.........................                                                [100%]
25 passed in 0.17s


todo

B2-Schema Review Items und Parser/Dry-Run-Pipeline implementieren

Import Wizard, Manual Review Queue, Data Quality Center und globalen Safe-Mode erweitern

Synthetische DOCX/XLSX Parser-Tests und Safety-Tests ergänzen


[todo] updated task list


terminal

set -euo pipefail
cd /home/agent/.hermes/repos/FinanceManager
PYTHONPATH=src python -m compileall -q src tests
PYTHONPATH=src pytest -q


[terminal] ran `set -euo pipefail
cd /home/agent/.hermes/repos/FinanceManager
PYTHONPATH=src ...` -> exit 0, 1 lines output


todo

Compile, pytest, Git-Safety, Commit und Push durchführen


terminal

set -euo pipefail
cd /home/agent/.hermes/repos/FinanceManager
rm -rf .pytest_cache src/jarvis_finance/**/__pycache__ tests/**/__pycache__
PYTHONPATH=src python -m jarvis_finance.cli.main git-safety-scan .
git status --short --branch


[terminal] ran `set -euo pipefail
cd /home/agent/.hermes/repos/FinanceManager
rm -rf .pytest_...` -> exit 0, 1 lines output


from pathlib import Path
from tempfile import TemporaryDirectory
import zipfile, json
from jarvis_finance.storage.database import connect_memory
from jarvis_finance.storage.migrations import apply_mig...[truncated]

Explanation:
**Considering dry run summary**

I think I need to create a sample dry run summary for my report. I should do a synthetic summary and maybe use a temporary setup to run tests. After that, it could be helpful to execute the code and gather the output for aggregation. I’m also wondering if I need to avoid committing changes that aren't really related to siblings—just warnings. Maybe a git diff would help clarify things?


[execute_code] `from pathlib import Path from tempfile import TemporaryDirec...` (1 lines output)


set -euo pipefail
cd /home/agent/.hermes/repos/FinanceManager
PYTHONPATH=src python - <<'PY'
from pathlib import Path
from tempfile import TemporaryDirectory
import zipfile, json
from jarvis_finance.s...[truncated]


set -euo pipefail
cd /home/agent/.hermes/repos/FinanceManager
PYTHONPATH=src python -m compileall -q src tests
PYTHONPATH=src pytest -q
rm -rf .pytest_cache src/jarvis_finance/**/__pycache__ tests/**/__pycache__
PYTHONPATH=src python -m jarvis_finance.cli.main git-safety-scan .
git status --short --branch

Explanation:
**Verifying and pushing changes**

I want to make sure I've double-checked everything after dealing with the pycache files. It's always good to be thorough, you know? Then, I'll go ahead and commit the changes and push them to the repository. I can’t help but feel that it’s important to keep everything organized, so nothing gets lost later on. Let's get this done!


set -euo pipefail
cd /home/agent/.hermes/repos/FinanceManager
git add src/jarvis_finance/storage/migrations.py src/jarvis_finance/imports/broker_mapping.py src/jarvis_finance/imports/broker_parsers.py...[truncated]


[terminal] ran `set -euo pipefail
cd /home/agent/.hermes/repos/FinanceManager
git add src/jar...` -> exit 0, 1 lines output


set -euo pipefail
cd /home/agent/.hermes/repos/FinanceManager
python - <<'PY'
import re, subprocess, sys
from pathlib import Path
repo=Path('/home/agent/.hermes/repos/FinanceManager')
text=Path.home()...[truncated]


Output:
REMOTE_BEFORE=cd2d954
LOCAL_HEAD=fa254a9
PUSH=success
REMOTE_AFTER=fa254a9
GIT_SAFETY_OK
## main...origin/main


Erledigt, Sir. Phase B2 ist implementiert, getestet, committed und gepusht.

## Status

- Commit: `fa254a9`
- Push Status: erfolgreich
- Remote HEAD: `fa254a9`
- Git Status: sauber, `main...origin/main`
- Compile: OK
- Tests: `144 passed`
- Git-Safety: `GIT_SAFETY_OK`
- Produktive Aktien-/ETF-/Cash-Imports: **nein**
- Echte Broker-/Bank-Dateien ins Repo: **nein**
- Echte Werte/Mengen im Chat/Repo: **nein**

## Implementierte Parser

### PostFinance Parser

Datei:

- `src/jarvis_finance/imports/broker_parsers.py`

Unterstützt synthetische DOCX-Tabellen.

Erkennt:

- Positionskandidaten
- Cash-/Account-Kandidaten
- Produkt/Name
- Menge vorhanden
- Währung
- Marktwert vorhanden
- Account-/Depot-Kontext
- Stichtag, falls erkennbar

Quality Flags:

- `missing_isin`
- `missing_ticker`
- `snapshot_only`
- `cost_basis_uncertain`
- `market_value_legacy`
- `needs_manual_review`

### True Wealth Parser

Unterstützt synthetische DOCX-Tabellen mit gemischter Struktur.

Erkennt:

- Cash-Zeilen
- positionsähnliche Zeilen
- Konto-/Portfolio-Kontext
- Währung
- Stichtag, falls erkennbar

Quality Flags:

- `missing_isin`
- `missing_ticker`
- `snapshot_only`
- `cost_basis_uncertain`
- `missing_fx`
- `needs_manual_review`

### Raiffeisen Parser

Unterstützt synthetische XLSX-Konto-/Anlagenübersichten.

Erkennt:

- Cash-/Account-Zeilen
- aggregierte Depot-/Anlagezeilen
- behandelt Raiffeisen defensiv als Cash-/Kontrollsnapshot
- importiert Aggregate **nicht** als Instrumentpositionen

Quality Flags:

- `cash_snapshot_candidate`
- `aggregate_only`
- `missing_instrument_details`
- `needs_manual_review`

## Runtime Review Items

Neue Runtime-/DB-Struktur:

- `broker_import_review_items`

Felder implementiert:

- `review_item_id`
- `dry_run_id`
- `source_platform`
- `source_file_type`
- `source_row_ref`
- `row_hash`
- `source_label`
- `normalized_name`
- `detected_asset_class`
- `detected_currency`
- `detected_quantity_present`
- `detected_market_value_present`
- `isin`
- `ticker`
- `exchange`
- `mapped_instrument_id`
- `mapped_account_id`
- `quality_flags_json`
- `review_status`
- `reviewer_note`
- `created_at`
- `updated_at`

Regeln umgesetzt:

- Details bleiben Runtime-DB-only.
- Keine Review Items in Git.
- Chat-Ausgabe nur aggregiert.
- Manual Review Queue liest Review Items.
- Mapping-Bestätigung für Review Items erzeugt Audit-Log.
- Produktive Positionen werden nicht geschrieben.

## Dry-Run Pipeline

Erweitert in:

- `src/jarvis_finance/imports/broker_mapping.py`

Pipeline-Schritte:

1. Datei parsen
2. Kandidaten normalisieren
3. Instrument-Mapping versuchen
4. Account-Mapping vorbereitet
5. Quality Flags setzen
6. Dry-Run Summary schreiben
7. Review Items schreiben
8. keine produktiven Positionen
9. kein Ledger Commit
10. kein Snapshot Commit

Output:

- `rows_total`
- `candidate_positions`
- `candidate_cash_rows`
- `mapped_positions`
- `blocked_positions`
- `review_items_created`
- `warnings_count`
- `errors_count`
- `quality_flags_summary`

## Import Wizard

Erweitert:

- Parser-/Dry-Run Summary sichtbar
- Quality Flags sichtbar
- Review Items sichtbar
- Link/Hinweis zur Manual Review Queue
- synthetischer Demo-Dry-Run als UI-Konzept vorbereitet
- Commit weiterhin deaktiviert
- Read-only Mode sichtbar

Datei:

- `src/jarvis_finance/dashboard/pages/page_14_import_wizard.py`

Status:

- vorhanden
- B2-fähig
- produktive Commit-Aktionen deaktiviert

## Manual Review Queue

Erweitert:

- zeigt offene Runtime Review Items
- zeigt Quelle
- Source Label
- Normalized Name
- Assetklasse
- Währung
- ISIN
- Ticker
- Exchange
- Account Mapping
- Quality Flags
- Status

Aktionen vorbereitet:

- ISIN ergänzen
- Ticker ergänzen
- Exchange ergänzen
- Instrument Mapping bestätigen
- Account Mapping bestätigen
- Ignore with note
- Mark resolved

Alle Buttons bleiben in Phase B2 deaktiviert / kontrolliert vorbereitet.

Datei:

- `src/jarvis_finance/dashboard/pages/page_15_manual_review_queue.py`

## Data Quality Center

Erweitert:

- offene Review Items nach Quality Flag
- `missing_isin`
- `missing_ticker`
- `snapshot_only`
- `cost_basis_uncertain`
- `missing_fx`
- `aggregate_only`
- stale/missing price alerts weiterhin sichtbar
- resolved/fixed alerts weiterhin sichtbar

Datei:

- `src/jarvis_finance/dashboard/pages/page_16_data_quality_center.py`

## Globaler Safe Mode

Erweitert:

- Command Center zeigt Safe Mode Status.
- Dashboard Sidebar zeigt:
  - DB-Modus: Runtime
  - Write Mode: Read-only
  - letzter Backup-Zeitpunkt, falls verfügbar
  - Hinweis: echte Daten liegen außerhalb Git

Dateien:

- `src/jarvis_finance/dashboard/main.py`
- `src/jarvis_finance/dashboard/pages/page_00_command_center.py`
- `src/jarvis_finance/dashboard/data.py`

Default:

- `Read-only`

## Neue/geänderte Dateien

Neu:

- `src/jarvis_finance/imports/broker_parsers.py`
- `tests/unit/test_broker_parser_dry_run_pipeline.py`

Geändert:

- `src/jarvis_finance/imports/broker_mapping.py`
- `src/jarvis_finance/storage/migrations.py`
- `src/jarvis_finance/dashboard/data.py`
- `src/jarvis_finance/dashboard/main.py`
- `src/jarvis_finance/dashboard/pages/page_00_command_center.py`
- `src/jarvis_finance/dashboard/pages/page_14_import_wizard.py`
- `src/jarvis_finance/dashboard/pages/page_15_manual_review_queue.py`
- `src/jarvis_finance/dashboard/pages/page_16_data_quality_center.py`
- `tests/unit/test_schema.py`

## Neue Tests

Abgedeckt:

- synthetischer PostFinance Parser erkennt Kandidaten
- synthetischer True Wealth Parser erkennt Kandidaten
- synthetischer Raiffeisen Parser erkennt Cash-/Kontrollsnapshot
- Parser erzeugen Quality Flags
- Dry Run schreibt Summary
- Dry Run schreibt Review Items
- Dry Run erzeugt keine produktiven Positionen
- Name ohne ISIN bleibt review-pflichtig
- Raiffeisen `aggregate_only` wird nicht als Instrumentposition importiert
- Manual Review Queue lädt Review Items
- Data Quality Center aggregiert Review Flags
- Safe Mode bleibt default Read-only
- Commit-Aktionen bleiben deaktiviert
- Audit-Log bei Review-Item-Mapping-Bestätigung
- keine echten Broker-Dateien im Repo

Testergebnis:

- `144 passed`

## Synthetische Dry-Run Summary

Aggregierte synthetische Ergebnisse:

### PostFinance

- rows_total: 3
- candidate_positions: 1
- candidate_cash_rows: 1
- mapped_positions: 0
- blocked_positions: 1
- review_items_created: 2
- warnings_count: 8
- errors_count: 0
- Quality Flags:
  - `missing_isin`
  - `missing_ticker`
  - `snapshot_only`
  - `cost_basis_uncertain`
  - `market_value_legacy`
  - `needs_manual_review`

### True Wealth

- rows_total: 3
- candidate_positions: 1
- candidate_cash_rows: 1
- mapped_positions: 0
- blocked_positions: 1
- review_items_created: 2
- warnings_count: 9
- errors_count: 0
- Quality Flags:
  - `missing_isin`
  - `missing_ticker`
  - `snapshot_only`
  - `cost_basis_uncertain`
  - `missing_fx`
  - `needs_manual_review`

### Raiffeisen

- rows_total: 3
- candidate_positions: 1
- candidate_cash_rows: 1
- mapped_positions: 0
- blocked_positions: 1
- review_items_created: 2
- warnings_count: 5
- errors_count: 0
- Quality Flags:
  - `cash_snapshot_candidate`
  - `aggregate_only`
  - `missing_instrument_details`
  - `needs_manual_review`

## Nicht durchgeführt

- Kein produktiver Import echter PostFinance-Daten
- Kein produktiver Import echter True-Wealth-Daten
- Kein produktiver Import echter Raiffeisen-Daten
- Keine produktiven `positions_snapshot` Writes
- Keine Ledger-Commits
- Keine Snapshot-Commits
- Keine echten Broker-/Bank-Dateien ins Repo kopiert
- Keine echten Werte/Mengen im Chat ausgegeben
- Keine Monte-Carlo-, Backtesting-, Rebalancing-, Score- oder Heatmap-Module begonnen

## Nächste Empfehlung

Phase B3 sollte sein:

**Controlled real-file dry-run, weiterhin ohne produktiven Import.**

Reihenfolge:

1. Vorher Runtime-DB Backup erzeugen/verifizieren.
2. Git-Safety vorab prüfen.
3. Echte PostFinance-Datei nur aus Runtime/Temp lesen.
4. Aggregierten Dry Run erzeugen.
5. Review Items in Runtime-DB schreiben.
6. Nur Counts/Quality Flags berichten.
7. Dann True Wealth.
8. Dann Raiffeisen.
9. Temp bereinigen.
10. Git-Safety danach erneut prüfen.

Noch kein produktiver Initial Snapshot, bevor die Review Queue mit echten Dry-Run-Ergebnissen sauber aussieht.


[M G] Sehr gut. Phase B2 ist akzeptiert.

Der Stand ist sauber:

* Commit fa254a9 gepusht
* Tests: 144 passed
* Git-Safety: OK
* Import Wizard vorhanden
* Manual Review Queue vorhanden
* Data Quality Center erweitert
* Globaler Safe Mode sichtbar
* Parser-/Dry-Run-Pipeline für synthetische Quellen vorhanden
* Keine produktiven Aktien-/ETF-/Cash-Imports
* Keine echten Broker-/Bank-Dateien im Repo
* Keine echten Werte/Mengen im Chat

Bitte fahre jetzt mit Phase B3 fort.

# Phase B3 — Controlled Real-File Dry Run

Ziel:
Die echten PostFinance-, True-Wealth- und Raiffeisen-Dateien sollen kontrolliert durch die neue Dry-Run-Pipeline laufen. Es darf weiterhin kein produktiver Initial Snapshot entstehen.

Wichtig:
Dieser Schritt darf Runtime-Review-Items erzeugen, aber keine produktiven Positionen, keine Ledger-Transaktionen und keine finalen Snapshots.

## 1. Reihenfolge

Bitte in dieser Reihenfolge arbeiten:

1. Preflight
2. Runtime-DB Backup erzeugen und verifizieren
3. Git-Safety prüfen
4. PostFinance real-file dry run
5. True Wealth real-file dry run
6. Raiffeisen real-file dry run
7. Review Queue / Data Quality Center prüfen
8. Git-Safety erneut prüfen
9. Ergebnisbericht ohne echte Werte/Mengen

## 2. Preflight

Bitte vorab prüfen:

* Git Status clean
* Git-Safety OK
* Runtime-DB liegt außerhalb Repo
* Backup-/Reports-/Imports-Ordner liegen außerhalb Repo
* keine echten Dateien im Repo
* aktueller Remote HEAD ist synchron
* Safe Mode / Read-only bleibt für produktive Imports aktiv

## 3. Backup vor Real-File Dry Run

Bitte zuerst ausführen:

* finance backup-runtime-db
* finance verify-backup --file <backup-file>

Bitte nur aggregiert berichten:

* Backup erzeugt: ja/nein
* Backup verifiziert: ja/nein
* Checksum OK: ja/nein
* Speicherort: Runtime-Backup-Ordner
* keine Pfade mit sensitiven Dateinamen nötig

## 4. Echte Dateien nur temporär lesen

Für jede Quelle:

* Datei aus Google Drive / Runtime-Kontext temporär lesen
* nicht ins Repo kopieren
* keine echten Werte/Mengen im Chat
* keine echten Werte/Mengen in Docs/Tests/Fixtures
* keine echten Dateien committen
* Temp-Dateien nach Verarbeitung entfernen
* Details nur in Runtime-DB / Review Items

## 5. Dry Run pro Quelle

### A. PostFinance

Bitte echten PostFinance-Auszug durch Parser/Dry Run laufen lassen.

Ziel:

* Positionskandidaten erkennen
* Cash-/Account-Kandidaten erkennen
* Instrument-Mappings versuchen
* fehlende ISIN/Ticker markieren
* Review Items erzeugen
* Dry-Run Summary speichern

Bitte keine produktiven Positionen schreiben.

Bericht nur aggregiert:

* rows_total
* candidate_positions
* candidate_cash_rows
* mapped_positions
* blocked_positions
* review_items_created
* warnings_count
* errors_count
* quality_flags_summary
* snapshot_date_status
* keine echten Namen/Werte/Mengen, außer nicht-sensitive generische Labels

### B. True Wealth

Analog PostFinance.

Besonders prüfen:

* ETF-Positionen
* Cash-/Kontozeilen
* ob ISIN/Ticker erkannt werden
* ob True Wealth eher als einzelne ETF-Positionen oder Plattformstrategie erscheint

Keine produktiven Positionen schreiben.

### C. Raiffeisen

Analog, aber defensiv:

* primär Cash-/Kontrollsnapshot
* aggregierte Depot-/Anlagezeilen nicht als Instrumentpositionen importieren, wenn ISIN/Ticker/Menge fehlen
* Review Items erzeugen, falls unklare Zeilen vorhanden

Keine produktiven Positionen schreiben.

## 6. Review Items

Bitte echte Dry-Run-Ergebnisse in broker_import_review_items speichern, aber nur Runtime-DB.

Regeln:

* jeder Review Item braucht dry_run_id
* source_platform
* source_file_type
* source_row_ref
* row_hash
* source_label
* normalized_name
* detected fields
* quality_flags_json
* review_status = open / blocked / ignored / mapped / resolved

Bitte alte Dry Runs nicht überschreiben.

Falls mehrere Dry Runs derselben Quelle existieren:

* neue Session erzeugen
* ältere Session nicht löschen
* optional Status/Notiz setzen, wenn eine Session ersetzt wurde

Bitte später vorbereiten:

* Dry Run archivieren/verwerfen
* Review Items pro Dry Run filtern

## 7. Parser-Anpassungen

Falls
echte Dateien leichte Parser-Anpassungen benötigen:

* Code darf angepasst werden
* aber nur generisch
* keine echten Daten in Tests/Fixtures
* synthetische Fixtures erweitern, falls ein neuer Strukturfall entdeckt wurde
* echte Werte/Namen nicht in Docs schreiben
* Tests ergänzen
* Commit/Push nur Code und synthetische Tests

## 8. Data Quality Center prüfen

Nach den Dry Runs bitte prüfen:

* Review Items erscheinen im Data Quality Center
* Quality Flags werden aggregiert
* missing_isin sichtbar
* missing_ticker sichtbar
* snapshot_only sichtbar
* cost_basis_uncertain sichtbar
* missing_fx sichtbar, falls vorhanden
* aggregate_only sichtbar, falls vorhanden
* keine produktiven Positionswerte angezeigt

## 9. Manual Review Queue prüfen

Bitte prüfen:

* Review Items sind sichtbar
* Quelle filterbar
* Status sichtbar
* Quality Flags sichtbar
* Mapping-Aktionen weiterhin kontrolliert / produktive Positionserstellung weiterhin deaktiviert
* keine Commit-Aktion für Initial Snapshot aktiv

## 10. Keine produktive Aktion

Bitte jetzt weiterhin nicht:

* keine produktiven Aktien-/ETF-Positionen schreiben
* keine produktiven Cash-Snapshots schreiben
* keine Ledger-Transaktionen aus Brokerdateien schreiben
* keine positions_snapshot aus Brokerdateien schreiben
* keine echten Dateien ins Git
* keine echten Werte/Mengen im Chat
* keine echten Reports mit Aktien-/ETF-Zahlen erzeugen

## 11. Nach Abschluss

Bitte ausführen:

* Compile
* pytest
* Git-Safety
* Git Status

Falls Code geändert wurde:

* Commit mit sinnvoller Message
* Push auf main

Bitte berichten:

* Backup erzeugt/verifiziert
* Git-Safety vor/nachher
* ob Code geändert wurde
* Commit Hash, falls vorhanden
* PostFinance Dry-Run Summary
* True Wealth Dry-Run Summary
* Raiffeisen Dry-Run Summary
* Anzahl Review Items pro Quelle
* Quality Flags pro Quelle
* ob Manual Review Queue funktioniert
* ob Data Quality Center funktioniert
* ob produktive DB-Positionen unverändert blieben
* nächste Empfehlung

Wichtig:
Der Output bleibt aggregate-only. Keine echten Werte, Mengen, Kontostände, Kurse, Dateiinhalte oder sensitiven Labels im Chat.


Explanation:
**Considering actions for finance**

I need to figure out our next steps. I should check if the finance tool is already loaded, which might save time. Skills are important here, and maybe I need to look into a GitHub workflow too. I think it’s essential to inspect my commands for clarity and ensure everything is set. Let’s keep things organized and make sure I’m following a structured approach for whatever task we’re handling!


[skill_view] name=finance-system-architecture (24,906 chars)


[skill_view] name=finance-system-architecture (5,838 chars)


todo

Preflight: Git, Git-Safety, Runtime-Pfade, Remote-Sync, Safe Mode prüfen

Runtime-DB Backup erzeugen und verifizieren

Echte PostFinance/True Wealth/Raiffeisen Dateien temporär finden und Dry-Runs ausführen

Review Queue/Data Quality prüfen und produktive Positionen unverändert verifizieren

Compile, pytest, Git-Safety, Git Status, optional Commit/Push falls Codeänderungen

Explanation:
**Searching for CLI backup commands**

I need to get started on running search queries to find the command line interface (CLI) backup commands. It sounds like a straightforward task, but I'm making sure to cover my bases and not miss any critical commands. I'm actually curious to see the variety of commands available for backups in CLI, as that might help me understand the best practices. Let’s dig into it and see what I can find!


[todo] updated task list


[search_files] files search for '*.py' in /home/agent/.hermes/repos/FinanceManager -> 150 matches


[read_file] read /home/agent/.hermes/repos/FinanceManager/src/jarvis_finance/config.py from line 1 (370 chars)


     1|from __future__ import annotations
     2|
     3|import hashlib
     4|import shutil
     5|from dataclasses import dataclass
     6|from datetime import datetime, timezone
     7|from pathlib import Path
     8|
     9|from jarvis_finance.config.paths import is_relative_to
    10|
    11|
    12|@dataclass(frozen=True)
    13|class BackupResult:
    14|    backup_file: Path
    15|    checksum_file: Path
    16|    sha256: str
    17|    size_bytes: int
    18|
    19|
    20|@dataclass(frozen=True)
    21|class VerifyBackupResult:
    22|    backup_file: Path
    23|    checksum_file: Path
    24|    sha256: str
    25|    checksum_ok: bool
    26|    exists: bool
    27|
    28|
    29|@dataclass(frozen=True)
    30|class RestoreResult:
    31|    restored_to: Path
    32|    source_file: Path
    33|    verified: bool
    34|
    35|
    36|def sha256_file(path: Path) -> str:
    37|    digest = hashlib.sha256()
    38|    with path.open("rb") as handle:
    39|        for chunk in iter(lambda: handle.read(1024 * 1024), b""):
    40|            digest.update(chunk)
    41|    return digest.hexdigest()
    42|
    43|
    44|def write_checksum_file(backup_file: Path, sha256: str) -> Path:
    45|    checksum_file = backup_file.with_suffix(backup_file.suffix + ".sha256")
    46|    checksum_file.write_text(f"{sha256}  {backup_file.name}\n", encoding="utf-8")
    47|    return checksum_file
    48|
    49|
    50|def read_expected_checksum(checksum_file: Path) -> str | None:
    51|    if not checksum_file.exists():
    52|        return None
    53|    text = checksum_file.read_text(encoding="utf-8", errors="ignore").strip()
    54|    if not text:
    55|        return None
    56|    return text.split()[0]
    57|
    58|
    59|def ensure_outside_repo(path: Path, repo_root: Path) -> None:
    60|    if is_relative_to(path.resolve(), repo_root.resolve()):
    61|        raise ValueError(f"Backup/restore path must be outside the Git repository: {path}")
    62|
    63|
    64|def backup_runtime_db(*, db_path: Path, backups_dir: Path, repo_root: Path, timestamp: str | None = None) -> BackupResult:
    65|    db_path = db_path.expanduser().resolve()
    66|    backups_dir = backups_dir.expanduser().resolve()
    67|    ensure_outside_repo(backups_dir, repo_root)
    68|    ensure_outside_repo(db_path, repo_root)
    69|    if not db_path.exists():
    70|        raise FileNotFoundError(f"Runtime DB not found: {db_path}")
    71|    backups_dir.mkdir(parents=True, exist_ok=True)
    72|    stamp = timestamp or datetime.now(timezone.utc).strftime("%Y%m%dT%H%M%SZ")
    73|    backup_file = backups_dir / f"finance-runtime-db-{stamp}.sqlite3"
    74|    shutil.copy2(db_path, backup_file)
    75|    checksum = sha256_file(backup_file)
    76|    checksum_file = write_checksum_file(backup_file, checksum)
    77|    return BackupResult(
    78|        backup_file=backup_file,
    79|        checksum_file=checksum_file,
    80|        sha256=checksum,
    81|        size_bytes=backup_file.stat().st_size,
    82|    )
    83|
    84|
    85|def verify_backup(*, backup_file: Path, repo_root: Path | None = None) -> VerifyBackupResult:
    86|    backup_file = backup_file.expanduser().resolve()
    87|    if repo_root is not None:
    88|        ensure_outside_repo(backup_file, repo_root)
    89|    exists = backup_file.exists() and backup_file.is_file()
    90|    if not exists:
    91|        return VerifyBackupResult(
    92|            backup_file=backup_file,
    93|            checksum_file=backup_file.with_suffix(backup_file.suffix + ".sha256"),
    94|            sha256="",
    95|            checksum_ok=False,
    96|            exists=False,
    97|        )
    98|    actual = sha256_file(backup_file)
    99|    checksum_file = backup_file.with_suffix(backup_file.suffix + ".sha256")
   100|    expected = read_expected_checksum(checksum_file)
   101|    return VerifyBackupResult(
   102|        backup_file=backup_file,
   103|        checksum_file=checksum_file,
   104|        sha256=actual,
   105|        checksum_ok=bool(expected and expected == actual),
   106|        exists=True,
   107|    )
   108|
   109|
   110|def restore_runtime_db(*, backup_file: Path, db_path: Path, repo_root: Path, yes: bool = False) -> RestoreResult:
   111|    if not yes:
   112|        raise PermissionError("Restore requires explicit confirmation via --yes")
   113|    backup_file = backup_file.expanduser().resolve()
   114|    db_path = db_path.expanduser().resolve()
   115|    ensure_outside_repo(backup_file, repo_root)
   116|    ensure_outside_repo(db_path, repo_root)
   117|    verification = verify_backup(backup_file=backup_file, repo_root=repo_root)
   118|    if not verification.exists:
   119|        raise FileNotFoundError(f"Backup file not found: {backup_file}")
   120|    if not verification.checksum_ok:
   121|        raise ValueError(f"Backup checksum verification failed: {backup_file}")
   122|    db_path.parent.mkdir(parents=True, exist_ok=True)
   123|    shutil.copy2(backup_file, db_path)
   124|    return RestoreResult(restored_to=db_path, source_file=backup_file, verified=True)
   125|


     1|from __future__ import annotations
     2|
     3|import argparse
     4|from pathlib import Path
     5|
     6|from jarvis_finance.config.settings import ensure_runtime_dirs, load_settings
     7|from jarvis_finance.market.providers import CoinGeckoClient, refresh_crypto_prices
     8|from jarvis_finance.quality.data_quality import check_crypto_data_quality
     9|from jarvis_finance.quality.git_safety import assert_safe, scan_path
    10|from jarvis_finance.runtime.backup import backup_runtime_db, restore_runtime_db, verify_backup
    11|from jarvis_finance.storage.database import connect
    12|from jarvis_finance.storage.migrations import apply_migrations, get_schema_version
    13|
    14|
    15|def main(argv: list[str] | None = None) -> int:
    16|    parser = argparse.ArgumentParser(prog="finance")
    17|    sub = parser.add_subparsers(dest="command", required=True)
    18|    sub.add_parser("init-db")
    19|    sub.add_parser("migrate")
    20|    scan = sub.add_parser("git-safety-scan")
    21|    scan.add_argument("path", nargs="?", default=".")
    22|    prices = sub.add_parser("update-crypto-prices")
    23|    prices.add_argument("--currency", default="CHF")
    24|    prices.add_argument("--max-age-seconds", type=int, default=3600)
    25|    prices.add_argument("--sleep-seconds", type=float, default=0.0)
    26|    prices.add_argument("--max-retries", type=int, default=2)
    27|    prices.add_argument("--initial-backoff", type=float, default=1.0)
    28|    prices.add_argument("--max-backoff", type=float, default=8.0)
    29|    prices.add_argument("--only-missing", action="store_true")
    30|    prices.add_argument("--only-stale", action="store_true")
    31|    prices.add_argument("--only-symbol")
    32|    prices.add_argument("--limit", type=int)
    33|    prices.add_argument("--dry-run", action="store_true")
    34|    dq = sub.add_parser("check-data-quality")
    35|    dq.add_argument("--scope", choices=["crypto"], default="crypto")
    36|    dq.add_argument("--resolve-fixed", action="store_true")
    37|    dq.add_argument("--dry-run", action="store_true")
    38|    dq.add_argument("--currency", default="CHF")
    39|    dq.add_argument("--max-age-seconds", type=int, default=86_400)
    40|    sub.add_parser("backup-runtime-db")
    41|    restore = sub.add_parser("restore-runtime-db")
    42|    restore.add_argument("--from", dest="backup_file", required=True)
    43|    restore.add_argument("--yes", action="store_true")
    44|    verify = sub.add_parser("verify-backup")
    45|    verify.add_argument("--file", required=True)
    46|    args = parser.parse_args(argv)
    47|
    48|    if args.command in {"init-db", "migrate"}:
    49|        settings = load_settings()
    50|        ensure_runtime_dirs(settings)
    51|        conn = connect(settings.db_path)
    52|        apply_migrations(conn)
    53|        print(f"SCHEMA_VERSION={get_schema_version(conn)}")
    54|        return 0
    55|    if args.command == "git-safety-scan":
    56|        findings = scan_path(Path(args.path))
    57|        if findings:
    58|            for finding in findings:
    59|                print(f"{finding.path}: {finding.reason}")
    60|            return 1
    61|        assert_safe(Path(args.path))
    62|        print("GIT_SAFETY_OK")
    63|        return 0
    64|    if args.command == "update-crypto-prices":
    65|        settings = load_settings()
    66|        ensure_runtime_dirs(settings)
    67|        conn = connect(settings.db_path)
    68|        try:
    69|            apply_migrations(conn)
    70|            result = refresh_crypto_prices(
    71|                conn,
    72|                provider=CoinGeckoClient(
    73|                    max_retries=args.max_retries,
    74|                    initial_backoff_seconds=args.initial_backoff,
    75|                    max_backoff_seconds=args.max_backoff,
    76|                ),
    77|                currency=args.currency,
    78|                max_age_seconds=args.max_age_seconds,
    79|                only_missing=args.only_missing,
    80|                only_stale=args.only_stale,
    81|                only_symbol=args.only_symbol,
    82|                limit=args.limit,
    83|                dry_run=args.dry_run,
    84|                sleep_seconds=args.sleep_seconds,
    85|            )
    86|            print(
    87|                "CRYPTO_PRICE_UPDATE "
    88|                f"currency={result.currency} total_assets={result.total_assets} "
    89|                f"updated={result.updated_count} cached={result.cached_count} "
    90|                f"skipped={result.skipped_count} stale={result.stale_count} "
    91|                f"warnings={result.warning_count} errors={result.error_count} "
    92|                f"assets_still_missing_local_price={result.missing_local_price_count} "
    93|                f"dry_run={str(result.dry_run).lower()}"
    94|            )
    95|            return 1 if result.error_count else 0
    96|        finally:
    97|            conn.close()
    98|    if args.command == "check-data-quality":
    99|        settings = load_settings()
   100|        ensure_runtime_dirs(settings)
   101|        conn = connect(settings.db_path)
   102|        try:
   103|            apply_migrations(conn)
   104|            result = check_crypto_data_quality(
   105|                conn,
   106|                currency=args.currency,
   107|                max_age_seconds=args.max_age_seconds,
   108|                resolve_fixed=args.resolve_fixed,
   109|                dry_run=args.dry_run,
   110|            )
   111|            print(
   112|                "DATA_QUALITY_CHECK "
   113|                f"scope={result.scope} active_crypto_assets={result.active_crypto_assets} "
   114|                f"fresh_price_assets={result.fresh_price_assets} "
   115|                f"active_stale_price_alerts_before={result.active_stale_price_alerts_before} "
   116|                f"active_stale_price_alerts_after={result.active_stale_price_alerts_after} "
   117|                f"resolved_alerts={result.resolved_alerts} "
   118|                f"warnings={result.warnings} errors={result.errors} "
   119|                f"dry_run={str(result.dry_run).lower()}"
   120|            )
   121|            return 1 if result.errors else 0
   122|        finally:
   123|            conn.close()
   124|    if args.command == "backup-runtime-db":
   125|        settings = load_settings()
   126|        ensure_runtime_dirs(settings)
   127|        result = backup_runtime_db(
   128|            db_path=settings.db_path,
   129|            backups_dir=settings.runtime_paths.backups_dir,
   130|            repo_root=Path.cwd(),
   131|        )
   132|        print(
   133|            "RUNTIME_DB_BACKUP "
   134|            f"backup_file={result.backup_file} checksum_file={result.checksum_file} "
   135|            f"sha256={result.sha256} size_bytes={result.size_bytes}"
   136|        )
   137|        return 0
   138|    if args.command == "verify-backup":
   139|        result = verify_backup(backup_file=Path(args.file), repo_root=Path.cwd())
   140|        print(
   141|            "BACKUP_VERIFY "
   142|            f"backup_file={result.backup_file} checksum_file={result.checksum_file} "
   143|            f"exists={str(result.exists).lower()} checksum_ok={str(result.checksum_ok).lower()} "
   144|            f"sha256={result.sha256}"
   145|        )
   146|        return 0 if result.exists and result.checksum_ok else 1
   147|    if args.command == "restore-runtime-db":
   148|        settings = load_settings()
   149|        ensure_runtime_dirs(settings)
   150|        result = restore_runtime_db(
   151|            backup_file=Path(args.backup_file),
   152|            db_path=settings.db_path,
   153|            repo_root=Path.cwd(),
   154|            yes=args.yes,
   155|        )
   156|        print(
   157|            "RUNTIME_DB_RESTORE "
   158|            f"source_file={result.source_file} restored_to={result.restored_to} "
   159|            f"verified={str(result.verified).lower()}"
   160|        )
   161|        return 0
   162|    return 2
   163|
   164|
   165|if __name__ == "__main__":
   166|    raise SystemExit(main())
   167|


     1|from __future__ import annotations
     2|
     3|from dataclasses import dataclass
     4|from pathlib import Path
     5|
     6|DEFAULT_RUNTIME_DIR = Path("~/jarvis_runtime/finance-system").expanduser()
     7|
     8|
     9|@dataclass(frozen=True)
    10|class RuntimePaths:
    11|    base_dir: Path
    12|    data_dir: Path
    13|    imports_dir: Path
    14|    exports_dir: Path
    15|    reports_dir: Path
    16|    backups_dir: Path
    17|    secrets_dir: Path
    18|    logs_dir: Path
    19|    db_path: Path
    20|
    21|    @classmethod
    22|    def from_base(cls, base_dir: str | Path = DEFAULT_RUNTIME_DIR) -> "RuntimePaths":
    23|        base = Path(base_dir).expanduser().resolve()
    24|        return cls(
    25|            base_dir=base,
    26|            data_dir=base / "data",
    27|            imports_dir=base / "imports",
    28|            exports_dir=base / "exports",
    29|            reports_dir=base / "reports",
    30|            backups_dir=base / "backups",
    31|            secrets_dir=base / "secrets",
    32|            logs_dir=base / "logs",
    33|            db_path=base / "data" / "finance.sqlite3",
    34|        )
    35|
    36|    def ensure_dirs(self) -> None:
    37|        for path in [
    38|            self.data_dir,
    39|            self.imports_dir,
    40|            self.exports_dir,
    41|            self.reports_dir,
    42|            self.backups_dir,
    43|            self.secrets_dir,
    44|            self.logs_dir,
    45|        ]:
    46|            path.mkdir(parents=True, exist_ok=True)
    47|
    48|
    49|def is_relative_to(path: Path, parent: Path) -> bool:
    50|    try:
    51|        path.resolve().relative_to(parent.resolve())
    52|        return True
    53|    except ValueError:
    54|        return False
    55|


     1|from __future__ import annotations
     2|
     3|import os
     4|from dataclasses import dataclass
     5|from pathlib import Path
     6|
     7|from .paths import DEFAULT_RUNTIME_DIR, RuntimePaths, is_relative_to
     8|
     9|
    10|@dataclass(frozen=True)
    11|class Settings:
    12|    env: str = "local"
    13|    base_currency: str = "CHF"
    14|    timezone: str = "Europe/Zurich"
    15|    runtime_paths: RuntimePaths = RuntimePaths.from_base(DEFAULT_RUNTIME_DIR)
    16|    log_level: str = "INFO"
    17|    block_runtime_inside_repo: bool = True
    18|
    19|    @property
    20|    def db_path(self) -> Path:
    21|        return self.runtime_paths.db_path
    22|
    23|
    24|def find_repo_root(start: Path | None = None) -> Path:
    25|    current = (start or Path.cwd()).resolve()
    26|    for candidate in [current, *current.parents]:
    27|        if (candidate / ".git").exists() or (candidate / "pyproject.toml").exists():
    28|            return candidate
    29|    return current
    30|
    31|
    32|def load_settings(repo_root: Path | None = None, environ: dict[str, str] | None = None) -> Settings:
    33|    env = environ or os.environ
    34|    runtime_dir = Path(env.get("JARVIS_FINANCE_RUNTIME_DIR", str(DEFAULT_RUNTIME_DIR))).expanduser()
    35|    paths = RuntimePaths.from_base(runtime_dir)
    36|    db_override = env.get("JARVIS_FINANCE_DB_PATH")
    37|    if db_override:
    38|        object.__setattr__(paths, "db_path", Path(db_override).expanduser().resolve())
    39|    settings = Settings(
    40|        env=env.get("JARVIS_FINANCE_ENV", "local"),
    41|        base_currency=env.get("JARVIS_FINANCE_BASE_CURRENCY", "CHF"),
    42|        timezone=env.get("JARVIS_FINANCE_TIMEZONE", "Europe/Zurich"),
    43|        runtime_paths=paths,
    44|        log_level=env.get("JARVIS_FINANCE_LOG_LEVEL", "INFO"),
    45|        block_runtime_inside_repo=env.get("JARVIS_FINANCE_BLOCK_RUNTIME_INSIDE_REPO", "1") not in {"0", "false", "False"},
    46|    )
    47|    validate_settings(settings, repo_root=repo_root)
    48|    return settings
    49|
    50|
    51|def validate_settings(settings: Settings, repo_root: Path | None = None) -> None:
    52|    root = (repo_root or find_repo_root()).resolve()
    53|    if settings.block_runtime_inside_repo and is_relative_to(settings.runtime_paths.base_dir, root):
    54|        raise ValueError(f"Runtime directory must be outside the Git repository: {settings.runtime_paths.base_dir}")
    55|    if settings.base_currency != "CHF":
    56|        raise ValueError("MVP 1 base currency must be CHF")
    57|
    58|
    59|def ensure_runtime_dirs(settings: Settings) -> None:
    60|    settings.runtime_paths.ensure_dirs()
    61|


Explanation:
**Searching for files**

I need to find real files, perhaps on Google Drive using a specific tool called ‘gog.’ I'll want to check if I can search within my memory or local runtime imports. Maybe files are in the drive mounted? I might consider using the terminal to find them. My search may pattern the files under runtime, but I need to avoid listing any sensitive names in the final output, even if the tool output may include them. Let's search!


terminal

set -euo pipefail
cd /home/agent/.hermes/repos/FinanceManager
python - <<'PY'
from pathlib import Path
import os, subprocess, re, json
repo=Path.cwd().resolve()
runtime=Path('/home/agent/jarvis_runtime/finance-system').resolve()
paths={
 'runtime_db': runtime/'data'/'finance.sqlite3',
 'backups': runtime/'backups',
 'reports': runtime/'reports',
 'imports': runtime/'imports',
}
def outside(p):
    try: p.resolve().relative_to(repo); return False
    except ValueError: return True
status=subprocess.check_output(['git','status','--short','--branch'], text=True).strip()
head=subprocess.check_output(['git','rev-parse','--short','HEAD'], text=True).strip()
origin=subprocess.check_output(['git','rev-parse','--short','origin/main'], text=True).strip()
base=subprocess.check_output(['git','merge-base','HEAD','origin/main'], text=True).strip()
local_full=subprocess.check_output(['git','rev-parse','HEAD'], text=True).strip()
origin_full=subprocess.check_output(['git','rev-parse','origin/main'], text=True).strip()
forbidden=[]
for suffix in ('.xlsx','.docx','.pdf'):
    forbidden.extend([str(p.relative_to(repo)) for p in repo.rglob('*'+suffix) if p.is_file()])
print(json.dumps({
 'git_clean': status=='## main...origin/main',
 'git_status': status,
 'head': head,
 'origin': origin,
 'remote_sync': local_full==origin_full,
 'runtime_db_exists': paths['runtime_db'].exists(),
 'runtime_paths_outside_repo': {k: outside(v) for k,v in paths.items()},
 'real_files_in_repo_count': len(forbidden),
 'safe_mode_default': 'Read-only',
}, sort_keys=True))
PY
PYTHONPATH=src python -m jarvis_finance.cli.main git-safety-scan .

Explanation:
**Checking preflight script**

I need to run a preflight check, and I’m thinking about using a terminal script to aggregate things. Oh, should I not use sed in this case? It seems like a terminal is necessary for git operations. I also need to verify the remote sync, possibly using a token, but I’m not sure if I should output the token. It feels important to double-check this process to ensure everything goes smoothly!


Output:
{"git_clean": true, "git_status": "## main...origin/main", "head": "fa254a9", "origin": "fa254a9", "real_files_in_repo_count": 0, "remote_sync": true, "runtime_db_exists": true, "runtime_paths_outside_repo": {"backups": true, "imports": true, "reports": true, "runtime_db": true}, "safe_mode_default": "Read-only"}
GIT_SAFETY_OK


set -euo pipefail
cd /home/agent/.hermes/repos/FinanceManager
backup_out=$(PYTHONPATH=src python -m jarvis_finance.cli.main backup-runtime-db)
printf '%s\n' "$backup_out" | python - <<'PY'
import sys,re,json
text=sys.stdin.read()
m=re.search(r'backup_file=(\S+)', text)
print(json.dumps({'backup_created': bool(m), 'backup_dir':'Runtime-Backup-Ordner', 'raw_has_checksum':'sha256=' in text}, sort_keys=True))
if m:
    open('/tmp/finance_b3_backup_path.txt','w').write(m.group(1))
PY
backup_file=$(cat /tmp/finance_b3_backup_path.txt)
verify_out=$(PYTHONPATH=src python -m jarvis_finance.cli.main verify-backup --file "$backup_file")
printf '%s\n' "$verify_out" | python - <<'PY'
import sys,re,json
text=sys.stdin.read()
print(json.dumps({'backup_verified':'exists=true' in text, 'checksum_ok':'checksum_ok=true' in text, 'storage':'Runtime-Backup-Ordner'}, sort_keys=True))
PY


Output:
{"backup_created": false, "backup_dir": "Runtime-Backup-Ordner", "raw_has_checksum": false}
cat: /tmp/finance_b3_backup_path.txt: Datei oder Verzeichnis nicht gefunden


set -euo pipefail
cd /home/agent/.hermes/repos/FinanceManager
PYTHONPATH=src python -m jarvis_finance.cli.main backup-runtime-db > /tmp/finance_b3_backup_out.txt
python - <<'PY'
from pathlib import Path
import re,json
text=Path('/tmp/finance_b3_backup_out.txt').read_text()
m=re.search(r'backup_file=(\S+)', text)
print(json.dumps({'backup_created': bool(m), 'backup_dir':'Runtime-Backup-Ordner', 'raw_has_checksum':'sha256=' in text}, sort_keys=True))
if not m: raise SystemExit(1)
Path('/tmp/finance_b3_backup_path.txt').write_text(m.group(1))
PY
backup_file=$(cat /tmp/finance_b3_backup_path.txt)
PYTHONPATH=src python -m jarvis_finance.cli.main verify-backup --file "$backup_file" > /tmp/finance_b3_verify_out.txt
python - <<'PY'
from pathlib import Path
import json
text=Path('/tmp/finance_b3_verify_out.txt').read_text()
print(json.dumps({'backup_verified':'exists=true' in text, 'checksum_ok':'checksum_ok=true' in text, 'storage':'Runtime-Backup-Ordner'}, sort_keys=True))
PY

Explanation:
**Evaluating input method**

It looks like there was an issue with how we piped things; the here-doc syntax is consuming stdin, which isn't what I intended. I definitely need to rerun the process, but it seems like I don’t really have a backup. Oops, I shouldn't have assumed that step would work seamlessly! So, it’s clear that using a temporary file might be the best way to handle this situation and ensure everything goes smoothly.


Output:
{"backup_created": true, "backup_dir": "Runtime-Backup-Ordner", "raw_has_checksum": true}
{"backup_verified": true, "checksum_ok": true, "storage": "Runtime-Backup-Ordner"}


set -euo pipefail
cd /home/agent/.hermes/repos/FinanceManager
PYTHONPATH=src python - <<'PY'
from pathlib import Path
from tempfile import TemporaryDirectory
import shutil, json, sqlite3
from jarvis_finance.config.settings import load_settings, ensure_runtime_dirs
from jarvis_finance.storage.database import connect
from jarvis_finance.storage.migrations import apply_migrations
from jarvis_finance.imports.broker_parsers import parse_postfinance_docx, parse_true_wealth_docx, parse_raiffeisen_xlsx
from jarvis_finance.imports.broker_mapping import run_broker_import_dry_run

settings=load_settings(); ensure_runtime_dirs(settings)
conn=connect(settings.db_path); apply_migrations(conn)

def count(table):
    try:
        return conn.execute(f'SELECT COUNT(*) AS n FROM {table}').fetchone()['n']
    except sqlite3.Error:
        return None
before={t:count(t) for t in ['positions_snapshot','transactions','cash_ledger','broker_import_dry_runs','broker_import_review_items']}
base=Path('/home/agent/jarvis_runtime/finance-system/imports/broker_mapping_analysis_20260515T082614Z')
sources=[
 ('PostFinance', base/'260514_Portfolio_Postfinance.docx', parse_postfinance_docx),
 ('True Wealth', base/'260514_Portfolio_True_Wealth.docx', parse_true_wealth_docx),
 ('Raiffeisen', base/'260514_uebersicht_alle_Anlagen.xlsx', parse_raiffeisen_xlsx),
]
out=[]
with TemporaryDirectory(prefix='finance_b3_real_dryrun_') as tmp:
    tmp=Path(tmp)
    for source,path,parser in sources:
        if not path.exists():
            out.append({'source':source,'file_present':False,'errors_count':1})
            continue
        temp_path=tmp / f'{source.lower().replace(" ","_")}{path.suffix.lower()}'
        shutil.copy2(path,temp_path)
        parsed=parser(temp_path)
        summary=run_broker_import_dry_run(conn, parsed, source_filename=f'{source.lower().replace(" ","_")}_runtime_file{path.suffix.lower()}')
        conn.commit()
        out.append({
            'source': source,
            'file_present': True,
            'rows_total': summary['rows_total'],
            'candidate_positions': summary['candidate_positions'],
            'candidate_cash_rows': summary['candidate_cash_rows'],
            'mapped_positions': summary['mapped_positions'],
            'blocked_positions': summary['blocked_positions'],
            'review_items_created': summary['review_items_created'],
            'warnings_count': summary['warnings_count'],
            'errors_count': summary['errors_count'],
            'snapshot_date_status': parsed.snapshot_date_status,
            'quality_flags_summary': summary['quality_flags_summary'],
        })
after={t:count(t) for t in before}
review_by_source={}
for row in conn.execute("SELECT source_platform, COUNT(*) AS n FROM broker_import_review_items WHERE dry_run_id IN (SELECT dry_run_id FROM broker_import_dry_runs ORDER BY created_at DESC LIMIT 3) GROUP BY source_platform").fetchall():
    review_by_source[row['source_platform']]=row['n']
flag_by_source={}
for row in conn.execute("SELECT source_platform, quality_flags_json FROM broker_import_review_items WHERE dry_run_id IN (SELECT dry_run_id FROM broker_import_dry_runs ORDER BY created_at DESC LIMIT 3)").fetchall():
    import json as _json
    d=flag_by_source.setdefault(row['source_platform'], {})
    for flag in _json.loads(row['quality_flags_json'] or '[]'):
        d[flag]=d.get(flag,0)+1
print(json.dumps({'dry_runs':out,'before_counts':before,'after_counts':after,'review_items_last_session':review_by_source,'review_flags_last_session':flag_by_source,'productive_positions_unchanged':before.get('positions_snapshot')==after.get('positions_snapshot'),'transactions_unchanged':before.get('transactions')==after.get('transactions'),'cash_ledger_unchanged':before.get('cash_ledger')==after.get('cash_ledger')}, ensure_ascii=False, sort_keys=True))
PY > /tmp/finance_b3_real_dryrun_summary.json
python -m json.tool /tmp/finance_b3_real_dryrun_summary.json


Output:
/usr/bin/bash: Zeile 133: Warnung: Das in der Zeile 70 beginnende Here-Dokument geht bis zum Dateiende (erwartet wird »PY«).
  File "<stdin>", line 62
    PY > /tmp/finance_b3_real_dryrun_summary.json
         ^
SyntaxError: invalid syntax


set -euo pipefail
cd /home/agent/.hermes/repos/FinanceManager
PYTHONPATH=src python - > /tmp/finance_b3_real_dryrun_summary.json <<'PY'
from pathlib import Path
from tempfile import TemporaryDirectory
import shutil, json, sqlite3
from jarvis_finance.config.settings import load_settings, ensure_runtime_dirs
from jarvis_finance.storage.database import connect
from jarvis_finance.storage.migrations import apply_migrations
from jarvis_finance.imports.broker_parsers import parse_postfinance_docx, parse_true_wealth_docx, parse_raiffeisen_xlsx
from jarvis_finance.imports.broker_mapping import run_broker_import_dry_run

settings=load_settings(); ensure_runtime_dirs(settings)
conn=connect(settings.db_path); apply_migrations(conn)

def count(table):
    try:
        return conn.execute(f'SELECT COUNT(*) AS n FROM {table}').fetchone()['n']
    except sqlite3.Error:
        return None
before={t:count(t) for t in ['positions_snapshot','transactions','cash_ledger','broker_import_dry_runs','broker_import_review_items']}
base=Path('/home/agent/jarvis_runtime/finance-system/imports/broker_mapping_analysis_20260515T082614Z')
sources=[
 ('PostFinance', base/'260514_Portfolio_Postfinance.docx', parse_postfinance_docx),
 ('True Wealth', base/'260514_Portfolio_True_Wealth.docx', parse_true_wealth_docx),
 ('Raiffeisen', base/'260514_uebersicht_alle_Anlagen.xlsx', parse_raiffeisen_xlsx),
]
out=[]
created_dry_run_ids=[]
with TemporaryDirectory(prefix='finance_b3_real_dryrun_') as tmp:
    tmp=Path(tmp)
    for source,path,parser in sources:
        if not path.exists():
            out.append({'source':source,'file_present':False,'errors_count':1})
            continue
        temp_path=tmp / f'{source.lower().replace(" ","_")}{path.suffix.lower()}'
        shutil.copy2(path,temp_path)
        parsed=parser(temp_path)
        summary=run_broker_import_dry_run(conn, parsed, source_filename=f'{source.lower().replace(" ","_")}_runtime_file{path.suffix.lower()}')
        conn.commit()
        created_dry_run_ids.append(summary['dry_run_id'])
        out.append({
            'source': source,
            'file_present': True,
            'rows_total': summary['rows_total'],
            'candidate_positions': summary['candidate_positions'],
            'candidate_cash_rows': summary['candidate_cash_rows'],
            'mapped_positions': summary['mapped_positions'],
            'blocked_positions': summary['blocked_positions'],
            'review_items_created': summary['review_items_created'],
            'warnings_count': summary['warnings_count'],
            'errors_count': summary['errors_count'],
            'snapshot_date_status': parsed.snapshot_date_status,
            'quality_flags_summary': summary['quality_flags_summary'],
        })
after={t:count(t) for t in before}
review_by_source={}
flag_by_source={}
if created_dry_run_ids:
    placeholders=','.join(['?']*len(created_dry_run_ids))
    for row in conn.execute(f"SELECT source_platform, COUNT(*) AS n FROM broker_import_review_items WHERE dry_run_id IN ({placeholders}) GROUP BY source_platform", created_dry_run_ids).fetchall():
        review_by_source[row['source_platform']]=row['n']
    for row in conn.execute(f"SELECT source_platform, quality_flags_json FROM broker_import_review_items WHERE dry_run_id IN ({placeholders})", created_dry_run_ids).fetchall():
        d=flag_by_source.setdefault(row['source_platform'], {})
        for flag in json.loads(row['quality_flags_json'] or '[]'):
            d[flag]=d.get(flag,0)+1
print(json.dumps({'dry_runs':out,'before_counts':before,'after_counts':after,'review_items_last_session':review_by_source,'review_flags_last_session':flag_by_source,'productive_positions_unchanged':before.get('positions_snapshot')==after.get('positions_snapshot'),'transactions_unchanged':before.get('transactions')==after.get('transactions'),'cash_ledger_unchanged':before.get('cash_ledger')==after.get('cash_ledger')}, ensure_ascii=False, sort_keys=True))
PY
python -m json.tool /tmp/finance_b3_real_dryrun_summary.json


Output:
{
    "after_counts": {
        "broker_import_dry_runs": 3,
        "broker_import_review_items": 59,
        "cash_ledger": null,
        "positions_snapshot": 0,
        "transactions": 0
    },
    "before_counts": {
        "broker_import_dry_runs": 0,
        "broker_import_review_items": 0,
        "cash_ledger": null,
        "positions_snapshot": 0,
        "transactions": 0
    },
    "cash_ledger_unchanged": true,
    "dry_runs": [
        {
            "blocked_positions": 33,
            "candidate_cash_rows": 1,
            "candidate_positions": 33,
            "errors_count": 0,
            "file_present": true,
            "mapped_positions": 0,
            "quality_flags_summary": {
                "cost_basis_uncertain": 33,
                "missing_isin": 33,
                "missing_ticker": 33,
                "needs_manual_review": 34,
                "snapshot_only": 34
            },
            "review_items_created": 34,
            "rows_total": 40,
            "snapshot_date_status": "missing",
            "source": "PostFinance",
            "warnings_count": 167
        },
        {
            "blocked_positions": 17,
            "candidate_cash_rows": 1,
            "candidate_positions": 17,
            "errors_count": 0,
            "file_present": true,
            "mapped_positions": 0,
            "quality_flags_summary": {
                "cost_basis_uncertain": 17,
                "missing_isin": 17,
                "missing_ticker": 17,
                "needs_manual_review": 18,
                "snapshot_only": 18
            },
            "review_items_created": 18,
            "rows_total": 19,
            "snapshot_date_status": "missing",
            "source": "True Wealth",
            "warnings_count": 87
        },
        {
            "blocked_positions": 4,
            "candidate_cash_rows": 3,
            "candidate_positions": 4,
            "errors_count": 0,
            "file_present": true,
            "mapped_positions": 0,
            "quality_flags_summary": {
                "aggregate_only": 4,
                "cash_snapshot_candidate": 3,
                "missing_instrument_details": 4,
                "needs_manual_review": 7
            },
            "review_items_created": 7,
            "rows_total": 8,
            "snapshot_date_status": "missing",
            "source": "Raiffeisen",
            "warnings_count": 18
        }
    ],
    "productive_positions_unchanged": true,
    "review_flags_last_session": {
        "PostFinance": {
            "cost_basis_uncertain": 33,
            "missing_isin": 33,
            "missing_ticker": 33,
            "needs_manual_review": 34,
            "snapshot_only": 34
        },
        "Raiffeisen": {
            "aggregate_only": 4,
            "cash_snapshot_candidate": 3,
            "missing_instrument_details": 4,
            "needs_manual_review": 7
        },
        "True Wealth": {
            "cost_basis_uncertain": 17,
            "missing_isin": 17,
            "missing_ticker": 17,
            "needs_manual_review": 18,
            "snapshot_only": 18
        }
    },
    "review_items_last_session": {
        "PostFinance": 34,
        "Raiffeisen": 7,
        "True Wealth": 18
    },
    "transactions_unchanged": true
}


Explanation:
**Checking for missing function values**

I need to take note that there’s a missing function value, which probably means the parser didn't detect the values accurately. I should verify the manual queue or dead-letter queue (DQ). Maybe I can use script counts if there are no labels present. It seems like I might also want to check out the dashboard functions to get more clarity on what's going wrong. It's a bit of a puzzle!


set -euo pipefail
cd /home/agent/.hermes/repos/FinanceManager
PYTHONPATH=src python - <<'PY'
import json, sqlite3
from jarvis_finance.config.settings import load_settings
from jarvis_finance.storage.database import connect
from jarvis_finance.storage.migrations import apply_migrations
from jarvis_finance.dashboard import data
settings=load_settings(); conn=connect(settings.db_path); apply_migrations(conn)
manual_items=data.get_broker_review_items(conn, limit=1000)
dq=data.get_data_quality(conn)
flag_counts=data.get_review_quality_flag_counts(conn)
dry_runs=data.get_broker_dry_runs(conn)
# no row labels printed
status_counts={}
source_counts={}
for row in conn.execute("SELECT source_platform, review_status, COUNT(*) AS n FROM broker_import_review_items GROUP BY source_platform, review_status").fetchall():
    source_counts.setdefault(row['source_platform'], {})[row['review_status']]=row['n']
print(json.dumps({
 'manual_review_queue_items_visible': len(manual_items)>0,
 'manual_review_queue_item_count': len(manual_items),
 'source_status_counts': source_counts,
 'data_quality_checks_count': len(dq),
 'data_quality_review_flag_counts': flag_counts,
 'dry_run_summaries_visible': len(dry_runs)>=3,
 'initial_snapshot_commit_action_active': False,
 'productive_position_rows': conn.execute('SELECT COUNT(*) AS n FROM positions_snapshot').fetchone()['n'],
 'transaction_rows': conn.execute('SELECT COUNT(*) AS n FROM transactions').fetchone()['n'],
}, ensure_ascii=False, sort_keys=True))
PY


Output: {"data_quality_checks_count": 16, "data_quality_review_flag_counts": [{"open_items": "3", "quality_flag": "cash_snapshot_candidate"}, {"open_items": "50", "quality_flag": "cost_basis_uncertain"}, {"open_items": "50", "quality_flag": "missing_isin"}, {"open_items": "50", "quality_flag": "missing_ticker"}, {"open_items": "55", "quality_flag": "needs_manual_review"}, {"open_items": "52", "quality_flag": "snapshot_only"}], "dry_run_summaries_visible": true, "initial_snapshot_commit_action_active": false, "manual_review_queue_item_count": 59, "manual_review_queue_items_visible": true, "productive_position_rows": 0, "source_status_counts": {"PostFinance": {"open": 34}, "Raiffeisen": {"blocked": 4, "open": 3}, "True Wealth": {"open": 18}}, "transaction_rows": 0}


patch

for row in conn.execute("SELECT quality_flags_json FROM broker_import_review_items WHERE review_status='open'").fetchall():

for row in conn.execute("SELECT quality_flags_json FROM broker_import_review_items WHERE review_status IN ('open','blocked')").fetchall():

Explanation:
**Considering code adjustments**

I’m noticing a bug where the aggregate_only blocked items aren’t being counted because the aggregate_review_quality_flags are only considering open items. The user is asking for DQ aggregate_only to be visible, but it won’t be visible because of the block. I believe I need a patch function that includes both open and blocked items. So, a code change is required, and I’ll need to run tests after implementing that patch!


--- a//home/agent/.hermes/repos/FinanceManager/src/jarvis_finance/imports/broker_mapping.py
+++ b//home/agent/.hermes/repos/FinanceManager/src/jarvis_finance/imports/broker_mapping.py
@@ -342,7 +342,7 @@

 def aggregate_review_quality_flags(conn: Connection) -> dict[str, int]:
     flags: Counter[str] = Counter()
-    for row in conn.execute("SELECT quality_flags_json FROM broker_import_review_items WHERE review_status='open'").fetchall():
+    for row in conn.execute("SELECT quality_flags_json FROM broker_import_review_items WHERE review_status IN ('open','blocked')").fetchall():
         try:
             for flag in json.loads(row["quality_flags_json"] or "[]"):
                 flags[str(flag)] += 1

/home/agent/.hermes/repos/FinanceManager/src/jarvis_finance/imports/broker_mapping.py was modified by sibling subagent '20260515_102855_64f760' but this agent never read it. Read the file before writing to avoid overwriting the sibling's changes.


    assert dq["Review Flag: missing_isin"] == "1"
    assert dq["Review Flag: missing_fx"] == "1"

def test_data_quality_includes_blocked_aggregate_only_review_flags(tmp_path: Path) -> None:
    path = tmp_path / "synthetic_raiffeisen.xlsx"
    _xlsx(path, [["Label", "Typ", "Währung"], ["Depot Anlagen gesamt", "Anlage", "CHF"]])
    c = conn()
    run_broker_import_dry_run(c, parse_raiffeisen_xlsx(path), source_filename=path.name)
    dq = {r["check"]: r["count"] for r in dashboard_data.get_data_quality(c)}
    assert dq["Review Flag: aggregate_only"] == "1"
    assert dq["Review Flag: missing_instrument_details"] == "1"

def test_safe_mode_default_read_only_and_commit_disabled() -> None:

    assert dq["Review Flag: missing_isin"] == "1"
    assert dq["Review Flag: missing_fx"] == "1"

def test_safe_mode_default_read_only_and_commit_disabled() -> None:


--- a//home/agent/.hermes/repos/FinanceManager/tests/unit/test_broker_parser_dry_run_pipeline.py
+++ b//home/agent/.hermes/repos/FinanceManager/tests/unit/test_broker_parser_dry_run_pipeline.py
@@ -130,6 +130,16 @@
     assert dq["Review Flag: missing_fx"] == "1"

+def test_data_quality_includes_blocked_aggregate_only_review_flags(tmp_path: Path) -> None:
+    path = tmp_path / "synthetic_raiffeisen.xlsx"
+    _xlsx(path, [["Label", "Typ", "Währung"], ["Depot Anlagen gesamt", "Anlage", "CHF"]])
+    c = conn()
+    run_broker_import_dry_run(c, parse_raiffeisen_xlsx(path), source_filename=path.name)
+    dq = {r["check"]: r["count"] for r in dashboard_data.get_data_quality(c)}
+    assert dq["Review Flag: aggregate_only"] == "1"
+    assert dq["Review Flag: missing_instrument_details"] == "1"
+
+
 def test_safe_mode_default_read_only_and_commit_disabled() -> None:
     c = conn()
     status = dashboard_data.get_safe_mode_status(c)

/home/agent/.hermes/repos/FinanceManager/tests/unit/test_broker_parser_dry_run_pipeline.py was modified by sibling subagent '20260515_104455_1e842f' but this agent never read it. Read the file before writing to avoid overwriting the sibling's changes.


terminal

set -euo pipefail
cd /home/agent/.hermes/repos/FinanceManager
PYTHONPATH=src pytest tests/unit/test_broker_parser_dry_run_pipeline.py -q
PYTHONPATH=src python - <<'PY'
import json
from jarvis_finance.config.settings import load_settings
from jarvis_finance.storage.database import connect
from jarvis_finance.storage.migrations import apply_migrations
from jarvis_finance.dashboard import data
conn=connect(load_settings().db_path); apply_migrations(conn)
print(json.dumps({'data_quality_review_flag_counts': data.get_review_quality_flag_counts(conn)}, sort_keys=True))
PY


Output:
...........                                                              [100%]
11 passed in 0.11s
{"data_quality_review_flag_counts": [{"open_items": "4", "quality_flag": "aggregate_only"}, {"open_items": "3", "quality_flag": "cash_snapshot_candidate"}, {"open_items": "50", "quality_flag": "cost_basis_uncertain"}, {"open_items": "4", "quality_flag": "missing_instrument_details"}, {"open_items": "50", "quality_flag": "missing_isin"}, {"open_items": "50", "quality_flag": "missing_ticker"}, {"open_items": "59", "quality_flag": "needs_manual_review"}, {"open_items": "52", "quality_flag": "snapshot_only"}]}


Output:
........................................................................ [ 49%]
........................................................................ [ 99%]
.                                                                        [100%]
145 passed in 3.92s
GIT_SAFETY_OK
## main...origin/main
 M src/jarvis_finance/imports/broker_mapping.py
 M tests/unit/test_broker_parser_dry_run_pipeline.py


set -euo pipefail
cd /home/agent/.hermes/repos/FinanceManager
git add src/jarvis_finance/imports/broker_mapping.py tests/unit/test_broker_parser_dry_run_pipeline.py
git commit -m "fix: include blocked broker review flags in data quality"
PYTHONPATH=src python -m compileall -q src tests
PYTHONPATH=src pytest -q
rm -rf .pytest_cache src/jarvis_finance/**/__pycache__ tests/**/__pycache__
PYTHONPATH=src python -m jarvis_finance.cli.main git-safety-scan .
git status --short --branch

Explanation:
**Preparing to commit code changes**

I need to commit and push my code changes. I’m making sure everything is ready before I do that — checking for any uncommitted edits and ensuring I have a clear message for the commit. It’s always good to double-check that my changes are functioning as expected. Alright, once I confirm everything’s in order, I’ll go ahead and push the changes to the repository. Exciting!


Output:
[main 4e808ed] fix: include blocked broker review flags in data quality
 2 files changed, 11 insertions(+), 1 deletion(-)
........................................................................ [ 49%]
........................................................................ [ 99%]
.                                                                        [100%]
145 passed in 4.06s
GIT_SAFETY_OK
## main...origin/main [voraus 1]


set -euo pipefail
cd /home/agent/.hermes/repos/FinanceManager
python - <<'PY'
import re, subprocess, sys
from pathlib import Path
repo=Path('/home/agent/.hermes/repos/FinanceManager')
text=Path.home().joinpath('jarvis_runtime/finance-system/secrets/Github_token_FinanceManager.csv').read_text()
m=re.search(r'(github_pat_[A-Za-z0-9_]+|ghp_[A-Za-z0-9_]+|gho_[A-Za-z0-9_]+)', text)
if not m:
    print('TOKEN_PARSE=failed'); sys.exit(12)
token=m.group(1); auth_url=f'https://{token}@github.com/Gamexgit/FinanceManager.git'
def run(args, check=True): return subprocess.run(args, cwd=repo, text=True, capture_output=True, check=check)
try:
    run(['git','fetch',auth_url,'main:refs/remotes/origin/main','--quiet'])
    local=run(['git','rev-parse','HEAD']).stdout.strip(); origin=run(['git','rev-parse','origin/main']).stdout.strip(); base=run(['git','merge-base','HEAD','origin/main']).stdout.strip()
    print('REMOTE_BEFORE='+origin[:7]); print('LOCAL_HEAD='+local[:7])
    if origin != base:
        print('REMOTE_HAS_NEW_COMMITS=yes'); sys.exit(3)
    push=run(['git','push',auth_url,'main:main'], check=False)
    if push.returncode:
        print('PUSH=failed'); print((push.stderr or push.stdout).replace(token,'[REDACTED]')[-1000:]); sys.exit(push.returncode)
    run(['git','fetch',auth_url,'main:refs/remotes/origin/main','--quiet'])
    print('PUSH=success'); print('REMOTE_AFTER='+run(['git','rev-parse','--short','origin/main']).stdout.strip())
finally:
    token=None; auth_url=None
PY
PYTHONPATH=src python -m jarvis_finance.cli.main git-safety-scan .
git status --short --branch


Output:
REMOTE_BEFORE=fa254a9
LOCAL_HEAD=4e808ed
PUSH=success
REMOTE_AFTER=4e808ed
GIT_SAFETY_OK
## main...origin/main


Erledigt, Sir. Phase B3 ist kontrolliert abgeschlossen.

## Gesamtstatus

- Phase B3: **erfolgreich durchgeführt**
- Produktiver Initial Snapshot: **nein**
- Produktive Aktien-/ETF-Positionen geschrieben: **nein**
- Produktive Cash-Snapshots geschrieben: **nein**
- Ledger-Transaktionen aus Brokerdateien geschrieben: **nein**
- Echte Dateien ins Git kopiert: **nein**
- Echte Werte/Mengen/Kontostände im Chat: **nein**
- Runtime Review Items erzeugt: **ja**
- Code geändert: **ja**, kleiner Fix für Data Quality Center
- Commit: `4e808ed`
- Push Status: **erfolgreich**
- Remote HEAD: `4e808ed`
- Git Status: **clean**, `main...origin/main`
- Tests: `145 passed`
- Git-Safety nach Abschluss: `GIT_SAFETY_OK`

## Preflight

Geprüft:

- Git Status clean: **ja**
- Remote HEAD synchron: **ja**
- Git-Safety vorab: **OK**
- Runtime-DB außerhalb Repo: **ja**
- Backup-/Reports-/Imports-Ordner außerhalb Repo: **ja**
- echte Broker-/Bank-Dateien im Repo: **0**
- Safe Mode / Read-only für produktive Imports: **aktiv**
- produktive Commit-Aktionen im UI: **weiterhin deaktiviert**

## Backup

Vor dem Real-File Dry Run ausgeführt:

- Runtime-DB Backup erzeugt: **ja**
- Backup verifiziert: **ja**
- Checksum OK: **ja**
- Speicherort: **Runtime-Backup-Ordner**
- Sensitive Pfade/Dateinamen: **nicht ausgegeben**

## Real-File Dry Runs

Die echten Dateien wurden nur temporär gelesen und nicht ins Repo kopiert. Temporärer Verarbeitungskontext wurde bereinigt.

### PostFinance Dry Run

- rows_total: `40`
- candidate_positions: `33`
- candidate_cash_rows: `1`
- mapped_positions: `0`
- blocked_positions: `33`
- review_items_created: `34`
- warnings_count: `167`
- errors_count: `0`
- snapshot_date_status: `missing`

Quality Flags:

- `missing_isin`: `33`
- `missing_ticker`: `33`
- `snapshot_only`: `34`
- `cost_basis_uncertain`: `33`
- `needs_manual_review`: `34`

Bewertung:

- Parser hat Positions- und Cash-Kandidaten erkannt.
- Keine produktiven Positionen geschrieben.
- Name-only / fehlende ISIN/Ticker bleiben korrekt review-pflichtig.

### True Wealth Dry Run

- rows_total: `19`
- candidate_positions: `17`
- candidate_cash_rows: `1`
- mapped_positions: `0`
- blocked_positions: `17`
- review_items_created: `18`
- warnings_count: `87`
- errors_count: `0`
- snapshot_date_status: `missing`

Quality Flags:

- `missing_isin`: `17`
- `missing_ticker`: `17`
- `snapshot_only`: `18`
- `cost_basis_uncertain`: `17`
- `needs_manual_review`: `18`

Bewertung:

- Parser hat ETF-/positionsähnliche Zeilen und Cash-/Kontozeilen erkannt.
- Aktuell keine ISIN/Ticker-Autoerkennung aus der echten Struktur.
- True Wealth wird dadurch korrekt in die Review Queue gelenkt, statt still importiert zu werden.
- `missing_fx` trat in diesem Lauf nicht auf.

### Raiffeisen Dry Run

- rows_total: `8`
- candidate_positions: `4`
- candidate_cash_rows: `3`
- mapped_positions: `0`
- blocked_positions: `4`
- review_items_created: `7`
- warnings_count: `18`
- errors_count: `0`
- snapshot_date_status: `missing`

Quality Flags:

- `cash_snapshot_candidate`: `3`
- `aggregate_only`: `4`
- `missing_instrument_details`: `4`
- `needs_manual_review`: `7`

Bewertung:

- Raiffeisen wurde defensiv verarbeitet.
- Cash-/Kontrollsnapshot-Kandidaten wurden erkannt.
- Aggregierte Anlage-/Depotzeilen wurden **nicht** als Instrumentpositionen importiert.
- Aggregate-only Items wurden blockiert/review-pflichtig gespeichert. Sehr unaufgeregt. So soll es sein.

## Review Items

Neue Runtime Review Items aus Phase B3:

- PostFinance: `34`
- True Wealth: `18`
- Raiffeisen: `7`
- Total: `59`

Statusverteilung:

- PostFinance:
  - open: `34`
- True Wealth:
  - open: `18`
- Raiffeisen:
  - open: `3`
  - blocked: `4`

Regeln erfüllt:

- jeder Review Item hat `dry_run_id`
- source platform vorhanden
- row hash vorhanden
- detected fields vorhanden
- quality flags vorhanden
- status `open` oder `blocked`
- alte Dry Runs wurden nicht überschrieben
- Details liegen nur in der Runtime-DB

## Manual Review Queue

Geprüft:

- Review Items sichtbar: **ja**
- Quelle sichtbar: **ja**
- Status sichtbar: **ja**
- Quality Flags sichtbar: **ja**
- produktive Initial-Snapshot-Commit-Aktion aktiv: **nein**
- Mapping-Aktionen bleiben kontrolliert/deaktiviert vorbereitet: **ja**

## Data Quality Center

Geprüft:

- Review Flags werden aggregiert: **ja**
- `missing_isin` sichtbar: **ja**
- `missing_ticker` sichtbar: **ja**
- `snapshot_only` sichtbar: **ja**
- `cost_basis_uncertain` sichtbar: **ja**
- `aggregate_only` sichtbar: **ja**
- `missing_instrument_details` sichtbar: **ja**
- `cash_snapshot_candidate` sichtbar: **ja**
- `missing_fx`: in diesem Lauf **nicht vorhanden**
- stale/missing price alerts weiterhin integriert: **ja**
- resolved/fixed alerts weiterhin integriert: **ja**
- produktive Positionswerte angezeigt: **nein**

Dabei wurde ein kleiner Code-Fix nötig:

- Vorher wurden im Data Quality Center nur `open` Review Items für Quality Flags aggregiert.
- Raiffeisen `aggregate_only` Items sind absichtlich `blocked`.
- Fix: Data Quality aggregiert nun `open` **und** `blocked` Review Items.
- Test ergänzt.

Commit:

- `4e808ed` — `fix: include blocked broker review flags in data quality`

## Produktive DB-Positionen unverändert

Verifiziert:

- `positions_snapshot`: unverändert
- `transactions`: unverändert
- Cash-Ledger: unverändert / nicht produktiv beschrieben
- Broker Dry Runs: neu erzeugt
- Broker Review Items: neu erzeugt

Heißt: Die Dry-Run-Pipeline hat Review-/Quality-Daten erzeugt, aber keine produktive Portfolio-Wahrheit verändert.

## Tests / Safety

Nach Abschluss ausgeführt:

- Compile: **OK**
- pytest: `145 passed`
- Git-Safety: `GIT_SAFETY_OK`
- Git Status: clean
- Push: erfolgreich

## Neue/geänderte Dateien

Geändert:

- `src/jarvis_finance/imports/broker_mapping.py`
- `tests/unit/test_broker_parser_dry_run_pipeline.py`

Keine echten Broker-/Bank-Dateien wurden committed.

## Nächste Empfehlung

Phase B4 sollte jetzt **Review Queue Hardening** sein, noch immer ohne produktiven Import:

1. Review Items pro Dry Run filterbar machen.
2. Dry Run archivieren/verwerfen vorbereiten.
3. Source-spezifische Review-Ansichten:
   - PostFinance
   - True Wealth
   - Raiffeisen
4. Manuelle Ergänzung von:
   - ISIN
   - Ticker
   - Exchange
   - Account Mapping
5. Jede Review-Aktion mit Audit-Log.
6. Noch kein Initial Snapshot Commit.
7. Danach erst kontrollierter, explizit freigegebener Import einer kleinen reviewed Safe Subset Position.

Kurz: Die echten Dateien sind jetzt trocken durch den Apparat gelaufen. Nichts Wertvolles wurde verschüttet, nichts Reales ins Git getragen, und die Review Queue hat endlich echte Arbeit vor sich.


[M G] Sehr gut. Phase B3 ist akzeptiert.

Der Stand ist sauber:

* Commit 4e808ed gepusht
* Tests: 145 passed
* Git-Safety: OK
* Backup vor Real-File-Dry-Run erzeugt und verifiziert
* echte Broker-/Bank-Dateien nur temporär gelesen
* keine echten Dateien ins Git
* keine echten Werte/Mengen im Chat
* Review Items in Runtime erzeugt
* keine produktiven Positionen, Cash-Snapshots oder Ledger-Transaktionen geschrieben

Die Ergebnisse zeigen klar:

* PostFinance und True Wealth liefern viele Positionskandidaten, aber aktuell 0 automatisch gemappte Positionen.
* Hauptgründe: fehlende/unklare ISIN, Ticker, Exchange, Stichtag und Cost-Basis-Unsicherheit.
* Raiffeisen wurde korrekt defensiv als Cash-/Kontrollsnapshot bzw. Aggregate behandelt.

Bitte fahre nun mit Phase B4 fort.

# Phase B4 — Review Queue Hardening und manuelles Mapping

Ziel:
Die echten Dry-Run-Ergebnisse sollen im Dashboard so bearbeitbar werden, dass ich fehlende Mappings manuell ergänzen und Positionen für einen späteren Import freigeben kann.

Wichtig:
Noch immer kein produktiver Initial Snapshot.
Noch keine produktiven Aktien-/ETF-/Cash-Imports.
Noch keine Ledger-Commits aus Brokerdateien.

## 1. Review Queue Filter und Detailansicht

Bitte erweitere die Manual Review Queue.

Filter:

* Quelle:

  * PostFinance
  * True Wealth
  * Raiffeisen
* Dry Run Session
* Status:

  * open
  * blocked
  * mapped
  * ignored
  * resolved
* Quality Flags:

  * missing_isin
  * missing_ticker
  * snapshot_only
  * cost_basis_uncertain
  * market_value_legacy
  * missing_fx
  * aggregate_only
  * missing_instrument_details
  * cash_snapshot_candidate
  * needs_manual_review
* Assetklasse
* Währung
* Account Mapping Status

Detailansicht pro Review Item:

* Quelle
* Dry Run ID
* Source Row Reference
* Source Label
* Normalized Name
* erkannte Assetklasse
* erkannte Währung
* erkannte ISIN
* erkannter Ticker
* erkannte Exchange
* Account Mapping
* Quality Flags
* Review Status
* Reviewer Note
* Audit History zu diesem Item

Wichtig:
Keine echten Mengen/Werte im Chat oder Git. In der Runtime-UI dürfen Details sichtbar sein, aber nicht exportiert/committed werden.

## 2. Manuelle Mapping-Aktionen aktivieren

Bitte aktiviere kontrollierte Mapping-Aktionen für Review Items.

Aktionen:

A. ISIN ergänzen

* Eingabefeld ISIN
* Validierung Format grob prüfen
* Status von missing_isin aktualisieren
* Audit-Log schreiben

B. Ticker ergänzen

* Eingabefeld Ticker
* optional Exchange
* Ticker ohne Exchange bleibt review-pflichtig
* Audit-Log schreiben

C. Exchange ergänzen

* z.B. SIX, XETRA, NASDAQ, NYSE, LSE usw.
* Audit-Log schreiben

D. Instrument Mapping bestätigen

* Wenn ISIN vorhanden: auf vorhandenes Instrument mappen oder neues Instrument vorbereiten
* Wenn nur Ticker + Exchange vorhanden: Mapping als review-pflichtig markieren, außer explizit bestätigt
* Name allein darf niemals automatisch auf mapped gesetzt werden
* Audit-Log schreiben

E. Account Mapping bestätigen

* Source Account auf internes Konto mappen
* PostFinance / True Wealth / Raiffeisen getrennt halten
* Cash und Brokerage nicht vermischen
* Audit-Log schreiben

F. Ignore with note

* Review Item ignorieren
* Notiz Pflicht
* Status ignored
* Audit-Log schreiben

G. Mark resolved

* Nur möglich, wenn Pflichtprobleme gelöst sind
* Status resolved
* Audit-Log schreiben

## 3. Import Readiness Status

Bitte berechne pro Review Item und pro Dry Run Session einen Import-Readiness-Status.

Vorschlag pro Review Item:

* not_ready
* review_needed
* ready_for_import
* ignored
* blocked

Ein Review Item darf nur ready_for_import sein, wenn:

Für Aktien/ETFs:

* source_platform vorhanden
* account mapping vorhanden
* asset_class equity oder etf
* ISIN vorhanden oder Ticker + Exchange explizit bestätigt
* currency vorhanden
* snapshot_date klar oder explizit manuell bestätigt
* Menge vorhanden
* keine blockierenden Quality Flags
* reviewer confirmation vorhanden

Für Cash:

* account mapping vorhanden
* currency vorhanden
* snapshot_date klar oder bestätigt
* Betrag vorhanden
* reviewer confirmation
vorhanden

Für Aggregate/Raiffeisen:

* aggregate_only darf nicht als Instrumentposition importiert werden
* nur Cash-/Kontrollsnapshot, falls eindeutig

## 4. Dry Run Session Management

Bitte baue eine Übersicht für Dry Run Sessions:

* Dry Run ID
* Quelle
* Dateityp
* Erstellungszeitpunkt
* Snapshot Date Status
* Anzahl Review Items
* Anzahl ready_for_import
* Anzahl blocked
* Anzahl ignored
* Anzahl open
* Warnings/Errors

Aktionen vorbereiten:

* Dry Run archivieren
* Dry Run verwerfen
* Dry Run als aktuell markieren
* nur aktuelle Session im Import Wizard priorisieren

Regeln:

* Archivieren/verwerfen löscht keine Audit-Historie.
* Kein produktiver Import durch diese Aktionen.
* Audit-Log für Statusänderung.

## 5. Data Quality Center erweitern

Bitte erweitere das Data Quality Center:

* Anzahl Review Items je Quality Flag
* Anzahl ready_for_import
* Anzahl not_ready
* Anzahl blocked
* Anzahl ignored
* Warnung, wenn eine Quelle 0 ready_for_import hat
* Warnung, wenn Snapshot Date fehlt
* Warnung, wenn viele Items missing_isin/missing_ticker haben
* Link zur Review Queue mit vorgefilterter Ansicht

Buttons:

* Data Quality Recheck
* Resolve Fixed Alerts
* Review Items aktualisieren

Produktive Import-Buttons bleiben deaktiviert.

## 6. Safe Mode / Write Enable

Bitte Safe Mode weiter stärken:

* Global sichtbar:

  * Runtime DB oder Demo DB
  * Read-only oder Write-enabled
  * letzter Backup-Zeitpunkt
  * Git-Safety Status, falls verfügbar
* Default bleibt Read-only.
* Mapping-Aktionen dürfen trotz Read-only nur dann erlaubt sein, wenn sie ausdrücklich als Review-Metadaten-Aktion gelten und keine produktive Portfolio-Position schreiben.
* Produktiver Import bleibt blockiert.

Bitte klar trennen:

* Review-Metadaten schreiben: erlaubt nach Bestätigung
* Portfolio-/Ledger-/Snapshot schreiben: weiterhin blockiert

## 7. Tests

Bitte Tests ergänzen für:

* Review Queue Filter nach Quelle
* Review Queue Filter nach Dry Run ID
* Review Queue Filter nach Quality Flag
* ISIN ergänzen erzeugt Audit
* Ticker/Exchange ergänzen erzeugt Audit
* Name allein bleibt nicht automatisch mapped
* Instrument Mapping mit ISIN kann ready_for_import werden
* Ticker ohne Exchange bleibt review_needed
* Account Mapping Bestätigung erzeugt Audit
* Ignore with note funktioniert
* Ignore ohne Note wird abgelehnt
* Mark resolved nur bei gelösten Pflichtproblemen
* ready_for_import Berechnung für Equity
* ready_for_import Berechnung für ETF
* ready_for_import Berechnung für Cash
* aggregate_only bleibt blockiert für Instrumentimport
* Dry Run archivieren/verwerfen erzeugt Audit
* Data Quality Center zeigt ready/not_ready/blocked/ignored Counts
* produktiver Import bleibt deaktiviert
* keine echten Dateien im Repo
* Git-Safety OK

## 8. Echte Daten

Du darfst die bestehenden echten Runtime Review Items verwenden, um die UI/Funktionen zu prüfen.

Aber:

* keine echten Werte/Mengen im Chat
* keine echten Dateien ins Repo
* keine produktiven Positionen
* keine Ledger-Commits
* keine Cash-Snapshots
* keine positions_snapshot-Writes
* keine echten Reports mit Aktien-/ETF-Zahlen

## 9. Nach Abschluss

Bitte ausführen:

* Compile
* pytest
* Git-Safety
* Git Status

Wenn sauber:

* Commit mit sinnvoller Message
* Push auf main

Bitte berichten:

* neue/geänderte Module
* implementierte Review-Aktionen
* Tests
* Testergebnis
* Git-Safety
* Commit Hash
* Push Status
* Review Item Counts nach Status
* Anzahl ready_for_import je Quelle
* ob produktive Portfolio-Daten unverändert blieben
* nächste Empfehlung

Wichtig:
Ziel von Phase B4 ist, die Review Queue bedienbar zu machen. Noch kein produktiver Aktien-/ETF-/Cash-Import.


Explanation:
**Inspecting code skill**

I need to take action and inspect the code. First, I’m wondering if the skill is already loaded; that’s crucial! The current message is prompting me to load the skill, which implies I might need to check its status. I should verify if everything is set up properly so I can proceed smoothly. Let’s make sure I have all the necessary components ready before moving forward!


finance-system-architecture

# Broker Parser Dry-Run Pipeline Pattern

Use this reference after the Import Wizard / Mapping Layer exists, when adding parser output to the review workflow for PostFinance, True Wealth, Raiffeisen, or similar broker/bank statements.

## Purpose

Turn broker/bank parser results into safe dry-run summaries and runtime-only review items. This phase must still avoid productive ledger, snapshot, holdings, or cash writes unless the user explicitly approves a later import phase.

## Sequencing and guardrails

1. Start with synthetic DOCX/XLSX fixtures that mimic real source structure.
2. Do not commit real broker/bank files, raw rows, quantities, market values, or generated runtime DBs.
3. Real files may only be read from runtime/temp for structural dry-runs; report aggregate counts and quality flags only.
4. Keep Import Wizard and review actions read-only by default.
5. Do not pivot into analytics modules such as Monte Carlo, backtesting, rebalancing, scores, projections, or heatmaps during import hardening.

## Parser behavior by source

### PostFinance DOCX/table parser

Extract candidates from DOCX table rows:

- product/name label
- quantity-present boolean, not quantity value in chat/docs
- currency
- market-value-present boolean
- account/depot context
- snapshot date when recognizable

Expected quality flags:

- `missing_isin`
- `missing_ticker`
- `snapshot_only`
- `cost_basis_uncertain`
- `market_value_legacy`
- `needs_manual_review`

### True Wealth DOCX/mixed-table parser

Extract:

- cash rows
- position-like rows
- portfolio/account context
- currency
- snapshot date when recognizable

Expected quality flags:

- `missing_isin`
- `missing_ticker`
- `snapshot_only`
- `cost_basis_uncertain`
- `missing_fx` for non-CHF rows without FX
- `needs_manual_review`

### Raiffeisen XLSX account/control parser

Extract:

- cash/account rows
- aggregate custody/investment rows

Rules:

- Treat Raiffeisen defensively as cash/control snapshot when detailed instrument fields are absent.
- Do not create instrument positions from aggregate-only rows.

Expected quality flags:

- `cash_snapshot_candidate`
- `aggregate_only`
- `missing_instrument_details`
- `needs_manual_review`

## Runtime-only review items

Add/maintain `broker_import_review_items` for row-level review data in the runtime DB only.

Fields:

- `review_item_id`
- `dry_run_id`
- `source_platform`
- `source_file_type`
- `source_row_ref`
- `row_hash`
- `source_label`
- `normalized_name`
- `detected_asset_class`
- `detected_currency`
- `detected_quantity_present`
- `detected_market_value_present`
- `isin`
- `ticker`
- `exchange`
- `mapped_instrument_id`
- `mapped_account_id`
- `quality_flags_json`
- `review_status`: open, mapped, ignored, blocked, resolved
- `reviewer_note`
- timestamps

Rules:

- Review item details are runtime-only and must not be written to Git fixtures/docs when they come from real files.
- In chat/reporting, output only aggregate counts and quality flag summaries.
- Manual mapping or status changes must require a non-empty note and create an audit-log entry.

## Generic dry-run pipeline

1. Parse source file.
2. Normalize candidates.
3. Attempt instrument mapping using existing mapping rules: ISIN primary; ticker+exchange fallback; name-only never auto-maps.
4. Attempt account mapping when account context exists.
5. Set quality flags.
6. Write `broker_import_dry_runs` aggregate summary.
7. Write `broker_import_review_items` row-level runtime review records.
8. Do not write `positions_snapshot`.
9. Do not write transactions, cash ledger, crypto holdings, or productive snapshots.
10. Return aggregate output: `rows_total`, `candidate_positions`, `candidate_cash_rows`, `mapped_positions`, `blocked_positions`, `review_items_created`, `warnings_count`, `errors_count`, `quality_flags_summary`.

## UI extensions

Import Wizard should show parser/dry-run summary, quality flag counts, runtime review items, and a visible link/path to Manual Review Queue. Productive commit buttons remain disabled in read-only mode.

Manual Review Queue should show review items with source, label, normalized name, asset class, currency, ISIN, ticker, exchange, mapped instrument/account, quality flags, and status. Prepare actions for adding identifiers, confirming mappings, ignoring with note, or resolving, but keep productive position creation disabled.

Data Quality Center should aggregate open review items by quality flag, including `missing_isin`, `missing_ticker`, `snapshot_only`, `cost_basis_uncertain`, `missing_fx`, `aggregate_only`, plus existing stale/missing price and alert lifecycle checks.

Global Safe Mode should be visible in the dashboard shell or Command Center: DB mode, write mode, last backup if known, Git-safety status if known, and reminder that real data lives outside Git. Default is read-only.

## Test matrix additions

Add tests for:

- synthetic PostFinance parser candidate detection and flags
- synthetic True Wealth parser candidate detection and flags
- synthetic Raiffeisen cash/control and aggregate-only detection
- dry-run summary creation
- review item creation
- no productive positions/ledger writes
- name-only rows stay review-gated
- Raiffeisen aggregate-only rows are blocked, not instrument positions
- Manual Review Queue loads review items
- Data Quality Center aggregates review flags
- default safe mode is read-only
- commit actions remain disabled
- audit log on review-item mapping/status change
- no real DOCX/XLSX/PDF files in the repo
- Git-safety remains OK


todo

Code-Struktur Review Queue/Dry Run/Data Quality/Audit inspizieren

B4 Schema/Mapping-Actions/Readiness/Session Management implementieren

Dashboard Seiten Filter/Details/Data Quality/Safe Mode erweitern

Tests für B4 ergänzen und produktive Daten-Unverändertheit prüfen

Compile, pytest, Git-Safety, Commit und Push