## Outcome - Inventoried FinanceManager at base commit `3aba7d2e7a9c55dece5c4d7c794a577ad4601903`. - Produced a minimal Sprint-13 design that imports PostFinance transaction history into the **existing PostFinance account, instruments, transaction ledger, and snapshot controls**. - No files were created or modified. - Verified clean tree: `git status --short` empty, `git diff --check` passed, HEAD unchanged. ## Architecture found ### Canonical model - Schema/migrations: `src/jarvis_finance/storage/migrations.py` - Current migration: `44 / 044_truewealth_verified_snapshot_v1`. - Existing canonical tables: - `accounts`: existing PostFinance E-Trading account must be reused. - `instruments`: Sprint-9’s 22 canonical ISIN identities must be reused. - `transactions`: canonical activities for performance/FIFO. - `positions_snapshot`: source-reported position controls. - `cash_balances`, `cash_account_snapshots`: source cash controls. - `account_value_snapshots`, `portfolio_valuation_snapshots`: account/performance valuations. - `portfolio_ingestion_batches`, `portfolio_ingestion_items`: immutable Preview → Confirm lineage. - Existing case-insensitive canonical identity: - `idx_instruments_unique_isin ON instruments(UPPER(isin))`. ### Prior sprint contracts - **Sprint 7** - Parser: `src/jarvis_finance/imports/postfinance_etrading.py` - Orchestration: `src/jarvis_finance/services/portfolio_data.py` - Imports a point-in-time DOCX, not transaction history. - Writes: - 22 `initial_position_snapshot` transactions, - 22 `positions_snapshot` rows, - cash controls, - account total and performance baseline. - **Sprint 9** - Activation: `scripts/sprint9_postfinance_activation.py` - Confirms 22 ISIN/provider mappings and imports exactly 22 positions. - Existing account resolver: `portfolio_data._postfinance_account()`. - Existing mapping resolver: `_instrument_for_source()` / `confirm_postfinance_instrument_mappings()`. - **Sprint 12** - `src/jarvis_finance/services/truewealth_service.py` - Demonstrates bounded parsing, digest identity, external archive, Preview → Confirm, immutable history, atomic rollback, and explicit precedence. - TrueWealth remains structurally separate and must not be touched by Sprint 13. - **API** - Existing generic endpoints in `api/routers/overview.py`: - `POST /api/portfolio/ingestion/preview` - `POST /api/portfolio/ingestion/confirm` - ingestion history and reconciliation GETs. - **UI** - Existing import surface: - `frontend/src/components/portfolio-data/DataIngestionReconciliationPanel.vue` - `frontend/src/api/portfolio.ts` - **CI/release** - `.github/workflows/portfolio-phase3-integration.yml` - `scripts/ci_portfolio_phase3_gate.py` - `Makefile` - Deployment scripts start services but do **not** apply migrations; migration must be an explicit pre-restart release step. ## Minimal Sprint-13 design ### 1. Source gate first The repository contains no verified PostFinance transaction-ledger layout. Do not invent CSV/PDF/XLSX columns or activity semantics. Add: - `docs/sprint13-postfinance-ledger-source-gate.md` - `docs/sprint13-postfinance-ledger-import.md` The gate must confirm file type, parser signatures, date/decimal rules, stable row identity, transaction references, currencies, fee/tax semantics, reversals, corporate actions, and whether history is complete through the Sprint-7 baseline date `2026-05-14`. ### 2. Small implementation surface **New files** - `src/jarvis_finance/imports/postfinance_etrading_ledger.py` - Pure, bounded, in-memory parser. - Returns immutable parsed rows and file digest. - No DB access, logging of source rows, or guessed fields. - `src/jarvis_finance/services/postfinance_ledger_service.py` - Resolves the existing account and existing ISIN instruments. - Classifies rows and computes baseline-replacement reconciliation. - Exposes record preparation/insertion helpers with no commits. - `tests/test_postfinance_sprint13.py` - Collected synthetic-format contract tests. **Modify** - `src/jarvis_finance/storage/migrations.py` - `src/jarvis_finance/services/portfolio_data.py` - Add `postfinance_etrading_ledger` dispatch. - Keep `confirm_ingestion()` as the sole transaction owner. - `src/jarvis_finance/api/schemas/portfolio_data.py` - Extend source literals and bounded upload contract. - `src/jarvis_finance/services/portfolio_performance.py` - Only if verified source activity names need explicit `ACTIVITY_MAP` entries. - `frontend/src/api/portfolio.ts` - `frontend/src/components/portfolio-data/DataIngestionReconciliationPanel.vue` - Add a transaction-history source option and preview counts. - Reuse the existing page; do not create a new PostFinance account/page. - `frontend/src/components/portfolio-data/DataIngestionReconciliationPanel.test.ts` - `tests/unit/test_portfolio_data_ingestion_reconciliation.py` - `tests/unit/test_schema.py` - `scripts/ci_portfolio_phase3_gate.py` - `.github/workflows/portfolio-phase3-integration.yml` No new router is needed. ## Recommended migration 45 ### Reuse rather than duplicate Do **not** add PostFinance position/account/instrument/transaction mirror tables. Canonical financial rows belong in existing tables. Add only generic provenance needed to close current audit gaps: ### `portfolio_ingestion_source_files` One immutable row per file-backed ingestion batch: - `batch_id` PK/FK → `portfolio_ingestion_batches` - `file_sha256` with length-64 check - `filename_sha256` - `media_type` - `parser_id` - `parser_version` - `archive_reference` - `created_at` - `UNIQUE(parser_id, parser_version, file_sha256)` Archive source bytes outside Git with `0700` directory / `0600` file permissions, following Sprint 12. ### `transaction_supersessions` Records exact replacement of each Sprint-7 baseline transaction: - `supersession_id` PK - `predecessor_transaction_id` UNIQUE FK → `transactions` - `superseding_batch_id` FK → `portfolio_ingestion_batches` - `account_id` FK → `accounts` - `reason_code CHECK(reason_code='postfinance_complete_history_replaces_baseline')` - `evidence_hash` UNIQUE - `audit_id` - `created_at` This preserves an exact database relationship for all 22 voided baseline transactions without inventing one-to-one replacement trades. ### Unique constraints/indexes After a duplicate preflight: ```sql CREATE UNIQUE INDEX ux_pf_ledger_file_revision ON portfolio_ingestion_batches(source_revision) WHERE source_key='postfinance_etrading_ledger'; CREATE UNIQUE INDEX ux_pf_ledger_transaction_row_hash ON transactions(row_hash) WHERE source_type='postfinance_etrading_ledger' AND row_hash IS NOT NULL AND trim(row_hash) <> ''; ``` Add an external-reference unique index only if the verified source proves that its transaction ID is unique per economic row. References may legitimately repeat across trade, fee, tax, and FX legs. ### Triggers - Update/delete immutability triggers for `portfolio_ingestion_source_files`. - Update/delete immutability triggers for `transaction_supersessions`. - `BEFORE DELETE` protection for `transactions` where `source_type='postfinance_etrading_ledger'`. - Restricted update trigger for those transactions: - permit only lifecycle fields such as `is_voided`, `voided_at`, `void_reason`, `voided_by`, and `updated_at`; - reject mutation of account, instrument, dates, amounts, currencies, source IDs, row hash, and correction lineage. - Insert-validation trigger on `transaction_supersessions`: - predecessor must be an active `postfinance_etrading_snapshot` / `initial_position_snapshot`; - batch must use `postfinance_etrading_ledger`; - predecessor and supersession account IDs must match. Existing ingestion batch/item/audit immutability triggers remain reusable. ## Double-counting contract Sprint 13 must support only two explicit modes: 1. **Complete-history replacement** - Source begins before the first relevant trade and reaches at least `2026-05-14`. - Reconstructed quantities at `2026-05-14` exactly match all 22 Sprint-7 snapshots. - Confirm atomically: - inserts historical canonical transactions, - voids exactly the 22 active baseline `initial_position_snapshot` transactions, - inserts 22 `transaction_supersessions`, - writes ingestion items, source-file provenance, and one audit. 2. **Append-after-baseline** - Every imported activity is strictly after `2026-05-14`. - Keep the 22 baseline transactions active. - Import only subsequent activities. Any partial history overlapping the baseline date must block Confirm. Do not delete or recreate: - the PostFinance account, - the 22 instruments/mappings, - Sprint-7 `positions_snapshot` rows, - cash/account-total controls, - the Sprint-9 valuation baseline. The snapshots remain authoritative source observations for reconciliation; only synthetic baseline ledger activities are voided when complete historical activity replaces them. ## Key pitfalls - Leaving the 22 baseline transactions active while importing pre-baseline buys doubles quantities and FIFO lots. - Deleting `positions_snapshot` rows removes the reconciliation control rather than fixing double counting. - Creating another PostFinance account splits holdings and portfolio totals. - Matching by product label/ticker can duplicate Sprint-9 instruments; resolve stocks/ETFs by normalized ISIN. - Mapping FX conversions to external deposits/withdrawals corrupts TWR/MWR; they are internal transfer legs. - Treating splits or corporate actions as buys/sells corrupts quantities and cost basis; block unsupported actions. - Fees, taxes, dividends, and trade rows may share a source reference; dedupe by normalized row hash unless verified otherwise. - Persisting signed sell quantities conflicts with existing FIFO/reconciliation code, which expects positive quantities and uses activity kind for direction. - Missing FX, gross, fee, or tax values must remain missing; do not infer them from net amounts. - `portfolio_data._existing_lineage()` is source-agnostic, so new lineage hashes must include source key and parser version. - API writes remain unavailable remotely under the current security baseline; Preview is the only write-like POST exempted in disabled mode. - Backend startup does not migrate automatically; migration 45 requires backup, rehearsal, explicit application, integrity check, then service restart. ## Required verification - Parser exact-format synthetic fixture and malformed/bounded-file tests. - Preview causes zero DB writes. - Full-history reconciliation matches all 22 baseline quantities before Confirm. - Confirm rolls back all transactions and baseline voids on a late failure. - Identical file and identical normalized row replay are idempotent. - Same filename with different bytes is distinct. - Account count and account ID remain unchanged. - Existing 22 instrument IDs/mappings remain unchanged. - Existing 22 position snapshots remain unchanged. - Exactly 22 baseline transactions are voided only in complete-history mode. - Append-only mode rejects any row on or before the baseline date. - FIFO, realized P&L, fees/taxes, transfers, and reconciliation tests. - Migration 44→45 and Sprint-5→45 preserve existing data digests. - Full backend/frontend/typecheck/build/safety/diff gates.