## Backend architecture map ### Finding: Schema 49 is sufficient No migration is required for a bounded v1 implementation: - **Official anchors:** `truewealth_snapshots`, `truewealth_snapshot_positions`, `truewealth_snapshot_cash` - **Confirmed external flows:** existing `transactions` with `source_type='truewealth_external_cashflow_v1'` - **Flow coverage:** `performance_cashflow_coverage` - **Modeled daily valuations:** `portfolio_valuation_snapshots` - **Run identity/status:** `market_data_runs` - **Input provenance and reconciliation evidence:** immutable `audit_log` - **Scope:** `performance_scope_classifications` role `canonical_truewealth_total_value` - **Prices/FX:** existing `market_prices`, `fx_rates` Schema 49 remains practical if model-to-official reconciliation is stored in the immutable import audit JSON. A migration would only be justified if reconciliation must become a heavily queried first-class resource with dedicated foreign keys/indexes. --- ## Proposed implementation ### 1. New source-isolated modeling service Add `src/jarvis_finance/services/truewealth_valuations.py` with: - `SOURCE_KEY = "daily_truewealth_modeled_v1"` - `TrueWealthValuationRunResult` - `resolve_confirmed_anchor(conn, as_of, exclude_snapshot_id=None)` - `resolve_anchor_positions(conn, anchor)` - `resolve_anchor_cash(conn, anchor)` - `resolve_confirmed_external_flows(conn, anchor_date, as_of)` - `resolve_exact_price_fx_inputs(conn, positions, currencies, as_of)` - `build_modeled_truewealth_valuation(...)` — pure/read-only calculation - `run_daily_truewealth_valuation(...)` — append-only materialization - `build_truewealth_reconciliation(...)` — read-only official-vs-model comparison The service must never read preview/staging rows, create transactions, or call a provider. It consumes only confirmed anchors, confirmed flows, and already persisted prices/FX. ### 2. Modeling formulas and provenance For latest confirmed official anchor \(A\) at date \(d_0\): - Position quantities remain frozen: \[ q_{i,t}=q_{i,A} \] - Security value: \[ S_t=\sum_i q_{i,A}\times P_{i,t}\times FX_{ccy_i\rightarrow CHF,t} \] - Cash by currency: \[ C_{k,t}=\left(c_{k,A}+\sum_j signedFlow_{j,k}\right)\times FX_{k\rightarrow CHF,t} \] - Account value: \[ V_t=S_t+\sum_k C_{k,t} \] Flow timing should match existing `ttwror_daily_v1`: - deposit on day \(d\): applied after the \(d\) boundary, hence affects modeled values after \(d\); - withdrawal on day \(d\): reflected in the closing value on \(d\). Each modeled account snapshot should use: - `scope_kind='account'` - `scope_id=account_id` - `source='daily_truewealth_modeled_v1'` - `quality_status='complete'` - `source_reference=f"{run_id}:{input_fingerprint}"` Store position-level snapshots for price/FX attribution, but do **not** invent synthetic cash instruments or trades. Cash-FX and unobserved managed activity must remain an explicit attribution residual. The fingerprint should bind: - official anchor snapshot/batch IDs; - position ISINs and quantities; - original cash amounts/currencies; - all confirmed external-flow IDs and row hashes; - exact market-price row IDs; - exact FX-rate row IDs; - valuation date and modeling contract version. ### 3. Instrument and market-data resolution Resolve every official True Wealth position through: 1. exact case-insensitive ISIN match to one active canonical `instruments` row; 2. exactly one confirmed `instrument_price_mappings` row; 3. persisted positive price at or before the cutoff using the current business-day staleness policy; 4. persisted positive FX to CHF with no future observation. Do not use the tax-statement CHF source price as a daily market quote. It remains anchor evidence only. Any missing/ambiguous mapping, currency mismatch, future input, stale input beyond policy, duplicate ISIN, non-positive quantity/price/FX, or unexplained negative modeled cash blocks the account snapshot. ### 4. Official versus modeled canonical selection Modify `src/jarvis_finance/services/portfolio_performance.py`: - Keep manual provisional values excluded from performance. - Add explicit True Wealth same-day source priority: 1. `truewealth_official_import` 2. `daily_truewealth_modeled_v1` - Never select by cross-source `snapshot_version` alone. - On non-official dates, use the latest complete modeled observation. - On an official date, the official value always wins even if the daily model ran later. - Preserve older modeled and official rows unchanged. This avoids the current generic non-PostFinance behavior where the highest version/capture tuple can win across unrelated sources. Continue using the existing `build_portfolio_performance()` path and therefore the existing: - `ttwror_daily_v1` - `xirr_v1` - `portfolio_performance_v2` - external-flow and coverage validation No second performance engine is needed. ### 5. Reconciliation on a new official statement Modify `confirm_truewealth_import()` in `truewealth_service.py`: 1. Before inserting the new anchor, model the statement date from the latest **prior** official anchor. 2. Exclude the statement being confirmed from anchor selection. 3. Prefer an already persisted complete modeled snapshot for that date; otherwise perform a read-only calculation from persisted inputs. 4. Record: \[ \Delta_{CHF}=OfficialTotal-ModeledTotal \] \[ \Delta_{\%}=\Delta_{CHF}/ModeledTotal \] 5. Also compare: - official new quantity versus prior anchor quantity by ISIN; - official new cash versus modeled cash by currency. 6. Persist the complete reconciliation payload in the immutable `truewealth_import_batch` audit JSON. Recommended statuses: - `matched`: absolute difference ≤ CHF 0.01; - `difference_observed`: larger difference, with no automatic correction; - `unavailable`: model inputs incomplete. Do not overload `truewealth_snapshots.reconciliation_*`; those fields already mean source-total versus source-components reconciliation. Most importantly: - do not convert quantity differences into buy/sell transactions; - do not distribute the total residual across positions; - do not invent dividends, fees, taxes, rebalances, or cash; - importing the new statement merely resets the future modeling anchor. Expose reconciliation separately in `get_truewealth_summary()` and label both value classes explicitly: - `official_confirmed` - `modeled_from_official_anchor` ### 6. Daily-job source isolation Current CLI orchestration runs market and crypto sequentially; an exception can prevent later sources from running. Introduce a small orchestrator, preferably in `performance_activation.py`: - `run_daily_investment_valuations(...)` - execute independently: - `daily_market_fx_v4` - `daily_crypto_valuation_v1` - `daily_truewealth_modeled_v1` - separate lock per source; - catch/finalize failures per source; - continue remaining sources; - return one aggregate result without changing source-specific run identity. Update `cli/main.py` to call the orchestrator under the existing: - command `run-daily-market-valuation` - environment gate `JARVIS_FINANCE_DAILY_VALUATION_ENABLED=1` - schema-version check - existing systemd timer No second timer, scheduler, worker, or provider architecture. Add True Wealth to `build_daily_valuation_job_status()`. A partial/failed True Wealth run must not change PostFinance or crypto status, and vice versa. For stricter isolation, let `run_daily_market_valuation()` accept explicit account IDs and pass only PostFinance component-role accounts. The True Wealth runner should consume persisted shared market data but write only `daily_truewealth_modeled_v1` valuation/run rows. ### 7. Backfill behavior Update `preview_performance_backfill()` and `confirm_performance_backfill()`: - Replace unconditional `truewealth_requires_new_confirmed_values`. - Permit modeled historical days only when: - a prior official anchor exists; - complete confirmed flow coverage spans the period; - every required historical price/FX input exists; - the preview fingerprint binds all inputs. - Historical confirm invokes `run_daily_truewealth_valuation()` without provider calls. - Missing historical inputs remain blockers; no carry-forward quote or total value. --- ## Fail-closed rules No modeled account snapshot when any of the following applies: - no unique active True Wealth account/scope; - no prior confirmed official anchor; - target date is before the anchor; - incomplete external-flow coverage; - unsupported or unconfirmed activity in the covered period; - missing/ambiguous ISIN or provider mapping; - missing, stale-beyond-policy, future, non-positive, or currency-mismatched price/FX; - duplicate anchor ISIN/cash currency; - negative unexplained modeled cash; - conflicting existing observation for the same run/fingerprint; - official/model source selection is ambiguous. Partial runs may be audited in `market_data_runs`, but must write **zero** account valuation rows. Existing confirmed values remain unchanged. --- ## Test map Add `tests/unit/test_sprint20d_truewealth_modeled_valuations.py` covering: 1. official anchor + frozen quantities + cash formula; 2. external deposit/withdrawal timing; 3. foreign cash and security FX; 4. exact persisted price/FX provenance; 5. preview/calculation is read-only; 6. no transaction rows created; 7. incomplete mapping/price/FX/coverage writes no valuation; 8. unconfirmed flow ignored and causes coverage/classification blocker where applicable; 9. idempotent rerun; 10. corrected input appends `snapshot_version` with `supersedes_snapshot_id`; 11. same-day official wins over modeled; 12. modeled dates feed existing TTWROR and XIRR; 13. model-to-official reconciliation audit; 14. quantity differences never create trades; 15. new official statement resets the future anchor without rewriting history; 16. one source failure does not prevent the other daily sources; 17. job status reports each source independently; 18. historical True Wealth backfill uses only persisted exact inputs; 19. no provider call during True Wealth model or backfill; 20. `MIGRATION_VERSION == 49`, integrity and FK checks. Retain the existing Sprint 12/20B/20C tests as regressions. --- ## Verification and repository state - Inspected the exact requested base: `107de783afac36bf48c95a996fc920aec2a46270`. - No source files were created or modified. - `git diff --check` passed. - Focused tests could not run because the worktree’s untracked `.venv` is a broken symlink to `/home/agent/.hermes/worktrees/FinanceManager-main-deploy/.venv`; system Python does not have `pytest`. - The only repository-status entry remains that pre-existing untracked `.venv` symlink.