finance-system-architecture

---
name: finance-system-architecture
description: "Design/implement the user's JARVIS Finance System: CHF ledger, portfolio accounting, dashboard, imports, reports, secrecy."
version: 1.0.0
author: Hermes Agent
license: MIT
metadata:
  hermes:
    tags: [finance, portfolio, accounting, dashboard, csv, fx, tax, privacy]
---

# Finance System Architecture

## When to use

Use this skill whenever the user asks about the **Finance System – Architektur & Scripts** topic or work involving the user's personal finance-management system: portfolio imports, transaction accounting, Bank/Broker/CSV processing, dashboards, reports, tax/income views, FX logic, market data jobs, alerts, or later implementation.

Do **not** use it for pure investment-strategy discussion; those belong in the separate finance strategy topic unless the user explicitly asks for technical implementation.

## Core principles

1. **No automated trading.** The system may prepare orders and recommendations, but never executes orders or connects to broker trading APIs for execution.
2. **Accounting first, opinions later.** MVP must prioritize correct transaction ledger, cash ledger, FX handling, income/tax data, and dashboard before complex scores, news, analyst ratings, or LLM analysis.
3. **Rule-based first.** Calculations, scores, alerts, rebalancing and reports must be deterministic and auditable. LLMs may only summarize/explain text or assist extraction with human review.
4. **CHF as base currency.** Store every transaction in original currency and CHF using historical FX at transaction/payment date; current valuations use current FX.
5. **Financial data secrecy.** Never expose real financial values, portfolio files, raw exports, reports with real numbers, DBs, API keys, or credentials in chat, GitHub, logs, or shared docs unless the user explicitly approves scope.
6. **Data ownership.** All data must be exportable: CSV, JSON, Parquet, SQLite/Postgres dump, Markdown/HTML/PDF reports.

## Current v0.3 decisions

See `references/spec-v0.2-decisions.md` for the baseline, `references/spec-v0.3-decisions.md` for the priority update, `references/spec-v0.4-implementation-blueprint.md` for the MVP 1 technical implementation blueprint, `references/spec-v0.4.1-corrections.md` for accepted corrections, `references/spec-v0.5-repo-blueprint.md` for the concrete repo/implementation plan, `references/mvp-safety-gate-implementation.md` for the implementation safety-gate sequence and verification commands, `references/mvp-import-pipeline-tasks-9-13.md` for the synthetic CSV import/FX/transaction pipeline pattern, `references/mvp-ledger-cash-wac-positions.md` for cash/WAC/position ledger calculation rules and tests, `references/mvp-crypto-wallet-ledger.md` for crypto wallet/assets/holdings/transfers/buy-sell/fee MVP rules, `references/crypto-xlsx-initial-snapshot-dry-run.md` for the controlled real-file XLSX crypto initial-snapshot dry-run pattern and mapping decisions, `references/crypto-xlsx-initial-snapshot-production-import.md` for productive safe-subset XLSX import operations, compatibility/partial-write recovery, and dashboard correction follow-up, `references/crypto-runtime-db-consistency-verification.md` for post-import runtime DB consistency checks, exact safe-set verification, dashboard no-live-API testing, and runtime-only report generation, `references/mvp-crypto-manual-management-ui.md` for audited dashboard add/correct/remove crypto holding workflows after an initial snapshot, `references/mvp-market-import-safety-hardening.md` for CoinGecko provider/backoff, CSV importer completion, alert lifecycle, Git-safety and crypto Decimal precision patterns, `references/git-safety-verification-coverage.md` for the verifier-warning workflow and minimum Git-safety test matrix, `references/mvp-streamlit-dashboard.md` for the safe local Streamlit dashboard shell/read-model/testing pattern, `references/mvp-crypto-report-export.md` for local-only crypto report context/export rules, and `references/github-token-push-from-drive.md` for the safe Google-Drive-token push workflow when local FinanceManager commits are ahead of GitHub. Key decisions:

- Platform/account views: Raiffeisen, PostFinance, True Wealth, Crypto-Coins, plus Gesamtansicht.
- Existing Google Drive files under `Finanzen/01 Aktueller Stand Portfolios` are **initial-snapshot reality checks**, not full ledgers.
- MVP means the smallest useful first version: correct data foundation, ledger, CHF/FX logic, platform views, crypto inventory, watchlist, reports, workflows and audit log.
- Stock/ETF tax documents are downgraded: optional reference/control module and dividend/distribution transparency, not MVP 1 and not a complete tax dossier replacement.
- Crypto inventory is promoted to MVP 1: wallet/platform-level holdings, Crypto PDF, CoinGecko valuation, stale-price warnings and manual verification.
- Crypto snapshot values are stale; use quantities only and recalculate current values via CoinGecko later.
- MVP dashboard recommendation: Streamlit + SQLite + SQLAlchemy/Pandas/Plotly; later migrateable to FastAPI + React/PostgreSQL.
- MVP inputs: manual dashboard entry, standard CSV import/export, later broker/bank export import. No direct broker integrations initially.
- Telegram crypto buys/sells/transfers should be parsed as pending transactions and only saved after explicit confirmation.
- Every portfolio or crypto change must create a History/Audit entry.

## Required data model areas

At minimum design for:

- `platforms`, `accounts`, `instruments`
- `transactions` with original currency and CHF fields
- `cash_ledger`
- derived `positions_current`
- `dividends_distributions`
- `market_prices`, `fx_rates`
- `watchlist_items`
- `alerts`
- `decision_journal`
- `reports`
- `data_quality_issues`

## CSV/import rules

- Use a canonical CSV format before broker-specific importers.
- Imports must be idempotent via external transaction ID or row hash.
- Importers must separate dry-run and commit modes, collect validation errors explicitly, and always create an `import_sessions` record even for dry runs or failed imports.
- Import results should classify rows as new, existing, duplicate, or failed; do not rely on `INSERT OR IGNORE` as the only duplicate-detection mechanism.
- Row hashes must be stable and deterministic: normalize key order, whitespace, and `None` versus empty strings while leaving domain-specific canonicalization to the importer.
- If a source file has only current holdings and no full history, import it as `initial_position_snapshot` / `initial_cash_snapshot`, not fake historical buys.
- Missing historic FX, fees, taxes, dates, investment case, target weights, or dividend details must be flagged as data-quality gaps.

## FX and return rules

For foreign currency assets, always distinguish:

- price/security performance in original currency
- FX effect to CHF
- realized P&L
- unrealized P&L
- dividends/distributions/income
- fees and taxes
- Total Return in CHF

Every transaction/income event needs:

- original amount and currency
- historical FX rate to CHF, or explicit `fx_status=missing` with a critical data-quality alert
- FX source and timestamp
- CHF equivalent at event date when FX is known

Manual FX overrides require a non-empty note and an audit-log entry.

## Ledger calculation rules

- Cash after system start must be calculated from `transactions`; `cash_balances` is only a snapshot/cache/control view.
- `initial_cash_snapshot` and `initial_position_snapshot` are explicit starting truths, not fabricated historical trades.
- Implement Weighted Average Cost as the MVP internal performance method: buy fees increase cost basis, partial sells remove proportional cost basis, realized P&L is calculated against average cost, full sells zero remaining quantity/cost basis, and dividends do not alter quantity or cost basis.
- WAC is not tax-complete; do not add FIFO/LIFO/tax-lot logic in MVP 1, but keep transaction granularity so it can be added later.
- Position calculations must keep zero-quantity historical positions visible when sale history exists.
- Missing market price or FX must degrade data quality and suppress precise Total Return CHF rather than inventing a value.
- Before productive dashboard use, add alert deduplication per rule/entity/date.
- Market-FX for non-CHF market prices belongs to the later Market/FX provider block.

## Crypto MVP rules

- Crypto inventory is MVP-critical but must remain dummy-first until mapper/dry-run layers are ready; do not touch real coin/wallet files during core implementation. After explicit user approval, real files may be read only from temp/runtime for controlled dry runs using `references/crypto-xlsx-initial-snapshot-dry-run.md`; report only aggregate counts/labels and never quantities or values. Productive safe-subset imports require a second explicit approval and must follow `references/crypto-xlsx-initial-snapshot-production-import.md`: external runtime DB only, no real CSV/XLSX/reports/DBs in Git, Git-safety before/after, import-session + audit-log creation, and aggregate-only reporting.
- Wallet names must be unique; wallet addresses are optional and must not appear in tests/fixtures.
- Validate wallet type against: Hardware Wallet, Software Wallet, Exchange, Bank/Broker, DeFi, Sonstiges.
- CoinGecko ID is strongly recommended but not hard-required; missing ID creates a data-quality warning and blocks reliable valuation.
- Crypto symbols are not globally unique; symbol conflicts require warning/manual selection, never silent merging.
- Initial crypto holdings are confirmed snapshots by wallet+asset. Store legacy values for provenance/control only, never current valuation.
- Crypto holdings should be calculated/controlled from initial snapshots plus crypto transactions, not silently overwritten.
- Crypto transfers require different source/target wallets, quantity > 0, and same asset; coin fees reduce source-wallet balance.
- Negative wallet balances should be blocked in MVP and produce a critical alert unless a future explicit correction workflow is implemented.
- Crypto buys/sells with fiat impact must create/reference general ledger transactions via `source_type='crypto'`/`source_id` so cash remains ledger-consistent.
- Fiat fees should be ledger fees when unambiguous; avoid double counting when a fee is already embedded in sell net proceeds.
- Unknown fee currency creates a data-quality warning instead of invented accounting.
- Crypto Cost Basis/WAC is not required for MVP 1; MVP 1 prioritizes crypto holdings, wallet structure, current valuation, reports and audit.
- Crypto report context/export must be renderer-neutral and local-only: build from SQLite, aggregate by coin and wallet, use local cached prices only, preserve Decimal/Text values, ignore legacy snapshot values for current valuation, include data-quality warnings, and write generated files only to the external runtime reports directory.
- Crypto PDF export should be optional-dependency tolerant: prefer PDF when a renderer is available, otherwise fall back to HTML/Markdown with a clear warning while still recording report metadata and audit.
- After productive real-data crypto imports, run the post-import runtime verification in `references/crypto-runtime-db-consistency-verification.md`: exact safe-set membership checks, aggregate-only counts, audit/import-session traceability, dashboard render with no live API, runtime-only report generation, and Git-safety before/after.
- Manual dashboard additions/corrections/removals after an initial snapshot must follow `references/mvp-crypto-manual-management-ui.md`: no silent `crypto_holdings` mutation, explicit confirmation, mandatory notes for adjustments/removals, audit-log entries with old/new values, Decimal/TEXT quantity handling, no negative holdings, and no live API lookup during render.
- Crypto tax logic, tax lots, FIFO/LIFO and detailed cost basis are later modules.
- Keep crypto assets separate from `instruments` for MVP 1, but later evaluate a shared Asset Master if it improves reporting, performance views or watchlists.

## Income/tax rules

For dividends and ETF distributions, separately store:

- gross dividend/distribution
- Swiss withholding tax 35%
- foreign withholding tax
- other tax/withholding if present
- net amount
- original currency
- historic FX to CHF
- gross/net CHF equivalents
- platform/account
- ISIN/ticker
- payment date
- ex-dividend date if available

Plan later ESTV/ICTax support for Swiss year-end values and tax reports, but label outputs as technical reports, not tax advice.

## Market-data rules

- Dashboard reads primarily from local DB, not directly from APIs on page load.
- First Streamlit MVP should be a safe, local, read-first shell over SQLite with explicit synthetic demo DB creation only; page render functions must not call CoinGecko/market APIs, and tests should prove that by monkeypatching provider methods to fail.
- Use scheduled jobs to update prices/FX/news.
- Store provider, timestamp, quality status and freshness for every price/FX point.
- Implement caching, local historical storage, batch requests where possible, provider rate-limit queues, and backoff on HTTP 429.
- CoinGecko integrations must be mockable and unit-tested without real API calls; treat HTTP 429 as non-fatal and return a non-fresh data-quality status plus alert after configured retries.
- Candidate providers: Finnhub, Financial Modeling Prep, Twelve Data, Stooq, Alpha Vantage selectively, Yahoo Finance fallback only; CoinGecko for crypto; later DefiLlama; ESTV/ICTax for Swiss tax values.

## Crypto Decimal precision rules

SQLite has no exact native DECIMAL. In the crypto context, store quantities, prices, crypto fees and valuation-relevant amounts as exact Decimal strings (`TEXT`) and compute with Python `Decimal`. Never convert crypto quantity/price/fee/valuation through float/REAL. Format Decimal writes with `format(value, "f")` so small quantities such as `0.00000001` do not become scientific notation. CSV importers must validate Decimal strings, and tests should assert high-precision preservation plus SQLite text storage.

## Alert lifecycle rules

Alerts must be deduplicated and lifecycle-aware before dashboards run repeated calculations. Use a dedup key from `rule_id`, `entity_type`, `entity_id`, `priority` and an error fingerprint. Alerts should carry `status` (`active`, `resolved`, `muted`), `resolved_at`, optional `muted_until`, `last_seen_at`, `occurrence_count`, `fingerprint` and `dedup_key`. Re-seeing an identical active/resolved/muted alert should update `last_seen_at`/`occurrence_count` rather than create a new active row; changed fingerprints may create a new alert.

## Git/GitHub safety

Code/scripts/docs may be stored in a private Git repo, but never commit financial data. Use the `github-repo-management` skill and its `references/sensitive-code-backup.md` guidance.

Minimum `.gitignore` for finance repos must block `data/`, `reports/`, `exports/`, `backups/`, DB files, spreadsheets, PDFs, DOCX, raw CSV/JSON, `.env`, tokens, credentials and client secrets. Only synthetic examples under `examples/` are acceptable.

## MVP implementation safety gate

Before any real broker/bank/portfolio/crypto data is imported, build and verify the safety foundation described in `references/mvp-safety-gate-implementation.md`: repo skeleton, `.gitignore`, `.env.example`, runtime outside repo, settings validation, SQLite schema, synthetic fixtures, audit log, CSV dry-run, duplicate protection, and Git-safety scan. Run the safety scan before staging, committing, or pushing. If GitHub credentials are needed from Drive, use them only in a temporary location and never echo token values.

## Common pitfalls

- Building a fancy dashboard before ledger correctness is verified.
- Treating a holdings snapshot as a real transaction history.
- Mixing price P&L and FX P&L.
- Letting dashboard refreshes hammer APIs.
- Committing realistic sample data to Git because the repo is private. Private GitHub is not a financial data vault.
- Showing real portfolio values back to the user unnecessarily in summaries.
- Designing Git-safety scans that block source package names like `src/.../imports` or `src/.../reports`; block top-level runtime/data directories instead.
- Running compile/tests before Git-safety and forgetting generated `__pycache__`; remove or ignore generated caches before safety scans. `pytest` may recreate `.pytest_cache`, so remove `__pycache__`/`.pytest_cache` again after test execution and before `git-safety-scan`.
- Productive real-data imports can partially write before surfacing schema/importer compatibility issues. Do not rerun blindly or dump row-level data; inspect aggregate counts and unique identifiers needed for idempotency, patch/complete via missing-only inserts, then verify committed import session, audit log, Git clean and Git-safety OK.
- After productive crypto imports, do a separate aggregate-only consistency pass before building new features: DB/runtime outside repo, exact safe-subset membership, no duplicate/negative/orphan holdings, correct snapshot date, null legacy-value fields when ignored, import sessions committed, holding-level audit traces present, dashboard local-read/no-live-API verified, and report generated only under runtime reports.
- Letting a dashboard correction UI mutate crypto holdings directly. Add/remove/edit operations must become audited transactions or `manual_adjustment` events with mandatory notes where appropriate; preserve history instead of silently overwriting balances.
- Treating verifier/file-mutation warnings as either automatically true or automatically false. Check the exact commit with `git show --name-status --patch <commit> -- <file>`, then separately verify whether the test matrix actually covers the promised cases.
- Adding broad Git-safety tests that only assert `len(findings) >= N`; prefer explicit assertions for token filename, PAT patterns, private keys, CSV/JSON path policy, runtime/data directories, allowed synthetic paths, `config.example/`, and normal `src/` false-positive protection.


systematic-debugging

---
name: systematic-debugging
description: "4-phase root cause debugging: understand bugs before fixing."
version: 1.2.0
author: Hermes Agent (adapted from obra/superpowers)
license: MIT
metadata:
  hermes:
    tags: [debugging, troubleshooting, problem-solving, root-cause, investigation]
    related_skills: [test-driven-development, writing-plans, subagent-driven-development]
---

# Systematic Debugging

## Overview

Random fixes waste time and create new bugs. Quick patches mask underlying issues.

**Core principle:** ALWAYS find root cause before attempting fixes. Symptom fixes are failure.

**Violating the letter of this process is violating the spirit of debugging.**

## The Iron Law

```
NO FIXES WITHOUT ROOT CAUSE INVESTIGATION FIRST
```

If you haven't completed Phase 1, you cannot propose fixes.

## When to Use

Use for ANY technical issue:
- Test failures
- Bugs in production
- Unexpected behavior
- Performance problems
- Build failures
- Integration issues

**Use this ESPECIALLY when:**
- Under time pressure (emergencies make guessing tempting)
- "Just one quick fix" seems obvious
- You've already tried multiple fixes
- Previous fix didn't work
- You don't fully understand the issue

**Don't skip when:**
- Issue seems simple (simple bugs have root causes too)
- You're in a hurry (rushing guarantees rework)
- Someone wants it fixed NOW (systematic is faster than thrashing)

## The Four Phases

You MUST complete each phase before proceeding to the next.

---

## Phase 1: Root Cause Investigation

**BEFORE attempting ANY fix:**

### 1. Read Error Messages Carefully

- Don't skip past errors or warnings
- They often contain the exact solution
- Read stack traces completely
- Note line numbers, file paths, error codes

**Action:** Use `read_file` on the relevant source files. Use `search_files` to find the error string in the codebase.

### 2. Reproduce Consistently

- Can you trigger it reliably?
- What are the exact steps?
- Does it happen every time?
- If not reproducible → gather more data, don't guess

**Action:** Use the `terminal` tool to run the failing test or trigger the bug:

```bash
# Run specific failing test
pytest tests/test_module.py::test_name -v

# Run with verbose output
pytest tests/test_module.py -v --tb=long
```

### 3. Check Recent Changes

- What changed that could cause this?
- Git diff, recent commits
- New dependencies, config changes

**Action:**

```bash
# Recent commits
git log --oneline -10

# Uncommitted changes
git diff

# Changes in specific file
git log -p --follow src/problematic_file.py | head -100
```

### 4. Gather Evidence in Multi-Component Systems

**WHEN system has multiple components (API → service → database, CI → build → deploy):**

**BEFORE proposing fixes, add diagnostic instrumentation:**

For EACH component boundary:
- Log what data enters the component
- Log what data exits the component
- Verify environment/config propagation
- Check state at each layer

Run once to gather evidence showing WHERE it breaks.
THEN analyze evidence to identify the failing component.
THEN investigate that specific component.

### 5. Trace Data Flow

**WHEN error is deep in the call stack:**

- Where does the bad value originate?
- What called this function with the bad value?
- Keep tracing upstream until you find the source
- Fix at the source, not at the symptom

**Action:** Use `search_files` to trace references:

```python
# Find where the function is called
search_files("function_name(", path="src/", file_glob="*.py")

# Find where the variable is set
search_files("variable_name\\s*=", path="src/", file_glob="*.py")
```

### Phase 1 Completion Checklist

- [ ] Error messages fully read and understood
- [ ] Issue reproduced consistently
- [ ] Recent changes identified and reviewed
- [ ] Evidence gathered (logs, state, data flow)
- [ ] Problem isolated to specific component/code
- [ ] Root cause hypothesis formed

**STOP:** Do not proceed to Phase 2 until you understand WHY it's happening.

---

## Phase 2: Pattern Analysis

**Find the pattern before fixing:**

### 1. Find Working Examples

- Locate similar working code in the same codebase
- What works that's similar to what's broken?

**Action:** Use `search_files` to find comparable patterns:

```python
search_files("similar_pattern", path="src/", file_glob="*.py")
```

### 2. Compare Against References

- If implementing a pattern, read the reference implementation COMPLETELY
- Don't skim — read every line
- Understand the pattern fully before applying

### 3. Identify Differences

- What's different between working and broken?
- List every difference, however small
- Don't assume "that can't matter"

### 4. Understand Dependencies

- What other components does this need?
- What settings, config, environment?
- What assumptions does it make?

---

## Phase 3: Hypothesis and Testing

**Scientific method:**

### 1. Form a Single Hypothesis

- State clearly: "I think X is the root cause because Y"
- Write it down
- Be specific, not vague

### 2. Test Minimally

- Make the SMALLEST possible change to test the hypothesis
- One variable at a time
- Don't fix multiple things at once

### 3. Verify Before Continuing

- Did it work? → Phase 4
- Didn't work? → Form NEW hypothesis
- DON'T add more fixes on top

### 4. When You Don't Know

- Say "I don't understand X"
- Don't pretend to know
- Ask the user for help
- Research more

---

## Phase 4: Implementation

**Fix the root cause, not the symptom:**

### 1. Create Failing Test Case

- Simplest possible reproduction
- Automated test if possible
- MUST have before fixing
- Use the `test-driven-development` skill

### 2. Implement Single Fix

- Address the root cause identified
- ONE change at a time
- No "while I'm here" improvements
- No bundled refactoring

### 3. Verify Fix

```bash
# Run the specific regression test
pytest tests/test_module.py::test_regression -v

# Run full suite — no regressions
pytest tests/ -q
```

### 4. If Fix Doesn't Work — The Rule of Three

- **STOP.**
- Count: How many fixes have you tried?
- If < 3: Return to Phase 1, re-analyze with new information
- **If ≥ 3: STOP and question the architecture (step 5 below)**
- DON'T attempt Fix #4 without architectural discussion

### 5. If 3+ Fixes Failed: Question Architecture

**Pattern indicating an architectural problem:**
- Each fix reveals new shared state/coupling in a different place
- Fixes require "massive refactoring" to implement
- Each fix creates new symptoms elsewhere

**STOP and question fundamentals:**
- Is this pattern fundamentally sound?
- Are we "sticking with it through sheer inertia"?
- Should we refactor the architecture vs. continue fixing symptoms?

**Discuss with the user before attempting more fixes.**

This is NOT a failed hypothesis — this is a wrong architecture.

---

## Red Flags — STOP and Follow Process

If you catch yourself thinking:
- "Quick fix for now, investigate later"
- "Just try changing X and see if it works"
- "Add multiple changes, run tests"
- "Skip the test, I'll manually verify"
- "It's probably X, let me fix that"
- "I don't fully understand but this might work"
- "Pattern says X but I'll adapt it differently"
- "Here are the main problems: [lists fixes without investigation]"
- Proposing solutions before tracing data flow
- **"One more fix attempt" (when already tried 2+)**
- **Each fix reveals a new problem in a different place**

**ALL of these mean: STOP. Return to Phase 1.**

**If 3+ fixes failed:** Question the architecture (Phase 4 step 5).

## Common Rationalizations

| Excuse | Reality |
|--------|---------|
| "Issue is simple, don't need process" | Simple issues have root causes too. Process is fast for simple bugs. |
| "Emergency, no time for process" | Systematic debugging is FASTER than guess-and-check thrashing. |
| "Just try this first, then investigate" | First fix sets the pattern. Do it right from the start. |
| "I'll write test after confirming fix works" | Untested fixes don't stick. Test first proves it. |
| "Multiple fixes at once saves time" | Can't isolate what worked. Causes new bugs. |
| "Reference too long, I'll adapt the pattern" | Partial understanding guarantees bugs. Read it completely. |
| "I see the problem, let me fix it" | Seeing symptoms ≠ understanding root cause. |
| "One more fix attempt" (after 2+ failures) | 3+ failures = architectural problem. Question the pattern, don't fix again. |

## Quick Reference

| Phase | Key Activities | Success Criteria |
|-------|---------------|------------------|
| **1. Root Cause** | Read errors, reproduce, check changes, gather evidence, trace data flow | Understand WHAT and WHY |
| **2. Pattern** | Find working examples, compare, identify differences | Know what's different |
| **3. Hypothesis** | Form theory, test minimally, one variable at a time | Confirmed or new hypothesis |
| **4. Implementation** | Create regression test, fix root cause, verify | Bug resolved, all tests pass |

## Hermes Agent Integration

### Investigation Tools

Use these Hermes tools during Phase 1:

- **`search_files`** — Find error strings, trace function calls, locate patterns
- **`read_file`** — Read source code with line numbers for precise analysis
- **`terminal`** — Run tests, check git history, reproduce bugs
- **`web_search`/`web_extract`** — Research error messages, library docs

### With delegate_task

For complex multi-component debugging, dispatch investigation subagents:

```python
delegate_task(
    goal="Investigate why [specific test/behavior] fails",
    context="""
    Follow systematic-debugging skill:
    1. Read the error message carefully
    2. Reproduce the issue
    3. Trace the data flow to find root cause
    4. Report findings — do NOT fix yet

    Error: [paste full error]
    File: [path to failing code]
    Test command: [exact command]
    """,
    toolsets=['terminal', 'file']
)
```

### With test-driven-development

When fixing bugs:
1. Write a test that reproduces the bug (RED)
2. Debug systematically to find root cause
3. Fix the root cause (GREEN)
4. The test proves the fix and prevents regression

## Real-World Impact

From debugging sessions:
- Systematic approach: 15-30 minutes to fix
- Random fixes approach: 2-3 hours of thrashing
- First-time fix rate: 95% vs 40%
- New bugs introduced: Near zero vs common

**No shortcuts. No guessing. Systematic always wins.**

## Advanced Pitfalls

### Variable Shadowing

If you import `text` from SQLAlchemy (`from sqlalchemy import text`) and then assign to a local variable named `text`, it shadows the import. This causes `UnboundLocalError` or silent misbehavior.

**Rule:** Never use imported names as local variable names. Use descriptive alternatives: `pdf_text`, `page_text`, `ocr_text`, `query_result`, etc.

**Detection:** After any `import`, scan the function body for variables with the same name.

### CHECK Constraint Failures

SQLite CHECK constraints can fail silently in ORM layers. An INSERT may raise `IntegrityError` but the actual constraint definition is hidden.

**Debug:** Always inspect the schema:
```sql
PRAGMA table_info(table_name);
SELECT sql FROM sqlite_master WHERE type='table' AND name='table_name';
```

### Daemon Process Management

Long-running daemons (watchdog, monitoring services) may not restart cleanly with `systemctl` if root permissions are unavailable.

**Pattern:** Use wrapper scripts with PID file management:
```bash
kill $(cat pidfile) 2>/dev/null
nohup python3.12 script.py > log 2>&1 &
echo $! > pidfile
```


finance-system-architecture

# MVP Crypto Manual Management UI

Session learning from implementing the manual Crypto Manage dashboard block after the productive safe-subset crypto initial snapshot.

## Trigger

Use this reference when adding or reviewing dashboard features that let the user manually add, correct, or remove crypto holdings after an initial snapshot.

## Required pattern

- Do not mutate `crypto_holdings` silently from the UI.
- Model manual changes as one of:
  - `initial_snapshot_addition` for a missing initial holding, only if the wallet+asset holding does not already exist.
  - `manual_adjustment` transaction/event for corrections, reductions, and zeroing out a balance.
  - future dedicated `crypto_buy`/sell/transfer workflows when ledger/cash integration is present.
- Always require explicit confirmation before writing.
- Require a non-empty note for `manual_adjustment` and all removal/reduction flows.
- Write an audit-log entry for every confirmed change, including `old_values` and `new_values` for corrections.
- Preserve history; zeroing a holding is a transaction/adjustment, not a row delete.
- Calculate holdings from initial snapshots plus crypto transactions, including `manual_adjustment` deltas.
- Keep quantities as exact `Decimal` strings/TEXT. Never parse through float; format writes with `format(decimal, "f")`.
- Block negative resulting wallet balances.

## CoinGecko/manual mapping rules

- Symbol alone is never automatically unique.
- Let the user manually supply/select CoinGecko ID, but allow saving without one only with a warning/quality alert.
- Missing or uncertain CoinGecko mapping should create a data-quality alert.
- Dashboard render must not perform live CoinGecko/API search. Prepare later cache/provider-backed search separately.

## UI safety checklist

- Show a review summary before save.
- Save only after explicit confirmation checkbox/button.
- No batch/mass updates without a separate review workflow.
- Clear validation errors: wallet required, coin name required, symbol required, date required, quantity > 0 for additions/reductions, new quantity >= 0 for corrections.
- After successful save, show only IDs/status/warnings or refreshed local read-model; avoid echoing real quantities/values into chat/logs.
- Provide a dashboard-visible manual-review/reminder list for skipped/unsafe coins, but do not auto-import them.

## Test checklist

- Add initial snapshot addition.
- Add coin with CoinGecko ID.
- Add coin without CoinGecko ID creates warning/alert.
- Decimal precision such as `0.00000001` remains exact text.
- Manual adjustment with note succeeds.
- Manual adjustment without note fails.
- Reduce/zero holding preserves row/history rather than deleting.
- Negative resulting holding is rejected.
- Every confirmed change has audit log.
- Corrections record `old_values` and `new_values`.
- Dashboard modules import/render without live API calls.
- Compile, full pytest, Git-safety, git status before commit/push.


# MVP Streamlit Dashboard Pattern

Use this reference for the first local dashboard block of the JARVIS Finance System.

## Scope

Build a safe, local, read-first Streamlit MVP over a local SQLite database with synthetic data only.

Required pages for the v0.5 shell:

- Command Center
- Portfolio
- Platforms
- Equities/ETFs
- Crypto
- Wallets
- Ledger
- Watchlist
- Reports
- Alerts
- Audit
- Settings/Data Quality

## Safety rules

- Never import real finance files, portfolio exports, wallet files, wallet addresses, secrets, runtime data, DBs, reports, or cache files into the repo.
- Do not call market-data providers from dashboard page render functions.
- CoinGecko/current crypto prices shown in the dashboard must come only from local DB/cache tables.
- If no DB exists, show a clear error and an explicit action/hint for creating a synthetic demo DB; do not automatically create or mutate real data.
- Runtime paths must remain outside the repo; keep a test that blocks in-repo runtime configuration.
- Alert lifecycle mutations (`acknowledge`, `resolve`, `mute`) should remain read-only in the first MVP unless explicit mutation code and tests exist.

## Implementation pattern

- Keep `dashboard/main.py` thin: load settings, validate runtime, connect to an existing SQLite DB, show navigation, dispatch to page modules.
- Put all SQL/derived read models in a separate dashboard data module so they can be unit-tested without Streamlit.
- Use simple page modules with a `PAGE_TITLE` and `render(st, conn)` signature.
- Use a tiny fake Streamlit object in tests for page smoke rendering; avoid requiring Streamlit to be installed for unit tests.
- Use Python `Decimal` for crypto quantities/prices and format via `format(value, "f")`; never format crypto amounts through float.
- Tables are enough for MVP. Prefer correctness, clear status/warnings, and data truth over polished UI.

## Demo data pattern

- Seed demo DB only by explicit user action or test helper.
- Use `tests/fixtures/` and domain APIs for synthetic records if importers are stricter than legacy fixtures.
- Store synthetic crypto prices in local DB/cache with provider metadata such as `synthetic-cache`.
- Add synthetic audit and alert records so Audit/Alerts pages can be exercised.

## Minimum tests

- Dashboard modules import and selected page modules smoke-render.
- Dashboard reader functions return expected synthetic data.
- Crypto page render does not call CoinGecko/API provider methods.
- Decimal crypto quantities preserve exact text formatting and avoid scientific notation.
- Alert lifecycle fields load readably (`status`, `priority`, `occurrence_count`, `last_seen_at`, `fingerprint`, `dedup_key`).
- Runtime path inside repo remains blocked.

## Verification sequence

Run from repo root before committing/pushing:

```bash
find . -type d \( -name __pycache__ -o -name .pytest_cache \) -exec rm -rf {} +
PYTHONPATH=src python -m compileall src tests
PYTHONPATH=src pytest tests -q
find . -type d \( -name __pycache__ -o -name .pytest_cache \) -exec rm -rf {} +
PYTHONPATH=src python -m jarvis_finance.cli.main git-safety-scan .
git status --short
```

Important: `pytest` may recreate `.pytest_cache`; remove it before Git-safety because the finance Git-safety scanner intentionally blocks runtime/cache files.


# Crypto Runtime DB Consistency Verification

Use after a productive crypto initial-snapshot import or after any recovery from an importer/schema compatibility issue. The goal is to prove the external runtime SQLite DB is consistent without exposing real quantities, values, wallet addresses, CSV rows, or report contents.

## Non-negotiables

- Do not build new features during verification.
- Do not print real quantities, CHF values, wallet addresses, raw rows, generated report contents, or real file contents in chat.
- The productive DB and generated reports must be under the configured runtime path outside the Git repo.
- Remove generated `__pycache__` / `.pytest_cache` before Git-safety checks so generated cache files do not pollute results.
- Run `git status --short` and `PYTHONPATH=src python -m jarvis_finance.cli.main git-safety-scan .` before and after checks.

## Verification workflow

1. **Runtime/Git safety**
   - Resolve repo root, runtime base, DB path, and reports dir.
   - Assert runtime base, DB path, and reports dir are outside the repo.
   - Search the repo for non-synthetic `.csv`, `.xlsx`, `.xls`, `.pdf`, `.sqlite`, `.sqlite3`, `.db` files; allow only known synthetic/example/test fixtures.

2. **Import consistency**
   - Report aggregate counts only: wallets, crypto assets, holdings, safe-subset assets, safe-subset holdings.
   - For each approved safe coin, report only the number of holdings, never quantity/value.
   - Assert zero duplicate holdings for `(asset_id, wallet_id)`.
   - Assert zero negative holdings.
   - Assert zero holdings with null/blank quantity.
   - Assert zero holdings without wallet or without asset.
   - Assert zero imported holdings for ignored coins or manual-review coins.
   - Assert all initial snapshot rows have `legacy_snapshot_date` / verification date matching the confirmed snapshot date.
   - Assert legacy CHF/value fields are null when the user instructed legacy values to be ignored.
   - Assert every imported crypto asset has a CoinGecko ID.

3. **Safe-subset detection**
   - Prefer exact checks by approved `coingecko_id` and exact upper-case symbol, not broad substring scans.
   - Broad `LIKE '%term%'` scans can produce false positives (`POL` inside unrelated labels, legacy substrings in names, etc.). Use them only as an exploratory smoke test, then confirm with exact-ID/symbol queries before reporting a problem.

4. **Audit and import sessions**
   - Confirm committed wallet and holdings import sessions exist.
   - Confirm no `started`, `failed`, or `partial` sessions remain unless they are intentionally documented as technical intermediate artifacts.
   - Confirm audit log entries exist.
   - Confirm every imported holding has a matching `audit_log` entry for the initial snapshot action/entity.

5. **Dashboard local-read test**
   - Exercise the dashboard read model and crypto page against the real runtime DB.
   - Block network sockets or monkeypatch providers during render to prove no live API call is needed.
   - Report only aggregate row counts and whether rendering works.

6. **Report generation**
   - Generate the crypto inventory report only into the runtime reports directory.
   - Prefer PDF if the renderer exists; accept HTML fallback with a warning.
   - Record/report only: generated yes/no, format, reports directory, data-quality status, coin count, wallet count, and whether ignored/manual-review coins are absent.
   - Re-run Git-safety and Git status after generation.

## Expected aggregate final report shape

- Consistencyprüfung bestanden: ja/nein
- Anzahl Wallets / Assets / Holdings
- Probleme/Warnings count only
- Dashboard reads runtime DB: ja/nein
- Crypto report generated: ja/nein
- Report format and runtime reports directory only
- Git-safety result and Git status
- Open risks such as missing local prices or PDF-renderer fallback
- Whether the next feature step is safe to start

## Common pitfalls

- Reporting a broad substring scan as a real violation. Always confirm exact `coingecko_id`/symbol membership first.
- Treating missing local price data as import corruption. It is a data-quality warning unless current valuation was part of the import scope.
- Printing the generated report path with a unique filename is usually unnecessary; the runtime reports directory is enough for the chat summary.
- Letting the report/dashboard test call CoinGecko or market APIs during page load. Dashboard/report generation should use cached/local DB data only.


Explanation:
**Checking Streamlit Availability**

I need to inspect the availability of Streamlit, so I should check the config paths and any commands I might need to use in the terminal. It's a straightforward task, but I want to make sure I'm thorough in my checks. I need to ensure that everything is set up correctly to avoid any issues later. I'll get started on that right away!

todo

Streamlit, Dashboard-Entry und Runtime-DB-Konfiguration prüfen

Einzel-Testcoin BTC vorbereiten und sicheren Dashboard-Startbefehl formulieren

Nach-Speichern-Prüfskript/-Checks ohne Mengen-Ausgabe vorbereiten

UI-Testablauf und Stop-Punkt vor produktiver Änderung berichten


terminal

set -euo pipefail
cd /home/agent/.hermes/repos/FinanceManager
python - <<'PY'
import importlib.util
for mod in ['streamlit','jarvis_finance']:
    print(f'{mod}=' + ('available' if importlib.util.find_spec(mod) else 'missing'))
PY
PYTHONPATH=src python -m jarvis_finance.cli.main --help | sed -n '1,160p'


Output:
streamlit=missing
jarvis_finance=missing
usage: finance [-h] {init-db,migrate,git-safety-scan} ...

positional arguments:
  {init-db,migrate,git-safety-scan}

options:
  -h, --help            show this help message and exit


     1|[build-system]
     2|requires = ["setuptools>=68"]
     3|build-backend = "setuptools.build_meta"
     4|
     5|[project]
     6|name = "jarvis-finance"
     7|version = "0.1.0"
     8|description = "JARVIS Finance System MVP - code, docs and synthetic fixtures only"
     9|requires-python = ">=3.11"
    10|dependencies = []
    11|
    12|[project.optional-dependencies]
    13|dev = ["pytest>=8", "ruff>=0.5"]
    14|dashboard = ["streamlit", "pandas", "plotly"]
    15|reports = ["jinja2"]
    16|
    17|[tool.pytest.ini_options]
    18|testpaths = ["tests"]
    19|pythonpath = ["src"]
    20|
    21|[tool.ruff]
    22|line-length = 100
    23|src = ["src", "tests"]
    24|


     1|[app]
     2|base_currency = "CHF"
     3|timezone = "Europe/Zurich"
     4|
     5|[runtime]
     6|base_dir = "~/jarvis_runtime/finance-system"
     7|data_dir = "~/jarvis_runtime/finance-system/data"
     8|reports_dir = "~/jarvis_runtime/finance-system/reports"
     9|imports_dir = "~/jarvis_runtime/finance-system/imports"
    10|exports_dir = "~/jarvis_runtime/finance-system/exports"
    11|backups_dir = "~/jarvis_runtime/finance-system/backups"
    12|
    13|[database]
    14|path = "~/jarvis_runtime/finance-system/data/finance.sqlite3"
    15|
    16|[security]
    17|require_git_safety_scan = true
    18|block_runtime_inside_repo = true
    19|


     1|from __future__ import annotations
     2|
     3|import os
     4|from dataclasses import dataclass
     5|from pathlib import Path
     6|
     7|from .paths import DEFAULT_RUNTIME_DIR, RuntimePaths, is_relative_to
     8|
     9|
    10|@dataclass(frozen=True)
    11|class Settings:
    12|    env: str = "local"
    13|    base_currency: str = "CHF"
    14|    timezone: str = "Europe/Zurich"
    15|    runtime_paths: RuntimePaths = RuntimePaths.from_base(DEFAULT_RUNTIME_DIR)
    16|    log_level: str = "INFO"
    17|    block_runtime_inside_repo: bool = True
    18|
    19|    @property
    20|    def db_path(self) -> Path:
    21|        return self.runtime_paths.db_path
    22|
    23|
    24|def find_repo_root(start: Path | None = None) -> Path:
    25|    current = (start or Path.cwd()).resolve()
    26|    for candidate in [current, *current.parents]:
    27|        if (candidate / ".git").exists() or (candidate / "pyproject.toml").exists():
    28|            return candidate
    29|    return current
    30|
    31|
    32|def load_settings(repo_root: Path | None = None, environ: dict[str, str] | None = None) -> Settings:
    33|    env = environ or os.environ
    34|    runtime_dir = Path(env.get("JARVIS_FINANCE_RUNTIME_DIR", str(DEFAULT_RUNTIME_DIR))).expanduser()
    35|    paths = RuntimePaths.from_base(runtime_dir)
    36|    db_override = env.get("JARVIS_FINANCE_DB_PATH")
    37|    if db_override:
    38|        object.__setattr__(paths, "db_path", Path(db_override).expanduser().resolve())
    39|    settings = Settings(
    40|        env=env.get("JARVIS_FINANCE_ENV", "local"),
    41|        base_currency=env.get("JARVIS_FINANCE_BASE_CURRENCY", "CHF"),
    42|        timezone=env.get("JARVIS_FINANCE_TIMEZONE", "Europe/Zurich"),
    43|        runtime_paths=paths,
    44|        log_level=env.get("JARVIS_FINANCE_LOG_LEVEL", "INFO"),
    45|        block_runtime_inside_repo=env.get("JARVIS_FINANCE_BLOCK_RUNTIME_INSIDE_REPO", "1") not in {"0", "false", "False"},
    46|    )
    47|    validate_settings(settings, repo_root=repo_root)
    48|    return settings
    49|
    50|
    51|def validate_settings(settings: Settings, repo_root: Path | None = None) -> None:
    52|    root = (repo_root or find_repo_root()).resolve()
    53|    if settings.block_runtime_inside_repo and is_relative_to(settings.runtime_paths.base_dir, root):
    54|        raise ValueError(f"Runtime directory must be outside the Git repository: {settings.runtime_paths.base_dir}")
    55|    if settings.base_currency != "CHF":
    56|        raise ValueError("MVP 1 base currency must be CHF")
    57|
    58|
    59|def ensure_runtime_dirs(settings: Settings) -> None:
    60|    settings.runtime_paths.ensure_dirs()
    61|


     1|from __future__ import annotations
     2|
     3|from dataclasses import dataclass
     4|from pathlib import Path
     5|
     6|DEFAULT_RUNTIME_DIR = Path("~/jarvis_runtime/finance-system").expanduser()
     7|
     8|
     9|@dataclass(frozen=True)
    10|class RuntimePaths:
    11|    base_dir: Path
    12|    data_dir: Path
    13|    imports_dir: Path
    14|    exports_dir: Path
    15|    reports_dir: Path
    16|    backups_dir: Path
    17|    secrets_dir: Path
    18|    logs_dir: Path
    19|    db_path: Path
    20|
    21|    @classmethod
    22|    def from_base(cls, base_dir: str | Path = DEFAULT_RUNTIME_DIR) -> "RuntimePaths":
    23|        base = Path(base_dir).expanduser().resolve()
    24|        return cls(
    25|            base_dir=base,
    26|            data_dir=base / "data",
    27|            imports_dir=base / "imports",
    28|            exports_dir=base / "exports",
    29|            reports_dir=base / "reports",
    30|            backups_dir=base / "backups",
    31|            secrets_dir=base / "secrets",
    32|            logs_dir=base / "logs",
    33|            db_path=base / "data" / "finance.sqlite3",
    34|        )
    35|
    36|    def ensure_dirs(self) -> None:
    37|        for path in [
    38|            self.data_dir,
    39|            self.imports_dir,
    40|            self.exports_dir,
    41|            self.reports_dir,
    42|            self.backups_dir,
    43|            self.secrets_dir,
    44|            self.logs_dir,
    45|        ]:
    46|            path.mkdir(parents=True, exist_ok=True)
    47|
    48|
    49|def is_relative_to(path: Path, parent: Path) -> bool:
    50|    try:
    51|        path.resolve().relative_to(parent.resolve())
    52|        return True
    53|    except ValueError:
    54|        return False
    55|


     1|from __future__ import annotations
     2|
     3|from jarvis_finance.dashboard.main import run_app
     4|
     5|
     6|if __name__ == "__main__":
     7|    run_app()
     8|


     1|from __future__ import annotations
     2|
     3|from pathlib import Path
     4|from sqlite3 import Connection
     5|
     6|from jarvis_finance.config.settings import ensure_runtime_dirs, find_repo_root, load_settings
     7|from jarvis_finance.dashboard.demo_data import load_synthetic_demo_data
     8|from jarvis_finance.dashboard.pages import PAGES
     9|from jarvis_finance.storage.database import connect
    10|from jarvis_finance.storage.migrations import apply_migrations
    11|
    12|
    13|def safe_connect_existing_db(db_path: Path) -> Connection | None:
    14|    if not db_path.exists():
    15|        return None
    16|    conn = connect(db_path)
    17|    apply_migrations(conn)
    18|    return conn
    19|
    20|
    21|def create_demo_db(db_path: Path) -> None:
    22|    db_path.parent.mkdir(parents=True, exist_ok=True)
    23|    conn = connect(db_path)
    24|    try:
    25|        load_synthetic_demo_data(conn)
    26|    finally:
    27|        conn.close()
    28|
    29|
    30|def run_app() -> None:
    31|    import streamlit as st
    32|
    33|    st.set_page_config(page_title="JARVIS Finance MVP", layout="wide")
    34|    st.sidebar.title("JARVIS Finance")
    35|    st.sidebar.caption("MVP: lokal, synthetisch, read-first")
    36|
    37|    try:
    38|        repo_root = find_repo_root()
    39|        settings = load_settings(repo_root=repo_root)
    40|    except Exception as exc:
    41|        st.error(f"Settings-/Runtime-Ladeprüfung fehlgeschlagen: {exc}")
    42|        st.info("Prüfe JARVIS_FINANCE_RUNTIME_DIR. Runtime muss ausserhalb des Repos liegen.")
    43|        return
    44|
    45|    ensure_runtime_dirs(settings)
    46|    db_path = settings.db_path
    47|    conn = safe_connect_existing_db(db_path)
    48|
    49|    if conn is None:
    50|        st.title("JARVIS Finance Dashboard")
    51|        st.error(f"Keine lokale SQLite-DB gefunden: {db_path}")
    52|        st.info("Das Dashboard erstellt oder verändert keine echte Datenbank automatisch.")
    53|        st.caption("Für lokale Entwicklung kann explizit eine synthetische Dummy-DB aus tests/fixtures geladen werden.")
    54|        if st.button("Synthetische Dummy-DB jetzt erstellen", type="primary"):
    55|            create_demo_db(db_path)
    56|            st.success("Synthetische Dummy-DB erstellt. Bitte Seite neu laden.")
    57|        return
    58|
    59|    try:
    60|        page_names = [page.PAGE_TITLE for page in PAGES]
    61|        selected = st.sidebar.radio("Navigation", page_names, index=0)
    62|        st.sidebar.write(f"DB: `{db_path}`")
    63|        st.sidebar.warning("Keine Live-API-Abfragen beim Seitenreload. Preise nur aus lokaler DB/Cache.")
    64|        page = PAGES[page_names.index(selected)]
    65|        page.render(st, conn)
    66|    finally:
    67|        conn.close()
    68|
    69|
    70|if __name__ == "__main__":
    71|    run_app()
    72|


     1|from __future__ import annotations
     2|
     3|from datetime import date
     4|
     5|from jarvis_finance.crypto.manage import (
     6|    add_crypto_position,
     7|    adjust_crypto_position,
     8|    get_crypto_management_options,
     9|    reduce_crypto_position,
    10|)
    11|from jarvis_finance.dashboard.components.tables import show_table
    12|from jarvis_finance.dashboard import data
    13|
    14|PAGE_TITLE = "Crypto Manage"
    15|
    16|
    17|def _option_label(row: dict[str, str]) -> str:
    18|    if "wallet_name" in row and "coin_name" in row:
    19|        return f"{row['coin_name']} ({row['symbol']}) / {row['wallet_name']}"
    20|    if "wallet_name" in row:
    21|        return f"{row['wallet_name']} [{row.get('wallet_type', '')}]"
    22|    return f"{row['coin_name']} ({row['symbol']})"
    23|
    24|
    25|def _selected_row(rows: list[dict[str, str]], label: str | None) -> dict[str, str] | None:
    26|    if not label:
    27|        return None
    28|    for row in rows:
    29|        if _option_label(row) == label:
    30|            return row
    31|    return None
    32|
    33|
    34|def render(st, conn) -> None:
    35|    st.title("Crypto Manage")
    36|    st.caption("Auditierbare manuelle Crypto-Ergänzungen. Keine Live-API-Abfragen beim Rendern; CoinGecko-Mapping ist manuell zu bestätigen.")
    37|    st.warning("Sicherheitsregel: Bestände werden nicht still gelöscht oder überschrieben. Änderungen laufen über Initial-Snapshot-Ergänzungen oder manual_adjustment mit Audit-Log.")
    38|
    39|    options = get_crypto_management_options(conn)
    40|    wallets = options["wallets"]
    41|    holdings = options["holdings"]
    42|    wallet_labels = [_option_label(row) for row in wallets]
    43|    holding_labels = [_option_label(row) for row in holdings]
    44|
    45|    tabs = st.tabs(["Hinzufügen", "Entfernen", "Korrigieren", "Nachtragsliste"])
    46|
    47|    with tabs[0]:
    48|        st.subheader("Coin/Bestand hinzufügen")
    49|        st.info("Symbol allein wird nie automatisch eindeutig behandelt. Wähle/erfasse CoinGecko-ID bewusst oder speichere ohne ID mit Warnung.")
    50|        coin_name = st.text_input("Coin-Name", key="crypto_manage_add_coin_name")
    51|        symbol = st.text_input("Symbol", key="crypto_manage_add_symbol")
    52|        coingecko_id = st.text_input("CoinGecko-ID (optional/manuell bestätigt)", key="crypto_manage_add_cg")
    53|        coingecko_status = st.selectbox("CoinGecko-Mapping-Status", ["selected", "uncertain", "missing_confirmed"], key="crypto_manage_add_cg_status")
    54|        wallet_label = st.selectbox("Wallet", wallet_labels, key="crypto_manage_add_wallet") if wallet_labels else None
    55|        quantity = st.text_input("Menge als Decimal-Text", key="crypto_manage_add_qty")
    56|        op_type = st.selectbox("Typ", ["initial_snapshot_addition", "manual_adjustment", "crypto_buy"], key="crypto_manage_add_type")
    57|        default_day = date(2025, 12, 31) if op_type == "initial_snapshot_addition" else date.today()
    58|        effective_date = st.date_input("Datum", value=default_day, key="crypto_manage_add_date")
    59|        note = st.text_area("Notiz (Pflicht bei manual_adjustment; empfohlen bei unsicherem Mapping)", key="crypto_manage_add_note")
    60|        confirm = st.checkbox("Ich bestätige diese einzelne Änderung", key="crypto_manage_add_confirm")
    61|        st.write({
    62|            "Aktion": op_type,
    63|            "Coin": coin_name,
    64|            "Symbol": symbol,
    65|            "CoinGecko-ID": coingecko_id or "<fehlt/unsicher>",
    66|            "Wallet": wallet_label,
    67|            "Datum": str(effective_date),
    68|            "Hinweis": "Keine Massenänderung; Speicherung erst nach Bestätigung.",
    69|        })
    70|        if st.button("Bestätigt speichern", key="crypto_manage_add_save", type="primary"):
    71|            try:
    72|                wallet = _selected_row(wallets, wallet_label)
    73|                result = add_crypto_position(
    74|                    conn,
    75|                    coin_name=coin_name or "",
    76|                    symbol=symbol or "",
    77|                    coingecko_id=coingecko_id or None,
    78|                    coingecko_status=coingecko_status or "selected",
    79|                    wallet_id=wallet["wallet_id"] if wallet else "",
    80|                    quantity_text=quantity or "",
    81|                    effective_date=str(effective_date),
    82|                    operation_type=op_type,
    83|                    note=note or "",
    84|                    confirm=bool(confirm),
    85|                )
    86|                st.success("Änderung gespeichert und auditierbar erfasst.")
    87|                st.write({"asset_id": result.asset_id, "wallet_id": result.wallet_id, "warnings": result.warnings})
    88|            except Exception as exc:
    89|                st.error(f"Speichern abgelehnt: {exc}")
    90|
    91|    with tabs[1]:
    92|        st.subheader("Coin/Bestand entfernen — via manual_adjustment, nicht löschen")
    93|        st.warning("Historie bleibt erhalten. Es wird nichts still gelöscht; die Reduktion wird als bestätigte Korrektur erfasst.")
    94|        holding_label = st.selectbox("Bestand", holding_labels, key="crypto_manage_reduce_holding") if holding_labels else None
    95|        reduction_mode = st.radio("Aktion", ["Bestand auf 0 setzen", "Teilmenge reduzieren"], key="crypto_manage_reduce_mode")
    96|        reduction_quantity = st.text_input("Teilmenge als Decimal-Text", key="crypto_manage_reduce_qty")
    97|        effective_date = st.date_input("Datum", value=date.today(), key="crypto_manage_reduce_date")
    98|        note = st.text_area("Notiz / Grund (Pflicht)", key="crypto_manage_reduce_note")
    99|        confirm = st.checkbox("Ich bestätige diese Reduktion", key="crypto_manage_reduce_confirm")
   100|        selected = _selected_row(holdings, holding_label)
   101|        st.write({"Aktion": reduction_mode, "Bestand": holding_label, "Datum": str(effective_date), "Historie": "bleibt erhalten"})
   102|        if st.button("Reduktion bestätigt speichern", key="crypto_manage_reduce_save"):
   103|            try:
   104|                if not selected:
   105|                    raise ValueError("holding selection is required")
   106|                qty = selected["quantity"] if reduction_mode == "Bestand auf 0 setzen" else (reduction_quantity or "")
   107|                result = reduce_crypto_position(
   108|                    conn,
   109|                    asset_id=selected["asset_id"],
   110|                    wallet_id=selected["wallet_id"],
   111|                    reduction_quantity_text=qty,
   112|                    effective_date=str(effective_date),
   113|                    note=note or "",
   114|                    confirm=bool(confirm),
   115|                )
   116|                st.success("Reduktion als manual_adjustment gespeichert.")
   117|                st.write({"asset_id": result.asset_id, "wallet_id": result.wallet_id})
   118|            except Exception as exc:
   119|                st.error(f"Speichern abgelehnt: {exc}")
   120|
   121|    with tabs[2]:
   122|        st.subheader("Coin/Bestand korrigieren")
   123|        holding_label = st.selectbox("Bestehender Bestand", holding_labels, key="crypto_manage_adjust_holding") if holding_labels else None
   124|        new_quantity = st.text_input("Neue Menge als Decimal-Text", key="crypto_manage_adjust_new_qty")
   125|        effective_date = st.date_input("Datum", value=date.today(), key="crypto_manage_adjust_date")
   126|        note = st.text_area("Notiz / Grund (Pflicht)", key="crypto_manage_adjust_note")
   127|        confirm = st.checkbox("Ich bestätige diese Korrektur", key="crypto_manage_adjust_confirm")
   128|        selected = _selected_row(holdings, holding_label)
   129|        st.write({"Aktion": "manual_adjustment", "Bestand": holding_label, "Neue Menge": new_quantity, "Datum": str(effective_date)})
   130|        if st.button("Korrektur bestätigt speichern", key="crypto_manage_adjust_save"):
   131|            try:
   132|                if not selected:
   133|                    raise ValueError("holding selection is required")
   134|                result = adjust_crypto_position(
   135|                    conn,
   136|                    asset_id=selected["asset_id"],
   137|                    wallet_id=selected["wallet_id"],
   138|                    new_quantity_text=new_quantity or "",
   139|                    effective_date=str(effective_date),
   140|                    note=note or "",
   141|                    confirm=bool(confirm),
   142|                )
   143|                st.success("Korrektur als manual_adjustment gespeichert.")
   144|                st.write({"asset_id": result.asset_id, "wallet_id": result.wallet_id})
   145|            except Exception as exc:
   146|                st.error(f"Speichern abgelehnt: {exc}")
   147|
   148|    with tabs[3]:
   149|        st.subheader("Ausgelassene/manuell zu prüfende Coins")
   150|        st.caption("Reminder-Liste aus dem Initial-Snapshot-Dry-Run. Diese Coins werden nicht automatisch importiert.")
   151|        show_table(st, [{"coin_label": label, "status": "manual_review"} for label in options["manual_review_coin_labels"]])
   152|
   153|    st.subheader("Aktuelle Crypto-Bestände nach lokaler DB")
   154|    show_table(st, data.get_crypto_by_wallet(conn))
   155|


     1|from __future__ import annotations
     2|
     3|from dataclasses import dataclass, field
     4|from decimal import Decimal, InvalidOperation
     5|from sqlite3 import Connection
     6|
     7|from jarvis_finance.audit.log import record_audit_event
     8|from jarvis_finance.crypto.assets import create_crypto_asset
     9|from jarvis_finance.crypto.holdings import calculate_crypto_holdings, create_initial_holding_snapshot
    10|from jarvis_finance.imports.common import stable_id, utc_now
    11|from jarvis_finance.quality.alerts import create_alert
    12|
    13|CRYPTO_MANUAL_REVIEW_COIN_LABELS = [
    14|    "ADA Cardano",
    15|    "Amp",
    16|    "Avalanche AVAX",
    17|    "Bitcoin BTC",
    18|    "Chainlink",
    19|    "DOGE",
    20|    "Decentraland MANA",
    21|    "Ethereum ETH",
    22|    "HIGH Highstreet",
    23|    "IOTA IOT",
    24|    "LUNA Terra",
    25|    "LUNC Terra Classic",
    26|    "NETVR",
    27|    "NuCypher NU",
    28|    "Ox ZRX",
    29|    "Polygon MATIC",
    30|    "Ripple XRP",
    31|    "SAND The Sandbox",
    32|    "SAROS",
    33|    "SKY",
    34|    "SOL Solana",
    35|    "Sundae",
    36|    "USTC Terra Classic",
    37|]
    38|
    39|VALID_ADD_TYPES = {"initial_snapshot_addition", "manual_adjustment", "crypto_buy"}
    40|
    41|
    42|@dataclass(frozen=True)
    43|class CryptoManageResult:
    44|    asset_id: str
    45|    wallet_id: str
    46|    holding_id: str | None = None
    47|    transaction_id: str | None = None
    48|    warnings: list[str] = field(default_factory=list)
    49|
    50|
    51|def parse_decimal_text(value: str, *, field_name: str = "quantity") -> Decimal:
    52|    text = str(value).strip()
    53|    if not text:
    54|        raise ValueError(f"{field_name} is required")
    55|    try:
    56|        dec = Decimal(text)
    57|    except (InvalidOperation, ValueError) as exc:
    58|        raise ValueError(f"{field_name} must be a Decimal string") from exc
    59|    if not dec.is_finite():
    60|        raise ValueError(f"{field_name} must be finite")
    61|    return dec
    62|
    63|
    64|def decimal_text(value: Decimal) -> str:
    65|    return format(value, "f")
    66|
    67|
    68|def _require_confirm(confirm: bool) -> None:
    69|    if not confirm:
    70|        raise ValueError("explicit confirm is required before saving")
    71|
    72|
    73|def _require_date(effective_date: str) -> str:
    74|    text = str(effective_date).strip()
    75|    if not text:
    76|        raise ValueError("date is required")
    77|    # Keep validation intentionally simple and deterministic; Streamlit date_input supplies ISO dates.
    78|    parts = text.split("-")
    79|    if len(parts) != 3 or not all(part.isdigit() for part in parts):
    80|        raise ValueError("date must be ISO YYYY-MM-DD")
    81|    return text
    82|
    83|
    84|def _require_wallet(conn: Connection, wallet_id: str) -> str:
    85|    wallet_id = str(wallet_id).strip()
    86|    if not wallet_id:
    87|        raise ValueError("wallet is required")
    88|    row = conn.execute("SELECT wallet_id FROM crypto_wallets WHERE wallet_id=?", (wallet_id,)).fetchone()
    89|    if row is None:
    90|        raise ValueError("wallet not found")
    91|    return wallet_id
    92|
    93|
    94|def _require_coin_fields(coin_name: str, symbol: str) -> tuple[str, str]:
    95|    coin_name = str(coin_name).strip()
    96|    symbol = str(symbol).strip().upper()
    97|    if not coin_name:
    98|        raise ValueError("coin name is required")
    99|    if not symbol:
   100|        raise ValueError("symbol is required")
   101|    return coin_name, symbol
   102|
   103|
   104|def _require_note(note: str, *, reason: str) -> str:
   105|    note = str(note or "").strip()
   106|    if not note:
   107|        raise ValueError(f"note is required for {reason}")
   108|    return note
   109|
   110|
   111|def _find_asset(conn: Connection, *, coin_name: str, symbol: str, coingecko_id: str | None) -> str | None:
   112|    if coingecko_id:
   113|        row = conn.execute("SELECT asset_id FROM crypto_assets WHERE coingecko_id=?", (coingecko_id.strip(),)).fetchone()
   114|        if row:
   115|            return row["asset_id"]
   116|    row = conn.execute(
   117|        "SELECT asset_id FROM crypto_assets WHERE lower(coin_name)=lower(?) AND upper(symbol)=upper(?)",
   118|        (coin_name, symbol),
   119|    ).fetchone()
   120|    return row["asset_id"] if row else None
   121|
   122|
   123|def _ensure_asset(
   124|    conn: Connection,
   125|    *,
   126|    coin_name: str,
   127|    symbol: str,
   128|    coingecko_id: str | None,
   129|    coingecko_status: str,
   130|    note: str | None,
   131|) -> tuple[str, list[str]]:
   132|    warnings: list[str] = []
   133|    cg = coingecko_id.strip() if coingecko_id and coingecko_id.strip() else None
   134|    existing = _find_asset(conn, coin_name=coin_name, symbol=symbol, coingecko_id=cg)
   135|    if existing:
   136|        asset_id = existing
   137|    else:
   138|        asset_id = create_crypto_asset(conn, coin_name=coin_name, symbol=symbol, coingecko_id=cg, notes=note)
   139|    if not cg:
   140|        warnings.append("missing_coingecko_id")
   141|    if coingecko_status in {"missing_confirmed", "uncertain"}:
   142|        rule = "crypto_coingecko_mapping_uncertain" if coingecko_status == "uncertain" else "missing_coingecko_id"
   143|        create_alert(
   144|            conn,
   145|            priority="warnung",
   146|            category="crypto",
   147|            entity_type="crypto_asset",
   148|            entity_id=asset_id,
   149|            rule_id=rule,
   150|            message="Crypto asset CoinGecko mapping needs manual confirmation.",
   151|            evidence={"symbol": symbol, "coin_name": coin_name, "coingecko_status": coingecko_status},
   152|            fingerprint=coingecko_status,
   153|        )
   154|        conn.commit()
   155|        if rule not in warnings:
   156|            warnings.append(rule)
   157|    return asset_id, warnings
   158|
   159|
   160|def current_wallet_asset_quantity(conn: Connection, *, wallet_id: str, asset_id: str) -> Decimal:
   161|    result = calculate_crypto_holdings(conn)
   162|    holding = result.wallet_holdings.get((wallet_id, asset_id))
   163|    return holding.quantity if holding else Decimal("0")
   164|
   165|
   166|def _insert_manual_adjustment_transaction(
   167|    conn: Connection,
   168|    *,
   169|    asset_id: str,
   170|    wallet_id: str,
   171|    delta: Decimal,
   172|    effective_date: str,
   173|    note: str,
   174|) -> str:
   175|    if delta == 0:
   176|        raise ValueError("manual adjustment delta must not be zero")
   177|    now = utc_now()
   178|    qty = abs(delta)
   179|    tx_id = stable_id("cryptotx", "manual_adjustment", asset_id, wallet_id, decimal_text(delta), effective_date, now)
   180|    from_wallet_id = wallet_id if delta < 0 else None
   181|    to_wallet_id = wallet_id if delta > 0 else None
   182|    conn.execute(
   183|        """
   184|        INSERT INTO crypto_transactions(
   185|            crypto_transaction_id, transaction_type, asset_id, quantity,
   186|            from_wallet_id, to_wallet_id, transaction_datetime, source,
   187|            confirmation_status, parse_confidence, notes, created_at
   188|        ) VALUES (?, 'manual_adjustment', ?, ?, ?, ?, ?, 'manual_dashboard', 'confirmed', 1, ?, ?)
   189|        """,
   190|        (tx_id, asset_id, decimal_text(qty), from_wallet_id, to_wallet_id, effective_date, note, now),
   191|    )
   192|    return tx_id
   193|
   194|
   195|def add_crypto_position(
   196|    conn: Connection,
   197|    *,
   198|    coin_name: str,
   199|    symbol: str,
   200|    wallet_id: str,
   201|    quantity_text: str,
   202|    effective_date: str,
   203|    operation_type: str,
   204|    note: str,
   205|    confirm: bool,
   206|    coingecko_id: str | None = None,
   207|    coingecko_status: str = "selected",
   208|) -> CryptoManageResult:
   209|    _require_confirm(confirm)
   210|    if operation_type not in VALID_ADD_TYPES:
   211|        raise ValueError("unsupported crypto management operation type")
   212|    if operation_type == "crypto_buy":
   213|        raise ValueError("crypto_buy requires the dedicated ledger buy workflow; use manual_adjustment or initial_snapshot_addition in this MVP")
   214|    effective_date = _require_date(effective_date)
   215|    wallet_id = _require_wallet(conn, wallet_id)
   216|    coin_name, symbol = _require_coin_fields(coin_name, symbol)
   217|    quantity = parse_decimal_text(quantity_text)
   218|    if quantity <= 0:
   219|        raise ValueError("quantity must be > 0")
   220|    if operation_type == "manual_adjustment":
   221|        note = _require_note(note, reason="manual_adjustment")
   222|    else:
   223|        note = str(note or "").strip() or "manual initial snapshot addition"
   224|    asset_id, warnings = _ensure_asset(conn, coin_name=coin_name, symbol=symbol, coingecko_id=coingecko_id, coingecko_status=coingecko_status, note=note)
   225|    existing_holding = conn.execute(
   226|        "SELECT crypto_holding_id FROM crypto_holdings WHERE asset_id=? AND wallet_id=?",
   227|        (asset_id, wallet_id),
   228|    ).fetchone()
   229|    if operation_type == "initial_snapshot_addition":
   230|        if existing_holding:
   231|            raise ValueError("initial snapshot addition would overwrite an existing holding; use manual_adjustment")
   232|        holding_id = create_initial_holding_snapshot(
   233|            conn,
   234|            asset_id=asset_id,
   235|            wallet_id=wallet_id,
   236|            quantity=quantity,
   237|            verification_status="verified",
   238|            last_verified_at=effective_date,
   239|            legacy_snapshot_date=effective_date,
   240|            note=note,
   241|        )
   242|        return CryptoManageResult(asset_id=asset_id, wallet_id=wallet_id, holding_id=holding_id, warnings=warnings)
   243|    before = current_wallet_asset_quantity(conn, wallet_id=wallet_id, asset_id=asset_id)
   244|    tx_id = _insert_manual_adjustment_transaction(conn, asset_id=asset_id, wallet_id=wallet_id, delta=quantity, effective_date=effective_date, note=note)
   245|    after = before + quantity
   246|    record_audit_event(
   247|        conn,
   248|        source="crypto_manage",
   249|        action="crypto_manual_adjustment",
   250|        entity_type="crypto_transaction",
   251|        entity_id=tx_id,
   252|        old_values={"asset_id": asset_id, "wallet_id": wallet_id, "quantity": decimal_text(before)},
   253|        new_values={"asset_id": asset_id, "wallet_id": wallet_id, "quantity": decimal_text(after), "delta": decimal_text(quantity)},
   254|        user_text_note=note,
   255|        confirmed=True,
   256|        created_by="dashboard",
   257|    )
   258|    conn.commit()
   259|    return CryptoManageResult(asset_id=asset_id, wallet_id=wallet_id, transaction_id=tx_id, warnings=warnings)
   260|
   261|

Use offset=261 to continue reading (showing 1-260 of 377 lines)


   260|
   261|
   262|def adjust_crypto_position(
   263|    conn: Connection,
   264|    *,
   265|    asset_id: str,
   266|    wallet_id: str,
   267|    new_quantity_text: str,
   268|    effective_date: str,
   269|    note: str,
   270|    confirm: bool,
   271|) -> CryptoManageResult:
   272|    _require_confirm(confirm)
   273|    effective_date = _require_date(effective_date)
   274|    wallet_id = _require_wallet(conn, wallet_id)
   275|    note = _require_note(note, reason="manual_adjustment")
   276|    new_quantity = parse_decimal_text(new_quantity_text, field_name="new quantity")
   277|    if new_quantity < 0:
   278|        raise ValueError("new quantity must be >= 0")
   279|    asset = conn.execute("SELECT asset_id FROM crypto_assets WHERE asset_id=?", (asset_id,)).fetchone()
   280|    if asset is None:
   281|        raise ValueError("asset not found")
   282|    before = current_wallet_asset_quantity(conn, wallet_id=wallet_id, asset_id=asset_id)
   283|    delta = new_quantity - before
   284|    if before + delta < 0:
   285|        raise ValueError("negative holding blocked")
   286|    tx_id = _insert_manual_adjustment_transaction(conn, asset_id=asset_id, wallet_id=wallet_id, delta=delta, effective_date=effective_date, note=note)
   287|    record_audit_event(
   288|        conn,
   289|        source="crypto_manage",
   290|        action="crypto_manual_adjustment",
   291|        entity_type="crypto_transaction",
   292|        entity_id=tx_id,
   293|        old_values={"asset_id": asset_id, "wallet_id": wallet_id, "quantity": decimal_text(before)},
   294|        new_values={"asset_id": asset_id, "wallet_id": wallet_id, "quantity": decimal_text(new_quantity), "delta": decimal_text(delta)},
   295|        user_text_note=note,
   296|        confirmed=True,
   297|        created_by="dashboard",
   298|    )
   299|    if before != 0 and (new_quantity / before >= Decimal("10") or new_quantity == 0):
   300|        create_alert(
   301|            conn,
   302|            priority="hinweis",
   303|            category="crypto",
   304|            entity_type="crypto_transaction",
   305|            entity_id=tx_id,
   306|            rule_id="crypto_manual_adjustment_review",
   307|            message="Manual crypto adjustment should be reviewed because it materially changes the holding.",
   308|            evidence={"asset_id": asset_id, "wallet_id": wallet_id},
   309|            fingerprint="manual_adjustment_review",
   310|        )
   311|    conn.commit()
   312|    return CryptoManageResult(asset_id=asset_id, wallet_id=wallet_id, transaction_id=tx_id)
   313|
   314|
   315|def reduce_crypto_position(
   316|    conn: Connection,
   317|    *,
   318|    asset_id: str,
   319|    wallet_id: str,
   320|    reduction_quantity_text: str,
   321|    effective_date: str,
   322|    note: str,
   323|    confirm: bool,
   324|) -> CryptoManageResult:
   325|    _require_confirm(confirm)
   326|    reduction = parse_decimal_text(reduction_quantity_text, field_name="reduction quantity")
   327|    if reduction <= 0:
   328|        raise ValueError("reduction quantity must be > 0")
   329|    current = current_wallet_asset_quantity(conn, wallet_id=wallet_id, asset_id=asset_id)
   330|    new_quantity = current - reduction
   331|    if new_quantity < 0:
   332|        raise ValueError("negative holding blocked")
   333|    return adjust_crypto_position(
   334|        conn,
   335|        asset_id=asset_id,
   336|        wallet_id=wallet_id,
   337|        new_quantity_text=decimal_text(new_quantity),
   338|        effective_date=effective_date,
   339|        note=note,
   340|        confirm=confirm,
   341|    )
   342|
   343|
   344|def get_crypto_management_options(conn: Connection) -> dict[str, list[dict[str, str]] | list[str]]:
   345|    wallets = [
   346|        {"wallet_id": row["wallet_id"], "wallet_name": row["wallet_name"], "wallet_type": row["wallet_type"]}
   347|        for row in conn.execute("SELECT wallet_id, wallet_name, wallet_type FROM crypto_wallets WHERE is_active=1 ORDER BY wallet_name").fetchall()
   348|    ]
   349|    assets = [
   350|        {"asset_id": row["asset_id"], "coin_name": row["coin_name"], "symbol": row["symbol"], "coingecko_id": row["coingecko_id"] or ""}
   351|        for row in conn.execute("SELECT asset_id, coin_name, symbol, coingecko_id FROM crypto_assets WHERE is_active=1 ORDER BY coin_name, symbol").fetchall()
   352|    ]
   353|    holdings = [
   354|        {
   355|            "asset_id": row["asset_id"],
   356|            "wallet_id": row["wallet_id"],
   357|            "coin_name": row["coin_name"],
   358|            "symbol": row["symbol"],
   359|            "wallet_name": row["wallet_name"],
   360|            "quantity": row["quantity"],
   361|        }
   362|        for row in conn.execute(
   363|            """
   364|            SELECT h.asset_id, h.wallet_id, a.coin_name, a.symbol, w.wallet_name, h.quantity
   365|            FROM crypto_holdings h
   366|            JOIN crypto_assets a ON a.asset_id=h.asset_id
   367|            JOIN crypto_wallets w ON w.wallet_id=h.wallet_id
   368|            ORDER BY a.coin_name, w.wallet_name
   369|            """
   370|        ).fetchall()
   371|    ]
   372|    return {
   373|        "wallets": wallets,
   374|        "assets": assets,
   375|        "holdings": holdings,
   376|        "manual_review_coin_labels": list(CRYPTO_MANUAL_REVIEW_COIN_LABELS),
   377|    }
   378|


     1|from __future__ import annotations
     2|
     3|from dataclasses import dataclass, field
     4|from decimal import Decimal
     5|from sqlite3 import Connection
     6|
     7|from jarvis_finance.audit.log import record_audit_event
     8|from jarvis_finance.imports.common import stable_id, utc_now
     9|from jarvis_finance.quality.alerts import create_alert
    10|
    11|VALID_VERIFICATION_STATUS = {"verified", "stale", "unverified", "estimated"}
    12|
    13|
    14|@dataclass
    15|class CryptoHolding:
    16|    wallet_id: str | None
    17|    asset_id: str
    18|    quantity: Decimal = Decimal("0")
    19|    verification_status: str = "unverified"
    20|    last_verified_at: str | None = None
    21|    legacy_snapshot_value_original: Decimal | None = None
    22|    legacy_snapshot_value_chf: Decimal | None = None
    23|    current_value_chf: Decimal | None = None
    24|    quality_warnings: list[str] = field(default_factory=list)
    25|
    26|
    27|@dataclass
    28|class CryptoHoldingsResult:
    29|    wallet_holdings: dict[tuple[str, str], CryptoHolding] = field(default_factory=dict)
    30|    total_by_asset: dict[str, CryptoHolding] = field(default_factory=dict)
    31|
    32|
    33|def _d(value: object, default: str = "0") -> Decimal:
    34|    if value is None or str(value).strip() == "":
    35|        return Decimal(default)
    36|    return Decimal(str(value))
    37|
    38|
    39|def _ds(value: Decimal | None) -> str | None:
    40|    return format(value, "f") if value is not None else None
    41|
    42|
    43|def _ensure_holding(result: CryptoHoldingsResult, wallet_id: str, asset_id: str) -> CryptoHolding:
    44|    return result.wallet_holdings.setdefault((wallet_id, asset_id), CryptoHolding(wallet_id=wallet_id, asset_id=asset_id))
    45|
    46|
    47|def _ensure_total(result: CryptoHoldingsResult, asset_id: str) -> CryptoHolding:
    48|    return result.total_by_asset.setdefault(asset_id, CryptoHolding(wallet_id=None, asset_id=asset_id))
    49|
    50|
    51|def create_initial_holding_snapshot(
    52|    conn: Connection,
    53|    *,
    54|    asset_id: str,
    55|    wallet_id: str,
    56|    quantity: Decimal,
    57|    verification_status: str,
    58|    note: str,
    59|    last_verified_at: str | None = None,
    60|    legacy_snapshot_value_original: Decimal | None = None,
    61|    legacy_snapshot_value_chf: Decimal | None = None,
    62|    legacy_snapshot_currency: str | None = None,
    63|    legacy_snapshot_date: str | None = None,
    64|) -> str:
    65|    if quantity < 0:
    66|        raise ValueError("initial holding quantity must be >= 0")
    67|    if verification_status not in VALID_VERIFICATION_STATUS:
    68|        raise ValueError("invalid verification_status")
    69|    if not note.strip():
    70|        raise ValueError("initial holding snapshot requires a note")
    71|    now = utc_now()
    72|    holding_id = stable_id("cryptoholding", asset_id, wallet_id)
    73|    conn.execute(
    74|        """
    75|        INSERT OR REPLACE INTO crypto_holdings(
    76|            crypto_holding_id, asset_id, wallet_id, quantity, acquisition_source,
    77|            last_verified_at, verification_status, legacy_snapshot_value_original,
    78|            legacy_snapshot_value_chf, legacy_snapshot_currency, legacy_snapshot_date,
    79|            notes, created_at, updated_at
    80|        ) VALUES (?, ?, ?, ?, 'initial_snapshot', ?, ?, ?, ?, ?, ?, ?,
    81|                  COALESCE((SELECT created_at FROM crypto_holdings WHERE crypto_holding_id=?), ?), ?)
    82|        """,
    83|        (
    84|            holding_id, asset_id, wallet_id, _ds(quantity), last_verified_at, verification_status,
    85|            _ds(legacy_snapshot_value_original),
    86|            _ds(legacy_snapshot_value_chf),
    87|            legacy_snapshot_currency, legacy_snapshot_date, note, holding_id, now, now,
    88|        ),
    89|    )
    90|    record_audit_event(
    91|        conn,
    92|        source="crypto_holdings",
    93|        action="initial_crypto_holding_snapshot",
    94|        entity_type="crypto_holding",
    95|        entity_id=holding_id,
    96|        new_values={"asset_id": asset_id, "wallet_id": wallet_id, "quantity": _ds(quantity), "verification_status": verification_status},
    97|        user_text_note=note,
    98|        confirmed=True,
    99|        created_by="system",
   100|    )
   101|    conn.commit()
   102|    return holding_id
   103|
   104|
   105|def _apply_delta(result: CryptoHoldingsResult, wallet_id: str, asset_id: str, delta: Decimal) -> None:
   106|    holding = _ensure_holding(result, wallet_id, asset_id)
   107|    holding.quantity += delta
   108|    total = _ensure_total(result, asset_id)
   109|    total.quantity += delta
   110|
   111|
   112|def calculate_crypto_holdings(conn: Connection) -> CryptoHoldingsResult:
   113|    result = CryptoHoldingsResult()
   114|    for row in conn.execute("SELECT * FROM crypto_holdings ORDER BY wallet_id, asset_id").fetchall():
   115|        holding = _ensure_holding(result, row["wallet_id"], row["asset_id"])
   116|        holding.quantity += _d(row["quantity"])
   117|        holding.verification_status = row["verification_status"]
   118|        holding.last_verified_at = row["last_verified_at"]
   119|        holding.legacy_snapshot_value_original = _d(row["legacy_snapshot_value_original"]) if row["legacy_snapshot_value_original"] is not None else None
   120|        holding.legacy_snapshot_value_chf = _d(row["legacy_snapshot_value_chf"]) if row["legacy_snapshot_value_chf"] is not None else None
   121|        total = _ensure_total(result, row["asset_id"])
   122|        total.quantity += _d(row["quantity"])
   123|    rows = conn.execute("SELECT * FROM crypto_transactions ORDER BY transaction_datetime, created_at, crypto_transaction_id").fetchall()
   124|    for row in rows:
   125|        asset_id = row["asset_id"]
   126|        qty = _d(row["quantity"])
   127|        fee_qty = _d(row["fee_quantity"])
   128|        t = row["transaction_type"]
   129|        if t == "transfer":
   130|            _apply_delta(result, row["from_wallet_id"], asset_id, -(qty + fee_qty))
   131|            _apply_delta(result, row["to_wallet_id"], asset_id, qty)
   132|        elif t == "buy":
   133|            _apply_delta(result, row["to_wallet_id"], asset_id, qty - fee_qty)
   134|        elif t == "sell":
   135|            _apply_delta(result, row["from_wallet_id"], asset_id, -(qty + fee_qty))
   136|        elif t == "fee":
   137|            _apply_delta(result, row["from_wallet_id"], asset_id, -fee_qty)
   138|        elif t == "manual_adjustment":
   139|            if row["to_wallet_id"]:
   140|                _apply_delta(result, row["to_wallet_id"], asset_id, qty)
   141|            elif row["from_wallet_id"]:
   142|                _apply_delta(result, row["from_wallet_id"], asset_id, -qty)
   143|    for asset in conn.execute("SELECT asset_id, symbol, coingecko_id FROM crypto_assets").fetchall():
   144|        if not asset["coingecko_id"]:
   145|            msg = "Crypto asset has no CoinGecko ID; current valuation is unavailable."
   146|            for holding in [h for h in result.wallet_holdings.values() if h.asset_id == asset["asset_id"]]:
   147|                holding.quality_warnings.append("missing_coingecko_id")
   148|            create_alert(
   149|                conn,
   150|                priority="warnung",
   151|                category="crypto",
   152|                entity_type="crypto_asset",
   153|                entity_id=asset["asset_id"],
   154|                rule_id="missing_coingecko_id",
   155|                message=msg,
   156|                evidence={"symbol": asset["symbol"]},
   157|                fingerprint="missing_coingecko_id",
   158|            )
   159|    conn.commit()
   160|    return result
   161|


set -euo pipefail
cd /home/agent/.hermes/repos/FinanceManager
PYTHONPATH=src python - <<'PY'
import sqlite3
from pathlib import Path
from jarvis_finance.config.settings import load_settings, find_repo_root
settings=load_settings(repo_root=find_repo_root())
db=settings.db_path
print('runtime_base=' + str(settings.runtime_paths.base_dir))
print('db_exists=' + str(db.exists()))
print('db_outside_repo=' + str(not str(db).startswith(str(find_repo_root().resolve()))))
if not db.exists():
    raise SystemExit(0)
conn=sqlite3.connect(db)
conn.row_factory=sqlite3.Row
# aggregate only, no quantities/wallet names
for table in ['crypto_wallets','crypto_assets','crypto_holdings','crypto_transactions','audit_log','import_sessions']:
    try:
        n=conn.execute(f'SELECT COUNT(*) AS n FROM {table}').fetchone()['n']
    except Exception as e:
        n='ERR'
    print(f'{table}_count={n}')
btc=conn.execute("SELECT asset_id, coingecko_id FROM crypto_assets WHERE upper(symbol)='BTC' OR coingecko_id='bitcoin' OR lower(coin_name)='bitcoin'").fetchall()
print('btc_asset_rows=' + str(len(btc)))
if btc:
    asset_ids=[r['asset_id'] for r in btc]
    qmarks=','.join('?' for _ in asset_ids)
    holdings=conn.execute(f'SELECT COUNT(*) AS n FROM crypto_holdings WHERE asset_id IN ({qmarks})', asset_ids).fetchone()['n']
    print('btc_holding_rows=' + str(holdings))
active_wallets=conn.execute('SELECT COUNT(*) AS n FROM crypto_wallets WHERE is_active=1').fetchone()['n']
print('active_wallets=' + str(active_wallets))
dups=conn.execute('SELECT COUNT(*) AS n FROM (SELECT asset_id,wallet_id,COUNT(*) c FROM crypto_holdings GROUP BY asset_id,wallet_id HAVING c>1)').fetchone()['n']
print('duplicate_holding_pairs=' + str(dups))
conn.close()
PY
PYTHONPATH=src python - <<'PY'
from jarvis_finance.dashboard.pages import PAGES
print('crypto_manage_page_available=' + str('Crypto Manage' in [p.PAGE_TITLE for p in PAGES]))
print('page_count=' + str(len(PAGES)))
PY
find . -type d \( -name __pycache__ -o -name .pytest_cache \) -prune -exec rm -rf {} +
PYTHONPATH=src python -m jarvis_finance.cli.main git-safety-scan .
git status --short --branch


Output:
runtime_base=/home/agent/jarvis_runtime/finance-system
db_exists=True
db_outside_repo=True
crypto_wallets_count=8
crypto_assets_count=19
crypto_holdings_count=25
crypto_transactions_count=0
audit_log_count=53
import_sessions_count=3
btc_asset_rows=0
active_wallets=8
duplicate_holding_pairs=0
crypto_manage_page_available=True
page_count=13
GIT_SAFETY_OK
## main...origin/main