## Outcome Read-only analysis completed against exact clean base `a5e8dd0e5b1330b588e5c6e501cd2af03f23818f`. No files were modified. ## Recommended Sprint 16.1 implementation map ### 1. Resolve categories by active DB identity, not display-name literals **Current risks** - `household_classification.py:_category()` correctly validates an exact active `category_id`. - Migros fallback incorrectly searches only `lower(name)='essen + haushalt'`; the canonical seeded category is `bcat_food_household / Essen & Haushalt`. - Bank-interest lookup also depends on mutable display names. - Cleanup can preserve an arbitrary existing ID or create `bcat_expense_essenundhaushalt`; therefore code must not assume a generated ID exists. - `budget_categories` has no canonical-alias column. **Implementation** In `src/jarvis_finance/services/household_classification.py` add a resolver such as: - `CATEGORY_ALIASES` - `food_household`: stable preferred ID `bcat_food_household`; names `Essen & Haushalt`, `Essen + Haushalt`, `Essen und Haushalt` - `interest_income`: names `Zinsertrag`, `Kapitalertrag`, then `Sonstige Einnahmen` - `_resolve_active_category(conn, canonical_alias, *, category_type)` 1. Prefer the configured stable ID when active and type-compatible. 2. Otherwise match normalized canonical names against active rows. 3. Return the actual DB `category_id` and current name. 4. Fail unresolved/ambiguous rather than silently selecting duplicate active categories. Expose `canonical_alias` alongside `category_id` where useful, but always write the actual active DB ID. No schema migration is necessary if aliases remain versioned code constants. ### 2. Migros matching: remove the CHF 50 gate The active threshold is in `household_import.py:762-797`, not `budget_csv_imports.py`: - `eligible = receipt_total >= Decimal("50.00")` - Current-request exact matching only links when `eligible`. - Small unlinked receipts avoid review solely because of this threshold. Replace it with amount-independent rules: 1. Same transaction date. 2. Absolute totals equal within CHF 0.01. 3. Exactly one unconsumed Migros money movement. 4. Link regardless of receipt amount. 5. Multiple exact matches → `review`. 6. Same-day non-exact candidates → `review`. 7. No candidate → `unmatched`. 8. Receipt detail never creates a second expense. Also remove the dead/legacy threshold surface from: - `budget_imports.py:seed_migros_candidates_from_rows()` (`threshold_chf`, `_threshold`, `legacy_threshold_chf`) - `api/routers/budget.py` transaction-candidate generation payload - Tests that still pass `threshold_chf` - `docs/sprint16-smart-household-version-bound-inventory.md:70` `budget_csv_imports.py` itself only detects/parses the profile. Its Migros `default_rules` strings are descriptive metadata and should be updated if they imply threshold or duplicate-expense behavior. ### 3. HMAC-bound unmatched-transfer decision contract `user_confirmed_unmatched_transfer` currently has downstream materialization support but **no producer**. No endpoint or test can currently create that semantic. Add an explicit request field, preferably: ```json { "decisions": [ { "row_token": "row_1", "decision": "user_confirmed_unmatched_transfer" } ] } ``` Implement in `household_import.py`: - `_validated_row_decisions(...)` - list/object validation - unique row tokens - allowlisted decision enum - token must resolve to the reconstructed row - only eligible unmatched, mapped, CHF, non-pending, non-duplicate money rows - reject safe/ambiguous pairs, receipts, refunds/reversals, and already categorized ordinary expenses - Bind canonical sorted decisions into `_input_fingerprint()`. - Resolve public row tokens to internal `source_row_fingerprint` before computing `preview_fingerprint`. - Apply the decision before classification output/materialization: - `classification = user_confirmed_unmatched_transfer` - `transaction_semantics = user_confirmed_unmatched_transfer` - `budget_effect_chf = 0.00` - `transaction_type = transfer` - no synthetic counterbooking - Confirm must reconstruct the exact decision set and compare: - input fingerprint - preview fingerprint - baseline fingerprint - Audit the decision type and masked row identity, never merchant text or raw reference. - Altered retry, removed decision, changed decision, changed row, and changed DB baseline must all return 409. This can use existing tables and audit notes; no schema migration is required. ### 4. Classification v2.1 contract and precedence Set: ```python CLASSIFICATION_VERSION = "household_classification_v2.1" ``` Recommended deterministic order: 1. Safe paired own-account transfer. 2. Card settlement/payment, refund, reversal, pending semantics. 3. Explicit fingerprint-bound user decision. 4. Exact active merchant or exact alias rule. 5. Conflict-free confirmed history for the same canonical merchant family, source, and account role. 6. Source-specific review rule. 7. Stable recurring-payment evidence. 8. Narrow canonical merchant-family fallback, including Migros. 9. Unresolved decision. Bank interest should remain explicit income semantics before generic expense categorization, but category resolution must use the active canonical alias resolver. **Merchant normalization/families** Split the current single normalized string into: - `normalized_merchant`: Unicode-normalized, case-folded, punctuation/whitespace collapsed. - `merchant_family`: stable identity from: 1. active `budget_merchants.normalized_name`; 2. active aliases; 3. narrow versioned family aliases such as Migros variants; 4. otherwise the normalized merchant itself. Do not strip arbitrary numbers globally; store/location numbers may distinguish merchants. Remove only explicitly versioned provider noise. Fix `_exact_rule()` so `match_type='regex'` executes bounded regex matching rather than currently behaving as `contains`. History must compare canonical family plus source/account role. Conflicts must remain `decision_needed`; similarity must never override explicit rules or conflicting history. ### 5. Separate technical confirmability from business readiness Current `confirmable` means only `not errors`; thus a preview with unresolved business decisions can still be confirmed into the review inbox. Return both: - `technically_confirmable` - supported/importable files - compatible active mappings - valid categories/decisions - reproducible HMAC-bound preview - no structural/security errors - `business_ready` - technically confirmable **and** - every writable money row is one of: - category-ready proposal, - safe transfer, - valid explicit unmatched-transfer decision, - supported refund/reversal, - intentionally non-productive receipt detail - no unresolved/ambiguous duplicate, FX, category, mapping, or pairing decision For compatibility, retain `confirmable` temporarily as an alias of `technically_confirmable`; backend confirm should gate technical safety, while UI can distinguish “importable into review” from “fully ready.” ## Exact tests to add/change ### `tests/unit/test_household_classification_v2_golden.py` Add or update: - `test_classification_version_is_v2_1` - `test_food_household_resolves_stable_active_id_across_ampersand_plus_and_und_aliases` - `test_inactive_preferred_category_id_falls_back_to_unique_active_canonical_alias` - `test_ambiguous_active_canonical_alias_fails_closed` - `test_exact_rule_precedes_history_and_source_rule` - `test_regex_alias_uses_regex_semantics_not_contains` - `test_merchant_family_unifies_versioned_provider_variants` - `test_merchant_family_does_not_merge_unrelated_numbered_merchants` - `test_conflicting_family_history_requires_decision` - precedence parameterization covering all nine levels. Update assertions currently expecting `household_classification_v2`. ### `tests/unit/test_household_import_v1_golden.py` Add: - `test_migros_exact_match_links_below_above_and_at_chf_50` - `test_migros_matching_has_no_amount_threshold` - `test_migros_multiple_exact_matches_require_review` - `test_migros_nonexact_same_day_match_requires_review` - `test_unlinked_migros_receipt_never_creates_second_expense` - `test_user_confirmed_unmatched_transfer_is_decision_bound_and_budget_neutral` - `test_unmatched_transfer_decision_rejects_ineligible_row` - `test_unmatched_transfer_decision_changes_input_and_preview_fingerprints` - `test_unmatched_transfer_confirm_rejects_altered_removed_or_added_decision` - `test_unmatched_transfer_confirm_rejects_stale_database_baseline` - `test_unmatched_transfer_retry_is_idempotent_only_for_identical_decision` - `test_preview_exposes_technical_confirmability_separately_from_business_readiness` - `test_unresolved_category_is_technically_confirmable_but_not_business_ready` - `test_fully_resolved_preview_is_business_ready` Replace `test_migros_links_only_high_value_receipts_and_keeps_small_receipts_out_of_review`, whose expected behavior is now obsolete. ### Legacy import tests Update: - `tests/unit/test_budget_phase15_source_aware_imports.py` - `tests/unit/test_budget_phase111_migros_monthly_analytics.py` - API coverage in `tests/unit/test_household_import_v1_golden.py` Remove threshold arguments/assertions and verify both small and large receipts follow the same categorization/linking rule. ### Frontend contract follow-up Update: - `frontend/src/api/household.ts` - add decision payload type - add `technically_confirmable` and `business_ready` - retain optional `confirmable` during compatibility window - `frontend/src/pages/HouseholdUx.test.ts` - verify explicit unmatched-transfer decision is re-previewed - confirm submits the latest decision-bound fingerprints - business-not-ready state is explained separately from technical failure. ## Notable current-field risks - Public import rows use `row_token`; internal identity uses `source_row_fingerprint`. - Import category overrides are `category_overrides: Record`. - Current fingerprints are `input_fingerprint`, `preview_fingerprint`, and `baseline_fingerprint`; only the latter two are public. - Review uses `item_token`, not import `row_token`. - `confirmable` currently conflates structural validity and business completeness. - `can_confirm` has different row/group UI meanings and should not replace either readiness field. - Public `classification_v2` details are deliberately stripped. - `_exact_rule()` mishandles declared regex aliases. - Merchant history requires exact normalized text and exact source despite additionally checking account role. - `budget_csv_imports.py` uses current profile names, while legacy import code/tests still use older source names such as `credit_card_csv` and `migros_receipt`. - Category cleanup does not repoint `budget_merchant_aliases` directly, though aliases follow their merchant; merchant/rule category references are repointed. - Existing schema 47 has no first-class canonical category alias or user-decision table; the proposed bounded implementation can remain schema-neutral through versioned code aliases and audit records. ## Verification / repository state - Repository remained clean; no files created or modified. - Focused pytest execution was attempted, but this worktree has no executable local `.venv` and system Python lacks `pytest` (`No module named pytest`). - No production data, raw imports, database contents, or secrets were accessed.