# Budget Import Duplicate Handling Hotfix Pattern

Use this reference when FinanceManager budget-import review shows duplicate or possible-duplicate candidates in the normal confirmation flow.

## Goal

Possible duplicates must never be offered as normal bookable expenses. They must be separated into a duplicate-review surface, blocked from normal confirm/batch confirm, and bookable only through an explicit override with reason and audit.

## Core invariants

- Normal review lists may include only new/open/review/auto-categorized/import-source candidates that are safe to decide normally.
- Normal review lists must exclude: `duplicate`, `duplicate_candidate`, `possible_duplicate`, `duplicate_blocked`, `superseded`, `covered_by_migros`, `covered_by_source`, `ignored`, `confirmed`, auto-ignored duplicates, and line-item/detail rows.
- Duplicate candidates are not bulk-confirmable and must not show a normal `Bestätigen` action.
- `covered_by_migros` and other covered/ignored states are terminal for normal confirm.
- Batch confirm may book only allowed statuses with category set and must return per-candidate blocker reasons for skipped items. No silent skipping.
- Override booking is separate from normal confirm and requires: warning, explicit reason, confirm, and audit event.

## Backend pattern

1. Centralize status sets in the import service:
   - user-open statuses
   - duplicate statuses
   - normal-review excluded statuses
   - confirm-blocked statuses
   - batch-confirm blocked statuses
2. Make `list_transaction_candidates(..., tab=None)` / normal status filtering exclude duplicate/covered/ignored/confirmed states by default.
3. Add or keep a dedicated duplicate tab/filter, e.g. `tab="duplicates"`, covering duplicate-like statuses.
4. Hard-guard normal confirm:
   - `duplicate_candidate`, `possible_duplicate`, `duplicate_blocked`, `duplicate`, `superseded`, `covered_by_migros`, `ignored` must raise a 409 with user-safe text.
5. Implement a separate `confirm_duplicate_override` path:
   - require non-empty reason
   - set `duplicate_override=True` internally
   - write audit action such as `budget_transaction_candidate_duplicate_override_confirmed`
   - do not expose this path through batch confirm.
6. Guard related write actions too, not just final confirm:
   - category edits and splits should reject duplicate states unless they are first explicitly reopened.
7. For Migros idempotency:
   - when `source_type` is `migros_receipt`/`migros_receipts` and `receipt_key` exists, treat `receipt_key` as the leading unique key.
   - reimport of the same receipt must not create a second open main candidate.
   - article rows remain in line-item/detail tables, never as main booking candidates.
   - credit-card Migros remains covered; Migros receipt CSV wins as the budget candidate.

## Runtime cleanup workflow

Before any runtime mutation:

1. Backup the runtime DB to a timestamped file under the runtime backup directory.
2. Dry-run counts only; do not print real amounts or sensitive raw data in chat:
   - open candidate count
   - duplicate candidate count
   - possible Migros duplicate count
   - identical `receipt_key` duplicate count
   - possible same date/merchant/amount duplicate count
3. Mark exact duplicates conservatively as `duplicate_blocked` or `superseded`.
4. Mark possible duplicates as `duplicate_candidate`.
5. Write cleanup audit events.
6. Verify no productive budget transactions were created.

## UI pattern

- Rename/label duplicate tab clearly: **Duplikate prüfen**.
- Normal review tab must not show duplicate candidates.
- Duplicate tab should show: candidate, possible existing transaction/candidate, date, merchant, amount, source, reason, confidence.
- Duplicate row actions:
  - Details
  - Als Duplikat ignorieren / superseded
  - Zurück in Review
  - Override mit Begründung
- Override action should be visually secondary/non-prominent, with a warning and required reason.
- Disable or hide normal confirm buttons for duplicate/covered/ignored rows.

## Tests to add

Backend:

- same Migros `receipt_key` creates no second open candidate
- same source fingerprint is idempotent
- credit-card Migros becomes `covered_by_migros` and cannot be booked normally
- article rows do not become main candidates
- duplicate/covered/ignored statuses cannot normal-confirm
- override requires reason and audits
- batch confirm skips duplicate states and reports per-candidate errors

Frontend:

- normal review does not render duplicates
- duplicate tab renders duplicate rows and badges
- normal confirm button absent/disabled for duplicate rows
- override dialog requires reason and calls the override endpoint
- batch blocker feedback remains visible

Verification:

- Python compile
- pytest
- frontend tests
- frontend build
- source secret scan excluding intentional fixtures only
- frontend build secret scan
- Git-safety: no runtime DB, CSV, XLS/XLSX, keys, or reports in Git
- `git diff --check`
- browser sanity on `/planning/budget/expenses/review`

## Reporting format

In the final report include yes/no status for:

- duplicates removed from normal review
- Migros receipt key/fingerprint checked
- existing runtime duplicates marked
- confirm guard for duplicates
- batch confirm guard
- duplicate tab working
- tests/build/Git-safety
- backup path
- commit hash and push status
- open limitations