# Grocery Product Matching Learning v2

Use this reference for Grocery/Migros Optimizer work where product alternatives must be reviewed, learned, reused, or manually mapped.

## Scope boundaries

- This is a product-matching and savings workflow, not a medical/nutrition workflow.
- Do not add Histamin, Behçet, trigger, recipe, nutrition-plan, auto-ordering, OCR/camera, login-scraping, newsletter, or general food-coach features unless explicitly authorized in a later sprint.
- Provider/web calls must only run on explicit user action; normal page render must use local DB/cache DTOs only.
- No runtime DB, real reports, product data exports, API keys, or credentials in Git/frontend.

## Persistent mapping model

Accepted/rejected product matches should be persisted in a durable mapping table, not only as transient provider candidates. Include at least:

- source product normalized/raw name and source retailer (`Migros` for current flow)
- target retailer/name/URL/brand/package/unit
- match type: `exact_match`, `close_match`, `cheaper_private_label`, `manual_match`
- status: `accepted`, `rejected`, `needs_review`, `deprecated`
- confidence, user note, created/updated timestamps, last sourced price timestamp
- health fields only as prepared placeholders: `health_analysis_status`, `health_flags_json`, `health_notes`

Accepted mappings are preferred on future receipts. Rejected mappings must block only the exact rejected target retailer+URL/name; do not suppress all products for the normalized source item, because a different retailer/product may still be valid.

## Price provenance rules

- Never invent prices or convert missing prices to `0.00`.
- A price is safe for savings only if it has a real source/cache row with URL and timestamp.
- Manual user-entered prices may be stored as notes/review data, but must not be treated as provider-checked or fresh. Tag them (e.g. `manual_price`) and exclude them from safe savings until a sourced cache/provider price exists.
- Refresh must not fake freshness. Only update `last_price_checked_at` when a sourced cache/provider result exists. Otherwise return a clear status such as `no_sourced_price_available`.
- No-price / non-cheaper / uncertain provider candidates still need a persisted `match_id` so the user can reject/accept/review them and the system can learn.

## Manual mapping validation

Manual product URLs are user-controlled input and may later be rendered or fetched. Validate before persisting:

- allow only `http`/`https`
- reject empty host, `localhost`, `*.localhost`
- reject loopback/private/link-local/multicast/unspecified/reserved IPv4/IPv6 addresses
- handle compact numeric loopback variants such as `127.1` via `socket.inet_aton`/equivalent, not only string prefix checks
- restrict retailer and match type to allowlists

Add regression tests for unsafe URL examples: `javascript:...`, `127.1`, `[::1]`, `169.254.169.254`, `172.31.0.1`, `localhost`.

## Search/reuse order

When analyzing/searching a receipt:

1. Load accepted mappings first.
2. If accepted mapping has sourced fresh/enough price, use it and avoid provider calls.
3. If accepted mapping lacks sourced price or is stale, show it as `needs_review` and offer price refresh; explicit provider search may still run.
4. Filter exact rejected targets out of future suggestions.
5. If no usable mapping exists, explicit web/provider search may run.
6. Provider errors/rate limits become `needs_review` candidates, not fake prices.

## UI workflow

For each alternative expose real actions or disabled-with-reason controls:

- Accept
- Reject
- Later/review
- Manual mapping
- Save as preferred alternative
- Refresh price
- Open source
- Cache status: fresh / stale / missing / error

After accept/reject, show feedback and update local mappings. Shopping lists and runtime reports should prefer accepted safe mappings.

## Real-Bon UAT v3 additions

Also consult `references/grocery-release-real-bon-uat-v3.md` before release/UAT work. Durable pitfalls from real-bon testing:

- If no real Migros receipt exists in `grocery_product_items`, create a runtime-only 5–10 item subset from existing import line items; never commit it.
- No-price/no-unit-price provider candidates must remain `needs_review` and must not be counted as safe savings.
- Accepting a no-price candidate should normalize the mapping `match_type` to an allowed value such as `manual_match`, keep it auditably reviewable when appropriate, and avoid setting a fake price-check timestamp.
- After a release tag has already been pushed, do not force-retag for a UAT hotfix; push a normal follow-up commit on `main` and report the distinction.

## Tests and review gates

Add backend and frontend tests for:

- accepted mapping persistence and reuse without provider call
- rejected exact target not recommended again, but other retailers still allowed
- manual mapping save and unsafe URL/type rejection
- no-price and non-cheaper candidates persisted with `match_id` and rejectable
- manual prices excluded from safe savings until sourced refresh
- fresh vs stale/missing cache behavior
- no provider call on render; provider call only after explicit button click
- max-3-store shopping list and safe-only savings
- runtime-only report with sources/timestamps/disclaimer
- health analysis remains prepared-only

Before commit/push, run full compile/pytest/frontend test/build, source and build secret scans, Git-safety, `git diff --check`, browser sanity through Tailscale, and remote-hash verification.