## Sprint 6E.1 audit — prioritized gaps

**Audit scope:** source-only review at `60ca752`. The worktree also contains concurrent/uncommitted WIP in `read_api.py`, `document_fts_migrate.py`, and new `dashboard_v5/document_chunks.py`; findings below reflect the currently observable implementation. **No files edited.**

### Critical

1. **Document filtering is applied after SQL sort/limit; results can be omitted or falsely empty.**  
   - **Location:** `scripts/health/dashboard_v5/read_api.py:_record_document_rows` lines **1243–1260**.  
   - FTS IDs are selected independently with `LIMIT 101` (line 1247), then the document query applies only metadata filters/order/`LIMIT 101` (line 1254), and finally applies the FTS membership in Python (line 1259). Thus a matching document beyond the first 101 sorted metadata rows is never returned. This violates “SQL filtering before sorting/limiting.”  
   - **Tests:** Seed >101 documents where only a later sorted row matches FTS plus category/date filters; verify it appears. Verify SQL joins/intersects FTS and `dokumente` with all filters before `ORDER BY … LIMIT`.

2. **No opaque cursor pagination on the documents API (and no reliable continuation contract).**  
   - **Location:** `_record_document_rows` **1216–1267** and `dispatch_api` documents parameter allowlist **1401–1402**.  
   - Endpoint hard-rejects >100 rows instead of returning a bounded page plus opaque `next_cursor`; `cursor`/`page_size` are not accepted. Newly added pagination constants at **1217–1221** are unused.  
   - **Tests:** deterministic forward traversal for every allowed sort, equal sort-key ties resolved by ID, malformed/tampered/cross-sort/cross-filter cursor rejection, page-size bounds, no duplicates/skips across pages.

### High

3. **FTS queries neither enforce *current* reviewed state at query time nor literal-token semantics; operator-like valid tokens cause a 503.**  
   - **Locations:** `_record_document_rows` **1243–1249**; `_record_document_matches` **1303–1316**; `_fts_tokens` **1270–1277**.  
   - FTS searches query `health_document_fts` alone, rather than joining `dokumente` and requiring `d.review_status='geprueft'`. A document demoted after the last rebuild remains searchable via stale index entries.  
   - `" AND ".join(tokens)` treats tokens such as `OR`, `AND`, `NOT` as FTS syntax rather than literals. Verified with a synthetic DB: `q=alpha%20OR%20beta` returns **`503 data_unavailable`**, through SQLite syntax failure, instead of a valid literal search or controlled client error.  
   - **Tests:** rebuild while reviewed, change document to unreviewed without rebuild, verify listing and matches return no FTS result; verify tokens `OR`, `AND`, `NOT`, repeated terms, and Unicode terms are literal and never produce a 503.

4. **Global search does not search document FTS content.**  
   - **Location:** `_search` **798–828**, especially `_safe_documents` **769–795**.  
   - `/api/v1/search` only scans reviewed document metadata (`category`, `institution`, date); it does not use `health_document_fts`, so text-only document queries are absent from the global search group.  
   - **Tests:** reviewed document whose unique term exists only in `extrahierte_inhalte`; assert `/api/v1/search?q=…` returns its opaque document result, while unreviewed/stale-index content does not.

### Medium

5. **Shared chunk work is not wired into read endpoints; detail still truncates at 28k and disagrees with FTS chunk boundaries.**  
   - **Locations:** new `scripts/health/dashboard_v5/document_chunks.py` **20–36** and updated migration **50–65**; but `_record_document_detail` remains legacy at `read_api.py` **1294–1300**.  
   - Detail still slices raw text and explicitly caps it with `min(len(text), 28_000)`. FTS uses normalized shared chunks, so returned detail section numbers/text can differ from match section/snippet offsets even below the cap.  
   - **Tests:** >28,000-character synthetic reviewed document with a target term after the old cap; verify it is retrievable through cursor-paginated sections and that match section/offset maps exactly to returned shared chunk text. Also test CRLF, blank lines, and whitespace normalization.

6. **Original-file route validates a path but then reopens and buffers it, leaving TOCTOU and memory/streaming gaps.**  
   - **Locations:** `scripts/health/health_dashboard_server.py:verified_original_path` **218–245** and `Handler._handle_api` **733–753**.  
   - `verified_original_path` resolves/checks size/type, but the route subsequently calls `found[0].read_bytes()` (line 740). A replacement between validation and read can bypass the checked inode/type/size; `read_bytes()` also buffers up to 20 MiB rather than streaming a pinned descriptor.  
   - **Tests:** deterministic descriptor-level test: replace the validated file before open/read and ensure neutral `404` (or safely serve only the verified inode); reject final/intermediate symlinks; reject non-regular files, over-limit files, unsupported magic/type; assert chunked streaming, exact `Content-Length`, no body on `HEAD`, and `no-store`/`nosniff` on every error/success path.

### Additional SQL-before-limit spillover

- `_record_medications` **1324–1330** and `_record_appointments` **1349–1346** fetch newest 101 records before applying date/institution filters in Python. Older valid in-range rows can disappear or return a misleading empty result. Apply predicates in SQL before ordering/limit, then include cursor pagination where the endpoint is list-like.
- Tests should seed >101 newer out-of-range rows plus an older in-range record and assert it is returned.

## Verification performed

- `PYTHONDONTWRITEBYTECODE=1 pytest -q tests/test_dashboard_v5_sprint6e_record.py` → **2 passed**.
- Synthetic FTS probe confirmed operator-token failure: **`503 data_unavailable`** for `q=alpha OR beta`.
- `git diff --check` passed. Repository was already non-clean due to concurrent WIP noted above; I made **no modifications**.