xj[8dZddlZddlZddlZddlmZmZejeZ dZ dZ Gdde Z Gdd ZGd d ZGd d ZdS)ub Per-session turn lease — serializes the [load history → run → flush] region. Why this exists (#64934): the gateway's busy guards are keyed by ROUTING KEY (``_active_sessions`` in the adapter, ``_running_agents`` in the runner), but the durable transcript is owned by SESSION_ID — and ``switch_session()`` makes the key→id mapping many-to-one (``/resume`` of a named session from a second chat/topic, CLI-continuity rebinding, async-delegation completion pinning, Telegram topic-binding tip-walks). Two routing keys mapped to one session_id run concurrent turns on two different agent objects, so no per-key guard ever sees the collision. The two turns then interleave their flushes on one transcript: rows persist in completion order instead of arrival order, the identity-marker dedup over shared history dicts can swallow a row outright, and the second turn runs on a history base that never saw the first turn's exchange — leaving a permanent ``user;user`` alternation wedge that ``repair_message_sequence`` re-repairs on every request forever. The lease closes that route by serializing per RESOLVED session_id: it is acquired after session resolution is final (post ``switch_session``/tip-walk), immediately before the transcript load, and released in the dispatch layer's ``finally`` on every exit path. Same-key messages never reach the acquisition point while a turn runs (both routing-key guards hold them), so the lock is uncontended everywhere except the alias-key route — where the second turn now waits for the first turn's flush and logs one WARNING naming the session and both routing keys (pairing with the cross-agent tripwire in ``agent/agent_runtime_helpers.note_turn_start``). Safety properties: - **Generation-scoped, identity-checked release.** A token records its owner (routing key, run generation) and release only frees the lease when that exact token is the current holder — a stale unwind can never release a newer turn's lease (the #28686 ownership lesson applied). Release is idempotent. - **Fail-closed on timeout.** A timed-out waiter raises :class:`TurnLeaseTimeoutError` and must be rejected by the dispatch layer with a visible resend notice. It never runs concurrently against the still-held lease and therefore cannot defeat the serialization invariant. - **Bounded registry.** The per-session lease map is size-capped; eviction only ever removes idle (unheld, uncontended) entries, never a live lease. Known limits (deliberate, flagged on #64934): - A CLI process sharing the session via CLI-continuity is outside any in-process lock — that pair needs a DB-level lease (separate design). - Mid-turn compression rotation leaves a small alias window: the tip-walk can resolve a fresh child id while the parent-holding turn is still in flight. The mid-turn binding-sync sites are the right place to alias the lease in a follow-up. N)DictOptionalig @c 8eZdZdZdededededdf fd ZxZS) TurnLeaseTimeoutErrorzThe session lease stayed held for the caller's full wait budget. This is a fail-closed signal: the caller did not acquire the lease and must not enter the transcript load/run/flush region for this turn. session_id owner_key generation wait_secondsreturnNc ||_||_||_||_t d|dd|d|d|d dS)Nz turn lease wait timed out after z.0fz s on session z for routing key z (gen ))rrr r super__init__)selfrrr r __class__s 6/home/agent/.hermes/hermes-agent/gateway/turn_lease.pyrzTurnLeaseTimeoutError.__init__Os%"$(  K| N K K K K,5 K K=G K K K     ) __name__ __module__ __qualname____doc__strintfloatr __classcell__)rs@rrrHsw                    rrc:eZdZdZdZdedededdfdZdefd ZdS) TurnLeaseTokenzHandle returned by :meth:`SessionTurnLeaseRegistry.acquire`. A timeout raises :class:`TurnLeaseTimeoutError` instead of returning a token, so every token handed out is a held lease. ``released`` makes release idempotent. rrr releasedrrr r Nc>||_||_||_d|_dS)NFr)rrrr s rrzTurnLeaseToken.__init__ks$ %"$ rc Hd|jd|jd|jd|jd S)NzTurnLeaseToken(session_id=z , owner_key=z , generation=z , released=r rrs r__repr__zTurnLeaseToken.__repr__vsL ) ) ) ) )8< ) )  ) ) ) r) rrrr __slots__rrrr#rrrras{FI          #      rrc6eZdZdZddZedefdZdS) _SessionLease)lockholder acquired_at last_usedpending_acquiresr Nctj|_d|_d|_t j|_d|_dS)Nr)asyncioLockr(r)r*timer+r,r"s rrz_SessionLease.__init__s9LNN 04  !rc^|jduo$|j o |jdkS)z?True when this lease can be evicted: nobody holds or awaits it.Nr)r)r(lockedr,r"s ridlez_SessionLease.idles< K4  +I$$&&& +%* rr N)rrrr$rpropertyboolr4r%rrr'r'~sVI"""" d   X   rr'c eZdZdZefdeddfdZdefdZdede fdZ dd Z dd ded ed ed e e de ef dZde ededefdZde edefdZdS)SessionTurnLeaseRegistryuAsyncio lease per resolved session_id serializing transcript turns. Process-local and single-event-loop by design — the same visibility scope as the routing-key guards it extends. All methods must be called from the gateway's event loop. max_entriesr NcXi|_tdt||_dS)N)_leasesmaxr _max_entries)rr:s rrz!SessionTurnLeaseRegistry.__init__s)13 3{#3#344rc*t|jSN)lenr=r"s r__len__z SessionTurnLeaseRegistry.__len__s4<   rrc|j|}|,|t}||j|<t j|_|SrA)r=get _evict_idler'r1r+)rrleases r_get_or_createz'SessionTurnLeaseRegistry._get_or_createsU   ,, =      !OOE',DL $)++ rctjjz dz}|dkrdStdjDfd}|d|D]}j|ddS)uDrop oldest idle entries so a new lease fits under the cap. Never evicts a held or contended lease — correctness beats the cap. r<rNc3.K|]\}}|j |VdSrA)r4).0sidrGs r z7SessionTurnLeaseRegistry._evict_idle..s- F FZS%5: FS F F F F F Frc(j|jSrA)r=r+)rLrs rz6SessionTurnLeaseRegistry._evict_idle..sDL-7r)key)rBr=r?sorteditemspop)roverflowidle_idsrLs` rrFz$SessionTurnLeaseRegistry._evict_idles t|$$t'881< q== F F F4<#5#5#7#7 F F F7777   IXI& ( (C L  S$ ' ' ' ' ( (rtimeoutrr rWc K|sdS|r|dkrt|nt}t||t|}||}|jr[|j}t d||||r|j nd|r|j nd|j rtj |j z nd|xjdz c_ tj|j|d{Vn`#tj$rN|j}td|||||r|j nd|r|j ndt)|||| dwxYw |xjdzc_n#|xjdzc_wxYw||_tj |_ |j |_|S) a>Acquire the turn lease for ``session_id``, waiting if held. Returns a held :class:`TurnLeaseToken`. Raises :class:`TurnLeaseTimeoutError` when the wait budget expires; the caller must reject rather than enter the serialized region. Returns ``None`` for a falsy ``session_id``. Nruturn lease contention on session %s: routing key %s (gen %s) waiting behind in-flight turn held by routing key %s (gen %s, held %.0fs) — two routing keys are mapped to one session_id (#64934); serializing this turn behind the previous turn's flush?gr<rVuturn lease wait timed out after %.0fs on session %s (waiter: routing key %s gen %s; holder: routing key %s gen %s) — failing closed: refusing to run this turn UNSERIALIZED against the still-held lease)rr r )rDEFAULT_LEASE_WAITrrrHr(r3r)loggerwarningrr r*r1r,r/wait_foracquire TimeoutErrorerrorrr+) rrrr rWwaittokenrGr)s rr^z SessionTurnLeaseRegistry.acquires. 4!(PWq[[uW~~~>Pz9c*ooFF##J// :     \F NN $*3  %+4!!383DN e///$   & !# ("5:#5#5#7#7FFF F F F F F F F F#   \F LL<$*3  %+4!!   (#%!      G,  " "a ' " " "E " "a ' " " " " " "  IKK+ s3DFAE..FFrbnew_session_idc ||js |r ||jkrdS|j|j}| |j|urdS|j|}|V||urR|jsK|j}t d|j||j|j |r|jnd|r|j nd|dS||j|<tj |_ ||_dS)uAlias a HELD lease onto ``new_session_id`` after mid-turn rotation. Compression can rotate the durable session_id while a turn is in flight (session-hygiene pre-compression, in-agent compression). The turn's flush then targets the NEW id — so the serialization boundary must follow it, or an alias routing key resolving the new id (e.g. a topic tip-walk landing on the fresh child) could start a concurrent turn the lease never sees. This closes the rotation-alias window flagged on #64934. Mechanism: the SAME ``_SessionLease`` object is registered under the new id (the old mapping stays until it goes idle and is evicted), so acquirers on either id serialize against one lock — no lock state is moved, no asyncio internals are touched. Only the current holder can rebind (identity-checked like release), and the token follows to the new id so release frees the shared object. Edge: if the new id already has a live lease of its own (another turn is running on the target session), the two serialization domains cannot be merged mid-wait — log loudly and keep the token on the old id. Fail-open, never deadlock: a holder cannot wait mid-turn. NFuturn lease rebind blocked: session %s rotated to %s mid-turn (holder: routing key %s gen %s) but the target session's lease is already live (holder: routing key %s gen %s) — keeping the lease on the old id; transcript writes on %s may interleave (#64934 rotation-alias edge)rYT) rrr=rEr)r4r[r\rr r1r+)rrbrcrGexistingr)s rrebindzSessionTurnLeaseRegistry.rebind s0 M~ ! !1115   !122 =EL555<##N33  HE$9$9(-$9_F NN>   $*3  %+4!!   5', ^$)++)trc||jrdSd|_|j|j}|dS|j|ur.t d|j|j|jdSd|_d|_ tj |_ |j r|j dS)uZRelease ``token``'s lease. Idempotent; ownership-checked. Returns True only when this exact token was the current holder and the lock was freed. A re-release or a stale token whose slot has since been granted to a newer turn are both safe no-ops — a stale unwind can never release a newer turn's lease. NFTzYturn lease release skipped on session %s: token (key %s gen %s) is not the current holderr.)rr=rErr)r[debugrr r*r1r+r(r3release)rrbrGs rriz SessionTurnLeaseRegistry.releaseDs =EN=5   !122 =5 rns00d !!!!!!!!  8 $ $      L   2        :        4HHHHHHHHHHr