D\j/ddlmZddlZddlZddlZddlZddlZddlZddl Zddl m Z m Z ddl m Z mZddlmZddlmZddlmZdZd d d d d dZdddZddDZddZddd?d"Zddd@d#Ze d$%Gd&d'ZeefdAd*ZdBdCd-Z eefd.d/dDd2Z!dEd3Z"d4d5dFd9Z#dd:dGd;Z$dS)H) annotationsN)asdict dataclass)datetimetimezone)Decimal)Path)Anyzexternal_market_context.v1bitcoinethereumsolana chainlink hyperliquid)BTCETHSOLLINKHYPEEthereumSolana)rrci|]}||d S)USDT.0coins 'src/research/external_market_context.py rs QQQ44DQQQ)rrrr0valuer defaultstrreturnrcz tt|S#t$rt|cYSwxYwN)rr# Exception)r!r"s r_dr(sJ s5zz"""    w s ::cbtjtjSr&)rnowrutc isoformatrrr_nowr-s < % % / / 1 11rtimeouturlr0intctj|ddd}tj||5}t j|cdddS#1swxYwYdS)NCryptoAgentResearch/1.0zapplication/jsonz User-AgentAcceptheadersr/)urllibrequestRequesturlopenjsonloadr1r0r:responses r _get_jsonrA!sn$$SAZfx2y2y$zzG     9 9#Xy""##################sA''A+.A+ctj|ddd}tj||5}|dcdddS#1swxYwYdS)Nr4ztext/csvr5r7r/utf-8)r9r:r;r<readdecoder?s r _get_textrF'sn$$SAZfp2q2q$rrG     9 9/X}}%%g..//////////////////s'A::A>A>T)frozenceZdZUded<ded<ded<ded<ded<ded <d Zd ed <eZded <dZded<dZded<dZ ded<ddZ dS)SourceEnveloper# source_id observed_atstatus reliabilitybool research_onlydict[str, Any]datartuple[str, ...]blockerssource_versionr2max_age_secondsg?floatreliability_scoreNz str | Noneprovider_observed_atr$cdt|}t|j|d<|j|d<d|d<d|d<t j|j|j|j|j dddt  d }d tj |z|d <|S) NrS collected_atr age_secondsTfresh)rJrTrYrQ),:) sort_keys separatorsr"rCzsha256: snapshot_id)rlistrSrKr=dumpsrJrTrYrQr#encodehashlibsha256 hexdigest)selfpayload fingerprints rto_dictzSourceEnvelope.to_dict;s,,"4=11 "&"2!" jt~QUQdCXbfbk"l"lx|IS]`aaahhipqq !*W^K-H-H-R-R-T-T!T r)r$rP) __name__ __module__ __qualname____annotations__rSSOURCE_VERSIONrTrVrXrYrlrrrrIrI-sNNNKKK "H""""(N((((O"""""'+++++rrIcoinsrRcd|D}tjdd|dd}t d|}d|D}i}g}|D]-}|t |d}|s|d |Ltt|d t|d pd tt|d tt|dtt|dd||</d|D} tdtt|t|krdn|rdnddd|t|dd| rt| nd S)Nc:g|]}|tv t|Sr) COINGECKO_IDSrs r z%collect_coingecko..Gs' J J J4DM4I4I= 4I4I4Irusdr^24h) vs_currencyidsprice_change_percentagez/https://api.coingecko.com/api/v3/coins/markets?c~i|]:}t|tt|d|;S)id isinstancedictr#getrrows rrz%collect_coingecko..Js; N N N 38M8M NS   N N Nrzcoingecko_missing: market_capmarket_cap_rankr total_volumeprice_change_percentage_24hcirculating_supply)market_cap_usdrtotal_volume_usdprice_change_24h_pctrcg|]N}t|t|d,t|dOS) last_updated)rrrr#rs rrvz%collect_coingecko..YsRxxxsjQTVZF[F[x`c`g`ghv`w`wxc#''.1122xxxrzcoingecko:coins_markets:v3okpartialdegradedmediumTi皙?)rVrXrY)r9parse urlencodejoinrArruappendr#r(r2rIr-lentuplemin) rrrzqueryrowsby_idrQrSrrprovider_timess rcollect_coingeckorFs[ J J5 J J JC L " "5#kp#q#q r rE NuNN O OD N N N N NEDH   ii ))$3344  OO777 8 8 8 !"SWW\%:%:";";<<"377+<#=#=#BCC #Bsww~'>'>$?$? @ @$'3773P+Q+Q(R(R$S$S"%b1E)F)F&G&G"H"H   T yxdxxxN 6D UXY\U]U]H]H]ptdEclcl{EGOQUW[]bck]l]l~BVZHVq`qtuCqDqDqD\` a a aarpreviousdict[str, Any] | Nonec td}d|D}t|tr|pidpini}i}g}tD]\}}||}|s|d|5t|d} t||pid} | dkr%| | z tdz td zntd } |t| t| d ||<td t|rd nddd|t|dd S)Nzhttps://api.llama.fi/v2/chainsc~i|]:}t|tt|d|;S)namer~rs rrz%collect_defillama.._s;RRRSJsDu>> ? ? ?   M%%d++1r66yAABBDIAII3;'#,,.'%..@@SZ[^S_S_$S\_`f\g\ghhT 2DFFD)?)?)I)ISZ)I)[)[\\*3/B/F/F~/V/V+W+W+_+_`cem+n+noo !TY[[9#6#6#8#8899  " "8 , , 4 4266w??E2 $( ! ! OO@ A A A 8886777778SS $$T**   S `X^ ` `aaGY[[\\B Gkq G G GHHF)3FD)A)ATfT6"::RTL*..t44:K #Bw{{3D'E'E$F$F G G!"W[[%=%=">">??"2gkk,&?&?#@#@AA+.r"&&2H2H/I/I+J+J+.r,2B2BCS2T2T/U/U+V+V$'<+;+;M+J+J(K(KgV[nn(\$]$]%(L,<,<^,L,L)M)MPWX]P^P^)^%_%_dy-DC;??C]3^3^0_0_,`,`,`@DZo'ys2koo>S.T.T+U+U'V'V'Vuy\q({B{?U/V/V,W,W(X(X(Xw{Wl+v3{}/M/M/RQR+S+S+Srv  DJJ S S S OOQdQQT#YY=OQQ R R R R R R R R S EtvvW[PTdlPTtt@DsTr{r{JTV^`dfjlqrzl{l{MPdh i i iis+DD D.-D.JO** P,4-P''P,c i}g} td}|dpidpdgdpi}|dpg}|dpidpigddpg}d t||D}|r||d \}}tjt |t j tt|d |d <n| d nA#t$r4} | dt| jYd} ~ nd} ~ wwxYw t!dd} t#t%jt)j| } t-dt/| Dd} | rI| dtt| dd |d<n| dnA#t$r4} | dt| jYd} ~ nd} ~ wwxYwt1dt3d |vrdnddd|t5|dd S) NzMhttps://query1.finance.yahoo.com/v8/finance/chart/%5EVIX?range=5d&interval=1dchartresultrr indicatorsquoteclosec g|] \}}|||f Sr&r)rtsr!s rrvz&collect_fred_macro..s%[[[UIZ"eIZIZIZrr)dater! vix_closevix_series_emptyzvix_series_error:z6https://fred.stlouisfed.org/graph/fredgraph.csv?id=DFFr.r/c3HK|]}|ddv|VdS)DFF>Nr.Nrrs r z%collect_fred_macro..s6^^so8]8]s8]8]8]8]^^robservation_daterfed_funds_effective_pctzfred_series_empty:DFFzfred_series_error:DFF:zmacro:yahoo_vix_fred_dff:v1rrhighTiQrr)rArzipr fromtimestampr2rr+rr,r#r(rr'rrmrFrccsv DictReaderioStringIOnextreversedrIr-r) rQrSyahoor timestampsclosespairsrr!rtextrlatests rcollect_fred_macror s?DH BijjIIg&&,"11(;;EvqIORZZ ,,2 ZZ --388AAIbT1MRRSZ[[a_a[[c*f.E.E[[[  0b IB)1)?B)V)V)[)[)])])g)g)i)itwxz|AyByBuCuC!D!DD   OO. / / / BBB@DII,>@@AAAAAAAAB GQ[]^^^CN2;t#4#45566^^htnn^^^`dee  57=zzBT7U7U`cdfgmgqgqrwgxgxdydy`z`z.{.{D* + + OO3 4 4 4 GGGEc1CEEFFFFFFFFG 7X\I\I\blntvz}ACHIQCRCRdi}A B B BBs1EE F!*FFC I%% J#/*JJ#i)rVpathrVrPc ht|}|sdddgidS tj|d}n#t $r dddgidcYSwxYw|d } td tjtj t| d d  z }n#t $rtd }YnwxYwt|dpg}|dpi}|D]\}} td tjtj t| dp| d d d  z } n#t $rtd } YnwxYw| | d<| t#| dpd k| d<| ds|d| ||kr|di||||ktt&|dS)NmissingFexternal_context_missing)rLr]rSsourcesrCrinvalidexternal_context_invalidrrrrinfrSrr[rKr\rVr]z source_stale:external_context_stale)r\r]rS)r existsr=rrr'rrrrrr#rrrWrcrr2rrfromkeys) r rVtargetrjrrrSrkeysource source_ages rload_external_contextrs  $ZZF ==??n#eB\A]jlmmmn*V--w-??@@ nnn#eB\A]jlmmmmmn;;~..L!TY[[8#9#l:K:K:S:STWYa:b:b#c#c#m#m#o#oopp EllGKK ++1r22Hkk)$$*G}}33 V &Q h.DSTbIcIcJAgmgqgqrhAhAFBFBFJFJKNPXFYFY/Z/Z/d/d/f/f!fggJJ & & &uJJJ & *}$FJJ7H,I,I,NQ(O(OOwg 3 OO1C11 2 2 2 _0111 vg vcC?4JX\]a]j]jks]t]tXuXu v v vvs7(AA+*A+A)C..D  D #BG55HHrci}g}ddfdfdfddfddff}|D]\}} |}n`#t$rS}t|td d d id t |jf}Yd}~nd}~wwxYw|||<||d pgtd| D}tt|t|krdn|rd ndd dd|tt|dS)N coingeckoctSr&)rrrrz build_snapshot..s /11r defillamacvtpidpidS)Nrr)rrrsrrz build_snapshot..s7/(.b1E1Ei1P1P1VTV0[0[\g0h0hiirbinance_crowdingctSr&)rrrrrz build_snapshot..s %=%?%?r fred_macroctSr&)r rrrrz build_snapshot..s 133rrlowTz source_error:rSc3DK|]}|ddVdS)rQNr)rrs rrz!build_snapshot..s3HHvVZZ5G5GHHHHHHHrr unavailableF)schemarrLrOlive_order_allowedmainnet_signed_actionrrS)rlr'rIr-rrmextendrsumvaluesrqrrcrr) rrrS collectorsr collectorenveloperusables ` rbuild_snapshotr3s GH 112 iiiij ??@ 334 J %88Y C y{{**,,HH C C C%c466:udBQuaefiajajasQuQuPwxxAACCHHHHHH C  Z006B7777 HH!1!1HHH H HF  CLL00$$F6]jjP]#!&x0011   s? B A BB)r )r!r r"r#r$r)r$r#)r1r#r0r2r$r )r1r#r0r2r$r#)rrrRr$rIr&)rrr$rI)rrrRrrr$rI)r$rI)r rrVr2r$rP)rrr$rP)% __future__rrrfrr=r urllib.parser9urllib.request dataclassesrrrrdecimalrpathlibr typingr rqrurrr(r-rArFrIrrrrr rr3rrrr;s""""""  ))))))))''''''''-!*X{dqrr %h77QQ3PQQQ     2222+-###### +-//////  $005u]/C/Caaaaa.ppppp$7