Si2wdZddlZddlZddlZddlZddlZddlZddlZddlZddlm Z m Z m Z ddl m Z ddlmZddlmZmZddlmZdd lmZmZdd lmZmZmZmZmZmZd Zd Z d Z!dZ"dZ#dZ$dZ%dZ&GddZ'y)z scholarly.pyN)DictListUnion) Navigator)ProxyGenerator) find_dotenv load_dotenv) AuthorParser)PublicationParser_SearchScholarIterator)Author AuthorSource CitesPerYearJournal PublicationPublicationSourcez4/citations?hl=en&view_op=search_authors&mauthors={0}z:/citations?hl=en&view_op=search_authors&mauthors=label:{0}z3/citations?hl=en&view_op=search_authors&mauthors={}z [-: #(),;]+z/scholar?hl=en&q={0}z/scholar?hl=en&cites={0}z)/citations?view_op=view_org&hl=en&org={0}zKhttps://scholar.google.com/citations?view_op=mandates_leaderboard_csv&hl=encbeZdZdZdZedZdeddfdZ dCde d e ddfd Z d e fd Z d efdZ dDdede de dedededededefdZdeeeffdZdEdede defdZdefdZgd dfd!ed"ed#edexsefd$Zd!edefd%Zed&efd'Zd!edefd(Zd!efd)ZdFd*ede d"ed#edef d+Z d,efd-Z!d.e"efd/Z#d0edefd1Z$d0edefd2Z%d!edefd3Z&d!exseddfd4Z'dEded5e de(fd6Z)d7efd8Z* dGd9ed:e d;e fd<Z+ dDd=ede de dedededededefd>Z,d?Z-dHd@e de.ee/ffdAZ0dHdBZ1y)I _Scholarlyz(Class that manages the API for scholarlyctttjj |_t |_|jj|_d|_ yN) r r osenvironcopyenvr_Scholarly__navlogger_journal_categoriesselfs L/home/agent/.friday_env/lib/python3.12/site-packages/scholarly/_scholarly.py__init__z_Scholarly.__init__s@KM"::??$[ jj'' #' c\|j|j|_|jSr)rget_journal_categoriesrs r!journal_categoriesz_Scholarly.journal_categories&s+  # # +'+'B'B'DD $'''r# num_retriesreturnNc8|jj|S)zSets the number of retries in case of errors :param num_retries: the number of retries :type num_retries: int )r _set_retries)r r's r! set_retriesz_Scholarly.set_retries,szz&&{33r#proxy_generatorsecondary_proxy_generatorc<|jj||y)a,Select which proxy method to use. See the available ProxyGenerator methods. This is used to get some pages that have strong anti-bot prevention. ``secondary_proxy_generator`` is used for other pages that do not have a strong anti-bot prevention. If not set, free proxies are used. :param proxy_generator: a proxy generator object, typically setup with a premium proxy service (ScraperAPI or Luminati) :type proxy_generator: ProxyGenerator :param proxy_generator: a second proxy generator object, optional :type proxy_generator: ProxyGenerator :Example:: .. testcode:: pg = ProxyGenerator() pg.ScraperAPI(YOUR_SCRAPER_API_KEY) scholarly.use_proxy(pg) N)r use_proxy)r r,r-s r!r/z_Scholarly.use_proxy5s2 _.GHr#enablec:|jj|y)zTEnable or disable the logger for google scholar. Enabled by default N)r set_logger)r r0s r!r2z_Scholarly.set_loggerQs f%r#timeoutc:|jj|y)z+Set timeout period in seconds for scholarlyN)r set_timeout)r r3s r!r5z_Scholarly.set_timeoutWs w'r#rquerypatents citationsyear_low year_highsort_byinclude_last_year start_indexc |jtjtjj ||||||||} |j j| S)a Searches by query and returns a generator of Publication objects :param query: terms to be searched :type query: str :param patents: Whether or not to include patents, defaults to True :type patents: bool, optional :param citations: Whether or not to include citations, defaults to True :type citations: bool, optional :param year_low: minimum year of publication, defaults to None :type year_low: int, optional :param year_high: maximum year of publication, defaults to None :type year_high: int, optional :param sort_by: 'relevance' or 'date', defaults to 'relevance' :type sort_by: string, optional :param include_last_year: 'abstracts' or 'everything', defaults to 'abstracts' and only applies if 'sort_by' is 'date' :type include_last_year: string, optional :param start_index: starting index of list of publications, defaults to 0 :type start_index: int, optional :returns: Generator of Publication objects :rtype: Iterator[:class:`Publication`] :Example:: .. testcode:: search_query = scholarly.search_pubs('Perception of physical stability and center of mass of 3D objects') scholarly.pprint(next(search_query)) # in order to pretty print the result :Output:: .. testoutput:: {'author_id': ['4bahYMkAAAAJ', 'ruUKktgAAAAJ', ''], 'bib': {'abstract': 'Humans can judge from vision alone whether an object is ' 'physically stable or not. Such judgments allow observers ' 'to predict the physical behavior of objects, and hence ' 'to guide their motor actions. We investigated the visual ' 'estimation of physical stability of 3-D objects (shown ' 'in stereoscopically viewed rendered scenes) and how it ' 'relates to visual estimates of their center of mass ' '(COM). In Experiment 1, observers viewed an object near ' 'the edge of a table and adjusted its tilt to the ' 'perceived critical angle, ie, the tilt angle at which ' 'the object', 'author': ['SA Cholewiak', 'RW Fleming', 'M Singh'], 'pub_year': '2015', 'title': 'Perception of physical stability and center of mass of 3-D ' 'objects', 'venue': 'Journal of vision'}, 'citedby_url': '/scholar?cites=15736880631888070187&as_sdt=5,33&sciodt=0,33&hl=en', 'eprint_url': 'https://jov.arvojournals.org/article.aspx?articleID=2213254', 'filled': False, 'gsrank': 1, 'num_citations': 23, 'pub_url': 'https://jov.arvojournals.org/article.aspx?articleID=2213254', 'source': 'PUBLICATION_SEARCH_SNIPPET', 'url_add_sclib': '/citations?hl=en&xsrf=&continue=/scholar%3Fq%3DPerception%2Bof%2Bphysical%2Bstability%2Band%2Bcenter%2Bof%2Bmass%2Bof%2B3D%2Bobjects%26hl%3Den%26as_sdt%3D0,33&citilm=1&json=&update_op=library_add&info=K8ZpoI6hZNoJ&ei=QhqhX66wKoyNy9YPociEuA0', 'url_scholarbib': '/scholar?q=info:K8ZpoI6hZNoJ:scholar.google.com/&output=cite&scirp=0&hl=en'} )r7r8r9r:r;r<r=)_construct_url _PUBSEARCHformatrequestsutilsquotersearch_publications) r r6r7r8r9r:r;r<r=urls r! search_pubsz_Scholarly.search_pubs[sbD!!*"3"3HNN4H4H4O"PZa,5T]*1EVdo"qzz--c22r#publication_idc |jtjt|fi|}|jj |S)aSearches by Google Scholar publication id and returns a generator of Publication objects. :param publication_id: Google Scholar publication id :type publication_id: int or str For the remaining parameters, see documentation of `search_pubs`. )r?_CITEDBYSEARCHrAstrrrE)r rHkwargsrFs r!search_citedbyz_Scholarly.search_citedbysA"d!!."7"7N8K"LWPVWzz--c22r# pub_titlefilledctjtjj |}|j j ||S)a&Search by scholar query and return a single Publication container object :param pub_title: Title of the publication to search :type pub_title: string :param filled: Whether the application should be filled with additional information :type filled: bool )r@rArBrCrDrsearch_publication)r rNrOrFs r!search_single_pubz_Scholarly.search_single_pubs: 4 4Y ?@zz,,S&99r#namectjtjj |}|j j |S)a%Search by author name and return a generator of Author objects :Example:: .. testcode:: search_query = scholarly.search_author('Marty Banks, Berkeley') scholarly.pprint(next(search_query)) :Output:: .. testoutput:: {'affiliation': 'Professor of Vision Science, UC Berkeley', 'citedby': 21074, 'email_domain': '@berkeley.edu', 'filled': False, 'interests': ['vision science', 'psychology', 'human factors', 'neuroscience'], 'name': 'Martin Banks', 'scholar_id': 'Smr99uEAAAAJ', 'source': 'SEARCH_AUTHOR_SNIPPETS', 'url_picture': 'https://scholar.google.com/citations?view_op=medium_photo&user=Smr99uEAAAAJ'} ) _AUTHSEARCHrArBrCrDrsearch_authors)r rSrFs r! search_authorz_Scholarly.search_authors80  !5!5d!;<zz((--r#citedbyobjectsortbypublication_limitc|ddk(r:t|j}|j||||}|dur td|S|ddk(r&t |j}|j|}|S)aFills the object according to its type. If the container type is Author it will fill the additional author fields If it is Publication it will fill it accordingly. :param object: the Author or Publication object that needs to get filled :type object: Author or Publication :param sections: the sections that the user wants filled for an Author object. This can be: ['basics', 'indices', 'counts', 'coauthors', 'publications', 'public_access'] :type sections: list :param sortby: if the object is an author, select the order of the citations in the author page. Either by 'citedby' or 'year'. Defaults to 'citedby'. :type sortby: string :param publication_limit: if the object is an author, select the max number of publications you want you want to fill for the author. Defaults to no limit. :type publication_limit: int Note: For Author objects, if 'public_access' is filled prior to 'publications', only the total counts from the Public Access section of the author's profile page is filled. If 'public_access' is filled along with 'publications' or afterwards for the first time, the publication entries are also marked whether they satisfy public access mandates or not. container_typerFzIncorrect inputr)r rfill ValueErrorr )r rYsectionsrZr[ author_parserpublication_parsers r!r^z_Scholarly.fills( " #x /(4M"''&BSTF !233 $ % 6!24::!> ',,V4F r#c|ddk(r&t|j}|j|S|jj dy)zReturns a bibtex entry for a publication that has either Scholar source or citation source :param object: The Publication object for the bibtex exportation :type object: Publication r]r+Object not supported for bibtex exportationN)r rbibtexrwarning)r rYrbs r!rez_Scholarly.bibtexsH " #} 4!24::!> %,,V4 4 KK   M N r#cites_per_yearcg}||}}d}t|dD]2}|||zdkr |||z }|}||}|j||f|}4|dkDr|j||f|S)NrT)reverse)sortedappend)rgyear_endyearsy_hiy_lo running_countys r!_bin_citations_by_yearz!_Scholarly._bin_citations_by_yearsxd 5 A~a00D8!22  .q 1  dD\*  1  LL$ & r#c&|ddk7r|jjdy|ddkr$t|jj |S|jj d|dd |dt tjjj}|d tjk(r4|j||j|jd i|}n: t |dd }t#t%||dz dt%||dz d}|j'||S#t $rI|jjd |dd t|jj |cYSwxYw)zSearches Google Scholar for other articles that cite this Publication and returns a Publication generator. :param object: The Publication object for the bibtex exportation :type object: Publication r]rrdN num_citationsrjzSSince the paper titled %s has %d citations (>1000), fetching it on an annual basis.bibtitlesourcergpub_yearzXUnknown publication year for paper %s, may result in incorrect number of citedby papers.r)rrfr rrXdebugintdatetimedatetodayyearrAUTHOR_PUBLICATION_ENTRYr^rsgetKeyErrorziprange _citedby_long)r rYrmrnr9s r!rXz_Scholarly.citedbys " #} 4 KK   M N  / "d *$TZZ088@ @ <=CE]7=SU[\kUl nx}}**,112 ( 0II I IIf // ;KR0PRZ[E Eve}Z89h B7xRSUW9XYE!!&// E ##%9:@-:PR(4<>AFFc#JKtjd|d}|jdd}|D]l\}}|j|||}|jr=|jdkDr.|j j d|dd|j||Ed{ny7w) Nz cites=[\d+,]* citedby_url)rHr9r:rjzThe paper titled %s has %d citations in the year %d. Due to the limitation in Google Scholar, fetching only 1000 results from that year.rvrw)researchgrouprM total_resultsrwarn)r rYrnmpub_idrorp sub_citationss r!rz_Scholarly._citedby_long6s IIovm'< =12 %JD$ //vPT`d/eM** 0K0Kd0R   "34:5M'4JMLgLgimo% $ $  % %sBB#B!B#idc>|jj||||S)uHSearch by author id and return a single Author object :param sortby: select the order of the citations in the author page. Either by 'citedby' or 'year'. Defaults to 'citedby'. :type sortby: string :param publication_limit: if the object is an author, select the max number of publications you want you want to fill for the author. Defaults to no limit. :type publication_limit: int :Example:: .. testcode:: search_query = scholarly.search_author_id('EmD_lTEAAAAJ') scholarly.pprint(search_query) :Output:: .. testoutput:: {'affiliation': 'Institut du radium, University of Paris', 'citedby': 2208, 'filled': False, 'interests': [], 'name': 'Marie Skłodowska-Curie', 'scholar_id': 'EmD_lTEAAAAJ', 'source': 'AUTHOR_PROFILE_PAGE', 'url_picture': 'https://scholar.googleusercontent.com/citations?view_op=view_photo&user=EmD_lTEAAAAJ&citpid=3'} )rsearch_author_id)r rrOrZr[s r!rz_Scholarly.search_author_idBs 6zz**2vv?PQQr#keywordctjtd|}tj t j j|}|jj|S)aSearch by keyword and return a generator of Author objects :param keyword: keyword to be searched :type keyword: str :Example:: .. testcode:: search_query = scholarly.search_keyword('Haptics') scholarly.pprint(next(search_query)) :Output:: .. testoutput:: {'affiliation': 'Postdoctoral research assistant, University of Bremen', 'citedby': 56666, 'email_domain': '@collision-detection.com', 'filled': False, 'interests': ['Computer Graphics', 'Collision Detection', 'Haptics', 'Geometric Data Structures'], 'name': 'Rene Weller', 'scholar_id': 'lHrs3Y4AAAAJ', 'source': 'SEARCH_AUTHOR_SNIPPETS', 'url_picture': 'https://scholar.google.com/citations?view_op=medium_photo&user=lHrs3Y4AAAAJ'} _) rsub_KEYWORDSEARCH_PATTERN_KEYWORDSEARCHrArBrCrDrrV)r r reg_keywordrFs r!search_keywordz_Scholarly.search_keyword_sK>ff3S'B ##HNN$8$8$EFzz((--r#keywordscd|D}|Dcgc]$}dtjj|z&}}dj|}tj |}|j j|Scc}w)aSearch by keywords and return a generator of Author objects :param keywords: a list of keywords to be searched :type keyword: List[str] :Example:: .. testcode:: search_query = scholarly.search_keywords(['crowdsourcing', 'privacy']) scholarly.pprint(next(search_query)) :Output:: .. testoutput:: {'affiliation': 'Cornell University', 'citedby': 40976, 'email_domain': '', 'filled': False, 'interests': ['Crowdsourcing', 'privacy', 'social computing', 'game theory', 'user-generated content'], 'name': 'Arpita Ghosh', 'scholar_id': '_cMw1IUAAAAJ', 'source': 'SEARCH_AUTHOR_SNIPPETS', 'url_picture': 'https://scholar.google.com/citations?view_op=medium_photo&user=_cMw1IUAAAAJ'} c3RK|]}tjtd|!yw)rN)rrr).0rs r! z-_Scholarly.search_keywords..s]5sGD]s%'zlabel:+)rBrCrDjoin_KEYWORDSEARCHBASErArrV)r r reg_keywordsrformated_keywordsrFs r!search_keywordsz_Scholarly.search_keywordsst>^T\] S_`Xhnn&:&:7&CC``HH%67 ''(9:zz((--as)A:rFc8|jj|S)aSearch by custom URL and return a generator of Publication objects URL should be of the form '/scholar?q=...' A typical use case is to generate the URL by first typing in search parameters in the Advanced Search dialog box and then use the URL here to programmatically fetch the results. :param url: custom url to seach for the publication :type url: string )rrEr rFs r!search_pubs_custom_urlz!_Scholarly.search_pubs_custom_urlszz--c22r#c8|jj|S)zSearch by custom URL and return a generator of Author objects URL should be of the form '/citation?q=...' :param url: url for the custom author url :type url: string )rrVrs r!search_author_custom_urlz#_Scholarly.search_author_custom_urlszz((--r#cb|ddk7r|jjdy|dtjk(rAd|j vr|j |}|j j|dS|dtjk(r|j j|dSy)z Search google scholar for related articles to a specific publication. :param object: Publication object used to get the related articles :type object: Publication r]rzNot a publication objectNrxurl_related_articles) rrfrrkeysr^rrEPUBLICATION_SEARCH_SNIPPET)r rYs r!get_related_articlesz_Scholarly.get_related_articless " #} 4 KK   : ;  ( 0II I%V[[]:6*::11&9O2PQ Q H !2!M!M M::11&9O2PQ QNr#czd|vr|jjdytj|}|ddk(rt |dj |d<n|ddk(rt |j}t|dj |d<|j|dk(rd|d<nd |d<d |vr-|d D]%}d |d<|d=t|dj |d<'d |vr(|d D] }t |dj |d<|d="|d=ttj|jd y) zPretty print an Author or Publication container object :param object: Publication or Author container object :type object: Author or Publication r]z Not a scholarly container objectNrrxrrOTF coauthors publicationszutf-8)rrfrdeepcopyrrSr rr _sectionsprintpprintpformatencode)r rYto_printparsercoauthor publications r!rz_Scholarly.pprintsh 6 ) KK   B C ==( $ % 6!28H3E!F!K!KHX  & '8 3!$**-F!-hx.@!A!F!FHX 8H#55%)"%*"h& ( 5OH).HX& !12)5hx6H)I)N)NHX&O )#+N#;6K,=k(>S,T,Y,YK)#$456 % & fnnX&--g67r# fromauthorctjtjj |}|j j ||S)a( Search by organization name and return a list of possible disambiguations :Example:: .. testcode:: search_query = scholarly.search_org('ucla') print(search_query) :Output:: .. testoutput:: [{'Organization': 'University of California, Los Angeles', 'id': '14108176128635076915'}, {'Organization': 'Universidad Centroccidental Lisandro Alvarado', 'id': '9670678584336165373'} ] )rUrArBrCrDrsearch_organization)r rSrrFs r! search_orgz_Scholarly.search_orgs:$  !5!5d!;<zz--c:>>r#organization_idcbtj|}|jj|S)a> Search for authors in an organization and return a generator of Authors ``organization_id`` can be found from the organization name using ``search_org``. Alternatively, they can be found in the ``Author`` object. The returned authors are typically in the decreasing order of total citations. The authors must have a verified email address and set their affiliation appropriately to appear on this list. :param organization_id: unique integer id for each organization :type organization_id: integer ) _ORGSEARCHrArrV)r rrFs r!search_author_by_organizationz(_Scholarly.search_author_by_organizations)0zz((--r#filename overwrite include_linksc H|s-tjj|rt|d|jj t d}|r|jjd}|jddd}|jdd D]}|jd d jd }|jj}|dk7r|jd } n|jdd}d} d|vr!|jd|dd|d| d|d}|j|d|d| d|d} t|d5} | j|ddd|S#1swY|SxYw#t$r |j j#d|Y|SwxYw#|ccYSxYw)z@ Download the CSV file of the current mandates. zc already exists. Either provide a different filename or allow overwriting by setting overwrite=TrueF)premiumz-/citations?hl=en&view_op=mandates_leaderboardzFunder,zFunder,Policy,Cached,rtd gsc_mlt_tclass_spangs_ahrefcachedNi,"z",wzError writing mandates as %s)rpathexistsr_r _get_page _MANDATES_URL _get_soupreplacefind_allfindatextopenwriteIOErrorrerror) r rrrrsoupagencyrrSpolicyfs r!download_mandates_csvz _Scholarly.download_mandates_csvs rww~~h7z*778 8zz##M5#A ::''(WXD<< +BAFD--[-A RVF;==fExx}}8##XXf-F!;;t,DF$;<r1sq  $$!,+'IccD MJ& # + 8 ] N N r#