U g|@sdZdddgZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl Z ddl ZddlZddlmZmZmZGdddeZGdddeZedZed Zed Zd d ZGd ddZddZddZdS)z!Defines the Revital class which manipulates tcvitals files. This module deals with rewriting TCVitals files to remove errors, change Invests to storms, and other such operations. Revital RevitalErrorRevitalInitErrorN)great_arc_dist to_fraction to_timedeltac@seZdZdZdS)rz2!Base class of errors related to rewriting vitals.N__name__ __module__ __qualname____doc__r r ;/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/tcutil/revital.pyrsc@seZdZdZdS)rzU!This exception is raised when an argument to the Revital constructor is invalid.Nrr r r rrsii`TcCs||k||kS)z# Python3 does not have cmp funtion r abr r rcmp&src @seZdZdZd?ddZdd Zd d Zd d Zd@ddZdAddZ ddZ ddZ dBddZ ddZ dCddZddZdd Zd!d"Zd#d$Zd%d&Zd'd(Zd)d*Zd+d,Zd-d.Zd/d0ZdDd1d2ZdEd4d5Zd6d7Zd8d9Zd:d;Zdd9ZdS)FrzP!This class reads one or more tcvitals files and rewrites them as requested.NFjATc Cs>| dk r| j| j| j| jf\|_|_|_|_| j| j| j|_|_|_t|_| j D]8\} } t|j| <| D]\} } | |j| | <qvqZt | j |_ dd| j D|_ dSt||_|dkrtn||_||_|o|dk |_t||_||_|dk rtj|std|ft|_t |_ t|_ d|_dS)a!Creates a Revital object: @param logger A logging.Logger object for logging or None to disable. Default: None @param invest_number_name Rename storms to have the last non-INVEST name seen @param stormid Ignored. @param adeckdir Directory with A deck files. This is used to read the storm type information from CARQ entries and append them to produce vitals with the storm type from vitals that lack it. @param renumberlog Ignored. @param search_dx Search radius in km for deciding whether two storms are the same. @param search_dt Search timespan for finding two storms that are the same. @param debug If True, enables DEBUG level logging. @param copy Used by copy() to make a shallow copy of a Revital. If specified, the other arguments are ignored, and the copy's contents are copied. Do not use this argument. If you need a copy, use copy() instead.NcSsg|] }|qSr copy).0vr r r Psz$Revital.__init__..z*Specified directory %s is not a directory.F) search_dx search_dtloggerdebuginvest_number_nameadeckdir is_cleaneddictcarqdatitemsrsetcarqfailvitalsfloat six_hoursboolospathisdirrlist)selfrrstormidr renumberlogrrrrkeycdatZymdhcardr r r__init__-s>     zRevital.__init__cCs&t|tjjstd|j|dS)zc!Appends a vital entry to self.vitals. @param vital an tcutil.storminfo.StormInfo to appendzDThe argument to Revital.append must be an tcutil.storminfo.StormInfoN) isinstancetcutil storminfo StormInfo TypeErrorr%appendr-vitalr r rr9szRevital.appendcCs|j|d|_dS)z!Given the specified iterable object, appends its contents to my own. @param vitals an iterable object filled with tcutil.storminfo.StormInfoFN)r%extendr)r-r%r r rr<s zRevital.extendcCs t|dS)z_!Returns a deep copy of this Revital. Modifying the copy will not modify the original.r)rr-r r rrsz Revital.copycCst|dkr|jdn|jd|dd|jtjj|||jdd|_ |jdk r||j r||j dt|jf|S) a!Same as readfiles except the tcvitals have already been parsed and the vitals are being passed in, not the filelist. This was created to handle the multistorm fake storm. This is being used to validate any self generated vitals by the fake storm. @param vitalslist a list of strings with tcvitals data @param raise_all if True, raise exceptions for any parsing errors. @returns selfrz1No parsed tcvitals provided to revital.readvitalsz%Processing parsed tcvitals, line1: %sN raise_allrFline count: %d) lenrerrorinfor%r<r5r6parse_tcvitalsrr)r-Z vitalslistr@r r r readvitalss  zRevital.readvitalsc CsDt|tr|g}t}d}|D]}|jdk r>|jd|fz,t|d}||W5QRXd}Wqtk r}zV|j t j ks|j t j kr|j |dt||r΂n|j |dt|W5d}~XYqXq|s|j d|jtjj|||jdd|_|jdk r@|jr@|jd t|jfdS) a:!Reads the list of files and parses them as tcvitals files. @param filelist a list of string filenames @param raise_all if True, all exceptions are raised. If False, then exceptions are ignored, and the function will attempt to process all files, even if earlier ones failed.FNz read file: %srtTz: cannot open: zANo message files or tcvitals files provided to revital.readfiles.r?rA)r4strr,rrDopenr< readlinesEnvironmentErrorerrnoENOENTZEISDIRwarningrCr%r5r6rErrrB)r-filelistr@linesopenedtcvitalsfer r r readfiless8     zRevital.readfilesc Cst|dd}t|dd}|dks@|dks@|dks@|dks@|dkrDdStjd}d}||||}|d|}|t|} |t|t|j|} |j| |j| fS) a!Returns a tuple containing the latitude and longitude of the storm at a different position according to the storm motion vector. @param vital the tcutil.storminfo.StormInfo for the storm fix being extrapolated @param dt the time difference in hours stormspeedNstormdirr)NNgf@g@TXAgV@)getattrmathpisincoslatlon) r-r;dtrVrWZpi180ZRearthkZ moveangleZdlatZdlonr r r move_latlons    zRevital.move_latlonc Cs||jkrdStj|jd|f}tj|r@tj|dkrP|j|dSt|d}dd| D}W5QRXt j j ||j d}t}|D]}|||j<q||j|<dS)ar!Tries to find the CARQ data for the specified storm. Reads it into the self.carqdat array, or adds the stormid to self.carqfail if the data cannot be read in. @param longstormid the long stormid of the storm to read @post the self.carqfail will contain longstormid OR self.carqdat[longstormid] will contain data for that stormNza%s.datrrGcSsg|]}|qSr r )rliner r rrsz$Revital.readcarq..)r)r$r)r*joinrexistsgetsizeaddrIrJr5r6Z parse_carqrr YMDHr!)r- longstormidfilenamerSdatacarqr1r2r r rreadcarqs    zRevital.readcarqcCs"tjj|j|||d|_d|_dS)aS!Calls the tcutil.storminfo.clean_up_vitals on this object's vitals. The optional arguments are passed to tcutil.storminfo.clean_up_vitals. @param name_number_checker a function like tcutil.storminfo.name_number_okay() for validating the storm name and number @param vitals_cmp a cmp-like function for ordering tcutil.storminfo.StormInfo objects @param basin_center_checker a function like tcutil.storminfo.basin_center_okay() for checking the storm basin and forecast center (RSMC) @post is_cleaned=True)name_number_checkerbasin_center_checker vitals_cmpTN)r5r6clean_up_vitalsr%r)r-rmrnror r rrpszRevital.clean_up_vitalscCsd}|jo|jdk }|j}|dks&t|dks2t|dkrz||d|\} } |jr| dk r| dk r|jd| | fn|j|j} } |r| dks| dkr|r|ddS|j}|jo|dk }d}t|D]} || } |r$t| dd} | dkr$| j |kr$|r|d | j | j fq|r<|d | j f| j |j }|t krX| }|t krdq|tkr|r~|d || =q||jkr|r|d q|r|d | jr`| j|jkr| j|jkr|r|d|j|j f|jr|dt|j|jf|| j| j|t| jddd}|rT|d|j f||| j<q|dkr|| d|\}}|r|dk r|dk r|d||fn| j| j}}|dks|dkr|r|dqt| | ||}||jkr|tks|r|d|dfq|r4|d| jf|jrV|dt|j|jf|| j| j|t| jddd}|r|d|j f||| j<q|r|dt|f|S)af!Internal function that handles renumbering of storms. @protected This is an internal implementation function that should never be called directly. It handles part of the work of renumbering storms in the list of vitals. You should call "renumber" instead. @param vital the vital being renumbered @param lastvit the last vital seen @param vit_motion time since vital @param other_motion time since other storms vital @param threshold cold start threshold, used to decide when to stop connecting an invest to a non-investFNrg @z -- lat=%.3f lon=%.3fz -- no lat,lon for search old_stnumZz@Old %s vit was a low intensity invest, so not considering it: %sz vs. %sz -- age out othervitz -- dt is too largez -- within dtz# -- continue renumbering (%s) %sz INVEST%02d%1sTz NOW %sz -- vs lat=%.3f lon=%.3fz3 -- cannot get other vitals location; moving on.z& -- not kinda near (distance %f km)g@@z* -- within dx: renumber to %s and storez - renumbered = %s)rrAssertionErrorrar]r^r,keysrXwmax old_stormid3rbwhen zero_timetwo_daysrZ has_old_stnumrqstnumbasin1stormid3r rename_stormintZ change_basinZ pubbasin2Zrenumber_stormrrr'repr)r-r;lastvitZ vit_motionZ other_motion threshold renumberedrrr]r^r.ZothervitZold_idr_ZotherlatZotherlondistr r r renumber_one s                  zRevital.renumber_onerc Cs|js|d|_|sd}t|}t}|jo8|jdk }|j}t|jD]}|rd|d|jf|j } |j dkr|j dkrqJn:|j dkr|||j <qJn"| ||dd|r|rJ|dqJ|r|d| ||dd |rqJ|r|d | ||d d|rqJ|r|d | ||d d|rJqJqJ|r6| |r|sd|rd|dk r\| d||dk rx| dtj|j|_dS)a!Renumbers storms with numbers 90-99, if possible, to have the same number as later 1-49 numbered storms. Loops over all vitals from last to first, renumbering 90-99 storms to have the same storm number as later 1-49 storms. @param threshold If a threshold is given, then a cycle will only be considered for renumbering if it is either above that threshold, or is not an Invest. @param unrenumber If unrenumber is True, the original storm numbers are restored after renumbering. @param discard_duplicate If True, discard invests that are duplicates of non-invests. This feature is disabled if unrenumber is enabled or cleaning is disabled. @param clean If clean is True (the default), then self.clean_up_vitals is called, which will (among other things) delete vitals lines that have the same time and storm ID. The cleaning is done after unrenumbering, so if both options are turned on, the result will contain only one entry per storm ID per time, but with all storm IDs that are available for a given storm at any one time.FrNzVITAL %s2rrz -- done renumbering this onez7 - SEARCH AGAIN: subtract storm motion from later cycleiz2 - SEARCH AGAIN: add storm motion to earlier cyclez; - SEARCH AGAIN: add half motion to later and earlier cyclez2Delete Invests that are duplicates of non-Invests.z.Clean up the vitals again after renumbering...)rrprr rrreversedr%rbr}r{r swap_numbersrDdelete_invest_duplicatesr5r6) r- unrenumbercleanrZdiscard_duplicatesrrrr;r0r r rrenumbersd        zRevital.renumbercCstt}|jD]8}|jdkrt|jdt|jdf}|||j|<q|jD]F}|jdkrPt|jdt|jdf}|||jkrP|||j|<qPt }| D]}| D]}| |qq||_dS)z[!Deletes Invest entries that have the same location and time as non-invest entries.rrrN) collections defaultdictr r%r{rr]r^rgr,valuesr9)r-orr`lZyvr r rrs       z Revital.delete_invest_duplicatescCs|jD] }|qd|_dS)z@!Calls swap_numbers on all vitals to swap old and new storm IDs.FN)r%rrr:r r rrs  zRevital.swap_numberscCsDt}|jD]&}||d|jkr ||q ||_d|_dS)z!Duplicates all vitals that have been renumbered, creating one StormInfo with the old number and one with the new number.rwFN)r,r%r9__dict__oldr)r-newvitr;r r rmirror_renumbered_vitalss   z Revital.mirror_renumbered_vitalscCs.t}|jD]}||r ||q ||_dS)aP!Discards all vitals except those for which the keep_condition function returns True. @param keep_condition A function that receives a StormInfo object as its only argument, returning True if the vital should be kept and False if not. @note The list will be unmodified if an exception is thrown.N)r,r%r9)r-keep_conditionrvitr r rdiscard_excepts   zRevital.discard_exceptcCs,|jD] }t|dr|j|j|_|_qdS)zT!This subroutine undoes the effect of renaming by swapping old and new names old_stormnameN)r%hasattr stormnamerr:r r r swap_namess   zRevital.swap_namescCs|j}|jo|dk }t}t|jD]p}|j}||kr||}|j|kr|r`|d||jf|||r|d|jfq$|jdd||<q$dS)zd!This subroutine renames storms so that they have the last name seen for their storm number.NzRename to %s: %szNow: %sr ) rrr rr%r}rrbr~)r-rrlastnamer;r0namer r rrenames  zRevital.renamecCs|j}|jo|dk }|jD]}|j}|j}||jkrB||||jkrp|dk rd|d|f| dq|j|}||kr|dk r|d||f| dq| ||qdS)zJ!Add the storm type parameter from the CARQ entries in the A deck.Nz2storm %s: no CARQ data found. Using stormtype XX.ZXXzDstorm %s cycle %s: no CARQ data for this cycle. Using stormtype XX.) rrr%rhlowerrgr!rlrNZ set_stormtype)r-rrr;Zlsidrxrkr r r add_stormtype!s.        zRevital.add_stormtypecCst|jt|d|_dS)z!Resorts the vitals using the specified cmp-like function. @param cmpfun a cmp-like function for comparing tcutil.storminfo.StormInfo objects)r0N)sortedr% functools cmp_to_key)r-Zcmpfunr r rsort_by_function;szRevital.sort_by_functioncCst|jtjjd|_dS)zi!Resorts the vitals by storm instead of date. See tcutil.storminfo.vit_cmp_by_storm for details.)rN)rr%r5r6Zvit_cmp_by_stormr=r r r sort_by_stormAszRevital.sort_by_stormccs|jD] }|VqdS)zK!Iterates over all vitals, yielding StormInfo objects for each one.N)r%)r-xr r r__iter__Es zRevital.__iter__c#sdkrdd}nttdrHfdd}|rfdd}nftdrrfd d}|rfd d}n.selectedz\A\d\d[a-zA-Z]\Zcs |jkSN)r}rr.r rrYrcsd|jko|jkS)Nrw)rrwrrr r old_selected[s z"Revital.each..old_selectedz\A[a-zA-Z]{2}\d\d\Zcs |jkSr)Zstormid4rrr rr_rcsd|jko|jkS)N old_stormid4)rrrrr rras z\A[a-zA-Z]{2}\d{6}\Zcs |jkSr)rhrrr rrercsd|jko|jkS)Nold_longstormid)rrrrr rrgs zNInvalid storm id %s. It must be one of these three formats: 04L AL04 AL042013)rHupperresearchrvalr%)r-r.rrrrr rreachIs0          z Revital.eachrbc Cs|j||dD]}|dk rh|j}t|d|}|j} t|d| } |d|j||| dd| ddf|dkrt||d q|d kr|} t|d|j}t|d|j} td || | f|d q|d krtd |j |jf|d qt|j |d qdS)a!Print the vitals to the given stream in a specified format. @param stream The stream (eg.: opened file) to receive the vitals. @param format Either "tcvitals" to reformat as tcvitals (cleaning up any errors); or "line" to simply print the original data for each line; or "HHS" to use the HHS output format. (Do not use the "HHS" option unless you are HHS.) @param renumberlog If given, sends information about renaming and renumbering of the vitals to a second stream. @param stormid The "stormid" argument is used to restrict printing to only a certain stormid. @param old If True, then vitals with an old_stormid that matches are also printed.)r.rNrwrz%10s %3s %3s %-9s %-9s rrrR)fileZ renumberingz %3s %9s => %sZHHSz %s %s "TCVT") rr}rXrwritergprint as_tcvitalsrhrrb) r-streamr/formatr.rrZ xstormid3Zoldidroldnamesr r r print_vitalsvs&  zRevital.print_vitalsc Cst|dd}t|dd}|jdkr&dnd}|jdkr8dnd}|jdkrJdnd}|jdkr\dnd}t||pt||pt||pt|j|j p|jdkr|jdkrt|j|jp|jdko|jdkot|j|j } | S)am!A drop-in replacement for "cmp" that can be used for sorting or comparison. Returns -1 if ab or 0 if a=b. Decision is made in this order: Edited by GJA on 08-29-2017 -- Added logic to prefer Atlantic storm over East Pacific storm if the class (i.e., invest vs. non-invest) is the same. 1) User priority (a.userprio): lower (priority 1) is "more important" than higher numbers (priority 9999 is fill value). 2) Invest vs. non-invest: invest is less important 3) Atlantic vs. East Pacific: Atlantic is more important 4) wind: stronger wind is more important than weaker wind 5) North Atlantic (L) storms: farther west is more important 6) North East Pacific (E) storms: farther East is more important If all of the above values are equal, 0 is returned.userprio'INVESTr>rEL)rXrr|rrvr^) r-rr a_userprio b_userprioa_investb_investZa_basinZb_basincr r rhrd_multistorm_sorters$    "zRevital.hrd_multistorm_sortercCsdSz!Does nothing.Nr r=r r rmultistorm_priorityszRevital.multistorm_prioritycCs t||Sr)hrd_multistorm_cmp)r-rrr r rrszRevital.hrd_multistorm_cmpcCs ttSr)rrr)r-rr r rhrd_multistorm_keyszRevital.hrd_multistorm_keycCsdSrr r=r r rrs) NFNNNrNTN)T)T)NNN)FTrT)NF)NrbNF)r r r r r3r9r<rrFrUrarlrprrrrrrrrrrrrrrrrrrr r r rr*sV Y  ! t H  - #&cCs||kr dS||krdSdS)Nr>rr rr r roldcmps rcCst|dd}t|dd}|jdkr&dnd}|jdkr8dnd}t||pt||pt|j|j p|jdkr|jdkrt|j|jp|jdko|jdkot|j|j }|S)a!A compares two storminfo objects for use in sorting or comparison. Returns -1 if ab or 0 if a=b. Decision is made in this order: 1. User priority (a.userprio): lower (priority 1) is "more important" than higher numbers (priority 9999 is fill value). 2. Invest vs. non-invest: invest is less important 3. wind: stronger wind is more important than weaker wind 4. North Atlantic (L) storms: farther west is more important 5. North East Pacific (E) storms: farther East is more important If all of the above values are equal, 0 is returned. @returns -1, 0 or 1 @param a,b the vitals to compare rrrr>rrr)rXrrrvr|r^)rrrrrrrr r rrs    "r)r __all__loggingdatetimegetoptsysos.pathr)rrYrLrrtcutil.storminfor5tcutil.numericsrrr Exceptionrrryrzr'rrrrr r r rs& H(