3 `Mcw@sdZdddgZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl Z ddlZ ddlZddlZddlZddlZddlmZmZmZddlmZddlmZGd ddeZGd ddeZedZedZedZddZGdddZdS)z!Defines the Revital class which manipulates tcvitals files. This module deals with rewriting TCVitals files to remove errors, change Invests to storms, and other such operations. Revital RevitalErrorRevitalInitErrorN)great_arc_dist to_fraction to_timedelta) isnonempty) HWRFErrorc@seZdZdZdS)rz2!Base class of errors related to rewriting vitals.N)__name__ __module__ __qualname____doc__rr:/lfs/h1/ops/prod/packages/hwrf.v13.2.5/ush/hwrf/revital.pyrsc@seZdZdZdS)rzU!This exception is raised when an argument to the Revital constructor is invalid.N)r r r r rrrrrsicCs||k||kS)z# Python3 does not have cmp funtion r)abrrrcmp'src @seZdZdZd:ddZdd Zd d Zd d Zd;ddZdddZddZdd Zd!d"Zd#d$Zd%d&Zd'd(Zd)d*Zd+d,Zd-d.Zd/d0Zd?d1d2Zd@d4d5Zd6d7Zd8d9ZdS)ArzP!This class reads one or more tcvitals files and rewrites them as requested.NFjATc CsF| dk r| j| j| j| jf\|_|_|_|_| j| j| j|_|_|_t|_xJ| jj D]<\} } t|j| <x&| j D]\} } | j |j| | <qzWq\Wt | j |_ dd| j D|_ dSt||_|dkrtn||_||_|o|dk |_t||_||_|dk r$tjj|s$td|ft|_t |_ t|_ d|_dS)a!Creates a Revital object: @param logger A logging.Logger object for logging or None to disable. Default: None @param invest_number_name Rename storms to have the last non-INVEST name seen @param stormid Ignored. @param adeckdir Directory with A deck files. This is used to read the storm type information from CARQ entries and append them to produce vitals with the storm type from vitals that lack it. @param renumberlog Ignored. @param search_dx Search radius in km for deciding whether two storms are the same. @param search_dt Search timespan for finding two storms that are the same. @param debug If True, enables DEBUG level logging. @param copy Used by copy() to make a shallow copy of a Revital. If specified, the other arguments are ignored, and the copy's contents are copied. Do not use this argument. If you need a copy, use copy() instead.NcSsg|] }|jqSr)copy).0vrrr Qsz$Revital.__init__..z*Specified directory %s is not a directory.F) search_dx search_dtloggerdebuginvest_number_nameadeckdir is_cleaneddictcarqdatitemsrsetcarqfailvitalsfloat six_hoursboolospathisdirrlist)selfrrstormidr renumberlogrrrrkeycdatymdhcardrrr__init__.s6$      zRevital.__init__cCs&t|tjjstd|jj|dS)za!Appends a vital entry to self.vitals. @param vital an hwrf.storminfo.StormInfo to appendzBThe argument to Revital.append must be an hwrf.storminfo.StormInfoN) isinstancehwrf storminfo StormInfo TypeErrorr'append)r/vitalrrrr<szRevital.appendcCs|jj|d|_dS)z!Given the specified iterable object, appends its contents to my own. @param vitals an iterable object filled with hwrf.storminfo.StormInfoFN)r'extendr!)r/r'rrrr>s zRevital.extendcCs t|dS)z_!Returns a deep copy of this Revital. Modifying the copy will not modify the original.)r)r)r/rrrrsz Revital.copycCst|dkr|jjdn|jjd|dd|jjtjj|||jdd|_ |jdk r||j r||jj dt|jf|S) a!Same as readfiles except the tcvitals have already been parsed and the vitals are being passed in, not the filelist. This was created to handle the multistorm fake storm. This is being used to validate any self generated vitals by the fake storm. @param vitalslist a list of strings with tcvitals data @param raise_all if True, raise exceptions for any parsing errors. @returns selfrz1No parsed tcvitals provided to revital.readvitalsz%Processing parsed tcvitals, line1: %sN) raise_allrFzline count: %d) lenrcriticalinfor'r>r8r9parse_tcvitalsr!r)r/Z vitalslistr@rrr readvitalss  zRevital.readvitalscCsHt|tr|g}t}d}x|D]}|jdk r@|jjd|fy,t|d}|j|jWdQRXd}Wq tk r}zX|j t j ks|j t j kr|jj |dt||rЂn|jj |dt|WYdd}~Xq Xq W|s|jj d|jjtjj|||jdd|_|jdk rD|jrD|jjd t|jfdS) a:!Reads the list of files and parses them as tcvitals files. @param filelist a list of string filenames @param raise_all if True, all exceptions are raised. If False, then exceptions are ignored, and the function will attempt to process all files, even if earlier ones failed.FNz read file: %srtTz: cannot open: zANo message files or tcvitals files provided to revital.readfiles.)r@rzline count: %d)r7strr.rrCopenr> readlinesEnvironmentErrorerrnoENOENTZEISDIRwarningrBr'r8r9rDr!rrA)r/Zfilelistr@linesZopenedtcvitalsferrr readfiless2      zRevital.readfilesc Cst|dd}t|dd}|dks@|dks@|dks@|dks@|dkrDdStjd}d}||||}|d|}|tj|} |tj|tj|j|} |j| |j| fS) a!Returns a tuple containing the latitude and longitude of the storm at a different position according to the storm motion vector. @param vital the hwrf.storminfo.StormInfo for the storm fix being extrapolated @param dt the time difference in hours stormspeedNstormdirrgf@g@TXAgV@)NN)getattrmathpisincoslatlon) r/r=dtrSrTpi180RearthkZ moveangledlatdlonrrr move_latlons    zRevital.move_latlonc Cs||jkrdStjj|jd|f}t|s<|jj|dSt|d}dd|jD}WdQRXt j j ||j d}t }x|D]}|||j<qW||j|<dS)as!Tries to find the CARQ data for the specified storm. Reads it into the self.carqdat array, or adds the stormid to self.carqfail if the data cannot be read in. @param longstormid the long stormid of the storm to read @post the self.carqfail will contain longstormid OR self.carqdat[longstormid] will contain data for that stormNza%s.datrFcSsg|]}|qSrr)rlinerrrrsz$Revital.readcarq..)r)r&r+r,joinr raddrHrIr8r9 parse_carqrr"YMDHr#)r/ longstormidfilenamerPdatacarqr3r5rrrreadcarqs    zRevital.readcarqcCs"tjj|j|||d|_d|_dS)aI!Calls the hwrf.storminfo.clean_up_vitals on this object's vitals. The optional arguments are passed to hwrf.storminfo.clean_up_vitals. @param name_number_checker a function like hwrf.storminfo.name_number_okay() for validating the storm name and number @param vitals_cmp a cmp-like function for ordering hwrf.storminfo.StormInfo objects @param basin_center_checker a function like hwrf.storminfo.basin_center_okay() for checking the storm basin and forecast center (RSMC) @post is_cleaned=True)name_number_checkerbasin_center_checker vitals_cmpTN)r8r9clean_up_vitalsr'r!)r/rmrnrorrrrps  zRevital.clean_up_vitalscCsd}|jo|jdk }|j}|dks&t|dks2t|dkrz|j|d|\} } |jr| dk r| dk r|jjd| | fn|j|j} } |r| dks| dkr|r|jddS|j}|jo|dk }d}xt|jD]} || } |r(t| dd} | dkr(| j |kr(|r|jd | j | j fq|r@|jd | j f| j |j }|t kr\| }|t krhq|tkr|r|jd || =q||jkr|r|jd q|r|jd | jrd| j|jkr| j|jkr|r|jd|j|j f|jr|jdt|j|jf|j| j| j|jt| jddd}|rX|jd|j f||| j<q|dkr|j| d|\}}|r|dk r|dk r|jd||fn| j| j}}|dks|dkr|r|jdqt| | ||}||jko|tks |r|jd|dfq|r8|jd| jf|jrZ|jdt|j|jf|j| j| j|jt| jddd}|r|jd|j f||| j<qW|r|jdt|f|S)af!Internal function that handles renumbering of storms. @protected This is an internal implementation function that should never be called directly. It handles part of the work of renumbering storms in the list of vitals. You should call "renumber" instead. @param vital the vital being renumbered @param lastvit the last vital seen @param vit_motion time since vital @param other_motion time since other storms vital @param threshold cold start threshold, used to decide when to stop connecting an invest to a non-investFNrg @z -- lat=%.3f lon=%.3fz -- no lat,lon for search old_stnumZz@Old %s vit was a low intensity invest, so not considering it: %sz vs. %sz -- age out othervitz -- dt is too largez -- within dtz# -- continue renumbering (%s) %sz INVEST%02d%1srTz NOW %sz -- vs lat=%.3f lon=%.3fz3 -- cannot get other vitals location; moving on.z& -- not kinda near (distance %f km)g@@z* -- within dx: renumber to %s and storez - renumbered = %s)rrAssertionErrorrbrZr[r.keysrUwmax old_stormid3rcwhen zero_timetwo_daysr has_old_stnumrqstnumbasin1stormid3r rename_stormint change_basin pubbasin2renumber_stormrrr)repr)r/r=lastvitZ vit_motionZ other_motion thresholdZ renumberedrrrZr[r0ZothervitZold_idr\ZotherlatZotherlonZdistrrr renumber_ones                 zRevital.renumber_onerc Cs|js|jd|_|sd}t|}t}|jo8|jdk }|j}xt|jD]}|rf|jd|jf|j } |j dkr|j dkrqLn:|j dkr|||j <qLn"|j ||dd|r|rL|jdqL|r|jd|j ||dd|rqL|r|jd |j ||d d|rqL|r|jd |j ||d d|rLqLqLW|r:|j |r| rj|rj|dk rb|j d |j|dk r~|j dtjj|j|_dS)a!Renumbers storms with numbers 90-99, if possible, to have the same number as later 1-49 numbered storms. Loops over all vitals from last to first, renumbering 90-99 storms to have the same storm number as later 1-49 storms. @param threshold If a threshold is given, then a cycle will only be considered for renumbering if it is either above that threshold, or is not an Invest. @param unrenumber If unrenumber is True, the original storm numbers are restored after renumbering. @param discard_duplicate If True, discard invests that are duplicates of non-invests. This feature is disabled if unrenumber is enabled or cleaning is disabled. @param clean If clean is True (the default), then self.clean_up_vitals is called, which will (among other things) delete vitals lines that have the same time and storm ID. The cleaning is done after unrenumbering, so if both options are turned on, the result will contain only one entry per storm ID per time, but with all storm IDs that are available for a given storm at any one time.FrNzVITAL %s2rrz -- done renumbering this onez7 - SEARCH AGAIN: subtract storm motion from later cyclerz2 - SEARCH AGAIN: add storm motion to earlier cyclez; - SEARCH AGAIN: add half motion to later and earlier cyclez2Delete Invests that are duplicates of non-Invests.z.Clean up the vitals again after renumbering...i)r!rprr"rrreversedr'rcr}r{r swap_numbersrCdelete_invest_duplicatesr8r9) r/Z unrenumbercleanrZdiscard_duplicatesrrrr=r2rrrrenumbers^       zRevital.renumbercCstjt}xB|jD]8}|jdkrt|jdt|jdf}|||j|<qWxP|jD]F}|jdkrVt|jdt|jdf}|||jkrV|||j|<qVWt }x,|j D] }x|j D]}|j |qWqW||_dS)z[!Deletes Invest entries that have the same location and time as non-invest entries.rrrN) collections defaultdictr"r'r{rrZr[rgr.valuesr<)r/orr_lZyvrrrrs     z Revital.delete_invest_duplicatescCs"x|jD] }|jqWd|_dS)z@!Calls swap_numbers on all vitals to swap old and new storm IDs.FN)r'rr!)r/r=rrrrs  zRevital.swap_numberscCsHt}x0|jD]&}|j|d|jkr|j|jqW||_d|_dS)z!Duplicates all vitals that have been renumbered, creating one StormInfo with the old number and one with the new number.rvFN)r.r'r<__dict__oldr!)r/newvitr=rrrmirror_renumbered_vitalss   z Revital.mirror_renumbered_vitalscCs2t}x |jD]}||r|j|qW||_dS)aT!Discards all vitals except those for which the keep_condition function returns True. @param keep_condition A function that receives a StormInfo object as its only argument, returning True if the vital should be kept and False if not. @note The list will be unmodified if an exception is thrown.N)r.r'r<)r/Zkeep_conditionrvitrrrdiscard_excepts  zRevital.discard_exceptcCs0x*|jD] }t|dr|j|j|_|_qWdS)zT!This subroutine undoes the effect of renaming by swapping old and new names old_stormnameN)r'hasattr stormnamer)r/r=rrr swap_namess  zRevital.swap_namescCs|j}|jo|dk }t}x~t|jD]p}|j}||kr||}|j|kr|rb|jd||jf|j||r|jd|jfq&|jdd||<q&WdS)zd!This subroutine renames storms so that they have the last name seen for their storm number.NzRename to %s: %szNow: %sr ) rrr"rr'r}rrcr~)r/rrZlastnamer=r2namerrrrenames  zRevital.renamecCs|j}|jo|dk }x|jD]}|jj}|j}||jkrD|j|||jkrr|dk rf|jd|f|j dq|j|}||kr|dk r|jd||f|j dq|j ||qWdS)zJ!Add the storm type parameter from the CARQ entries in the A deck.Nz2storm %s: no CARQ data found. Using stormtype XX.XXzDstorm %s cycle %s: no CARQ data for this cycle. Using stormtype XX.) rrr'rhlowerrgr#rlrM set_stormtype)r/rrr=Zlsidrwrkrrr add_stormtype"s*          zRevital.add_stormtypecCst|jtj|d|_dS)z!Resorts the vitals using the specified cmp-like function. @param cmpfun a cmp-like function for comparing hwrf.storminfo.StormInfo objects)r2N)sortedr' functools cmp_to_key)r/Zcmpfunrrrsort_by_function<szRevital.sort_by_functioncCst|jtjjd|_dS)zg!Resorts the vitals by storm instead of date. See hwrf.storminfo.vit_cmp_by_storm for details.)rN)rr'r8r9vit_cmp_by_storm)r/rrr sort_by_stormBszRevital.sort_by_stormccsx|jD] }|VqWdS)zK!Iterates over all vitals, yielding StormInfo objects for each one.N)r')r/xrrr__iter__Fs zRevital.__iter__c#sdkrdd}ntjtjdrHfdd}|rfdd}nftjdrrfd d}|rfd d}n.selectedz\A\d\d[a-zA-Z]\Zcs |jkS)N)r})r=)r0rrrZscsd|jko|jkS)Nrv)rrv)r=)r0rr old_selected\s z"Revital.each..old_selectedz\A[a-zA-Z]{2}\d\d\Zcs |jkS)N)stormid4)r=)r0rrr`scsd|jko|jkS)N old_stormid4)rr)r=)r0rrrbs z\A[a-zA-Z]{2}\d{6}\Zcs |jkS)N)rh)r=)r0rrrfscsd|jko|jkS)Nold_longstormid)rr)r=)r0rrrhs zNInvalid storm id %s. It must be one of these three formats: 04L AL04 AL042013)rGupperresearchrvalr')r/r0rrrrr)r0reachJs.          z Revital.eachrcc Csx|j||dD]}|dk rj|j}t|d|}|j} t|d| } |jd|j||| dd| ddf|dkrt|j|d q|d kr|j} t|d|j}t|d|j} td || | f|d q|d krtd |jj |jf|d qt|j |d qWdS)a!Print the vitals to the given stream in a specified format. @param stream The stream (eg.: opened file) to receive the vitals. @param format Either "tcvitals" to reformat as tcvitals (cleaning up any errors); or "line" to simply print the original data for each line; or "HHS" to use the HHS output format. (Do not use the "HHS" option unless you are HHS.) @param renumberlog If given, sends information about renaming and renumbering of the vitals to a second stream. @param stormid The "stormid" argument is used to restrict printing to only a certain stormid. @param old If True, then vitals with an old_stormid that matches are also printed.)r0rNrvrz%10s %3s %3s %-9s %-9s rrrO)fileZ renumberingz %3s %9s => %sZHHSz %s %s "TCVT") rr}rUrwritergprint as_tcvitalsrhrrc) r/streamr1formatr0rrZ xstormid3ZoldidrZoldnamesrrr print_vitalsws$  $zRevital.print_vitalsc Cst|dd}t|dd}|jdkr&dnd}|jdkr8dnd}|jdkrJdnd}|jdkr\dnd}t||pt||pt||pt|j|j p|jdkr|jdkrt|j|jp|jdko|jdkot|j|j } | S)am!A drop-in replacement for "cmp" that can be used for sorting or comparison. Returns -1 if ab or 0 if a=b. Decision is made in this order: Edited by GJA on 08-29-2017 -- Added logic to prefer Atlantic storm over East Pacific storm if the class (i.e., invest vs. non-invest) is the same. 1) User priority (a.userprio): lower (priority 1) is "more important" than higher numbers (priority 9999 is fill value). 2) Invest vs. non-invest: invest is less important 3) Atlantic vs. East Pacific: Atlantic is more important 4) wind: stronger wind is more important than weaker wind 5) North Atlantic (L) storms: farther west is more important 6) North East Pacific (E) storms: farther East is more important If all of the above values are equal, 0 is returned.Zuserprioi'ZINVESTr?rEL)rUrr|rrur[) r/rrZ a_userprioZ b_userprioZa_investZb_investZa_basinZb_basincrrrhrd_multistorm_sorters     "$zRevital.hrd_multistorm_sortercCsdS)z!Does nothing.Nr)r/rrrmultistorm_priorityszRevital.multistorm_priority) NFNNNrNTN)T)T)NNN)FTrT)NF)NrcNF)r r r r r6r<r>rrErRrbrlrprrrrrrrrrrrrrrrrrrrrr+s> W  ! t G  - "&iQii`T) r __all__loggingdatetimegetoptsysZos.pathr+rrVrKrprodutil.sigsafetyprodutilprodutil.fileophwrf.storminfor8 hwrf.numericshwrf.exceptionsrrrrrr rrrxryr)rrrrrrs H