U  gu@sdZdddgZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl Z ddl ZddlZddlmZmZmZGdddeZGdddeZedZed Zed ZGd ddZd d ZddZdS)z!Defines the Revital class which manipulates tcvitals files. This module deals with rewriting TCVitals files to remove errors, change Invests to storms, and other such operations. Revital RevitalErrorRevitalInitErrorN)great_arc_dist to_fraction to_timedeltac@seZdZdZdS)rz2!Base class of errors related to rewriting vitals.N__name__ __module__ __qualname____doc__r r 9/lfs/h1/ops/prod/packages/hmon.v3.2.7/ush/hwrf/revital.pyrsc@seZdZdZdS)rzU!This exception is raised when an argument to the Revital constructor is invalid.Nrr r r rrsii`Tc @seZdZdZdddZ ddZ ddZ d?ddZ ddZ d@ddZddZdd Zd!d"Zd#d$Zd%d&Zd'd(Zd)d*Zd+d,Zd-d.Zd/d0ZdAd1d2ZdBd4d5Zd6d7Zd8d9Zd:d;ZdS)CrzP!This class reads one or more tcvitals files and rewrites them as requested.NFjATc Cs>| dk r| j| j| j| jf\|_|_|_|_| j| j| j|_|_|_t|_| j D]8\} } t|j| <| D]\} } | |j| | <qvqZt | j |_ dd| j D|_ dSt||_|dkrtn||_||_|o|dk |_t||_||_|dk rtj|std|ft|_t |_ t|_ d|_dS)a!Creates a Revital object: @param logger A logging.Logger object for logging or None to disable. Default: None @param invest_number_name Rename storms to have the last non-INVEST name seen @param stormid Ignored. @param adeckdir Directory with A deck files. This is used to read the storm type information from CARQ entries and append them to produce vitals with the storm type from vitals that lack it. @param renumberlog Ignored. @param search_dx Search radius in km for deciding whether two storms are the same. @param search_dt Search timespan for finding two storms that are the same. @param debug If True, enables DEBUG level logging. @param copy Used by copy() to make a shallow copy of a Revital. If specified, the other arguments are ignored, and the copy's contents are copied. Do not use this argument. If you need a copy, use copy() instead.NcSsg|] }|qSr copy).0vr r r Ksz$Revital.__init__..z*Specified directory %s is not a directory.F) search_dx search_dtloggerdebuginvest_number_nameadeckdir is_cleaneddictcarqdatitemsrsetcarqfailvitalsfloat six_hoursboolospathisdirrlist)selfrrstormidr renumberlogrrrrkeycdatZymdhcardr r r__init__(s>     zRevital.__init__cCs&t|tjjstd|j|dS)za!Appends a vital entry to self.vitals. @param vital an hwrf.storminfo.StormInfo to appendzBThe argument to Revital.append must be an hwrf.storminfo.StormInfoN) isinstancehwrf storminfo StormInfo TypeErrorr!appendr)vitalr r rr5szRevital.appendcCs|j|d|_dS)z!Given the specified iterable object, appends its contents to my own. @param vitals an iterable object filled with hwrf.storminfo.StormInfoFN)r!extendr)r)r!r r rr8s zRevital.extendcCs t|dS)z_!Returns a deep copy of this Revital. Modifying the copy will not modify the original.r)rr)r r rrsz Revital.copycCst|dkr|jdn|jd|dd|jtjj|||jdd|_ |jdk r||j r||j dt|jf|S) a!Same as readfiles except the tcvitals have already been parsed and the vitals are being passed in, not the filelist. This was created to handle the multistorm fake storm. This is being used to validate any self generated vitals by the fake storm. @param vitalslist a list of strings with tcvitals data @param raise_all if True, raise exceptions for any parsing errors. @returns selfrz1No parsed tcvitals provided to revital.readvitalsz%Processing parsed tcvitals, line1: %sN raise_allrFline count: %d) lenrcriticalinfor!r8r1r2parse_tcvitalsrr)r)Z vitalslistr<r r r readvitalss  zRevital.readvitalsc CsDt|tr|g}t}d}|D]}|jdk r>|jd|fz,t|d}||W5QRXd}Wqtk r}zV|j t j ks|j t j kr|j |dt||r΂n|j |dt|W5d}~XYqXq|s|j d|jtjj|||jdd|_|jdk r@|jr@|jd t|jfdS) a:!Reads the list of files and parses them as tcvitals files. @param filelist a list of string filenames @param raise_all if True, all exceptions are raised. If False, then exceptions are ignored, and the function will attempt to process all files, even if earlier ones failed.FNz read file: %srtTz: cannot open: zANo message files or tcvitals files provided to revital.readfiles.r;r=)r0strr(rr@openr8 readlinesEnvironmentErrorerrnoENOENTZEISDIRwarningr?r!r1r2rArrr>)r)Zfilelistr<linesZopenedtcvitalsfer r r readfiless8     zRevital.readfilesc Cst|dd}t|dd}|dks@|dks@|dks@|dks@|dkrDdStjd}d}||||}|d|}|t|} |t|t|j|} |j| |j| fS) a!Returns a tuple containing the latitude and longitude of the storm at a different position according to the storm motion vector. @param vital the hwrf.storminfo.StormInfo for the storm fix being extrapolated @param dt the time difference in hours stormspeedNstormdirr)NNgf@g@TXAgV@)getattrmathpisincoslatlon) r)r7dtrPrQZpi180ZRearthkZ moveangleZdlatZdlonr r r move_latlons    zRevital.move_latlonc Cs||jkrdStj|jd|f}tj|r@tj|dkrP|j|dSt|d}dd| D}W5QRXt j j ||j d}t}|D]}|||j<q||j|<dS)ar!Tries to find the CARQ data for the specified storm. Reads it into the self.carqdat array, or adds the stormid to self.carqfail if the data cannot be read in. @param longstormid the long stormid of the storm to read @post the self.carqfail will contain longstormid OR self.carqdat[longstormid] will contain data for that stormNza%s.datrrCcSsg|]}|qSr r )rliner r rrsz$Revital.readcarq..)r)r r%r&joinrexistsgetsizeaddrErFr1r2Z parse_carqrrYMDHr)r) longstormidfilenamerMdatacarqr-r.r r rreadcarqs    zRevital.readcarqcCs"tjj|j|||d|_d|_dS)aE!Calls the hwrf.storminfo.clean_up_vitals on this object's vitals. The optional arguments are passed to hwrf.storminfo.clean_up_vitals. @param name_number_checker a function like hwrf.storminfo.name_number_okay() for validating the storm name and number @param vitals_key a key generator for ordering hwrf.storminfo.StormInfo objects @param basin_center_checker a function like hwrf.storminfo.basin_center_okay() for checking the storm basin and forecast center (RSMC) @post is_cleaned=True)name_number_checkerbasin_center_checker vitals_keyTN)r1r2clean_up_vitalsr!r)r)rgrhrir r rrjszRevital.clean_up_vitalscCsd}|jo|jdk }|j}|dks&t|dks2t|dkrz||d|\} } |jr| dk r| dk r|jd| | fn|j|j} } |r| dks| dkr|r|ddS|j}|jo|dk }d}t|D]} || } |r$t| dd} | dkr$| j |kr$|r|d | j | j fq|r<|d | j f| j |j }|t krX| }|t krdq|tkr|r~|d || =q||jkr|r|d q|r|d | jr`| j|jkr| j|jkr|r|d|j|j f|jr|dt|j|jf|| j| j|t| jddd}|rT|d|j f||| j<q|dkr|| d|\}}|r|dk r|dk r|d||fn| j| j}}|dks|dkr|r|dqt| | ||}||jkr|tks|r|d|dfq|r4|d| jf|jrV|dt|j|jf|| j| j|t| jddd}|r|d|j f||| j<q|r|dt|f|S)af!Internal function that handles renumbering of storms. @protected This is an internal implementation function that should never be called directly. It handles part of the work of renumbering storms in the list of vitals. You should call "renumber" instead. @param vital the vital being renumbered @param lastvit the last vital seen @param vit_motion time since vital @param other_motion time since other storms vital @param threshold cold start threshold, used to decide when to stop connecting an invest to a non-investFNrg @z -- lat=%.3f lon=%.3fz -- no lat,lon for search old_stnumZz@Old %s vit was a low intensity invest, so not considering it: %sz vs. %sz -- age out othervitz -- dt is too largez -- within dtz# -- continue renumbering (%s) %sz INVEST%02d%1sTz NOW %sz -- vs lat=%.3f lon=%.3fz3 -- cannot get other vitals location; moving on.z& -- not kinda near (distance %f km)g@@z* -- within dx: renumber to %s and storez - renumbered = %s)rrAssertionErrorr[rWrXr(keysrRwmax old_stormid3r\when zero_timetwo_daysrZ has_old_stnumrkstnumbasin1stormid3r rename_stormintZ change_basinZ pubbasin2Zrenumber_stormrrr#repr)r)r7lastvitZ vit_motionZ other_motion threshold renumberedrrrWrXr*ZothervitZold_idrYZotherlatZotherlondistr r r renumber_ones                  zRevital.renumber_onerc Cs|js|d|_|sd}t|}t}|jo8|jdk }|j}t|jD]}|rd|d|jf|j } |j dkr|j dkrqJn:|j dkr|||j <qJn"| ||dd|r|rJ|dqJ|r|d| ||dd |rqJ|r|d | ||d d|rqJ|r|d | ||d d|rJqJqJ|r6| |r|sd|rd|dk r\| d||dk rx| dtj|j|_dS)a!Renumbers storms with numbers 90-99, if possible, to have the same number as later 1-49 numbered storms. Loops over all vitals from last to first, renumbering 90-99 storms to have the same storm number as later 1-49 storms. @param threshold If a threshold is given, then a cycle will only be considered for renumbering if it is either above that threshold, or is not an Invest. @param unrenumber If unrenumber is True, the original storm numbers are restored after renumbering. @param discard_duplicate If True, discard invests that are duplicates of non-invests. This feature is disabled if unrenumber is enabled or cleaning is disabled. @param clean If clean is True (the default), then self.clean_up_vitals is called, which will (among other things) delete vitals lines that have the same time and storm ID. The cleaning is done after unrenumbering, so if both options are turned on, the result will contain only one entry per storm ID per time, but with all storm IDs that are available for a given storm at any one time.FrNzVITAL %s2rlz -- done renumbering this onez7 - SEARCH AGAIN: subtract storm motion from later cycleiz2 - SEARCH AGAIN: add storm motion to earlier cyclez; - SEARCH AGAIN: add half motion to later and earlier cyclez2Delete Invests that are duplicates of non-Invests.z.Clean up the vitals again after renumbering...)rrjryrrrreversedr!r\rwrur swap_numbersr@delete_invest_duplicatesr1r2) r) unrenumbercleanr|Zdiscard_duplicatesr{rrr7r,r r rrenumber|sd        zRevital.renumbercCstt}|jD]8}|jdkrt|jdt|jdf}|||j|<q|jD]F}|jdkrPt|jdt|jdf}|||jkrP|||j|<qPt }| D]}| D]}| |qq||_dS)z[!Deletes Invest entries that have the same location and time as non-invest entries.rrlN) collections defaultdictrr!ruryrWrXrar(valuesr5)r)orrZlZyvr r rrs       z Revital.delete_invest_duplicatescCs|jD] }|qd|_dS)z@!Calls swap_numbers on all vitals to swap old and new storm IDs.FN)r!rrr6r r rrs  zRevital.swap_numberscCsDt}|jD]&}||d|jkr ||q ||_d|_dS)z!Duplicates all vitals that have been renumbered, creating one StormInfo with the old number and one with the new number.rqFN)r(r!r5__dict__oldr)r)newvitr7r r rmirror_renumbered_vitalss   z Revital.mirror_renumbered_vitalscCs.t}|jD]}||r ||q ||_dS)aP!Discards all vitals except those for which the keep_condition function returns True. @param keep_condition A function that receives a StormInfo object as its only argument, returning True if the vital should be kept and False if not. @note The list will be unmodified if an exception is thrown.N)r(r!r5)r)keep_conditionrvitr r rdiscard_excepts   zRevital.discard_exceptcCs,|jD] }t|dr|j|j|_|_qdS)zT!This subroutine undoes the effect of renaming by swapping old and new names old_stormnameN)r!hasattr stormnamerr6r r r swap_namess   zRevital.swap_namescCs|j}|jo|dk }t}t|jD]p}|j}||kr||}|j|kr|r`|d||jf|||r|d|jfq$|jdd||<q$dS)zd!This subroutine renames storms so that they have the last name seen for their storm number.NzRename to %s: %szNow: %sr ) rrrrr!rwrr\rx)r)rrZlastnamer7r,namer r rrename s  zRevital.renamecCs|j}|jo|dk }|jD]}|j}|j}||jkrB||||jkrp|dk rd|d|f| dq|j|}||kr|dk r|d||f| dq| ||qdS)zJ!Add the storm type parameter from the CARQ entries in the A deck.Nz2storm %s: no CARQ data found. Using stormtype XX.ZXXzDstorm %s cycle %s: no CARQ data for this cycle. Using stormtype XX.) rrr!rblowerrarrfrJZ set_stormtype)r)rrr7Zlsidrrrer r r add_stormtypes.        zRevital.add_stormtypecCst|j|d|_dS)z!Resorts the vitals using the specified key generator. @param keyfun a key generator for comparing hwrf.storminfo.StormInfo objectsr,N)sortedr!)r)Zkeyfunr r rsort_by_function6szRevital.sort_by_functioncCst|jtjjd|_dS)zg!Resorts the vitals by storm instead of date. See hwrf.storminfo.vit_key_by_storm for details.rN)rr!r1r2Zvit_key_by_stormr9r r r sort_by_storm;szRevital.sort_by_stormccs|jD] }|VqdS)zK!Iterates over all vitals, yielding StormInfo objects for each one.N)r!)r)xr r r__iter__?s zRevital.__iter__c#sdkrdd}nttdrHfdd}|rfdd}nftdrrfd d}|rfd d}n.selectedz\A\d\d[a-zA-Z]\Zcs |jkSN)rwrr*r rrSrcsd|jko|jkS)Nrq)rrqrrr r old_selectedUs z"Revital.each..old_selectedz\A[a-zA-Z]{2}\d\d\Zcs |jkSr)Zstormid4rrr rrYrcsd|jko|jkS)N old_stormid4)rrrrr rr[s z\A[a-zA-Z]{2}\d{6}\Zcs |jkSr)rbrrr rr_rcsd|jko|jkS)Nold_longstormid)rrrrr rras zNInvalid storm id %s. It must be one of these three formats: 04L AL04 AL042013)rDupperresearchrvalr!)r)r*rrrrr rreachCs0          z Revital.eachr\c Cs|j||dD]}|dk rh|j}t|d|}|j} t|d| } |d|j||| dd| ddf|dkrt||d q|d kr|} t|d|j}t|d|j} td || | f|d q|d krtd |j |jf|d qt|j |d qdS)a!Print the vitals to the given stream in a specified format. @param stream The stream (eg.: opened file) to receive the vitals. @param format Either "tcvitals" to reformat as tcvitals (cleaning up any errors); or "line" to simply print the original data for each line; or "HHS" to use the HHS output format. (Do not use the "HHS" option unless you are HHS.) @param renumberlog If given, sends information about renaming and renumbering of the vitals to a second stream. @param stormid The "stormid" argument is used to restrict printing to only a certain stormid. @param old If True, then vitals with an old_stormid that matches are also printed.)r*rNrqrz%10s %3s %3s %-9s %-9s rrrL)fileZ renumberingz %3s %9s => %sZHHSz %s %s "TCVT") rrwrRrwriteraprint as_tcvitalsrbrr\) r)streamr+formatr*rrZ xstormid3ZoldidrZoldnamesr r r print_vitalsps&  zRevital.print_vitalscCs t||Sr)hrd_multistorm_cmp)r)abr r rrszRevital.hrd_multistorm_cmpcCs ttSr) functools cmp_to_keyr)r)rr r rhrd_multistorm_keyszRevital.hrd_multistorm_keycCsdS)z!Does nothing.Nr r9r r rmultistorm_priorityszRevital.multistorm_priority) NFNNNrNTN)T)T)NNN)FTrT)NF)Nr\NF)r r r r r/r5r8rrBrOr[rfrjrrrrrrrrrrrrrrrrrr r r rr%sR Y  ! t H  - #cCs||kr dS||krdSdS)Nr:rr )rrr r roldcmps rcCst|dd}t|dd}|jdkr&dnd}|jdkr8dnd}t||pt||pt|j|j p|jdkr|jdkrt|j|jp|jdko|jdkot|j|j }|S)a!A compares two storminfo objects for use in sorting or comparison. Returns -1 if ab or 0 if a=b. Decision is made in this order: 1. User priority (a.userprio): lower (priority 1) is "more important" than higher numbers (priority 9999 is fill value). 2. Invest vs. non-invest: invest is less important 3. wind: stronger wind is more important than weaker wind 4. North Atlantic (L) storms: farther west is more important 5. North East Pacific (E) storms: farther East is more important If all of the above values are equal, 0 is returned. @returns -1, 0 or 1 @param a,b the vitals to compare Zuserprioi'ZINVESTr:rLE)rRrrrprvrX)rrZ a_userprioZ b_userprioZa_investZb_investcr r rrs    "r)r __all__loggingdatetimegetoptsysZos.pathr%rrSrHrrhwrf.storminfor1 hwrf.numericsrrr Exceptionrrrsrtr#rrrr r r rs" H|