U Ó gf¯ã@sîdZdddddddgZdd lZdd lZdd lZdd lZdd lZdd lZdd lZdd l Z dd l Z dd l Z dd l Z dd lZdd lZdd lZdd lZdd lZdd lZdd lZdd lZdd lmZdd lmZmZmZdd lmZdd lm Z m!Z!m"Z"m#Z#m$Z$ddl%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+m,Z,ddlm-Z-m.Z.m/Z/m0Z0ddl1m2Z2m3Z3ddlm4Z4m5Z5m6Z6m7Z7ddlm8Z8m9Z9m:Z:e;ƒZGdd„deƒZ?Gdd„de?ƒZ@Gdd„de?ƒZAGdd„de?ƒZBGdd„deƒZCGdd„de?ƒZDd S)aJ!This module implements Python classes that run the 2014 HWRF Relocation. The HWRF relocation is divided into four parts: * Stage1 --- remove the prior cycle's vortex * Stage2 --- remove the parent vortex * Stage3 --- Relocate and paste the fields together * Merge --- merge the relocation output and GSI. This is only used when GSI is enabled. In addition, due to the requirement of splitting the relocation into three stages, there are "relocation info" files for passing information between the three. The RelocationInfo object provides a means to read and write these files.ÚStage1ÚStage2ÚStage3ÚMergeÚRelocationInfoÚWARMÚCOLDéN)Úset_ecflow_label)Ú ConfigParserÚSafeConfigParserÚRawConfigParser)ÚHWRFTask)ÚcheckrunÚrunÚexeÚbigexeÚalias)Úpartial_orderingÚ TimeArrayÚ to_timedeltaÚwithin_dt_epsilonÚto_datetime_relÚ to_datetimeÚ to_fraction)Ú FileProductÚ COMPLETEDÚFAILEDÚRUNNING)ÚTempDirÚNamedDir)Ú deliver_fileÚ make_symlinkÚ isnonemptyÚ remove_file)ÚRelocationErrorÚRelocateOutputMissingÚStormRadiusErrorc@sHeZdZdZddd„Zdd„Zddd„Zd d „Zdd d „Zdd d„Z dS)raú!Passes information about relocation status between relocation stages. This class is used to pass information between the relocate stages. There are three public member variables that ARE meant to be written depending on logic within the relocation Stage1, Stage2 and Stage3 classes. One then writes out the relocation status information to an intermediate file at the end of each relocation stage using self.write_info. It is read in by a later step of the relocation, or by the merge, via RelocationInfo(filename,...) * iflag_cold = 1 or 0 used by several of the fortran relocation programs * warm_cold_flag = the constants WARM, COLD or None, depending on whether this is a warm start, a cold start, or an unknown state. * cold_ok = True only if the relocation intentionally vetoed a warm start, such as for a weak storm * initopt = 0, full vortex initialization; 1, relocation only * ensda_relocate = True relcocation for ensemble memberNcCs@d|_d|_d|_d|_d|_d|_|dk r6| |¡||_dS)zU!Creates a new RelocationInfo object by reading in the specified *.info file.rNF)Ú iflag_coldÚwarm_cold_flagÚcold_okÚinitoptÚensda_relocateÚensda_relocate_continueÚ read_infoÚ from_file)ÚselfÚfilename©r1ú:/lfs/h1/ops/prod/packages/hmon.v3.2.7/ush/hwrf/relocate.pyÚ__init__Zs zRelocationInfo.__init__cCs6dt|jƒ| ¡t|jƒt|jƒt|jƒt|jƒfS)z1!A Pythonic string representation of this object.zrRelocationInfo(iflag_cold=%s,warm_cold_flag=%s,cold_ok=%s,initopt=%s,ensda_relocate=%s,ensda_relocate_continue=%s))Úreprr'Ú warm_cold_strr)r*r+r,©r/r1r1r2Ú__str__{sþþzRelocationInfo.__str__cCsZt|ƒ ¡}|dkrtS|dkr$tS|dkr0dS|dk rR| dt|ƒt|ƒf¡dSdS)ap!Returns the module-level constants COLD or WARM, or None, for the specified string value. @param value A string that is "WARM", "COLD" or "NONE" and is case-insensitive. If it has any other value, None is returned and a message is logged at ERROR level (if a logger is given). @param logger a logging.Logger for log messages.rrÚNONENz?Invalid value %s (string %s) for warm_cold_flag. Assuming None)ÚstrÚupperrrÚerrorr4)r/ÚvalueÚloggerÚvr1r1r2Úmake_warm_cold‚s ÿÿzRelocationInfo.make_warm_coldcCs$|jtkrdS|jtkrdSdSdS)z«!This is the opposite of the make_warm_cold routine: it returns "COLD", "WARM", or "None" for the constants COLD, WARM or None based on self.warm_cold_flagrrÚNoneN)r(rrr6r1r1r2r5”s   zRelocationInfo.warm_cold_strc Cs¼t|tƒs$tdt|ƒjt|ƒfƒ‚t|dƒ(}tdddddddœd}| |¡W5QRX|  d d ¡|_ |  |  d d d¡¡|_ | d d ¡|_|  d d ¡|_| d d¡|_| d d¡|_dS)aJ!Reads the relocation information into this object from the specified filename. This is called automatically by the constructor. Error messages are logged to the given logger, if one is provided. @param filename the name of the file to read @param logger a logging.Logger for log messagesz\RelocationInfo.read_info expects a string for its filename parameter. You provided a %s %s.ÚrtÚ0r@ÚFalseÚTrue)r'r(r)r*r+r,)ÚdefaultsÚinfor'r(r)r*r+r,N)Ú isinstancer9Ú TypeErrorÚtypeÚ__name__r4Úopenr ÚreadfpÚgetintr'r?Úgetr(Ú getbooleanr)r*r+r,)r/r0r=ÚfZscpr1r1r2r-›s0 þÿ ü  ÿzRelocationInfo.read_infoc Csø|dk r| d¡t|ƒ}tj |¡}tjj||dtƒ}|  d¡|  ddt|j ƒ¡|  dd|  ¡¡|  ddt|j ƒ¡|  ddt|jƒ¡|  dd t|jƒ¡|  dd t|jƒ¡|dk rÔ| d |f¡t|d ƒ}| |¡W5QRXdS) zõ!Writes this object's relocation information to the specified *.info file. Logs errors to the specified logger, if one is present. @param filename the file to write @param logger a logging.Logger for log messagesNz;Generating in-memory ConfigParser object for RelocationInfo©r=rFr'r(r)r*r+r,z"Writing RelocationInfo to file: %sÚwt)rFr9ÚosÚpathÚdirnameÚprodutilÚfileopÚmakedirsr Ú add_sectionÚsetr'r5r)r*r+r,rKÚwrite)r/r0r=ÚthedirÚcrPr1r1r2Ú write_info¶s"    zRelocationInfo.write_info)N)N)N)N) rJÚ __module__Ú __qualname__Ú__doc__r3r7r?r5r-r^r1r1r1r2rCs !  csxeZdZdZdM‡fdd„ ZdNdd „Zd d „Zd d „Zdd„Zdd„Z dd„Z dd„Z dd„Z dOdd„Z dPdd„ZdQdd„ZdRd d!„Zed"d#„ƒZed$d%„ƒZed&d'„ƒZed(d)„ƒZed*d+„ƒZed,d-„ƒZed.d/„ƒZed0d1„ƒZed2d3„ƒZed4d5„ƒZed6d7„ƒZed8d9„ƒZed:d;„ƒZdd?„Z d@dA„Z!dBdC„Z"dDdE„Z#dFdG„Z$ee"e#ddHƒZ%dIdJ„Z&dSdKdL„Z'‡Z(S)TÚRelocationTaskz™!This is a HWRF task that forms the base class for all vortex relocation tasks, including the Merge. It exists solely to reduce code complexity.NÚGDAS1Útrack0é «ÿÿc s:t|tƒrt‚d|_|dkr |}|dkr.tƒn||_d|krTtj |  d¡|¡|d<||_ | }|dkrn| }t |ƒ|_ t t|ƒj||||f|Ž||_| ¡}|dk röt||jjƒ}|D]:}t||ƒ}ttt|ƒƒƒ}|dkrºtdt|ƒfƒ‚qºˆ|_| dt|ƒ¡‡fdd„|Dƒ|_t d ƒ|_||_||_ ||_!|dk rn| "|jd ¡|_#| "|jd ¡|_$n d|_#d|_$|jdk rÞ|jD]N}| %¡r²|jj&||jjd |_'n&|jj"||jjd |_(|j(dk sŒt‚qŒn d|_'d|_(||_)| dk r|  &¡|_*nd|_*ˆdk rœ| d t| ƒ¡| dtdd„ˆ +¡Dƒƒ¡‡fdd„| Dƒ|_,ˆ -| d ¡|_.|j.dk s|t‚ˆ -| d ¡|_/|j/dk sœt‚ˆ|_0| |_1| |_2||_3|dkrX| dt|j)ƒ¡| dt|j) +¡ƒ¡| dt|j) +¡ 4¡ƒ¡| dt|j) +¡ 4¡ 5d¡ƒ¡tj |  d¡| 6¡d|j) +¡ 4¡ 5d¡¡}||_7| d|j7¡|j 4¡|jj}d|j8d|j9d|_:||_;|  d¡}|  d¡}tj <|¡rÖd|_=||_>||_?nd|_=d |_>d |_?|j@d!krüd!|_An:|j@d"krd"|_An&|j dk s(|j!dk r0d#|_And$|_AdS)%aê!RelocationTask constructor. @param dstore the produtil.datastore.Datastore for database storage @param conf the hwrf.config.HWRFConfig for configuration info @param section the configuration section to use @param sim the hwrf.wrf.WRFSimulation describing the simulation being relocated @param domains the hwrf.wrf.WRFDomains being relocated @param taskname the taskname in the database @param modin input model: "GFS" or "GDAS1". @param wrfanl the wrfanl input source, which should have a get_wrfanl() function that accepts an hwrf.wrf.WRFDomain and returns an hwrf.datastore.Product @param wrfghost the ghost file input source, which should have a get_ghost() function that accepts an hwrf.wrf.WRFDomain and returns an hwrf.datastore.Product @param wrfinput the wrfinput datasource, which must have a get_wrfinput() function that returns an hwrf.datastore.Product for the wrfinput file @param parentTrack the parent track file input source, which must have a products() function that takes the track product name and returns the produtil.datastore.Product @param trackName the parent track name to pass to parentTrack.products() @param ghost_domains the list of ghost domains for relocation input @param dest_dir output directory for the relocation @param gsi_d02 the hwrf.gsi.FGATGSI that will run the GSI for the intermediate domain. @param gsi_d03 the hwrf.gsi.FGATGSI that will run the GSI for the innermost domain. @param gsi_d01 the hwrf.gsi.GSIBase that will run GSI for the outermost domain. @param cycling_interval negative number of seconds between cycles (-6*3600) @param info the RelocationInfo object to use @param fgat_times the list of FGAT times, datetime.datetime objects @param centrack the product for the center FGAT time track @param kwargs passed to hwrf.hwrftask.HWRFTask.__init__NÚlocationÚintercomrztIn RelocationTask.__init__, all fgat_times must be 0 or greater (times relative to the parent model). You gave: %s.z domains: csg|] }ˆ|‘qSr1r1©Ú.0Údomain)Úsimr1r2Ú sz+RelocationTask.__init__..i,éé©Úatimezghost domains: zghost domains iterated: cSsg|]}|‘qSr1r1©riÚdr1r1r2rl=scsg|]}ˆ ¡|‘qSr1)Úwrfrh)Úwrfghostr1r2rl>sÿz self._wrfanl=zself._wrfanl.wrf()=zself._wrfanl.wrf().simstart()=z5self._wrfanl.wrf().simstart().strftime(%%Y%%m%%d%%H)=ú%Y%m%d%HÚWORKhwrfÚ.zself.dest_dir=éééÚoldcomÚoldsidTFÚÚGFSÚENKFZGDASZHDAS)BrGr9ÚAssertionErrorÚ_RelocationTask__rinforrFrSrTÚjoinÚgetdirÚ fgat_timesrÚcycling_intervalÚsuperrbr3Ú_ensdaÚlogrÚconfÚcyclerÚintÚfloatÚroundÚ ValueErrorr4rkÚdebugÚdomainsÚ dt_epsilonZ_gsi_d01Ú_gsi_d02Ú_gsi_d03Ú get_wrfanlÚ _wrfanl_d02Ú _wrfanl_d03Úis_moadÚ get_wrfinputÚ_ensda_wrfinput_d01Ú_ensda_wrfinput_d02Ú_wrfanlÚ _wrfinputrsÚ ghost_domainsÚ get_ghostÚ _ghost_d02Ú _ghost_d03Ú _wrfghostÚ _parentTrackÚ _trackNameÚ _centrackÚsimstartÚstrftimeÚlowerÚdest_dirÚdaysÚsecondsÚ_fhrÚ_modinÚisdirÚ_warmÚ_prev_cycle_dirÚ_prev_cycle_sidÚmodinÚ_vortex)r/Údstorer‰ÚsectionrkrÚtasknamer±ÚwrfanlrtÚwrfinputÚ parentTrackÚ trackNamerr¨Úgsi_d02Úgsi_d03Úgsi_d01r…rFr„ÚcentrackÚensdaÚkwargsr=Z parent_cycleÚtrjÚdtr{r|©Ú __class__)rkrtr2r3Ôsä" ÿ þÿ     ÿ ÿ   ÿ ÿ ÿÿÿÿ ÿÿ    zRelocationTask.__init__cCsdS)zþ!Returns the wrfinput output Product for the specified domain, or None if no such domain is known. @returns None @param domain the domain of interest @note This is a abstract function that should be replaced in subclasses.Nr1©r/rjr1r1r2r˜zszRelocationTask.get_wrfinputcCsdS)a!Returns the wrfanl output Product for this Task for the specified domain or None if no such product exists @param domain the domain of interest @returns None @note This is a abstract function that should be replaced in subclasses.Nr1rÄr1r1r2r”œszRelocationTask.get_wrfanlcCsdS)zÑ!Returns the wrfghost output Product for this Task for the specified domain @param domain the domain of interest @note This is a abstract function that should be replaced in subclasses.Nr1rÄr1r1r2rž¤szRelocationTask.get_ghostcCsL|dk rBt||j ¡|jƒsB| ¡ d| d¡| d¡f¡dS| |¡S)zË!Returns the wrfinput output file for the specified time and domain, or returns None if no such file exists. @param atime the time of interest @param domain the domain of interestNú1wrfinput_at_time: atime=%s is not near my time %sru)rrkr¥r‘rˆrFr¦r˜©r/rprjr1r1r2Úwrfinput_at_time«s ÿÿÿÿzRelocationTask.wrfinput_at_timecCsF|dk rBt||j ¡|jƒsB| ¡ d| d¡| d¡f¡dSdS)zÁ!Returns the wrfanl output file for the specified time and domain, or None if no such file exists. @param atime the time of interest @param domain the domain of interestNú/wrfanl_at_time: atime=%s is not near my time %sru)rrkr¥r‘rˆrFr¦rÆr1r1r2Úwrfanl_at_time¹s ÿÿÿÿzRelocationTask.wrfanl_at_timecCsN| ¡}| dd¡}|dkr&| d¡}|s.t‚t|ƒs:t‚t|dd|ddS)z/!Copies the fixed files to the local directory.Útblr}z#{FIXhwrf}/hwrf_eta_micro_lookup.datzeta_micro_lookup.datT©Úforcer=N)rˆÚconfstrÚ confstrinterpr€r"r!)r/r=rÊr1r1r2Ú copy_fixedÆs   ÿzRelocationTask.copy_fixedc s| ¡‰|j}|dkr"ˆ d¡nê|dkr6ˆ d¡nÖˆ t|ƒd¡tj |¡sjˆ t|ƒd¡dS‡fdd„}ztj||d Wn‚t k rÈ}z"ˆjt|ƒd t|ƒd d W5d}~XYnDt k r }z$ˆjt|ƒd t|ƒd d ‚W5d}~XYnXdS)z.rmerr)Úonerrorz;: unable to delete this directory tree; continuing anyway: T©rÑz4: unhandled exception deleting this directory tree: ) rˆr¨Úwarningr9rSrTÚexistsÚshutilÚrmtreeÚEnvironmentErrorÚ Exception)r/ÚddrÒÚeeÚer1rQr2Ú delete_tempÑs<    ÿ  ÿþ ÿþzRelocationTask.delete_tempccsdS)z„!Iterates over all products generated by this task. @note This is a abstract function that should be replaced in subclasses.Nr1r6r1r1r2ÚproductsðszRelocationTask.productsFc Ks¸|dkr| ¡}| d|jf¡tjj|j|d| ¡D]v}|j}tj   |¡}tj   |¡rŒ| d|j |f¡|j f|||dœ|—Žq<| d|j |f¡|dk r<|||ƒq.copiercWst|j|d|ddS©NTrË©r!rfrr1r1r2Úlinker—sz4RelocationTask._make_plist_and_names..linkerÚ wrfinput_d01Ú wrfinput_d02Ú wrfinput_d03Ú wrfghost_d02Ú wrfghost_d03cSsg|]}|‘qSr1r1©riÚkr1r1r2rlŸsz8RelocationTask._make_plist_and_names..c3s|]}|ˆfVqdSrúr1©riÚn©rr1r2Ú  sz7RelocationTask._make_plist_and_names..)Údictrœr•r–rŸr Úkeys)r/rÚnamesÚplistÚactionsr1r r2Ú_make_plist_and_namesŒs      z$RelocationTask._make_plist_and_namescsŽ| ¡}| ¡\}‰‰‡fdd„}‡fdd„}|D]"}| d|jt|jƒ|jf¡q2t|ƒtj j ||||ddkr€t j   d¡‚| d ¡d S) z+!Copies, or makes, one or more input files.csˆ|Srúr1)rær=r)r$r1r2Únamer§óz)RelocationTask.copy_inputs..namercsˆ||||f|žŽdSrúr1r)Úactionr1r2Úactor¨r)z)RelocationTask.copy_inputs..actorz,Need product %s at location=%s, available=%srn)ÚmaxtimezSSome inputs to the relocation are missing. The previous init job must have failed.réN)rˆr'rFrãr4rfÚ availableÚlenrVÚ datastoreÚwait_for_productsÚhwrfÚ exceptionsÚRelocationInputErrorrí)r/r=r%r(r+rær1)r*r$r2Ú copy_inputs£s  ÿ  zRelocationTask.copy_inputscCs ||_dSrú)r‡)r/r¾r1r1r2Ú set_ensda·szRelocationTask.set_ensdacCs|jdkr|jS|jS)zz!Returns the Product for the center FGAT time track file if available, or otherwise the parent track file Product.N)r¤rör6r1r1r2Ú get_centrackºs zRelocationTask.get_centrackcCs|dkrtj d¡‚||_dS)zl!Sets the Product for the center FGAT time track file. @param centrack the center FGAT track productNz#You must specify central track file)r1r2ÚRelocationConfigurationErrorr¤)r/r½r1r1r2Ú set_centrackÁs ÿzRelocationTask.set_centrackcCs d|_dS)zq!Unsets the center FGAT time track file so that get_centrack() will return the parent track file instead.N)r¤r6r1r1r2Ú del_centrackÉszRelocationTask.del_centrackz(The track file for the center FGAT hour.c Csò| ¡}|dkr|j}n|j}| dt|ƒt|ƒf¡tjj||ddd„dd„d|j   ¡  d¡}|  d d ¡}|j rÌ|jrÌ|j}| d ¡| d |j|f¡|dkrºt|d |dnt|d|dn"|dk rÄ|d krÄ| d¡|jjj}|  dd¡}d|||f} | d|f¡t|dƒŠ} | dt|ƒt|ƒt|ƒf¡td dƒT} tD]H} t | | ¡rŒ| dt| ƒf¡|  | ¡n| dt| ƒf¡qZW5QRXW5QRX| d¡n*| d¡| d¡td dƒ} W5QRXdS)a !Gets the parent vortex track file, either from a specified directory or from the tracker, run by a previous hwrf.init.HWRFInit object's tracker member. @param case 1 or 2: 1 for creating the atcfunix file, 2 for the atcfunix_cen file.rmz case %d:tp %srncSsdS)Nzgfs-anl.atcfunixr1)ræÚlr1r1r2Úár)z,RelocationTask.create_atcf..cSst|j||dS)NrQr)rærr:r1r1r2r;âr))r,Úrenamerr*ruÚgfs_atcfr}z/Using parent vortex from provided tracker data.z%s (%s): parent vortex locationÚatcfunixrQÚ atcfunix_cenNz6Using parent vortex from parent model's own track fileZ track_namezAVNO|PRE1|PRD1z(?=%s)(?=%s)(?=%s)z%s: track fileÚrz.Parsing track for: start=%s basin=%s fields=%sÚwz Keep : %s z Discard: %s zDone parsing track file.z7Could not find a track file for parent vortex location.z3Will proceed assuming parent vortex is at tcvitals.Úa)rˆrör½rFr‹r4rVr/r0rkr¥r¦rÍr-rfrÕrãr r‰ÚsyndatrþrKÚfileÚreÚsearchr[r;)r/Úcaser=ÚtpÚstartr=Zta_atcfÚbasinÚfieldsZpatnÚifileÚofileÚlineÚor1r1r2Ú create_atcfÓs^ þ   ÿÿ   ÿ  , ÿÿ zRelocationTask.create_atcfcCs&t|ƒ}| ¡}| |¡} d|j|jj|f} |rÂd} |D]J} t| tƒrX| d| f7} q:t| tƒrr| d| f7} q:| dt| ƒf7} q:|  dt |ƒt | ƒf¡| d7} t j   t j  | ¡¡| >}nt j   t j  | ¡¡}|jrä|| k}i} |dkrô| }|dkr| }|dkr| }tt t| ¡ƒt| ¡ƒt| ¡ƒ¡ƒ}|D]}t jjd t|ƒ|d q>|rvt jj|d |d |rt jj|d d |d | t |ƒ¡t j j||d |r | ¡D]N\}}d t|ƒ}tj |¡rðt||d|dn| d||t|ƒf¡q¼|rj| ¡D]N\}}d t|ƒ}tj |¡rNt||d|dn| d||t|ƒf¡q| ¡D]R\}}tj  d t|ƒ¡rr|  dd t|ƒ|f¡t jjd t|ƒ|d qr| ¡D]R\}}tj d t|ƒ¡rÎ|  dd t|ƒ|f¡t jjd t|ƒ|d qÎdS)aõ!Helper function for running Fortran programs that need fort.* files for inputs and outputs. Run an external command linking in fort.X files for input and output. If self.redirect=True, redirect logs to a separate file. @param cmd The command to execute. This function will use "self.getexe()" on the command to find the external program to execute. @param echo If a list is passed in as the echo variable, then the contents will be sent to the stdin of the program as a string. @param inputs Input dictionary for files to link. See below. @param incopies Input dictionary for files to copy. See below. @param outputs Output dictionary for files to link. See below. If the output is not present, a message is logged at ERROR level. @param opt_outputs Optional outputs dictionary for files to link. See below. If the outputs are not present, it is not considered an error. The dictionary arguments should consist of a fortran file number and the source file. @code inputs = {11:tcvitals, 12:wrfout_d01} @endcode would produce symbolic links: @code{.unformatted} fort.11 -> tcvitals fort.12 -> wrfout_d01 @endcode input files can also be copied using incopies: @code incopies = {11:tcvitals, 12:wrfout_d01} @endcode would create files instead of links. The outputs and opt_outputs (optional outputs) should be of the dictionary as the inputs. As in: outputs = {56:new_data_4x, 85:storm_radius} this would mean the "fort.56" file would be renamed to "new_data_4x" and the "fort.85" renamed to "storm_radius". If opt_outputs is given then the fortran file is tested to see if it exists and only if it does is it renamed to the output filename. A log file will be created consisting of the stdout and stderr of the command run. It will be named consisting of the taskname and command. For example, if this is relocation stage 1 and the command is hwrf_pert_ct then the log file is "rel_stage_1_hwrf_pert_ct.log" z%s/logs/%s_%s.logr}z%g z%d z%s z6Converted %s to %s for stdin input to fortran command.rêNzfort.rQTrË)rÌÚonly_log_errorsr=F©rár=z)%s: did not make file %s (would mv to %s)z*%s: did not make file %s (would mv to %s).z,%s: deleting input fort file (symlink to %s)z)%s: deleting input fort file (copy of %s))!r9rˆÚgetexer¨rÃrJrGrŒr‹rFr4rVrÚopenmprÚredirectr"Ú itertoolsÚchainÚlistÚitemsrWr#ÚfortlinkÚfortcopyrÕrrSrTrÖr r;Úislink)r/ÚcmdÚechoÚinputsÚincopiesÚoutputsÚ opt_outputsZcmdnamer=ÚprogZlogfZechostrÚsÚemptyÚiofrr>Zffiler1r1r2Úrun_ext s˜> ÿ  ÿÿ   ÿ ÿ  ÿ   ÿÿÿÿzRelocationTask.run_ext)NrcNNNNrdNNNNNreNNNN)N)NNFN)réN)N)N)NNNNN))rJr_r`rar3r˜r”ržrÇrÉrÏrÞrßrèrírîrôÚpropertyrör±r÷rørùrûrýrÿrrrr r r'r4r5r6r8r9r½rPrgÚ __classcell__r1r1rÂr2rbÐs”û' "  ÿ                ÿ7ÿrbcszeZdZdZd‡fdd„ Zdd„Zdd„Zd d „Zd d „Zd d„Z dd„Z dd„Z dd„Z dd„Z dd„Zdd„Z‡ZS)rzL!This is a HWRF task that encapsulates stage 1 of the vortex relocation.Nc s"tt|ƒj||||||f|ŽdS)z‚!Stage1 constructor. @param dstore,conf,section,sim,domains,taskname,kwargs Passed to RelocationTask.__init__()N)r†rr3©r/r³r‰r´rkrrµr¿rÂr1r2r3s ÿÿzStage1.__init__c Cs¾| ¡}d}zŒzH|j}tj |¡r.|  ¡t j   |¡|  dd¡|j_|j dd¡}|jdkrˆ| d d¡rˆt|ƒr€d |j_nd|j_d|j_t|ƒ²}| d t ¡¡t d t ¡¡rÂt‚t j   |d ¡|jdk sæ|jdk rî| ¡|jdkr¶| ¡}|s’| d¡d |j_t|j_ | d¡}|rR| d¡t!dd|ƒd|j_"nPrior cycle %s forecast does not exist. This is a cold start.z'Prior cycle data is present for time %sNzRFgat wrfout_d%02d file does not exist for forecast time %s. This is a cold start.z,Have wrfout_d%02d file for forecast time %s.z.Prior cycle data is present for all fgat timesz9No fgat_times specified. Prior %s forecast is available.T)rˆrÕrkr¥r¦rûrr§rÚ get_grid_idrùrFr"r„) r/r=ÚftimestrÚsidrrÚidrLrÀZ ftimestr2r1r1r2rx*s^     ÿÿÿ    ÿ ÿÿÿ  ÿzStage1.check_prior_cyclecs| ¡ d¡d}| ¡‰|j ¡ d¡}| dd¡‰ˆsJtjjddd‰ˆr^‡‡fdd „}nd }d d „|j Dƒ}|j dkrˆ|j   ¡}n |j   ¡}|D]p}|j d krÂ| ¡}d |j|||f}ndt| ¡ƒ} | ¡rðd}|j j||jjd} nd}|j j||jjd} ˆ dt|ƒ| jf¡| j}d|} d|} tj |¡dkrlˆ d|f¡|d krlˆ d¡t|| dˆ|d| |¡} d|j|jj||f}tj  | ¡d| d| f}|j!rÆ||k}tjj"|| ¡dt#j$ %d¡ròt# &d¡t#j$ %d¡r–t# &d¡q–d S)z2!Runs the hwrf_3dvar to paste the relocated storm.zstage1 relocate_stormÚ hwrf_3dvarrˆÚncksr}F)Ú raise_missingcs.tj |ˆ¡ttˆƒdd||fˆddS)Nz-6z-OrQ)rVrWr#rr)rdrÀÚx©r=rr1r2rasz%Stage1.relocate_storm..copierNcSsg|]}|‘qSr1r1rqr1r1r2rlgsz)Stage1.relocate_storm..r‰rm)rjrprnzdomain %s prod %sz wrfout_d%02dzold_hwrf_d%02dÚHDF5z`%s: file is HDF5, so I will assume it is compressed and convert back to 64-bit indexing NetCDF3.z*ncks not found; things will probably breakT)rár=rú%s/logs/%s_%s_d%02d.logÚstorm_relocateÚflnm3rQúfort.50úfort.73)'rˆrFrkr¥r¦rSrVrWÚfind_exerrûrr§r‡rŠrùr‹r—r˜r‰rŠr”r9rãrfÚ netcdfverryr r¨rÃrJrrrUrrSrTÚisfileÚremove)r/Úfprogr‹rrrŒrjrrLrãÚprodÚwrfoutZ old_wrfoutrcrˆr]r1r’r2r|Wsp      ÿ þ   ÿÿ  zStage1.relocate_stormcCs|| ¡ d¡|jdkrd}nd}d|jd|j|jg}|jdkrPddd d d œ}n ddd d œ}d ddœ}|j||||ddS)ú%!Runs the fortran merge_nest program.zstage1 merge_nestNÚhwrf_merge_nestZhwrf_merge_enkfrxrréÚ old_hwrf_d01Z old_hwrf_d02Z old_hwrf_d03©é éé$é.©r¤r¥r¦Ú data_4x_hwrfÚ roughness1©é8éB©r^r_ra)rˆrFr‡rýrrrg©r/rÚevarsÚinsÚousr1r1r2r}–s0 ü ýþÿzStage1.merge_nestc Csè| ¡}| d¡|jdkr"d}nd}|jdkr<|j ¡}n |j ¡}t|j|jj ƒ}d|j ||  d¡|f}t j  |¡sÐ| d|¡| d ¡| d ¡| d |f¡| d ¡td dƒW5QRXdSt|d d| ¡ddS)z%!Get the previous cycle's HDAS track.zstage1 copy_hdasé ÚcombineÚparentFz%s/%s.trak.hwrf.atcfunix.%s.%sruz!Prior cycle atcf does not exist: zWill use an empty track.z"PRIOR HWRF CYCLE PROBABLY FAILED!!z=Prior cycle has wrfout files, but no 12hr track file here: %szbCheck the prior cycle JHWRF_PRODUCTS job for errors. Check to see if NHC received the track file.Ú hdas_atcfunixrRTrR)rˆrFrýrûrr§rr…r‰rŠrùr¦rSrTrÖrÕryrKr )r/r=ÚdomrŒZ prev_timeZ hdas_atcfr1r1r2r~´s8     ÿÿ   þ   zStage1.copy_hdascCs`| ¡}| d¡|jj|jjd}|dkrB| d|jj¡dS|j}t|dd|ddSdS) z1Get the previous cycle's ensemble forecast track.zstage1 copy_ensda_trackroNzNo track for member %s.FÚensda_atcfunixTrR) rˆrFr‡Ú get_trackr‰rŠZ_Stage1__enkfmemrfr )r/r=Z ensda_atcfÚfromfiler1r1r2rƒßs zStage1.copy_ensda_trackc Csæ| ¡}tƒ}tt|j ƒƒ}d}| dt|jjƒ¡|jjrFd}nd}t|dƒ¸}|D]¬}|  ¡}zbt |dd…ƒ} |  | d¡t || dƒdkr°| d ||f¡d }| d t| ƒ|f¡WqZt ttfk r} z| d ||f¡W5d } ~ XYqZXqZW5QRX|jd ks(|jjr@|r2d S| d¡dS|jjsâd } t|j|jjƒ} |jD]L} t| ƒ| }t|ƒ}t tt|ƒƒƒ}||krd| dt|ƒf¡d} qd| rÆ| d¡d S| d¡t|j_d |_dSd S)zR!Checks to see if all FGAT hours have tracks of a required minimum length.Fzself.info.ensda_relocate is %sr¸r¶rAéé!rzz$Found cycling interval hour %d in %sTzFound hour %s in %sz"Cannot parse hour from %s line: %sNzNDid not find cycling interval hour %d in hdas_atcfunix. This is a cold start.z'Could not find hour %s in hdas_atcfunixz%All FGAT hours found in hdas_atcfunixz*Some FGAT hours not found in hdas_atcfunix)rˆrZrrr…rFr4r+rKÚrstripr‹ÚaddÚabsÚ IndexErrorrŽrHrÕr„rr‰rŠrrŒrr(r))r/r=ÚseenZicycZ found_icycr>ZharNÚrlineÚhourrÝZ all_foundÚ parent_atimeÚftimerÁÚfdtZfgathr1r1r2rísh ÿÿÿÿ"      ÿ  zStage1.check_atcf_hourscCsd| ¡ d¡d}|jd|jjjfg}|jjr}z|jd t|ƒdd‚W5d}~XYnXW5|rf|jjtj |jd¡|dXdS)z !Runs stage 2 of the relocation.Fú stage2.inforQzStage 2 starting in directory: rmrkTrnrmZ roughnessÚ roughness2z Stage 2 completed in directory: zStage 2 failed: rÔN) rˆrFr^rSrTr‚r rVrWrXr¨rrtrurErvr€r-rírÏr4r|rPÚ create_trackr#Ú merge_nestsÚ wrf_splitrr…rÚryr9)r/r-r=rlrÝr1r1r2r‡sF ÿ     ÿÿz Stage2.runc Csl| ¡ d¡d}|j d¡}| ¡}| d¡}|jD]”}| ¡}d|}d|}d|j|jj ||f} t j   |¡d|d|f} |j rŽ| | k} t j j| |d tj d ¡r´t d ¡tj d ¡r6t d ¡q6td d ƒD]}d|}d|}d|j|jj ||f} t j   |¡d|d|f} |j r(| | k} t j j| |d tj d ¡rPt d ¡tj d ¡rÖt d ¡qÖdS)zs!Runs the hwrf_diffwrf_3dvar program on all inputs to create binary file for input to the Fortran programs.zstage2 relocate_stormrŽrgúwrfinput_d%02dú new_gfs_d%02dr”r•r–rQr—r˜rnéúwrfghost_d%02dú new_ght_d%02dú%s/logs/%s_%s_ghost_d%02d.logN)rˆrFr‰rƒrSrrŠr¨rÃrJrVrrrUrrSrTr›rœÚrange) r/rÚicom_dirr=rcrrrÚfinÚfourˆr]r1r1r2r|«sP   ÿ    ÿ  zStage2.relocate_stormcCshd}d|jg}dddœ}ddi}dd i}|j|||||d | ¡}tdd d |d tddd |d dS)z+!Runs the fortran hwrf_create_nest program.Zhwrf_create_nestrxÚ new_gfs_d01Ú new_gfs_d02)r¥r§é9Z new_data_d01r¬Z new_data_1xrÕZnew_gfs_d01_orgFrRN)rÿrgrˆr )r/rr°r±r²rÖr=r1r1r2Ú create_nest×s*ÿÿ ÿÿÿzStage2.create_nestcCsT| ¡ d¡d}|j|j ¡ d¡|jg}dddœ}ddi}|j||||d d S) z*!Runs the fortran create_trak_fnl program.zstage2 create_trackÚhwrf_create_trak_fnlú%Yrér>rÇr»útrak.fnl.all_gfsr®N©rˆrFrrkr¥r¦rÿrgr¯r1r1r2râêsþÿzStage2.create_trackcCsV| ¡ d¡d}ddd|j|j|jg}dddd d œ}d d d œ}|j||||ddS)r zstage2 merge_nestsr¡rxrrmrérïrðÚ new_gfs_d03r£Ú data_4x_gfsrár«r®N)rˆrFrrrÿrgr¯r1r1r2rãøs$ûýÿzStage2.merge_nestsc Cs¨| ¡ d¡tj d¡r d}nd}d|j ¡ d¡}d|j ¡ d¡}d}|j||j |j |jj d g}d d d d ddœ}|ddddœ}d|i}|j |||||ddS)z$!Runs the fortran split_wrf program.zstage2 wrf_splitrÛrmrnzrel_inform_gfs.%sruzvital_syn_gfs.%srËrÌrérørõrïrÏ)r¤r¥r»r§rØÚgfs_envÚstorm_pert_gfsÚstorm_radius_gfs)rÓr¬rÐrÑrÔrÕN) rˆrFrSrTr›rkr¥r¦r rýrÿr'rg) r/ZibgsÚrelÚvitalrr°r±r²rÖr1r1r2rä sB ÿÿûüý ÿzStage2.wrf_split)N) rJr_r`rar3rr|ròrârãrärir1r1rÂr2r|s$,csèeZdZdZd6‡fdd„ Zdd„Zdd„Zd d „Zd d „Zd d„Z d7dd„Z dd„Z dd„Z d8dd„Z dd„Zdd„Zdd„Zdd„Zdd „Zd9d"d#„Zd$d%„Zd&d'„Zd(d)„Zd*d+„Zd,d-„Zd.d/„Zd0d1„Zd2d3„Zd4d5„Z‡ZS):rz–!This is a HWRF task that encapsulates stage 3 of the vortex relocation which relocates and pastes the vortexes together from various sources.Nc s8tt|ƒj||||||f|Ž| ¡}t|d|jtj |j d¡d|_ t|d|jtj |j d¡d|_ t|d|jtj |j d¡d|_ t|d|jtj |j d¡d|_ t|d|jtj |j d¡d|_t|d|jtj |j d¡d|_t|d|jtj |j d¡d|_t|d |jtj |j d ¡d|_W5QRXd S) z…!Stage3 constructor. @param dstore,conf,section,sim,domains,taskname,kwargs Passed to the RelocationTask.__init__()r©rfrrrrrÏZ wrfout_d01Ú wrfout_d02N)r†rr3Ú transactionrrµrSrTr‚rfÚ_prod_ghost_d02Ú_prod_ghost_d03Ú_prod_wrfinputÚ_prod_wrfanl_d02Ú_prod_wrfanl_d03Ú_prod_storm_radiusÚ_prod_ens_wrfout_d01Ú_prod_ens_wrfout_d02) r/r³r‰r´rkrrµr¿rÀrÂr1r2r3:slÿÿ  ÿÿ ÿÿ ÿÿ ÿÿ ÿÿ ÿÿ ÿÿ ÿÿzStage3.__init__cCsÐ| ¡}| dt|ƒ¡||jdkrX|jdk rL| dt|jdƒ¡|jS| d¡n||jdkr¤|jdk rŒ| dt|jdƒ¡|jS| dt|jdƒ¡n| d t|jdƒ¡| d t|ƒ¡dS) zˆ!Returns Product objects for the ghost domain output file for the specified domain. @param domain the domain of interestržrmNú is domain 2zno prod_ghost_d02rnz is domain 3zno prod_ghost_d03zis not domain 3zget_ghost: no ghost for domain )rˆrr4rrrrFr9©r/rjr=r1r1r2ržZs   zStage3.get_ghostcCs| ¡}||jkr>| dt|ƒd dd„|jDƒ¡f¡dS| dt|ƒ¡||jdkrŽ|jdk r‚| dt|jdƒ¡|jS| d¡n||jd krÚ|j dk rÂ| d t|jd ƒ¡|j S| d t|jd ƒ¡n| d t|jd ƒ¡| d t|ƒ¡dS)NzInvalid domain: %s not in %sz, cSsg|] }t|ƒ‘qSr1)r9)rir‘r1r1r2rlvsz%Stage3.get_wrfout..Ú get_wrfoutrz is domain 1zno prod_ens_wrfout_d01rmr zno prod_ens_wrfout_d0zis not domain 2z!get_wrfout: no wrfout for domain ) rˆrrFr9r‚Zomainsrr4rrr r1r1r2r rs( ÿ   zStage3.get_wrfoutcCsd||jkrdS|j|}|dk rZt||j ¡|jƒsZ| ¡ d| d¡| d¡f¡dS| |¡S)zí!Returns a Product object for the wrfinput output file for the specified domain if the atime matches this object's self.sim.simstart() @param atime the time of interest @param domain the domain of interestNrÅru)rkrr¥r‘rˆrFr¦r˜rÆr1r1r2rÇ‹s   ÿÿÿÿzStage3.wrfinput_at_timecCsL|dk rBt||j ¡|jƒsB| ¡ d| d¡| d¡f¡dS| |¡S)zì!Returns a Product object for the wrfanl output file for the specified domain if the atime matches this objects' self.sim.simstart(). @param atime the time of interest @param domain the domain of interestNrÈru)rrkr¥r‘rˆrFr¦r”rÆr1r1r2rÉœs ÿÿÿÿzStage3.wrfanl_at_timecCs@||jdkr|jdk r|jS||jdkr<|jdk r<|jSdS)zƒ!Returns a Product object for the wrfanl output file for the specified domain. @param domain the domain of interestrmNrn)rr•rr–rrÄr1r1r2r”«szStage3.get_wrfanlcCs |dk r||jdkrdS|jS)zÃ!Returns a Product object for the wrfinput output file. If a domain is specified, and is not the correct MOAD, then None is returned. @param domain the domain of interestNr©rrrÄr1r1r2r˜´szStage3.get_wrfinputcCs|jS)z-!Returns a Product for the storm radius file.)rr6r1r1r2Úget_storm_radius¼szStage3.get_storm_radiuscCs|jS)z&!Returns a Product for the track file.)rör6r1r1r2r¹ÀszStage3.get_trackccs|| ¡}|dkrV| ¡Vtƒ}|jD]}| |¡q&|jdk rV|jD]}| |¡qFtƒ}|D]}||kr`| |¡nq`|jdkr2||jdkr¨|jdk r¨|j V||jdkrÈ|j dk rÈ|j V||jdkrè|j dk rè|j V||jdkr |jdk r |jV||jdkrv|jdk rv|jVq`||jdkrV|jdk rV|jV||jdkr`|jdk r`|jVq`dS)z¨!Iterates over all products, or all selected products. @param domains If an iterable of domains is given, only iterates over products for those domains.Nrrmrn)rˆr rZrr¾r¡rr‡rœrr•rr–rrŸrr rr™rršr)r/rr=rrZhitr1r1r2rßÄs:        zStage3.productscCstdt|ƒfƒ‚dS)a«!Internal function that raises an exception when a product is missing. This is an internal implementation function. It should not be called directly. This is called by deliver_products when an expected input file is missing. It either returns, or raises an exception. See deliver_products for details. @param prod the Product @param basename the basename of the missing filez#Mandatory output file %s is missingN)r%r4)r/ržrâr1r1r2Ú_missing_productès  ÿzStage3._missing_productc CsZ| ¡}d}z"zÞtj  |j ¡t |j ƒ´}|  dt  ¡¡t dt  ¡¡rVt‚|jjtj |jd¡|dd}|jjsž|jjtj |jd¡|dd}tj  |j d ¡| ¡|jjræd |_tj d ¡rä| d ¡| ¡nÐd |_tj d ¡r| d ¡t|j_nT| d¡t|j_| d|j¡|jdksJ|jjd krb| d¡| ¡d|_tj d ¡r„| d¡| ¡n| d¡| ¡| ¡|  ¡t!dƒ| "¡| #¡|j$|j%d|  dt  ¡¡W5QRXt&|_'Wn<t(k r,}z|j)dt*|ƒdd‚W5d}~XYnXW5|rT|jjtj |jd¡|dXdS)z'!Runs stage 3 of the vortex relocation.Fz stage3.inforQzStage 3 running in directory: rmrkTràrnrmrÛzhave storm_pert_new (check 1)rxz$do not have storm_pert_new (check 1)zstorm intensity is %04dézintensity <20 or initopt=1rzhave storm_pert_new (check 2)z$do not have storm_pert_new (check 2)Ú flag_file)råz Stage 3 completed in directory: zStage 3 failed: rÔN)+rˆrFr^rSrTr‚r rVrWrXr¨rrtrurErvr€r-r+rÏÚgfs_flagrÖÚensda_relocate_runrr(rrýr*Ú weak_cold_runÚcycled_or_weak_runÚ anl_bogus_10mÚ inter_2to2Úinter_2to2_againr#Ú inter_4to6Ú update_3dvarrèrrr…rÚryr9)r/r=r-rlrÝr1r1r2rôsvÿÿ            ÿÿz Stage3.runcCs2| ¡ d¡d|_| ¡| d¡| ¡dS)zO!Runs the portion of the relocation that is used for weak, cold storms.zstage3 cold_runrrnN)rˆrFrÚ pert_ct_weakrPrâr6r1r1r2r:s  zStage3.weak_cold_runcCsª| ¡ d¡|jdd|jdkrbtj d¡rb|jdkrb| ¡ d¡d|_| ¡|jddtj d ¡r„| ¡ d ¡|  ¡tj d¡r¦| ¡ d ¡|  ¡d S) zR!Runs the portion of the relocation that is run for cycled or weak storms.ústage3 cycled_or_weak_runrm©rGrnÚ flag_file2r~z,gfs_flag>2, have flag_file2 and modin is GFSrrzhave flag_filezhave flag_file2N) rˆrFÚanl_4xrrSrTr›r±Ú pert_ct_gfsÚ anl_cs_10mrr6r1r1r2rHs ÿ   zStage3.cycled_or_weak_runcCs| ¡ d¡|jdddS)z#Runs relocation for ensemble memberrérN)rˆrFrr6r1r1r2rZszStage3.ensda_relocate_runrncCs| ¡ d¡| ¡ d|j¡d}tdƒtdƒ|j|j|j|jjg}|dkr„ddd d d d d dœ}tdƒrzd|d<q¶d|d<n2|dkr¢ddd d d dd dœ}nddd d dd d dœ}ddi}ddi}|j|||||dt ddd| ¡dt j  d¡sþt ‚t j d¡s| ¡ d ¡d!S)"zO!Runs the anl_4x programs. @param case 1 or 2: why is anl_4x being run.z stage3 anl_4xz self.fhr %sZ hwrf_anl_4xrrrmrér¶rÙrÚrùrªrÛ)r¤rÈrprÝr¥r§rÐrÉr»útrak.fnl.all_gfs_cenrnr>rár¸rÍr¦Ú wrf_env_newr¬Ú new_data_4xrÕrÏZstorm_radius_1TrRzCNO FLAG FILE!!! The hwrf_anl_4x program did not make the flag_file.N)rˆrFr r#rÿrr*r"rgr rVrWr€rSrTrÖrÕ)r/rGrr°r±r²Zoousr1r1r2r_shýú  ú ú ÿÿÿz Stage3.anl_4xc CsÂ| ¡ d¡d}|jjdk s"t‚d|jt|jjƒg}d| d¡}d| d¡}dd d d d d œ}||||||||dœ}ddi}tj  d ¡st‚|j |||||dt j   d¡s¾| ¡ d¡dS)z%!Runs the anl_cs_10m fortran program.zstage3 anl_cs_10mZ hwrf_anl_csNrxú%s/hwrf_storm_cyn_axisy_47ÚFIXhwrfú%s/hwrf_storm_20rérÚr#rÏrª)r¤rÝr¥rÑr§©rÐéHéIéJéKéLéMéNr¬r$©r^r_r`rarzGNO FLAG FILE!!! The hwrf_anl_cs_10m program did not make the flag_file.)rˆrFr'r€rÿr‹rƒrVrWr"rgrSrTrÖrÕ©r/rr°Zaxisy_47Zstorm_20r±Úincr²r1r1r2r ™s> þüù  ÿzStage3.anl_cs_10mc Cs€| ¡ d¡d}d|jg}d| d¡}d| d¡}ddd d d d d œ}||||||||dœ}ddi}|j|||||ddS)z(!Runs the anl_bogus_10m fortran program.zstage3 anl_bogus_10mZhwrf_anl_bogusrxr%r&r'rérùrørárúrû)r¤r¥r¦r§é=rÑr(r¬r$r0N)rˆrFrÿrƒrgr1r1r1r2r¼s2ÿûù zStage3.anl_bogus_10mcsž| ¡ d¡| ¡‰‡fdd„}|ddƒ|ddƒ|dd ƒ|d d ƒd }d |jdg}dddd dddœ}ddddœ}|j||||dtj d¡sšt‚dS)z+!Runs hwrf_pert_ct for the weak storm case.zstage3 pert_ct_weakcs4tj |¡rt||dˆdnˆ d||f¡dS©NTrRz'%s: does not exist; will not copy to %s©rSrTrÖr rÕ©rBÚbrQr1r2ÚcpÝs   ÿzStage3.pert_ct_weak..cprúrÎrûrÏr>r¶rárªr×rxrrérù©r¤rÈr¥r§rØrÐrÙrÚrÛrÜr®N)rˆrFrÿrgrVrWr"r€©r/r8rr°r±r²r1rQr2rÙs2     þûþzStage3.pert_ct_weakcsÀ| ¡ d¡| ¡‰‡fdd„}tdˆdtdˆdtdˆd|dd ƒ|d d ƒ|d d ƒ|ddƒd}d|jdg}dd ddd d dœ}dddœ}|j||||dtj d ¡s¼t‚dS)z+!Runs hwrf_pert_ct for the gfs vortex case.zstage3 pert_ct_gfscs4tj |¡rt||dˆdnˆ d||f¡dSr4r5r6rQr1r2r8s   ÿzStage3.pert_ct_gfs..cprrQrÛrrúrÎrûrÏr>r¶rárªr×rxrrérùr9rÚ)rÝrÞr®N) rˆrFr#rÿrgrVrWr"r€r:r1rQr2rýs6        þûþzStage3.pert_ct_gfscCsv| ¡ d¡d}ddg}ddddœ}|jd k rBd |d <d d i}n| ¡ d¡d|d <d di}|j||||dd S)z"!Runs the hwrf_inter_2to2 program.zstage3 iter_2to2Úhwrf_inter_2to2rxrmrér$rï©r¤r¥r§NÚ new_ght_d02r¦r¬Údata_merge_g02z#gsi_d02 not run, interpolate to d02rðÚdata_merge_d02r®©rˆrFr’rgr¯r1r1r2r%s ÿþ  zStage3.inter_2to2cCs„| ¡ d¡d}ddg}ddddœ}|jd k rP| ¡ d ¡d |d <d di}n| ¡ d¡d|d <d di}|j||||dd S)z(!Runs the hwrf_inter_2to2 program again.zstage3 inter_2to2againr;rxrmrér$rïr<Nzstage3 inter_2to2Ú new_ght_d03r¦r¬Zdata_merge_g03z#gsi_d03 not run, interpolate to d03r÷Údata_merge_d03r®)rˆrFr“rgr¯r1r1r2r:s"ÿþ  zStage3.inter_2to2_againcCs^| ¡ d¡d}d|jg}ddddddœ}d d i}|j||||d tdd d | ¡ddS)z"!Runs the hwrf_inter_4to6 program.z stage3 4to6Zhwrf_inter_4to6rxrérïr$rû)r¤r¥r¦r§rÑr¬Údata_merge_d01r®rÏTrRN)rˆrFrÿrgr r¯r1r1r2rPs"ÿüÿzStage3.inter_4to6c Csð| ¡ d¡| ¡}d}| |¡}|jD]À}| ¡ dt|ƒ¡| ¡}|jjsŽd|}d|}d|}t||dƒdk r d|}d |}n| ¡r˜q*d }d }d |j |j j ||f} t j  |¡d ||f} |jrÚ| | k} t j j| |dq*dS)z8!Runs the hwrf_diffwrf_3dvar to update the output files.zstage3 update_3dvarrŽzstage3 update_3dvar domain=údata_merge_d%02dråz _gsi_d%02dNzdata_merge_g%02drèr$rÿr”Ú 3dvar_updaterQ)rˆrFrSrr9rŠr+Úgetattrr—r¨rÃrJrVrrrUr) r/r=rrcrrrrLrMÚgsirˆr]r1r1r2rbs8   ÿzStage3.update_3dvarcCsX| ¡}| d¡d}|j|j ¡ d¡|jg}dddœ}ddi}|j||||d d S) z"!Runs the create_trak_fnl program.zstage3 create_trackrórôrér?rÇr»r"r®Nrö©r/r=rr°r±r²r1r1r2râs þÿzStage3.create_track)N)N)N)rn)rJr_r`rar3ržr rÇrÉr”r˜r r¹rßrrrrrrr rrrrrrrrârir1r1rÂr2r5s4   $ F :#$(c@s"eZdZdZddd„Zdd„ZdS)Ú Relocationzê!This represents all three stages of the relocate. The individual stages may be accessed by rstage1, rstage2 and rstage3. The RelocationInfo object that is shared between them can be accessed by the "info" member variableNc Ks¢d|ks t‚tƒ|_| |d¡}t|||||f||jdœ|—Ž|_| |d¡}t|||||f||jdœ|—Ž|_| |d¡}t|||||f||jdœ|—Ž|_ dS)ah!Relocation constructor. Creates Stage1, Stage2 and Stage3 @param dstore the produtil.datastore.Datastore for database storage @param conf the hwrf.config.HWRFConfig for configuration info @param section the configuration section to use @param sim the hwrf.wrf.WRFSimulation describing the simulation being relocated @param domains the hwrf.wrf.WRFDomains being relocated @param taskname_pattern Pattern for generating the subtask tasknames, which is passed through make_taskname() @param kwargs passed to hwrf.hwrftask.HWRFTask.__init__() rFrm)rµrFrnr!N) r€rrFÚ make_tasknamerÚrstage1rZrstage2rÚrstage3) r/r³r‰r´rkrÚtaskname_patternr¿r r1r1r2r3˜s(  ÿÿ ÿÿ ÿÿzRelocation.__init__cCst|ƒ}t|ƒ}||S)a!!Creates the task name for relocation stage istage based on the pattern taskname_pattern. @param taskname_pattern The string format for the taskname, which must contain exactly one %d. @param istage the integer 1, 2 or 3 to substitute into taskname_pattern)r‹r9)r/rMZistager1r1r2rJÁszRelocation.make_taskname)N)rJr_r`rar3rJr1r1r1r2rI“sÿ )rIcs¬eZdZdZd'‡fdd„ Zdd„Zdd„Zd d „Zd d „Zd d„Z dd„Z d(dd„Z dd„Z dd„Z dd„Zdd„Zdd„Zdd„Zdd „Zd!d"„Zd#d$„Zd%d&„Z‡ZS))rz8!This is a HWRF task that merges the WRF analysis files.Nc  sŒ|j}|j}|dk st‚|dk s$t‚|dkr0dn| ¡|_| dkrFdn |  |d¡|_| dkrbdn |  |d¡|_| | | ¡|jdk s–|jdk s–t‚|dk rèt t |ƒj ||||j |j|f|j ||j||j|| | dœ| —Ž| ¡|_n,t t |ƒj |||| d| d|f| Žd|_| ¡f}t|d|jtj |jd¡d|_t|d |jtj |jd ¡d|_t|d |jtj |jd ¡d|_W5QRXdS) aÉ!Merge constructor @param dstore the produtil.datastore.Datastore for database storage @param conf the hwrf.config.HWRFConfig for configuration info @param section the configuration section to use @param relocate the Stage3 of the middle FGAT time relocation @param wrfinput The source of parent model data wrfinput files. @param wrfanl The source of parent model data wrfanl files. @param taskname the task name in the database @param gsi_d01,gsi_d02,gsi_d03 hwrf.gsi.FGATGSI classes for the GSI @param ges_d02,ges_d03 Ghost files for the first guess to GSI. @param kwargs passed to hwrf.hwrftask.HWRFTask.__init__ Nrmrn)r±r¶rtr·rr¼rºr»rkrrrþrr)rrr€r˜Z_gsi_d01_inputržÚ_gsi_d02_ghostÚ_gsi_d03_ghostÚset_gesr†rr3rkr¬r¡r Ú_input_storm_radiusrrrµrSrTr‚rfrrr)r/r³r‰r´Úrelocater·r¶rµr¼rºr»Úges_d02Úges_d03r¿rrrÀrÂr1r2r3Ðs|   ÿ  ÿ  ÿ  ÿûû ÿÿ  ÿÿ ÿÿ ÿÿzMerge.__init__cCs,|dk|dkkrtj d¡‚||_||_dS)z=!Sets the ges_d02 and ges_d03 first guess ghost file sources.NzjYou must specify both d02 and d03 first guess files OR neither. You cannot specify only one of the files.)r1r2r7Ú_ges_d02Ú_ges_d03)r/rSrTr1r1r2rP s ÿz Merge.set_gescCs |dk r||jdkrdS|jS)z—!Returns the wrfinput output product for the specified domain or None if no such data is available @param domain the domain of interestNrr rÄr1r1r2r˜ szMerge.get_wrfinputcCs | |¡S)zÉ!Returns the wrfanl output product for the specified domain and time or None if no such data is available. @param atime the time of interest @param domain the domain of interest)r”rÆr1r1r2rÉ szMerge.wrfanl_at_timecCsn||jdkr$|j dt|ƒf¡||jdkr8|jS||jdkrL|jS|j dt|ƒt|jƒf¡dS)z!Returns the wrfanl product for the specified domain or None if no such data is available. @param domain the domain of interestrzERequested domain %s, which is the moad. The MOAD has no wrfanl file.rmrnz;ERROR: requested domain %s, which is not in self.domains=%sN)rrˆr;r4rrrÄr1r1r2r” sÿÿÿzMerge.get_wrfanlcCsl| ¡}|jdkrJtj | d¡d¡}| d¡| d¡t|dd|dtdƒshd }|  |¡t |ƒ‚dS) zÝ!If no relocate was given, gets the storm radius file from a fix file. Also checks to see if the storm_radius file is present and non-empty, regardless of whether it came from the fix or relocate.Nr&Zhwrf_storm_radiusz2Could not get storm_radius from the relocate jobs.z9Will use the fix file $FIXhwrf/hwrf_storm_radius instead.rÏTrËzstorm_radius file is missing) rˆrQrSrTr‚rƒrÕr!r"r;r&)r/r=rÏr‡r1r1r2Úcheck_storm_radius# s  ÿÿÿ zMerge.check_storm_radiusc Cs^| ¡}|jdks|jdkr*| d¡dS|jdk r.copiercWst|j|d|ddSrrrr1r1r2r§ sz+Merge._make_plist_and_names..linkerrrrNrÏZ gsiges_d02Z gsiges_d03rFrcSsg|]}|‘qSr1r1rr1r1r2rlà sz/Merge._make_plist_and_names..c3s|]}|ˆfVqdSrúr1rr r1r2r!Ä sz.Merge._make_plist_and_names..) rˆr"rœr•r–rQrUrVrNr€rŸrOr#)r/r=rr$r%r&r1r r2r'˜ s,               zMerge._make_plist_and_namesc Cs| ¡}| d¡d}|j d¡}| d¡}tddƒD]”}d|}d|}t|ƒs6d|j|jj ||f}t j   |¡d |d |f} |j rŽ| |k} t j j| |d tj d ¡r´t d ¡tj d ¡r6t d ¡q6|jD]¤} |  ¡}d|}d|}|dkrúd}d|j|jj ||f}t j   |¡d |d |f} |j r8| |k} t j j| |d tj d ¡r`t d ¡tj d ¡rÒt d ¡qÒtddƒD]”}d|}d|}d|j|jj ||f}t j   |¡d |d |f} |j rÔ| |k} t j j| |d tj d ¡rüt d ¡tj d ¡r‚t d ¡q‚dS)z-!Runs the hwrf_diffwrf_3dvar for all domains.zrelocate stormrŽrgrnrçz gsiges_d%02dú new_ges_d%02dz%s/logs/%s_%s_gsiges_d%02d.logr•r–rQr—r˜rårærmÚ new_hdas_d01r”rèrérêN)rˆrFr‰rƒrSrër"r¨rÃrJrVrrrUrrSrTr›rœrrŠ) r/r=rrìrcrrírîrˆr]rrr1r1r2r|Ç sx   ÿ     ÿ   ÿ zMerge.relocate_stormcCsP| ¡ d¡d}ddg}d|d|ddœ}d d |i}|j||||d d S) ú@!Runs the hwrf_inter_2to1 Fortran program to interpolate fields.r^Úhwrf_inter_2to1rxrmrérærÏ©r¥r¦rÑr¬rDr®N©rˆrFrg)r/rjrr°r±r²r1r1r2r^ sÿþ zMerge.inter_2to1c Cs°| ¡ d¡| ¡}d}ddg}d|d|ddœ}d d |i}|j||||d td d ddd}| |¡}d}d } tj |¡d| |f} |jrœ| tk} tjj | |ddS)rerarfrxrrcrærÏrgr¬zges_merge_d%02dr®rZ wrfges_d02T©rárŽZ ges_merge_d02rErQN© rˆrFrgr rSrVrrrUr) r/rjr=rr°r±r²rcrLrMr]r1r1r2ra s*ÿþ  zMerge.inter_2to1gescCsD| ¡ d¡d}ddg}ddddœ}d d i}|j||||d d S) z@!Runs the hwrf_inter_2to2 Fortran program to interpolate fields.rr;rxrmrArðrd)r¥r¦r§r¬r?r®Nrhr¯r1r1r2r- sÿþzMerge.inter_2to2cCsN| ¡ d¡| ¡}d}ddg}ddddd œ}d d i}|j||||d d S)za!Interpolates gsi_d02 analysis increment to d03 and adds the increment to d03 first guessr`r;rxrnZ new_ges_d02r=Z new_ges_d03rd)ér¥r¦r§r¬rBr®NrhrHr1r1r2r`; sÿýzMerge.inter_2to3cCsb| ¡ d¡d}dddg}ddddœ}d d i}|jd k rDd |d <nd|d <|j||||dd S)z@!Runs the hwrf_inter_2to6 Fortran program to interpolate fields.rbÚhwrf_inter_2to6rxrmrðrdrÏ)r¥r§rÑr¬rCNr=r¦rAr®r@r¯r1r1r2rbM sþþ  zMerge.inter_2to6c Cs¶| ¡ d¡d}dddg}dddd d œ}d d i}|j||||d td dddtddddd}| |¡}| ¡}d}d}tj |¡d||f} |jr¢| tk} tjj | |ddS)z@!Runs the hwrf_inter_3to2 Fortran program to interpolate fields.r_rlrxrmrnr÷rAr=rÏ)r¥r¦r§rÑr¬r>r®TrirZ newghost_d02rŽrErQNrj) r/rr°r±r²rcr=rLrMr]r1r1r2r_` s0þý zMerge.inter_3to2c CsÄ| ¡ d¡| ¡}d}| |¡}|jD]”}||jdkr\|jdkr\|jdkr\| d¡q*| ¡}d|}d|}d|j|j j ||f}t j   |¡d ||f} |jr®| |k} t j j| |d q*dS) zJ!Runs the hwrf_diffwrf_3dvar program to update the output domains.rrŽrNz4Not updating MOAD: GSI is disabled for domains 2 & 3rDrår”rErQ)rˆrFrSrr’r“rÕrŠr¨rÃrJrVrrrUr) r/r=rrcrrrrLrMrˆr]r1r1r2r| s2  ÿÿÿzMerge.update_3dvar)NNNNNN)N)rJr_r`rar3rPr˜rÉr”rWrZrßrr'r|r^rarr`rbr_rrir1r1rÂr2rÍs2þ1 ' 1/?)EraÚ__all__rSr×ÚglobÚtimeÚmathrErVÚstringÚ urllib.parseÚurllibÚdatetimeÚpdbÚ subprocessÚloggingÚ hwrf.namelistr1Úhwrf.exceptionsÚprodutil.datastorerVÚ produtil.runÚprodutil.lockingÚprodutil.fileopÚprodutil.ecflowr Ú configparserr r r Ú hwrf.hwrftaskr rrrrrÚ hwrf.numericsrrrrrrrrrrrÚ produtil.cdrrr r!r"r#r$r%r&ÚobjectrrrrbrrrrIrr1r1r1r2Úsl ÿ  $Nf:b: