U  g@sdZgZddlZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl Z ddlZddlZddlZddlmZddlZddlZddlZddlmZddlmZmZmZmZmZmZmZddl m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&ddl'm(Z(m)Z)m*Z*m+Z+ddl,m-Z-m.Z.dd lTdd lm/Z/m0Z0m1Z1m2Z2dd lm3Z3e!d d dddgZ4e!dddddddddg e!dddddddd d!d"d#d$d%d&d'd(ge!d)d*ge!d+dd,ge!d-gd.Z5Gd/d0d0eZ6Gd1d2d2e6Z7Gd3d4d4e6Z8Gd5d6d6e8Z9Gd7d8d8e6Z:dS)9a!This module contains Tasks to run the WRF Preprocessing System (WPS): Geogrid, Ungrib and Metgrid. This module contains wrappers round the WRF Preprocessing System executables ungrib.exe, metgrid.exe and geogrid.exe. The Geogrid and Metgrid classes handle their corresponding programs, while there are two options for ungrib. The Ungrib and SteppingUngrib are wrappers around ungrib which function in different ways. The Ungrib class runs ungrib on all GRIB files at the same time, after all are available (the "traditional way"). The SteppingUngrib runs ungrib on one file at a time, allowing it to run in parallel with the parent model.N)set_ecflow_meter)HWRFTask)WPSErrorUngribSubsetErrorGeogridNoOutputTimestepTooShort GeogridNoLog UngribNoInputUngribInputUnknown)partial_ordering TimeArray to_timedeltato_datetime_rel to_fraction to_datetime) FileProduct COMPLETEDFAILEDRUNNING)TempDirNamedDir)*)runstrcheckrunaliasbigexe) defaultdictsharegeogridungribmetgridmod_levswrf_coremax_dom start_dateend_dateinterval_secondsio_form_geogridio_form_metgridnocolons parent_idparent_grid_ratioi_parent_startj_parent_starte_wee_sn geog_data_resdxdymap_projref_latref_longeog_data_pathopt_geogrid_tbl_pathZref_xZref_y out_formatprefixfg_nameopt_metgrid_tbl_pathZpress_pa)rrrr r!cseZdZdZd/fdd ZddZeedddZdd Zd d Z ee ddd Z d dZ ee dddZ ddZ ddZddZddZee dddZddZd0ddZddZd d!Zd1d"d#Zd$d%Zd2d&d'Zd3d)d*Zd+d,Zd-d.ZZS)4WPSTaska[!This subclass of HWRFTask represents a WPS Task. Multiple WPS jobs run in the same work directory. This class allows the jobs to be represented together as a set, with one work directory specified at the top level. This class exists only to reduce code duplication by letting Metgrid, Geogrid and Ungrib share a superclass.Nc  sh|dkr |}tt|j||||f| |j|_|_fdd|D|_d|_d|_| dk rt | t st dt | j t| f| |_tj||d|_|dkr}t||_| dkrƈ} t| ||_| dk r| nd|_t|j|_| dkr|} | |_|jtdkr&tdt|jf|d |j t!|jt!|jt!|jf|"|#dS) aZ!Create a new WPSTask. @param dstore The produtil.datastore.Datastore for database information @param conf the hwrf.config.HWRFConfig for configuration info @param section the section to use in conf @param sim The "sim" is the WRFSimulation for which we are preparing input. @param domains The domains must be a list of WRFDomain objects. @param taskname Optional: the name of this Task. Default: config section name. @param wpsdir Optional: the directory in which to run WPS. Default: taskname subdirectory of conf.getdir("WORKhwrf") @param starttime,endtime Optional: simulation length. Default: same as sim. @param increment Optional: boundary input interval. Default: get from sim. @param parent_atime Optional: analysis time of parent model. Default: starttime @param geogrid_from Optional: a Geogrid object whose output should be linked before running this WPS step. This is used when the WPS Ungrib and Metgrid are run more than once on the same domain. For example, one may run multiple analysis times or multiple forecast lengths off of the same geogrid output. @param kwargs Other options are sent to the hwrf.hwrftask.HWRFTask.__init__ constructor. Ncsg|] }|qSr=).0domainsimr=5/lfs/h1/ops/prod/packages/hmon.v3.2.7/ush/hwrf/wps.py sz$WPSTask.__init__..zXThe geogrid_from parameter to WPSTask.__init__ must be a Geogrid. You provided a %s %s.namelisti`Ti,z0Geogrid timestep %s is smaller than 300 seconds.z&%s times: start=%s end=%s increment=%s)$superr<__init__workdirlocation_sim_domains _products _geogrid_from isinstanceGeogrid TypeErrortype__name__reprhwrfrD Conf2Namelistconfstrnlsimstartr starttimesimendrendtime incrementr parent_atimerlogdebugtasknamestr _wps_namelist make_products)selfdstoreconfsectionrAdomainsr_ZwpsdirrXr[rZr\Z geogrid_fromkwargs __class__r@rBrFosT      zWPSTask.__init__cCs|j}|S)z!Guesses whether we should use colons in filenames. This information is obtained from sim.get_nocolons(). Note that this requires re-scanning all input and output io_forms, so this function is moderately expensive.)rI get_nocolons)rcncr=r=rB_guess_nocolonss zWPSTask._guess_nocolonszgA property that is True if WPS will omit colons from the output filenames, and False otherwise.cCs\|dks|dkr|jdS|dks,|dkr8|jdS|d|f|jdSdS)aJ!Guesses the file suffix (.nc, etc.) for the specified stream. Tries to guess what io suffix WPS will use for a stream. If the stream is not one of the known streams, this function logs a warning and tries to continue, but will likely fail. @param stream the stream of interest. r auxinput1r auxinput2z9caller requested unknown stream "%s" in WPSTask.io_suffixinputN)rI get_io_suffixr]warningrcstreamr=r=rB io_suffixs   zWPSTask.io_suffixcCs |jS)z6!The number of WRF domains, and highest domain number.)rImaxdomrcr=r=rB_maxdomszWPSTask._maxdomzThe number of WRF domains.cCs |jdS)z*!The mother of all domains as a WRFDomain.)rIgetrwr=r=rB_MOADsz WPSTask._MOADz(Returns the Mother of All Domains (MOAD)cCs|jS)z;!The WRFSimulation object for which we are preparing input.)rIrwr=r=rBgetsimszWPSTask.getsimcCs2|d}d|f}tj|s.td|fdS)z!Checks to see if the geogrid MOAD output file is present and non-empty in the current working directory. Raises GeogridNoOutput if the output is missing or empty.rzgeo_nmm.d01.%s%s: does not exist or is emptyN)ruprodutilfileop isnonemptyr)rcsuffixfilenamer=r=rB check_geogrids    zWPSTask.check_geogridc Cs6|}|j}|dkr$|ddS|d|jt|j|jf|D]}|j}|j}| ddkr|||d|n||dqJ|j }|j }d}|s|d}nV|dks|d kr|d }nWs z0WPSTask.make_product_structure..cstjjjjSr)rSrr rXrZr[r=rwr=rBr[sN)rZrX collectionsrrKrsr=rwrBmake_product_structureQs  zWPSTask.make_product_structurecCsdS)a!This subroutine should be implemented in subclasses. It should call self.make_product_structure(), and then add products by doing: @code self._products[domain.moad_ratio()][time]=product @endcode where the domain is a WRFDomainBase subclass, the time is anything accepted by to_datetime_rel's first argument, and the product is any subclass of Product.Nr=rwr=r=rBrb]szWPSTask.make_productsccs|jdk r|dk r||jkr|j|}|dkr`|D]\}}|}|dk sVt|Vq>q||kr||}|dk s|t|Vnb|dk rtjd}|jD]B}|D]4\}}|dk rtj|||sq|dk st|VqqdS)z!Iterates over all products @param domain the hwrf.wrf.WRFDomain @param time the time @note One, both or neither argument can be specified. All matching products are yielded.N ) rK moad_ratiovaluesAssertionErrorrSrr itemswithin_dt_epsilon)rctimer?ZtprodrZ localprodepsilonZthetimer=r=rBrgs2      zWPSTask.productscCs|jttS)z/!Returns the namelist.wps contents as a string.)rV remove_traits set_sorters_wps_namelist_order_wps_nl_var_order make_namelistrwr=r=rBrs zWPSTask.make_namelistcCs@|}|j}dd|j||dD}|D]}|j|dq*dS)z!Deletes all delivered products and marks them as unavailable. @param time,domain passed to products() to get the list of products @param fromloc Ignored.cSsg|]}|qSr=r=r>pr=r=rBrCsz.WPSTask.undeliver_products..rr?rN)r]rXr undeliver)rcrr?fromlocrprodlistrr=r=rBundeliver_productss zWPSTask.undeliver_productsTc Cs|dd}|o|}|}|dkr*|j}d}|dkr>tjj}dd|j||dD} | D]} z| j} | dks|t| dkrd}d | j f} | | t | || } t | t st| j| ||d |s0tj| s0| }| j}|d ||f|r|d ||ft||d|d n|d||fWqZtk rn}z|j d| fddd}W5d}~XYqZXqZ|r|ddS)a>!This is called from self.run to deliver all products to the intercom directory. @param time,domain The optional time and domain arguments are sent to self.products. @param fromloc By default, this routine assumes that the file to be delivered is in the current working directory with the same name as the destination file. To change that, give a lambda function in "fromloc", which converts the destination filename (the only argument) to a local filename. @param keep The "keep" argument has the same meaning as in deliver_file: if False, the file may be moved to the destination. @param relink If True, and the file is moved, then a symlink will be made from the original file location to the destination. link_wrf_fixTNFcSsg|]}|qSr=r=rr=r=rBrCsz,WPSTask.deliver_products..rryz.%s: product has no location; cannot deliver it)frominfokeeprz$%s: file was moved to destination %sz!%s: relink from delivered file %srz"%s: not relinking. File now at %sz%s: cannot deliver fileexc_infoz1some output files were empty or missing; aborting)confboolr]rXrrbasenamerrHlenrrrrrMr`rdeliverrrrEnvironmentErrorcritical)rcrr?rrrelink link_filesrbadrrfrflZlinkfileZdestfilerr=r=rBdeliver_productssV    zWPSTask.deliver_productscCst||j}t||}dS)a!Adjusts self.nl to set the start and end times of the wps namelist. @param start the start time, a datetime.datetime or a time relative to self.start @param end the end time, a datetime.datetime or a time relative to start @note This routine does not write out the namelist. Make sure you call make_namelist() to get the namelist as a string and then use an open/write block to write it to namelist.wps. N)rrX)rcstartrr=r=rBset_namelist_timess zWPSTask.set_namelist_timescCs6|jj}|jj}|jj}|j}|jd}|j}|dd||dd||dd||ddt|j ddt|j d d }t|j d d }|d krd }|d krd }|dd||dd||dd|j g} g} g} g} g} g}g}g}g}g}|dkr dnd}|D] }|jdkrv| d| d||jd||jd|d|dn| |j| |jdt|jdd}|dkr|t|jd|t|jdn|d|d| || |jd | |jd!||q|d"d#| |d"d| |d"d$| |d"d%| |d"d||d"d||d"d&||d"d'||d"d(d)|d"d*d+|d"d,d-|d"d.||d"d/|jd0d1d2|d"d3|jd4d1d2|dd5d-|dd6d7|d8d9d7|d8d:d;|ddz!Fills the self.nl namelist with correct information. This must be called before make_namelist, and hence it is called from the constructor.%Y-%m-%d_%H:%M:%Srr#r$r%r&wrfbdysteprodrn r'r r(r)Z10mZ2mNryr1r2r+rautofixedistartjstartrnxnyrr*r.r/r,r0r3Z rotated_llr6z ./geog-data/r7z./r-r4domlatconfig)rfr5domlonr;r:ZFILErr9r8ZWPSr"ZNMM)rVnl_setnl_set_if_unset trait_getrvrXstrftimerZint_confrzrA io_form_forr)rgparentappend get_grid_idr`lower conffloat)rcssiutrvrrr'r(pidgidZpgrrrr1r2rrres resolutiondr=r=rBras                              zWPSTask._wps_namelist)NNNNNNN)FN)NN)NNN)NNNTF)rQ __module__ __qualname____doc__rFrmpropertyr)rurxrvr{MOADr|rrrrArgrrrbrrrrrra __classcell__r=r=rirBr<hsLM  4      ; r<cs8eZdZdZfddZddZddZdd ZZS) rNz!This is a HWRF task that pre processes the geogrid to define the model domains and interpolates static geographical data to the grids.cstt|j||dS)z|!Creates a new Geogrid. @param args,kwargs All arguments are passed to the WPSTask.__init__() constructor.N)rErNrFrcargsrhrir=rBrF:szGeogrid.__init__cCs|d|d}|j}|D]}|}|jdkrFd||f}nd|d|f}tj|j |}t |j ||j d|j i|d}||_|dt|jt|jf||j||<q"dS) a!Creates the FileProduct objects for this Geogrid. @note Implementation note: If you change the list of products, make sure all geogrid outputs needed as input to ungrib or metgrid have a prodname that is the same as the destination file.rNzgeo_nmm.d%02d.%szgeo_nmm_nest.l%02d.%sryr?)metarHz(geogrid made product %s with location %s)rrurXrgrrrrjoinoutdirr_dstorer_rrHr]r^rRrrKr)rcrrridrdestprodr=r=rBrb@s$      zGeogrid.make_productscCs&|}||jkst|j||jS)a1!Returns the FileProduct for the geogrid data for the specified nesting ratio. The specified domain does not have to be one of the known domains. It just has to have the same nest:parent ration (WRFDomain.moad_ratio) as one. @param dom The hwrf.wrf.WRFDomain of interest.)rrKrrX)rcdomZratior=r=rBgeodatWszGeogrid.geodatc Csd|}ztj|jt|j}|dt t dt rNt t dD]L}ztjj||dWqXtk r}z|d|fW5d}~XYqXXqX|jddd td d }||W5QRX|d }|jd }tjjtj|dd}|jr||k}|d|jt|ftjj||dd}d} dD]*} tj| r`| } n|d| fqH| dkrd} | | t!|_"t#| |d| |ft$| |st%d| |f|j&dddW5QRXWnlt%k r} z t!|_"W5d} ~ XYnBt'k rN}z"|j(dt)|ddt!|_"W5d}~XYnXt*|_"|+ddS)zK!Copies inputs, links fix files, runs geogrid and delivers results.zGeogrid running in directory: \A/tmpzgeo*rz.%s: did not remove file, but continuing anywayNTz GEOGRID.TBL)rr namelist.wpsw hwrf_geogrid.logallranks%s command: %szSuccessful completion)z geogrid.logzgeogrid.log.0000zgeogrid.log.00000z%s: does not exist.z4WPS Geogrid failed: could not find geogrid log file.z%s: will check for %sz%s: did not find "%s"F)rrzWPS Geogrid failed: rzWPS Geogrid completed.),r]r~rmakedirsrrrHrrgetcwdrematchrglob remove_filerrrropenwritergetexe_sectionrunmpirunmpiredirectr_rRrrrerrorrstatercheck_last_linesrrrrr`rpostmsg) rcrdirrrprogr]cmdfindmeZ geogrid_logZglogrZwer=r=rBr`sf      z Geogrid.run) rQrrrrFrbrrrr=r=rirBrN5s   rNcseZdZdZfddZeddZeddZdd Zdd d Z dddZ ddZ dddZ ddZ ddZddZddZZS) Ungribz!This is a HWRF task that extracts the meteorological fields from GRIB formatted files and write the fields to intermediate files. It is a wrapper around the WPS ungrib.exe program.cstt|j||d|kr.|dr.|d|_n&|dd}tj|j||j |_t | dd|_ d|krv|dn |dd}|d ks|dkr|j |_nt||j |_d |kr|d nd }|d kr|d d }t|tjr||}t|ts,t|ts,t|tjs,td |jjt|ft||_t| d|dd|_t| d|d|j|_t| d|dd|_t| d|dd|_ |!"dt|jt|jt|j#fd |_$d S)aM!Creates a new Ungrib. All arguments are passed to the WRFTask constructor. The only new arguments are: * in_ftime - Optional: the parent model forecast hour to use for analysis time data in this Ungrib. * in_item, in_dataset - Optional: the item and dataset, in hwrf.input terminology, to use to request the GRIB1/2 input files. * in_item2, in_dataset2 - Optional: same as in_item and in_dataset, but for a second GRIB file that is appended to the first. * inputs - Optional: an hwrf.input.DataCatalog to provide input file locations. Default: construct one using the catalog name from the "catalog" option in this objects config section. @param args,kwargs passed to WPSTask.__init__ inputscataloghwrfdataone_timeFin_atimeatimerNin_ftimeftimerzGin_ftime must be an int, a float or a timedelta, not a %s (in_ftime=%s) in_datasetdataset in_dataset2Zdataset2in_itemitemZgfs_pgrbin_item2Zitem2z%self.in_item=%s dataset=%s section=%s)%rEr#rFr$rUrSrp DataCatalogrerArWboolrz_Ungrib__one_timer(rconfintrMdatetimerfloat timedeltarOrjrQrRr r*r`r,r.r/r1r]r^rf _n_gribfiles)rcrrhZhdr(r*rir=rBrFsh            zUngrib.__init__cCs|jS)z!If True, we are pretending that hour 0 is valid for all times. This is equivalent to constant boundary conditions. If in_ftime is non-zero, then that hour is used instead.)r4rwr=r=rBr'szUngrib.one_timecCs|jdk r|jdkrdSdS)z!How many grib files are processed for each time? This is 2 if an item2 or in_item2 were given in the config or constructor, respectively. Otherwise it is 1.Nrrry)r1rwr=r=rBgribs_per_timeszUngrib.gribs_per_timec cs|j}|D]z}||}|jr$d}t|t|j|j}t|j|j|j ||jdV|j r|j sbqt|j|j |j ||j| dddVqdS)z!Iterates over all input files needed. This is meant to be plugged in to an hwrf.input.InputSource to obtain input data in the scripts.exhwrf_input job.r)r-r0r+r)item2_optionalF)r-r0r+r)optionalN) rXrr'rr*r(dicttaskvarsr,r/r1r.r)rcrrrr+r=r=rB inputiter s*      zUngrib.inputiterryFc Cs|jr d}|}|j}|j}t|t|j|j}|jd}|d|||d|t |j f| dt |j |dkr|j j |j|jf||jd|j } n$|j j |j|jf||jd|j } |dt | |r| dks| d krtd |||d|t |j |f| S) a!Finds input data for a specific time and GRIB file @param dt the forecast time as a datetime.datetime, relative to the in_ftime @param igrib 1 or 2, which input file is used (in_item or in_item2 This is needed for models like GFS and GEFS that split their GRIB files into two parts. @param require if True, then hwrf.exceptions.UngribNoInput is raised when inputs cannot be found.rz%Y%m%d%Hz4Check for dataset=%s item=%s ftime=%s atime=%s in %szinputs: ry)r+r)z Got back: NrzICannot find input for: dataset=%s item=%s ftime=%s igrib=%datime=%s in %s)r4r]r/r,rr*r(rrrRr$r^locater>r.r1r ) rcrigribrequirerr0r-r+Zstratimeretr=r=rBinput_atsT    zUngrib.input_atNc' CsP|}d}g}|j}|j}|dk r4t||jg}ndd|D}|D]} | |} ttt| d} | dt | t |t | t | t |f| dd} t |D]f} |j | | d d }| d kr| rq|dks|d krtd | f||| d t | t |fqqJ|jrlt|||dd|dd|dd |dddddd sl|dtdt}|jd kr| d|D]r} | |} ttt| d} |j | d d }|j | dd }||kst|}||| <| dt |t | ft|d}| dt |t |ft|d}t||W5QRXz@| dt |t |ft|d}t||W5QRXWnRtk r}z2| dd}|r|jdt|fd d!nW5d}~XYnXW5QRX| d"t |fqnP| d#|D]@} | |} |j | d d }|}||| <t||d |d$q|d%d }|d&|}|d'|}|s|rL| d(tt |!d)d)}tt |!d*}t"}t"}|rt|d+&}|D]}|r||#qW5QRXt$%d,&|} nd} |r`t|d+&}|D]}|r(||#q(W5QRXt$%d,&|}!nd}!|D]} || }"t'j()|"d-}#d}$d}%t*j+,|"}&|&dkr|r|}$|!}%n(|&d kr|r|}$| }%nt-d.|"f|$dk r:|%dk r:| d/|#|"f|.|"|#|$|%| d0|"|#ft'/|"t'0|#|"n| d1|"qhdS)2a!Link or copies all the input GRIB files to the current working directory. Note that if two grib files are requested, then this is done by copying the data. @param require if True, then hwrf.exceptions.UngribNoInput is raised when inputs cannot be found. @param at if specified and not None, then only process this time, otherwise process all timesrNcSsg|]}|qSr=r=)r>rr=r=rBrCOsz#Ungrib.get_grib..z:Need to get input for t-start=%s-%s=%s=hour %s igribmax=%sr;Fry)rArzCannot find input for hour %dzInput for hour %s is %sZ max_grib_waitiZgrib_sleep_timeZ min_grib_sizeZ min_grib_ageg?)maxwait sleeptimemin_size min_mtime_age min_atime_age min_ctime_age min_fractionz/Some input GRIB files do not exist. Giving up.z6Some GRIB files are missing. See the log for details.zMerging GRIB files.rz %s: merge GRIBs for time %s herewbz%s: copy from %srbz%s: ignoring exceptionTrz%s: donez$Not subsetting or merging GRIB filesrsubsetZ subset_grib1Z subset_grib2zSubsetting GRIB filesZwgrib2wgribrt|z .subsettedz%s: is neither GRIB1 nor GRIB2.z%s: subsetting from %sz%%s: delete and replace with subset %sz%s: not subsetting)1r]rXr:rrrmathceilrrrRrrangerDr rrealtimewait_for_filesr5rr=r _rename_gribrshutil copyfileobjrrrr`rrUrrrlistrstriprcompilerrrrr~rgribverr _subset_gribunlinkrename)'rcrBatrjfilesrZigribmaxrrrfhrZopt2rArnamesin1in2outoutfZinf1Zinf2roptZsubset0Z subset1fileZ subset2fileZcmd2Zcmd1Zsubset1Zsubset2lineZ subset1_reZ subset2_resrcfiletgtfiler!Z subset_rer`r=r=rBget_grib?s                            zUngrib.get_gribc Csd}d\}}t|||ddD](}||rF||7}|d7}q&|d7}q&|sbtd|f|d||||ft||dd d ||f|>} |d t| ft| |dt j |std |fd S)a!Runs wgrib on a GRIB1 input file to get its content listing. Then runs wgrib again to subset it. * srcfile - the input file, to scan and subset * tgtfile - the location for the new, subset file * cmd - a produtil.prog.ImmutableRunner for wgrib * matcher - a regular expression, or anything else that has a .search method. Each call to search takes one line and returns True if it should be included in the subset, or False otherwise. r)rrrTryz%s: no matching records in filez+%s => %s: keeping %d records, discarding %dz-iz-gribz-ozrun: %sz!%s: file is non-existent or emptyN) rr] splitlinessearchrrrrRrr~rr) rcrorpr!ZmatcherrQkrrnZrunmer=r=rBras2      zUngrib._subset_gribcCs:ddtjtjddD}d||j}|jd7_|S)ay!Generates a GRIB filename using the input name expected by WPS: GRIBFILE.AAA for the first, GRIBFILE.AAB for the second, and so on. An internal counter self._n_gribfiles keeps track of the number of files requested so far. The optional filename argument is ignored. @param filename Ignored. @returns the new filename chosen.cSsg|]\}}}|||qSr=r=)r>abcr=r=rBrCsz'Ungrib._rename_grib..r)repeatz GRIBFILE.%sry) itertoolsrstringascii_uppercaser9)rcrZsufsZ new_filenamer=r=rBrZs  zUngrib._rename_gribc CsN|}ztj|jt|j}|dt ft dt rLt t dd}||W5QRX|jdd||d}|jd}tj|}|jr||k}|d |jt|ftjj||d ||W5QRXWnBtk r8}z"|jd t|d d t|_ W5d}~XYnXt!|_ |"ddS)!Links inputs and runs ungrib. Ungrib has no deliverables: it only places files in the local directory for a later Metgrid.run() to pick up and use.Ungrib starting in %srrrZVtabler hwrf_ungribr r rWPS Ungrib failed: TrNWPS Ungrib completed)#r]r~rr rrrHrrrrrrrrrrrqrrrexerr_rRrrcheck_outfilesrrr`rrrr)rcrrrr r]r!rr=r=rBrs2      z Ungrib.runcCs~d}|}|D]J}d|j|df}tj|rJ|d|fq|d|fd}q|rz| dt d|d S) z_!Checks the expected ungrib output files to make sure they all exist and are non-empty.Fz %s/FILE:%sz %Y-%m-%d_%Hz%s: exists, is non-empty.r}TzeWPS Ungrib failed: some output files did not exist or were empty. See stdout/stderr log for details.z4WPS Ungrib output file %s does not exist or is emptyN) r]rrHrr~rrrrrrr)rcrrirr=r=rBrs   zUngrib.check_outfilescOsdS)zV!Does nothing. Ungrib has no products to deliver. @param args,kwargs Ignored.Nr=rr=r=rBr0szUngrib.deliver_productscksdS)a!Ungrib delivers no products. Everything is kept in the WPS temporary directory and reused by metgrid. The Metgrid implementation assumes it is running in the same directory as Ungrib, so no call to products() is needed. @param kwargs Ignored.Nr=)rcrhr=r=rBr3szUngrib.products)ryF)FN)N)rQrrrrFrr'r:r?rDrqrarZrrrrrr=r=rirBr#s  \   #  !r#c@s"eZdZdZddZdddZdS)SteppingUngribzk!This subclass of ungrib processes one input file at a time, running in parallel with its parent model.c Csr|}ztj|jt|j}|dt f| D]}||j j }t tt|d}td||||tdd}||W5QRX|j|d|d}|jd}tj|} |jr| |k} |d |jt| ftjj| |d qB|W5QRXWnBt k r\} z"|j!d t"| d d t#|_$W5d} ~ XYnXt%|_$|&ddS)r|r}rErprr)rdrr r rrTrNr)'r]r~rr rrrHrrrrrecyclerrUrVrrrrrrrqrrrrrr_rRrrrrr`rrrr) rcrrrrrgrr r]r!rr=r=rBr@s6          zSteppingUngrib.runNcCsdS)za!Returns GRIBFILE.AAA @param filename Ignored @returns the string "GRIBFILE.AAA" z GRIBFILE.AAAr=)rcrr=r=rBrZ_szSteppingUngrib._rename_grib)N)rQrrrrrZr=r=r=rBr=srcs8eZdZdZfddZddZddZdd ZZS) Metgridz!This is a HWRF task that horizontally interpolates the meteorological fields extracted by ungrib to the model grids defined by geogrid. It is a wrapper around the WPS metgrid.exe Fortran program.cstt|j||d|_dS)z~!Creates a new Metgrid. @param args,kwargs All arguments are passed to the WPSTask.__init__() constructor.rN)rErrFr9rrir=rBrFlszMetgrid.__init__c Cs|}z~tj|jt|j|j dR}|dt t dt rVt tdd}||W5QRX|||jdd|d}|jd }tjjtj|d d }|jr||k}|d |jt|ftjj||d }|D]z}|d|d|jdd |d|j |j!kr|j"d|} |j!|j#} |j"d| } t$| j| jd |dd | _%qW5QRXWnDt&k r} z$|j'dt(| fd dt)|_*W5d} ~ XYnXt+|_*|,ddS)a!Copies inputs, runs metgrid, delivers outputs. @note Ungrib must have been run in the same directory beforehand. The Geogrid must have been run in the same directory, unless it was passed as input via geogrid_from= in the constructor.)rzWPS Metgrid running in dir: rrrz METGRID.TBLr~Z hwrf_metgridr Tr r rz#%s: deliver products for this time.z%Y-%m-%d %H:%M:%SF)rrrryrzWPS Metgrid failed: %srNzWPS Metgrid completed)-r]r~rr rrrHscrubrrrrrrrrrrrrrrrrrrr_rRrrrrrZrXrKr[rrrrr`rrrr) rcrrrr r]r!okrrtimebZpbrr=r=rBrrsR       z Metgrid.runcCs$||jdkr|jd|SdSdS)z!Returns a FileProduct for the Metgrid file at the specified time or None if no such product is found @param when the time of interestryN)rK)rcwhenr=r=rB met_at_timeszMetgrid.met_at_timec Cs|d|j}|d}|jr$dnd}|D]N}d|||f}tj|j |}t |j ||j |d}||_ ||jd|<q0|j|jkr|j|j}d|||f} tj|j | } t |j | |j | d} | |jd|<dS)zl!Generates FileProduct objects for this Metgrid. This is called automatically from the constructor.r z%Y-%m-%d_%H_%M_%Srzmet_nmm.d01.%s.%s)rHryN)rrrur)rrrrrrrrdr_rHrKrZrXr[) rcrrformatrrrrrZfbZlocbZprodbr=r=rBrbs$      zMetgrid.make_products) rQrrrrFrrrbrr=r=rirBrfs  /r);r__all__rr[rrrrUrryrz urllib.parseurllibr6 hwrf.namelistrShwrf.exceptionsprodutil.ecflowrprodutil.fileopr~ produtil.runprodutil.locking hwrf.hwrftaskrrrrrrr r hwrf.numericsr r r rrrprodutil.datastorerrrr produtil.cdrrrrrrrrrr<rNr#rrr=r=r=rBs@   $    .Pe&)