U  g"_@sdZdgZddlZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl Z ddlZ ddlZ ddlZ ddlZ ddlZddlZddlZddlZddlZddlZddl mZddlmZmZmZmZmZmZddlmZm Z m!Z!ddlm"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(ddlm)Z)m*Z*m+Z+m,Z,m-Z-m.Z.m/Z/m0Z0m1Z1dd lm2Z2m3Z3m4Z4m5Z5dd lm6Z6Gd d d e7Z8Gd dde8Z9Gddde8Z:Gddde8Z;Gddde8ZGddde8Z?Gdddej@jAZBGdddeBZCGdddeCZDeEd d!ZFGd"d#d#eCZGGd$d%d%eGZHe7ZIe7ZJGd&d'd'eGZKGd(d)d)eKZLGd*d+d+eCZMGd,d-d-eHZNdS).a!Runs the real_nmm or wrf executables. This module, contrary to its name, implements HWRFTask classes to run the real_nmm and WRF, including tasks to make the the wrfanl and ghost files. The uncoupled forecast is found in the WRFAtmos class. The coupled forecast is not in this module: it is hwrf.mpipomtc instead. All classes you will want to use derive from WRFTaskBase and FcstTask, classes that exist solely to simplify later code. These are the classes you will want to use: * RealNMM - to run the real_nmm program * WRFAnl, WRFAnl4Trak - 1 minute simulation to generate wrfanl files * WRFGhost, WRFGhostForPost - 1 minute simulation to generate wrfghost files * WRFAtmos - full-length uncoupled forecast * AnalysisCycle - short (1-6 hour) forecast that outputs wrfinput and wrfghost files, suitable for input to another AnalysisCycle * hwrf.mpipomtc.WRFCoupledPOM - coupled WRF+POM forecast, in the hwrf.mpipomtc module All of these classes take input from one another, or from other sources. The abstract base class FcstTask keeps track of those through its FcstTask.add_input() function. The WRFTaskBase class provides simple functions to call add_input for you. Those inputs are stored in objects of class Input2Fcst and its subclasses: * Geog2WRF (WRFTaskBase.add_geogrid()) - geogrid data from hwrf.wps.Geogrid * Met2WRF (WRFTaskBase.add_metgrid()) - metgrid data from hwrf.wps.Metgrid * WRFInput2WRF (WRFTaskBase.add_wrfinput()) - wrfinput data from RealNMM, AnalysisCycle, hwrf.relocate.Merge or hwrf.ensda.FromPriorCycle * Fort652WRF (WRFTaskBase.add_fort65()) - coupler fort.65 file from RealNMM * WRFAnl2WRF (WRFTaskBase.add_wrfanl()) - wrf analysis (time 0 restart) files from WRFAnl, WRFAnl4Trak, WRFGhost, WRFGhostForPost, AnalysisCycle, hwrf.relocate.Merge or hwrf.relocate.RelocationTask (and its subclasses) * WRFBdy2WRF (WRFTaskBase.add_wrfbdy()) - wrf boundary (wrfbdy) files from RealNMM * Prep2WRF (WRFTaskBase.add_prep_hybrid()) - prep_hybrid files from hwrf.prep.PrepHybrid FcstTaskN)NamedDir)realcwd deliver_file make_symlinkcheck_last_lines isnonempty remove_file) FileProduct UpstreamFile UNSTARTED)checkrunmpimpirunbigexeExitStatusExceptionrunbatchexe) WRFGeogridMissingWRFMetgridMissingWRFPrepMissingWRFInputMissing WRFBdyMissing WRFAnlMissingForecastInputError SetNestFailed RealNMMError) to_fractionwithin_dt_epsilonto_datetime_rel to_timedelta)ExternalWRFTaskc@s,eZdZdZddZd ddZd dd ZdS) Input2Fcsta!abstract base class of wrf/real_nmm input providers This is the abstract base class of anything that gets, or creates, input files for a WRF simulation without running another Task. For example, something that copies the geogrid output would be a subclass of Input2FcstcCs ||_dS)z!creates a new Input2Fcst with the specified src. The src is implementation-dependent and can be interpreted as desired by subclasses. If no "src" is needed, None is acceptable.N)src)selfr#r%:/lfs/h1/ops/prod/packages/hmon.v3.2.7/ush/hwrf/fcsttask.py__init__DszInput2Fcst.__init__FcKsdS)a!copies or links input files. This function is unimplemented: subclasses are expected to replace it. If just_check=True, checks to see if inputs are available, returning True if they are and False otherwise. If just_check=False, then the files are actually copied or linked. This default implementation does nothing and returns True. @return True on success, False otherwiseTr%)r$ just_checkkwargsr%r%r& get_inputsKs zInput2Fcst.get_inputsNc Cs|dkrtd}t|tjjs$t|j|j}}|sL| |j|j}}|r|r|st| dd}|stt j |}|rt|rdS|dk rd|f}||dSt|||dddSd t|jt|t|f}|dk r|||rdS||dS) a5!helper function that links data If just_check=True, checks to see if data is available, returning True if it is, and False otherwise. If just_check is False, then the file is linked from the given product to the target location (basename(product.location) if no target is provided). If the product is not yet available or has no location, then the given exception class excclass is raised. @return True on success, False otherwise @param product the produtil.datastore.Product to link @param excclass the class of Exception to raise on error @param logger the logging.Logger to use for logging @param just_check if True, just check for data, but link nothing @param target the name of the link Nz hwrf.fcsttaskbasenameTz!%s: file is empty or non-existentFloggerforcez*%s: unavailable (available=%s location=%s))logging getLogger isinstanceprodutil datastoreProductAssertionErrorlocation availablecheckmetaospathr+rwarningrstrdidrepr) r$productZexcclassr.targetr(LAmsgr%r%r& link_productWs>      zInput2Fcst.link_product)F)NF)__name__ __module__ __qualname____doc__r'r*rFr%r%r%r&r"=s  r"c@seZdZdZdddZdS)Geog2WRFan!Links Geogrid data to the current directory for a wrf or real_nmm run. This class links geogrid data to the current directory for a wrf or real_nmm run, taken from some source (src) object sent to __init__. That src must have a geodat(domain) function that can provide a Product with geogrid output data (such as hwrf.wps.Geogrid.geodat())FcKsJ|jdk rF|dk rF|j|}|rB|js0||j|t||dSdSdS)z!Links geogrid data if any is available for the specified domain. If just_check=True, checks for geogrid input data. Otherwise, links the data using Input2Fcst.link_product. Raises WRFGeogridMissing if the data is missing.Nr(F)r#Zgeodatr8r9rFr)r$r.domainr(r)pr%r%r&r*s  zGeog2WRF.get_inputsN)FrGrHrIrJr*r%r%r%r&rKsrKc@seZdZdZdddZdS)Met2WRFa!Links Metgrid data to the current directory for a wrf or real_nmm run. This class links metgrid.exe output data to the current directory for a wrf or real_nmm execution. The output data is taken from some source object, sent to the src argument of __init__. That object must have a met_at_time(ftime) function that can provide a Product for metgrid data at a specific forecast time (such as hwrf.wps.Metgrid.met_at_time())FcKsJ|jdk rF|dk rF|j|}|rB|js0||j|t||dSdSdS)z!Links metgrid data if any is available for the specified time. If just_check=True, checks for metgrid input data. Otherwise, links the data using Input2Fcst.link_product. Raises WRFMetgridMissing if the data is missing.NrLF)r#Z met_at_timer8r9rFr)r$r.ftimer(r)rNr%r%r&r*s  zMet2WRF.get_inputsN)FrOr%r%r%r&rPs rPc@seZdZdZdddZdS) WRFInput2WRFa!Links real_nmm wrfinput_d01 files the current directory for a wrf run. This class links real_nmm wrfinput_d01 files the current directory for a wrf run. It gets those files from a source object specified in the src argument to __init__. The src must have a wrfinput_at_time(ftime) function that can provide a Product for metgrid data at a specific forecast time (such as hwrf.fcsttask.RealNMM.wrfinput_at_time()).FcKsV|jdk rR|dk rR|dk rR|j||}|rN|js:||j|t|d|dSdSdS)z!Links wrfinput_d01 files. If just_check=True, checks for wrfinput_d01 data. Otherwise, links the data using Input2Fcst.link_product. Raises WRFInputMissing if the data is missing.N wrfinput_d01rBr(F)r#wrfinput_at_timer8r9rFrr$r.atimerMr(r)rNr%r%r&r*s zWRFInput2WRF.get_inputsN)FrOr%r%r%r&rRs rRc@seZdZdZdddZdS) Fort652WRFa!Links real_nmm fort.65 files the current directory for a wrf run. This class links real_nmm fort.65 files the current directory for a wrf run. The files are taken from some source object, specified in the src argument to __init__. The src must have a fort65_at_time(atime,domain) function that provides a Product for the fort.65 file. See hwrf.fcsttask.RealNMM.fort65_at_time() for an example.FcKsV|jdk rR|dk rR|dk rR|j||}|rN|js:||j|t|d|dSdSdS)ak!Links coupler input data for the specified domain and time. If just_check=True, checks for a coupler "fort.65" input product. Otherwise, links the data using Input2Fcst.link_product to the filename "fort.65" in the current working directory. Raises WRFInputMissing if the data is missing. @param atime the analysis time @param domain the wrf domain (a number, string name or WRFDomain) @param logger a logging.Logger for logging messages @param just_check if True, just check for data, otherwise link it @param kwargs ignoredNfort.65rTF)r#fort65_at_timer8r9rFrrVr%r%r&r*s  zFort652WRF.get_inputsN)FrOr%r%r%r&rXsrXcs*eZdZdZfddZdddZZS) WRFAnl2WRFa[!Links wrfanl or ghost files the current directory for a wrf run. This class links wrfanl or ghost files the current directory for a wrf run. The files come from some source object, sent to the src argument of __init__. That object must have a wrfanl_at_time(atime,domain) function like hwrf.fcsttask.WRFAnl.wrfanl_at_time()cstt||||_dS)z9!creates a WRFAnl2WRF for the specified source and domainN)superr[r'rM)r$r#rM __class__r%r&r'szWRFAnl2WRF.__init__wrfanlFc Ks|jdk r||dk r||dk r|||jkr||j||}|rx|jsD|tjt|d| ||j }|j |t |||dSdSdS)a !Links wrfanl files for the specified domain and time. If just_check=True, checks to see if there should be wrfanl files for the specified domain and time. Otherwise, links the data using Input2Fcst.link_product to the filename "fort.65" in the current working directory. Raises WRFAnlMissing if the data is missing. @param atime the analysis time @param domain the wrf domain (a number, string name or WRFDomain) @param logger a logging.Logger for logging messages @param wrfanl the beginning of the link filename. Usually this is "wrfanl", "wrfghost" or "ghost". @param just_check if True, just check for data, otherwise link it @param kwargs ignoredNz_d_rTF) r#rMwrfanl_at_timer8r9hwrfwrfbaseparse_wrf_outnamer> get_grid_idZnocolonsrFr) r$r.rWrMr_r(r)rN localnamer%r%r&r*s* zWRFAnl2WRF.get_inputs)r_F)rGrHrIrJr'r* __classcell__r%r%r]r&r[s  r[c@seZdZdZdddZdS) WRFBdy2WRFas!Links real_nmm wrfbdy_d01 files the current directory for a wrf run. This class links real_nmm wrfbdy_d01 files the current directory for a wrf run. Those files are taken from some source object specified in the src argument to __init__. The source must have a wrfbdy_at_time(atime,domain) function like hwrf.fcsttask.RealNMM.wrfbdy_at_time() FcKsV|jdk rR|dk rR|dk rR|j||}|rN|js:||j|t|d|dSdSdS)aP!Links a wrfbdy file for the specified domain and time. If just_check=True, checks for a wrfbdy input product. Otherwise, links the data using Input2Fcst.link_product to the filename "fort.65" in the current working directory. Raises WRFBdyMissing if the data is missing. @param atime the analysis time @param domain the wrf domain (a number, string name or WRFDomain) @param logger a logging.Logger for logging messages @param just_check if True, just check for data, otherwise link it @param kwargs ignorexN wrfbdy_d01rTF)r#wrfbdy_at_timer8r9rFrrVr%r%r&r**s  zWRFBdy2WRF.get_inputsN)FrOr%r%r%r&rg!srgc@seZdZdZdddZdS)Prep2WRFa!Links prep_hybrid files to the current directory for a real_nmm run. This class links prep_hybrid files to the current directory for a real_nmm run. The files must come from some source object, specified in the src argument to __init__. That object must have a products() function that behaves like hwrf.prep.PrepHybrid.products() when called with a time= and name= argument.Fc Ks|jdk r|rtt|D]}|dt|||}|dkrDdnd}dd|jj||dD}|s|rpd Std ||d f|d}|js| |j |t|d |d |ds|rd Sqn| dt tt |f|rd SdS)ah!Links prep_hybrid output files for the specified time index. If just_check=True, checks for the input product's availability. Otherwise, links the data using Input2Fcst.link_product to the filename "fort.65" in the current working directory. Raises WRFPrepMissing if the data is missing. @param times a list of integers, 0 for the initial time, 1 for the first boundary input time, and so on. @param logger a logging.Logger for logging messages @param just_check if True, just check for data, otherwise link it @param kwargs ignoredNzLook for prep at time rinitZbdycSsg|]}|qSr%r%).0rNr%r%r& Zsz'Prep2WRF.get_inputs..)timenameFzNo prep %s data for t=%sz %Y%m%d-%H%M%Sz fort.%03dirLzIWhen looking for prep data, src is none or times is false:src=%s times=%sT) r#rangeleninfor>productsrstrftimer8r9rFr=r@) r$r.timesr(r)itwhatprodr%r%r&r*Is<    zPrep2WRF.get_inputsN)FrOr%r%r%r&rj?s rjcsDeZdZdZdfdd ZddZddZd d Zdd d ZZ S)ra$!abstract base class of anything that runs or prepares input for a forecast model Abstract base class of anything that runs a forecast model, or prepares input to a forecast model. This should not be instantiated directly. It exists just to simplify code in subclasses.Nc sJ|dk s t|dkr|}tt|j|||f||d|tt|_dS)z!Creates a FcstTask object. Creates a new FcstTask object. All parameters are passed to the superclass constructor, hwrf.hwrftask.HWRFTask.__init__()N)tasknameoutdir)r6r\rr' collections defaultdictlistinputs)r$dstoreconfsectionr{rzr)r]r%r&r'us zFcstTask.__init__cCs ||jkS)a!is there any input data of this type to this task? Returns True if there is at least one input source for the specified input type. That is, if someone called add_input(typename,(something)). Returns False otherwise. @param typename the typename value that was sent to add_input @returns True if add_input was called with that typename, or False otherwise.)r)r$typenamer%r%r& has_inputs zFcstTask.has_inputcCs|j|||S)a3!add input of a specified type Adds an input of the given type typename that should be provided by the given object. The object should be a subclass of Input2Fcst. @param typename the type of input @param inobj the Input2Fcst object that will provide the input)rappend)r$rZinobjr%r%r& add_inputszFcstTask.add_inputcKs|j|fddi|S)aj!check if input data is available Checks all inputs of the given typenames to make sure link_input would work if called with the same parameters. Returns True if link_input should succeed, and False if it would fail. This is a simple wrapper around link_input with just_check=True. However, subclasses may override this to perform additional checks, such as for a coupled model. @param typenames a list of typenames that were sent to add_input @return True if all inputs are available, and False otherwise @param kwargs passed to link_input()r(T) link_input)r$ typenamesr)r%r%r& check_inputs zFcstTask.check_inputFc Ksft|tr|f}|}|D]B}|d|t|f||jkr|j|}d}d}|D]} |d| |f|d7}zd| j|fd|i|dk}|r|dt||t| fWq*n|d t||t| fWnBtt fk r} z|jd |t| fdd W5d } ~ XYnX|rXq*qX|sd |f} |rN|| dS| | t| qdS)a!link or check for inputs Links all inputs of types given in typenames (an iterable) by calling obj.get_inputs on anything sent to self.add_input. If multiple input sources are available for a given input type, then only the first one that has input is used. Do not use the just_check option: it is part of the internal implementation of check_input; if you need to just check the inputs, use check_input instead. If just_check=True, then nothing is linked. Instead, the routine just checks to see if the inputs are available. That is the same as calling check_input. However, subclasses may override check_input to check additional inputs as part of a coupled model. @param typenames a list of typenames to check, as sent to add_inputs() @param kwargs passed to Input2Fcst.get_inputs() @param just_check if True, just check for data, do not link. Do not use this argument - it is part of the internal implementation of this class. If you want to check for inputs, call check_input() instead, as subclasses may override that function to provide additional checks.z(Look for input of type %s with kwargs=%sFrzCheck %s for input of type %sr(Tz(Found input type %s in inputter #%d (%s)z0Could not get input type %s in inputter #%d (%s)z(cannot get %s files due to exception: %sexc_infoNz8%s: could not find input files of this type. Giving up.) r2r>logrrr@rr*r=rKeyErrorerror) r$rr(r)r.rZthelistfounditryZinputtererEr%r%r&rsr          zFcstTask.link_input)NN)F) rGrHrIrJr'rrrrrfr%r%r]r&rms    cs eZdZdZd?fdd Zd@ddZdd Zd d Zd d ZddZ e ddZ ddZ ddZ ddZddZddZddZddZd d!ZdAd#d$Zd%d&Zd'd(Zd)d*ZdBd,d-Zd.d/Zd0d1Zd2d3Zd4d5Zd6d7ZdCd9d:Zd;d<Ze d=d>Z Z!S)D WRFTaskBasez!base class of classes that run wrf or real_nmm This is the abstract base class of tasks that run real or WRF. The purpose of this class is simply to reduce code duplication.Tc sHtt|j|||f||||_|||_t|_ ||_ dS)a2!constructor Creates a WRFTaskBase for the specified datastore, conf and section. The wrf argument is a WRFSimulation. It will not be used directly: instead the wrf is copied, and the copy is used. If keeprun=True, then the output of the simulation, after running it, will not be scrubbed. Other keyword arguments are passed to the superclass FcstTask. @param dstore the produtil.datastore.Datastore to use @param conf the hwrf.config.HWRFConfig that provides configuration ifnormation @param section the config section in conf @param wrf the hwrf.wrf.WRFSimulation object that is being run @param keeprun if True, the simulation temporary files are not deleted @param kwargs passed unmodified to FcstTask.__init__()N) r\rr'make_wrf_WRFTaskBase__wrf make_productsZ bdyepsilon dt_epsilonr~_WRFTaskBase__run_exe_callbackskeeprun)r$rrrwrfrr)r]r%r&r's   zWRFTaskBase.__init__NcCs|dkr|}|dk sttd|dd}|ddd}d|kr\|d}n|}d|}d}t|dkr|D]}|t |7}qntd|dd}|}| dd }||dkrt|dkr| d |t |t |fn| d ||t |f| |||n | d |d d} |dd} |jdddd} |ddd} |ddd} |ddd}| dkr|dkr| dkrt | }t |}t | }nd| kr| d}n| }d|} t |d}d| kr(| d}n| }d|} t |d}d|kr\|d}n|}d|}t |d}| dkrd}n&d| krt| d}n t| }|dkr|dkr| dks|dkr2|dkr2|dkr2|dkr|dkr|dkr|jd|ddd}|jd|ddd}|dkr|dkr|dt |t |fd}d}d}np|} |}d| kr| d}n| }d|} t |d}d|kr|d}n|}d|}t |d}| dt | t | t |f|j| | ||||dnb| dkrF| dksZ| dkrz| dkrz| d| | f|| | n| dt | t | fdS)a!sets nproc_x, nproc_y and I/O server settings This is a protected member function meant to be used by the make_wrf implementation in subclasses. This class does not use it. It sets the WRF nest_pes_x and nest_pes_y, and sets I/O server settings if desired based on config options: @param nest_pes_x,nest_pes_y compute grid dimensions per domain @param nproc_x,nproc_y compute grid dimensions @param nio_groups number of I/O server groups @param nio_tasks_per_group number of servers per group @param poll_servers set to True to poll I/O servers. This generally decreases the number of I/O server groups needed for a given WRF run. Nr nio_groupsrnio_tasks_per_group0, poll_serversTzlUsing %d nio_groups. With a possible nio_tasks_per_group domain scheme up to %s domains with poll_servers=%sz3Using %d groups of %d io tasks with poll_servers=%szNot setting io_server settings.nproc_xnproc_yconfigZmultistorm_sidsz-999zdm_task_split.comm_startzdm_task_split.nest_pes_xzdm_task_split.nest_pes_yz wrf_%sstorm nest_pes_x nest_pes_yzTrying to set nest_pes x and y for %s storms, but no '[wrf_%sstorm]' section in conf file. Will set dm_task_split values to -1, which will cause wrf to decide the task mesh .zASetting dm_task_split comm_start=%s, nest_pes_x=%s, nest_pes_y=%s)comm_start_d01nest_pes_x_d01nest_pes_y_d01z1No dm_task_split so Setting nproc_x=%d nproc_y=%dz;No dm_task_split and Not setting nproc_x or nproc_y (%s,%s))rr6maxconfintconfstrstripsplitjoinrqintconfbooldebugr@Zset_io_serversrgetstrrZset_dm_task_splitZ set_nprocs)r$rr.rrZ nio_tpg_splitZ total_nio_tpgnumrrrZ multi_sidsZ comm_startrrrrrZcomm_start_splitZnest_pes_x_splitZnest_pes_y_splitZstormsZ inest_pes_xZ inest_pes_yr%r%r&_set_wrf_proc_configs                                 (z WRFTaskBase._set_wrf_proc_configcCs|S)a!creates a WRFSimulation object for this class Given the wrf object passed to the constructor, creates a new WRFSimulation that is modified for this object. For example, this class may need to add output of a restart and wrfinput file at the end of the simulation, or enable hourly output of history files. @param wrf the wrf argument to __init__)copyr$rr%r%r&rs zWRFTaskBase.make_wrfcCs|dt|S)as!adds a geogrid input source Adds an input source (via self.add_input) that will provide the output of WPS geogrid.exe. The given object must have a geodat member function which takes a WRFDomain as its argument and returns a Product to link. Returns self. @param g the src argument to Geog2WRF.__init__() @returns selfgeogrid)rrK)r$gr%r%r& add_geogrids zWRFTaskBase.add_geogridcCs|dt|S)ag!adds a metgrid input source Adds an input source (via self.add_input) that will provide the output of WPS metgrid.exe. The given object must have a met_at_time function that returns a Product to link for a specified forecast time. Returns self. @param m the src argument to Met2WRF.__init__() @returns selfmetgrid)rrP)r$mr%r%r& add_metgrids zWRFTaskBase.add_metgridcCs|dt|S)a!adds a prep_hybrid input source Adds an input source (via self.add_input) that will provide the output of the prep_hybrid program. The given object must have a products function that iterates over products for a given name='bdy' or name='init' and a time=F for a given forecast time F. Returns self. @param p the src argument to Prep2WRF.__init__() @returns selfprep)rrj)r$rNr%r%r&add_prep_hybrids zWRFTaskBase.add_prep_hybridcCs |dS)zH!returns True if prep_hybrid was requested, and False otherwise.r)rr$r%r%r&use_prep_hybridszWRFTaskBase.use_prep_hybridcCs|dt|S)a!adds a wrfinput_d01 input source Adds an input source (via self.add_input) that will provide the wrfinput output file from real_nmm. The given object must have a wrfinput_at_time(atime,domain) function that returns a Product for a given analysis time and WRFDomain object. Returns self. @param r the src argument to WRFInput2WRF.__init__() @returns selfwrfinput)rrRr$rr%r%r& add_wrfinputs zWRFTaskBase.add_wrfinputcCs|dt|S)a!adds a wrfbdy_d01 input source Adds an input source (via self.add_input) that will provide the wrfbdy output of a real_nmm run. The given object must have a wrfbdy_at_time(atime,domain) function that returns a Product to link for a specified analysis time and WRFDomain object. Returns self. @param r the src argument to WRFBdy2WRF.__init__() @returns selfwrfbdy)rrgrr%r%r& add_wrfbdys zWRFTaskBase.add_wrfbdycCs|dt|S)a!adds a coupler fort.65 input source Adds an input source (via self.add_input) that will provide the fort.65 output from real_nmm. given object must have a fort65_at_time(atime,domain) function that returns a Product to link for a specified analysis time and WRFDomain object. Returns self. @param r the src argument to Fort652WRF.__init__ fort65)rrXrr%r%r& add_fort65s zWRFTaskBase.add_fort65cCs||||||S)am!add a fort.65, wrfinput_d01 and wrfbdy_d01 input source This is a convenience function that simply passes its argument to self.add_fort65, add_wrfinput and add_wrfbdy in that order. Returns self. @param r the src argument to Fort652WRF.__init__(), WRFBdy2WRF.__init__() and WRFInput2WRF.__init__() @return self)rrrrr%r%r&add_reals  zWRFTaskBase.add_realcCsd|jf}||t||S)a!add a wrfanl input source for a specific domain Adds an input source (via self.add_input) that will provide the wrfanl output file from a prior run of wrf.exe. The given object must have a wrfanl_at_time function that returns a Product to link for a specified analysis time and domain. Returns self. @param r the src argument to WRFAnl2WRF.__init__() @param domain the domain for whom this is the wrfanl product @return self wrfanl-%s)rorr[)r$rrMror%r%r& add_wrfanls zWRFTaskBase.add_wrfanlcCsdS)as!called from constructor, creates Products This is called from the WRFTaskBase constructor. Subclasses should re-implement this method to generate internal information about what products this class can provide, so that self.products() and self.deliver_products() will function properly. The default implementation does nothing.Nr%rr%r%r&rszWRFTaskBase.make_productsc Cs|dd}|rdnd}dD]}||d}|}|sJ|d|fq|d||ft|D]J}td dtj |}|stj |rt |||dd qft |||d qfqd S) z!links or copies fixed data files Links or copies fix files needed by WRF. Will copy if the link_wrf_fix=no in this task's config section. Otherwise, the files are linked.Z link_wrf_fixTlinkr)zfix.eta_lookupz fix.trackz fix.wrf_otherr,z7%s: no conf entry for this fix file or set of fix filesz3Will %s WRF fix files from %s to current directory.z^hwrf_r-r.N)rrrr=rrglobresubr;r<r+isdirrr)r$Z link_filesZactZconfitempatternr.r#tgtr%r%r&link_fixs(   zWRFTaskBase.link_fixcCs |jddS)z0!Checks to see if all needed input is available.TrL)link_all_inputsrr%r%r&check_all_inputs0szWRFTaskBase.check_all_inputsFcCsnd|jk}d}|r6|o4|jddd|jD|d}|jD]}|oR|jd||d}q<|jD](}|ov|jd||d }|r|r`qq`d |jkr|o|jd |j|j|d }d |jkr|o|jd |j|j|d }d |jkr|o|jd |j|j|d }|j}|jD]F}||kr"d|jf}||jkr"|od|j|||j|d }q"|S)z!Links all inputs provided by the various add_* member functions. @param just_check if True, just check to see if data is available. If False, actually copy.rTcSsg|]}|qSr%r%)rlrwr%r%r&rm=sz/WRFTaskBase.link_all_inputs..)rur(r)rMr(r)rQr(r)rMrWr(rrr)rrrZbdytimesneed_all_metgridget_moadsimstartro)r$r(Zuse_prepokayrMrQMOADror%r%r&r3sn              zWRFTaskBase.link_all_inputscCsdS)zK!Returns True if all metgrid files are needed as input to this TaskFr%rr%r%r&r[szWRFTaskBase.need_all_metgridcCs|j|dS)a !Adds a callable object exe=f(self,exe) to the list of calls to make in run_exe() before executing the command. @param callback a function f(task,exe,comp,ranks) that takes a Runner "exe" as input and returns a Runner as output. The resulting Runner will be run. The comp and ranks are a list of coupled components, and a dict mapping from component to the number of ranks for that component. The special value 0 for ranks means "use all ranks". N)rr)r$callbackr%r%r&run_exe_callback_s zWRFTaskBase.run_exe_callbackcCs|}|dt|tjjs$t|jD]R}|dt|t|f|||||}t|tjjsht|dt|fq*|dt|f|S)a!Calls the callbacks provided to run_exe_callback, in the order they were provided, giving each the return value of the last. @param exe the Runner to execute @param comp a list of coupled component names @param ranks a dict mapping from component name to the number of ranks. The special value 0 for ranks means "use all ranks" @returns The Runner returned by the last callback function.zCalling run_exe callbacks.zCalling callback %s on %s...zResult of callback is %szFinal result is %s) rrrr2r3progRunnerr6rr@)r$execompranksr.cr%r%r&call_run_exe_callbacksks  z"WRFTaskBase.call_run_exe_callbacksrc Cs|dkr|dd}|}|dkrd||}|r@tt|}ntt|dd}||dgddi}t|||d}|d ||ftd d |d sd |f}| |t |n|d|fdS)a!runs the executable Runs the executable this task is responsible for running. Determines if the program ran correctly. The exename is the name of the argument in the [exe] section of the HWRFConfig. Options: @param not_allranks By default (False) all ranks are used to run the executable. Pass not_allranks=True to run on only one rank. @param runner pass a produtil.prog.Runner object if desired. This overrides any decision of what to run: the exename and not_allranks will be ignored, and whatever is supplied in runner is simply passed to produtil.run.run. @param sleeptime passed to produtil.run.run to determine how often to check the child process. By default, the sleeptime option in this task's config section is used, or if that is absent, 30 seconds. @param exename if no runner is specified, self.getexe(exename) is called to get the executable path, and it is run as an MPI program. @note The call_run_exe_callbacks() will NOT be called if the runner argument is specified and non-null. N sleeptimeT)allranksrr)r.rz%s: exit status %d rsl.out.0000zSUCCESS COMPLETErz0%s: did not see SUCCESS COMPLETE in rsl.out.0000z$%s: SUCCESS COMPLETE in rsl.out.0000) conffloatrgetexerrrrrrrrr) r$exename not_allranksrunnerrr.rstatrEr%r%r&run_exe}s$     zWRFTaskBase.run_execCsdS)z! last step before running executable Called by self.run() just before calling run_exe. The default implementation does nothing. This is intended to be overridden by subclasses.Nr%rr%r%r& final_prerunszWRFTaskBase.final_preruncCsdS)a!called immediately after cding to a scrub directory Called by self.run() after CDing to the new directory, but before doing anything else. The default implementation does nothing. This is intended to be overridden by subclasses.Nr%rr%r%r&initial_prerunszWRFTaskBase.initial_prerunc Cs.|}|j}tj|rL|d|ftj|d|rBtt ||j }t ||j|dd}z`|d|jtf||_||||||||WnBtk r }z"|jd|jt|fddW5d}~XYnXW5QRX|d |jfdS) a!run the wrf or real_nmm Performs all work needed to run the program. Creates the work directory, CD's to it, runs the initial_prerun(), link_fix(), link_all_inputs(), make_namelist(), final_prerun(), run_exe(), postrun() and deliver_products().z!%s: directory exists; will deleteWORKhwrfT)keepr. keep_on_errorz%s running in directory %sz %s failed: %srNz %s: completed)rworkdirr;r<existsr=samefilegetdirr6shutilrmtreerrrrrrrzrr7rrr make_namelistrrpostrundeliver_products Exceptioncriticalr>postmsg)r$r.ZrunhererWZrundirrr%r%r&rs>     zWRFTaskBase.runcCsdS)z!called just after run_exe() succeeds. Called by self.run() just after run_exe returns successfully. The default implementation does nothing; this is intended to be overridden in subclasses.Nr%rr%r%r&rszWRFTaskBase.postruncCs tddS)a1!delivers products to their destinations Called by self.run() after postrun(), just before CDing out of the work directory. This should deliver products to their destinations. The default implementation raises NotImplementedError. This MUST be overridden in subclasses.z9A WRFTaskBase subclass did not implement deliver_productsN)NotImplementedErrorrr%r%r&rszWRFTaskBase.deliver_productsnamelist.inputc Csn|dkr|}|jdd}|jdd}||d|j|||}t|d}||W5QRXdS)a%!generates the wrf namelist.input file Runs set_ij_start (swcorner_dynamic) to generate the i & j start locations for domain 2, then generates the namelist. @param filename the namelist.input filename @param logger the logging.Logger to log messages (optional)Nrdomlatdomlonswcorner_dynamicwt) rrgetfloatrrr storminfoopenwrite)r$filenamer.rrsZnlinr%r%r&rs zWRFTaskBase.make_namelistcCs|jS)z4!Returns the WRFSimulation object used by this task.rrr%r%r&rszWRFTaskBase.wrfcCs|jS)z!Returns the WRFSimulation object used by this task. This property has the same effect as self.wrf(), but this is a property instead. Hence, you can type self.sim instead of self.wrf()rrr%r%r&simszWRFTaskBase.sim)T)N)F)rFNN)rN)"rGrHrIrJr'rrrrrpropertyrrrrrrrrrrrrrrrrrrrrrrrfr%r%r]r&rsD           (  0! rcseZdZdZddZfddZddZdd Zd d Zd fdd Z ddZ d!ddZ d"ddZ d#ddZ ddZddZd$fdd ZZS)%RealNMMz!a HWRFTask subclass that runs real_nmm This subclass of WRFTaskBase runs the real_nmm to generate inputs for the provided WRFSimulation.cCst|jd|jtj|jdd|_t|jd|jtj|jdd|_t|jd|jtj|jdd|_ t|jd|jtj|jdd|_ dS)z!prepares products for deliver_products() and products() Generates produtil.datastore.FileProduct objects for the files that should be delivered in deliver_products() and listed by products().rSr7rhrrYN) r rrzr;r<rr{ prod_wrfinput prod_wrfbdyprod_log prod_fort65rr%r%r&rs   zRealNMM.make_productscs&tt||}|jjddd|S)aC!adds a prep_hybrid data source Modifies the simulation namelist generator so that it specifies use_prep_hybrid=T. Calls WRFTaskBase.add_prep_hybrid(p) to add the specified object as a prep_hybrid data source. @param p the src parameter to Prep2WRF.__init__() @return selfdomainsrT)r\rrrnlnl_set)r$rNretr]r%r&rs zRealNMM.add_prep_hybridcCs|}|jdd|j|S)aA!creates a WRFSimulation object for this class Creates a copy of the specified WRFSimulation wrf. The copy has a simulation length of one minute or one physics timestep, whichever is larger. Also, it produces wrfanl files for all domains. @param wrf the wrf argument to __init__rr)rrrrrr%r%r&r*szRealNMM.make_wrfcCsdS)zK!returns True if all metgrid files are needed as input to this TaskTr%rr%r%r&r5szRealNMM.need_all_metgridc Csb|}dD]P}zt||Wq tk rZ}z|jd|t|fddW5d}~XYq Xq dS)z)!deletes the wrfinput_d01 and wrfbdy_d01.)rSrhz8%s: cannot remove, but continuing anyway. Error was: %sTrN)rr EnvironmentErrorr=r>)r$r.frr%r%r&r9s zRealNMM.initial_prerunreal_nmmrc s6|}|dt|j|dd}|dt|td|}|dt||d|ft|D]}z2tt |j |d|j k|d |d Wq2Wqnt t tfk r.}zd|d|kr|jd |d|t|fd d t||dn|jd|t|fd d W5d}~XYqnXqndS)a!runs the real_nmm program Tries several times to run the real_nmm. This is a workaround for problems encountered on Zeus when running the experimental 9:3:1 HWRF. The real_nmm would exit for no apparent reason about 50% of the time. Eventually that was tracked down to a memory error caused by NetCDF, which forced us to use intio for some files, NetCDF for others, and PNetCDF for yet more files. @param exename the name of the real_nmm executable in the configuration file "exe" section. @param sleeptime passed to produtil.run.run to determine how often to check the child process.zreal section is max_triesrzreal max_tries is rz$after max(1,...), real max_tries is zWill try to run real %d timesr)rzReal succeeded. Hurrah!zEReal failed %d time(s). Will retry after %d second sleep. Error: %sTrz-Returned from sleeping. Will now retry real.z-Real failed %d time(s). Aborting. Error: %sN)rrrr@rrrr>rpr\rrrrrrr=rnsleepr)r$rrr.ZmaxtriesZtrynumrr]r%r&rBsL       zRealNMM.run_execCs|jS)zC!returns the wrfinput file regardless of the time or domainrrr%r%r& get_wrfinputmszRealNMM.get_wrfinputNcCs@|r||krdS|dk rr$rMr%r%r& get_wrfanls  zWRFAnl.get_wrfanlcCs@||jkrdS||krdS||}|r6dS|j|S)a'!get a wrfanl file for a specified domain and analysis time Returns the wrfanl product for the specified domain and analysis time, if one exists, and otherwise, None. @param domain the domain (an integer, string name or WRFDomain) @param atime the analysis time N)r<rrBr>)r$rWrMdr%r%r&r` s   zWRFAnl.wrfanl_at_timecCs~|}|d|jfd}|D]R}||jkr6q&|j|}|r^d}tjtj |j |j tj |j |ddq&dS)z!delivers products during a call to run() Delivers the products (FileProduct objects) that were identified by make_products.z%s: make directoryTFr#r.rN)rrrr{rr>r3r$r%r;r<r&r7r'r+)r$r.firstrMrNr%r%r&rs   zWRFAnl.deliver_productsNccsD|r||jkr@|j|Vn$|D]}||jkr$|j|Vq$dS)z!iterates over products Iterates over all Product objects that were identified by make_products. @param domain if provided, only this domain's Products are yieldedN)r>rrCr%r%r&rs's    zWRFAnl.products)N) rGrHrIrJrrr<rrDr`rrsr%r%r%r&r+s   r+cs(eZdZdZfddZddZZS)WRFGhostz!runs a short WRF simulation to generate wrfanl files named "ghost" This is the same as the superclass, WRFAnl, but it renames wrfanl files to "ghost" instead of "wrfanl". Also, history output is disabled.cs0tt||}|d|jddddd|S)z!makes the WRFSimulation object for this class. Creates a copy of the WRFSimulation object wrf. This first calls the WRFAnl.make_wrf, and then disables the history stream. @param wrf the wrf argument to __init__ghost_dr/i0*i~)r0r.start)r\rHrset_wrfanl_outnamer9rr]r%r&r<s zWRFGhost.make_wrfcCs:|dkr dS||jkrdS|j|}|r0dS|j|S)zB!same as get_wrfanl() @param domain the domain of interestNrArCr%r%r& get_ghostGs  zWRFGhost.get_ghost)rGrHrIrJrrLrfr%r%r]r&rH5s rHcsfeZdZdZedffdd ZddZddZd d Zd d Z fd dZ dddZ dddZ Z S) WRFAnl4Traka!runs wrfanl and generates a track file This subtask of WRFAnl modifies the time of the 1 minute forecast wrfout file from the outer domain to have the analysis time instead so that the tracker can be run to get the initial storm location.ztrankin_dc s*||_||_tt|j||||f|dS)aF!constructor Constructor for WRFAnl4Trak. @param trakdoms what domains should be used for the tracks? either JUST_MOAD or ALL_DOMS or a list of WRFDomain objects @param trakname the track output filename. @param dstore,conf,section,wrf,kwargs passed to superclass constructorN) _trakdoms_trackin_name_patternr\rMr'r$rrrrZtrakdomsZtraknamer)r]r%r&r'gs zWRFAnl4Trak.__init__ccsR|jtkr|jVn6|jtkr6|jD] }|Vq(n|jD]}|j|Vq". That file is intended to be used to generate the parent domain vortex information.rr/streamN)r+rrr= trackin_name trackin_prodrOrSrdrarbrcrZ get_nocolonsr;r<rr{r rrzr7)r$rZtrackin_name_patternrMZidomror@ryr%r%r&r~s&    zWRFAnl4Trak.make_productscCs|D]}||qdS)a!modifies trakin_* files to say they're at time 0 Calls retime_wrfout for all domains whose trackin_d0X files are requested. This produces the modified 1 minute forecast wrfout files that lie and say they're from the analysis time.N)rS retime_wrfoutrCr%r%r&rs zWRFAnl4Trak.postrunc Cs>d}|}|}||}|j|}|dt|dt||j|||d}|dt|| |d}|dkr| d||ft ||d Sz8t ||tt|d ||d fWnBtk r8} z"|d ||t| fW5d } ~ XYnXd S) a !modifies a trakin file's internal timestamp If possible, modifies the stated output time in the one minute forecast trackin_d file to the analysis time. Does this for one domain. For intio files, it does not modify the file, but instead simply renames it. That is done because, at last test, the post does not actually look in an intio wrfout file for the time, so no modification is necessary. @param domain the domain whose trakin file is being modifiedr/z simend = z = rz infile = drz^%s: renaming instead of running wrfout_newtime since file is (probably) not NetCDF: io_form=%dNZwrfout_newtimez%Y-%m-%d_%H:%M:%Sz6%s (from %s): unable to modify time in NetCDF file: %s)rrrUrrr@r8r> get_outputr< io_form_forr=r;renamercopy2r rrrrtrr) r$rMrTr.rZdomroinfileio_formrr%r%r&rWs2  . zWRFAnl4Trak.retime_wrfoutcs@tt||jD]"\}}|j|j||ddqdS)z!delivers products Delivers all products (FileProduct objects) created by make_products, including the new trackin_d added by this subclass, and all products added by the superclass WRFAnl.FrFN)r\rMrrVitemsr'rUr)r$rErNr]r%r&rs  zWRFAnl4Trak.deliver_productsNcCs*|dkr|j}||jkr&|j|SdS)aV!returns a trakin (trackin) Product Returns a trakin (trackin) Product. If a domain is specified, returns the trackin file for that domain. Otherwise, the MOAD (outermost domain) is assumed. @param domain None, or the domain of interest @return a Product for the trakin file for the specified domainN)rrrVrCr%r%r& get_trackins    zWRFAnl4Trak.get_trackinccs|s&|s&|s&tj||dD] }|Vq|r6|dkr6dS|rN||krNdS|jD]8\}}|j|}|rx||krxqX|dk r||krqX|VqXdS)aB!iterates over all products from this task. Iterates over all products from this task. This class adds the trackin_d0* files. @param domain the domain of interest @param domains a list of domains of interest @param time the analysis time @param stream the output stream)rMr/N)r+rsrrrVr_r)r$rMrrnrTrNrEZsimdr%r%r&rss    zWRFAnl4Trak.products)N)NNNN)rGrHrIrJrQr'rSrrrWrr`rsrfr%r%r]r&rM`s   " rMcsBeZdZdZedffdd ZfddZddZd d ZZ S) WRFGhostForPosta!runs wrf to generate ghost (wrfanl) and wrfout files This class generates wrfghost files, and wrfout files, for each domain. The wrfout files happen at the end of a roughly one minute forecast so that they can be correctly post-processed. However, their internal timestamp has been changed to be for the analysis time. This class derives from WRFAnl4Trak instead of WRFGhost to reuse the wrfout renamer functionality, but it may be used in place of a WRFGhost object. Note that a wrf ghost file is a wrfanl file. There is no difference whatsoever, except in the nomenclature in the HWRF initialization.zghout_dc s"tt|j||||||f|dS)zh!constructor Creates a WRFGhostForPost, passing all arguments to WRFAnl4Trak.__init__()N)r\rar'rPr]r%r&r' s zWRFGhostForPost.__init__cstt||}|d|S)zS!creates a WRFSimulation that calls its wrfanl files "ghost" files instead.rI)r\rarrKrr]r%r&rs zWRFGhostForPost.make_wrfcCs|dk s t||S)z;!returns the ghost wrfout product for the specified domain.N)r6r`rCr%r%r& get_ghouts zWRFGhostForPost.get_ghoutcCs:|dkr dS||jkrdS|j|}|r0dS|j|S)zp!same as get_wrfanl() This works exactly like get_wrfanl() @param domain the domain of interestNrArCr%r%r&rLs  zWRFGhostForPost.get_ghost) rGrHrIrJrRr'rrbrLrfr%r%r]r&ras  racseZdZdZdfdd ZfddZdd Zfd d Zdd dZddZ ddZ ddZ dfdd Z ddZ ddZZS) WRFAtmosa!The uncoupled HWRF forecast Task. This class runs an atmosphere-only WRF run, using wrfanl files from a prior WRFAnl simulation. It encapsulates an ExternalWRFTask, which performs the actual tracking of products. This allows the post-processors and wrf copy tasks to keep track of the model's output while the model is running. That subtask shows up as ".mon" (for "monitor") relative to this task (so if this task is wrfatmos, then the external task is wrfatmos.mon).Tauxhist1c sh||_tt|j||||fd|i|tjj|j|j|j ||j d|j |j d|_ |j |j d<dS)a!WRFAtmos constructor The constructor for WRFAtmos. @param dstore the produtil.datastore.Datastore object @param conf the HWRFConfig object for configuration information @param section the section name within that HWRFConfig object @param wrf the hwrf.wrf.WRFSimulation that is to be executed @param keeprun True means the output directory should NOT be deleted @param wrfdiag_stream stream that generates wrfdiag files @param kwargs passed to the parent class constructor.rz.mon)rrrrrzr7r{r{N) Z_wrfdiag_streamr\rcr'rarr!rrrrzr _WRFAtmos__ex)r$rrrrrwrfdiag_streamr)r]r%r&r'3s zWRFAtmos.__init__cs0tt||}||}||||S)a!creates a WRFSimulation for an uncoupled forecast run Creates a WRFSimulation for an uncoupled forecast run. Runs the superclass WRFAtmos.make_wrf. Instructs the resulting WRFSimulation to take analysis files as input, and calls the _set_wrf_proc_config to set I/O server and compute grid dimensions. @param wrf the wrf argument to __init__)r\rcr analysis_inrr)r$rr.r]r%r&rKs  zWRFAtmos.make_wrfcCs|jdS)zq!deletes output files Deletes output files. See the hwrf.wrf.ExternalWRFTask.unrun for details.N)reunrunrr%r%r&rhYszWRFAtmos.unruncs t|_t|j_tt|dS)aH!runs the uncoupled WRF forecast Performs all work needed to run the program. Sets the state to produtil.datastore.UNSTARTED. Creates the work directory, CD's to it, runs the initial_prerun, link_fix, link_all_inputs, make_namelist, final_prerun, run_exe, postrun and deliver_products.N)r staterer\rcrrr]r%r&r_sz WRFAtmos.runFcCs|jj||dS)a!Determines the last output time at which all streams have completed their output. Asks the underlying hwrf.wrf.ExternalWRFTask.last_completed_time() the last time at which all streams have completed their output. If check=True, then all products are checked, otherwise cached information is used. @param streams a list of WRF stream names @param check if True, call produtil.datastore.Datum.check() on any products that are unavailable. Otherwise, cached information is used. @returns None if no times are complete, or a datetime.datetime of the last forecast time at which all streams are complete. )r9)relast_completed_time)r$streamsr9r%r%r&rjjszWRFAtmos.last_completed_timecks|jjf|D] }|VqdS)z!iterates over all WRF products. Iterates over all WRF products. See the hwrf.wrf.ExternalWRFTask.products() for details. @param kwargs passed to hwrf.wrf.ExternalWRFTask.products()Nrersr$r)rNr%r%r&rszszWRFAtmos.productscks|jjf|D] }|VqdS)z!iterates over all WRF products. Synonym for products(). Iterates over all WRF products. See the hwrf.wrf.ExternalWRFTask.products() for details. @param kwargs passed to hwrf.wrf.ExternalWRFTask.products()Nrlrmr%r%r& exproductsszWRFAtmos.exproductscKs|jjf|dS)a!checks the status of the WRF simulation. Checks the status of the WRF simulation. Should only be used while the simulation is running. This is intended to be run by jobs other than the WRF job, such as the post-processors, to monitor the WRF simulation as it progresses. @param kwargs passed to hwrf.wrf.ExternalWRFTask.wrf_check()N)re wrf_check)r$r)r%r%r&ros zWRFAtmos.wrf_checkrNcs0|dkr|dd}tt|j||||ddS)a-!runs the wrf program Called from run(), this runs the wrf program after the directory is set up. @param exename sent to getexe() to get the path to wrf if runner is unspecified @param not_allranks if True, only use one MPI rank. Do not use. @param runner an alternative produtil.prog.Runner to execute. @param sleeptime seconds to sleep between checks of the wrf executables completion. The default sleeptime, if none is specified is 60 seconds rather than the usual 30.Nr<)rrrr)rr\rcr)r$rrrrr]r%r&rs  zWRFAtmos.run_exec Cs.|j}|j|jj|_W5QRXdS)a!checks the wrf state and updates it in the HWRF database file Checks the state of the WRF simulation and copies that information to self.state in the produtil.datastore.Datastore. See hwrf.wrf.ExternalWRFTask.update_state() for details.N)r transactionre update_stateri)r$rwr%r%r&rrs  zWRFAtmos.update_statecOsdS)ab!does nothing Has no effect. This is present only because it is a requirement of the parent class. No delivery is required because the products are all UpstreamFile objects, so the delivery state is set by the post-processors when calling the "check" function of each product. @param args,kwargs ignoredNr%)r$argsr)r%r%r&rszWRFAtmos.deliver_products)Trd)F)rFNN)rGrHrIrJr'rrhrrjrsrnrorrrrrfr%r%r]r&rc)s      rccseZdZdZdfdd ZddZedd Zed d Zed d Z ddZ dddZ ddZ ddZ dddZdfdd ZZS) AnalysisCyclea!runs wrf for an analysis cycle This class is similar to a WRFGhost run, except that it runs a longer simulation (typically 1-6 hours), and provides wrfghost and wrfinput files at the end of it. It also requires wrfghost and wrfinput files as input. Hence, one can run another AnalysisCycle after this one, using the output of the previous. Note that this implementation relies on the fact that wrfghost, wrfanl and restart files are all exactly the same file format (just different times and domains).NFc s>t|_|dk rt|}||_tt|j|||||f|dS)a!constructor for AnalysisCycle Constructor for the AnalysisCycle class. @param dstore the produtil.datastore.Datastore to use @param conf the hwrf.config.HWRFConfig that provides configuration ifnormation @param section the config section in conf @param wrf the hwrf.wrf.WRFSimulation object that is being run @param keeprun if True, the simulation temporary files are not deleted @param simlen simulation length in seconds @param kwargs passed unmodified to FcstTask.__init__()N)r=_AnalysisCycle__prodcacher _AnalysisCycle__simlenr\rtr')r$rrrrsimlenrr)r]r%r&r's zAnalysisCycle.__init__cCst|_|}|}|D]\}d|f}tj|j |}||krft |j ||j |d|_ q$t |j ||j |d|j|<q$dS)z!called from constructor, creates Products This is called from the WRFTaskBase constructor. It creates products that will be used by deliver_products() and products()wrfinput_d%02drN)r=r>rrrrdr;r<rr{r rrzr)r$rr.rMr?r@r%r%r&rs&   zAnalysisCycle.make_productscCs |S)z!analysis cycle end time The time at end of the analysis cycle, at which the output wrfanl and wrfinput files are available.)rr8rr%r%r& anlouttimeszAnalysisCycle.anlouttimecCs |S)z!analysis cycle start time The time at beginning of the analysis cycle, at which the input wrfanl and wrfinput files must be provided.r;rr%r%r& anlintimeszAnalysisCycle.anlintimecCs |jdkrt|d|_|jS)z!simulation length The requested length of the simulation. Note that this may not be the same as the actual simulation length (self.anlouttime-self.anlintime) due to the model timestep.Nrw)rvr rrr%r%r&rws zAnalysisCycle.simlencCs|}|t|j}|j|d|jd|||d|jd|||dd|jdd}|dksft|D] }| ds|t| dsjtqj|D]}|j d d d q| }| d s|jd |d ||dddD],}| |}|D]}| |r| |qq|S)an!creates the WRFSimulation object Called from the constructor. Generates and returns a new WRFSimulation object that meets these requirements: 1. Read in wrfanl and wrfinput files 2. Output a restart and wrfinput file after self.simlen time 3. Disable history and auxhist 1-3. @param wrf the wrf argument to __init__r-restart)r0rJr.inputoutzwrfinputout_d)r0rJr.ZoutnameZ time_control input_outnamer,Zntrackir/r)r*)rJr0r.)rdZauxhist2Zauxhist3r/)rrgr rwr7r9rr1r6Z has_outputrrrZZ hide_output)r$rrwr}rMr.rTr^r%r%r&rs0        zAnalysisCycle.make_wrfcCs:|r||krdS|dk r6t||j|jr6|jSdS)a,!the wrfinput_d01 output Product for the specified time and domain @return the wrfinput output Product for the specified time and domain or None if that time and domain are not valid. @param atime the time @param domain the domain (integer, string name or WRFDomain)N)rrrr<rrr"r%r%r&rUJszAnalysisCycle.wrfinput_at_timecCs|jS)zN!the wrfinput_d01 output product @returns the output wrfinput Productr rr%r%r&r!WszAnalysisCycle.get_wrfinputc Cs`|}|}|j}|dt|ftjj|j |d|D]}d| f}t j |j |}||kr|jd||d }|dt||f|jj||ddqH|jd||d }|d t||f|j|j||dd|d D]\}| }|d t||ft j |j |} |d t|| ftjj|| d |d qqHdS)z!delivers products from temporary areas Copies output products from temporary areas to long-term storage locations.z-Deliver products: rst and wrfinput at time %srrxr|zDeliver moad %s from %sFrFr{zDeliver nest %s from %sr/zDeliver nest %s to %sT)rr.N)rrrryrrr@r3r$r%r{rdr;r<rrYr>rr'r> get_outputsr) r$rr.whenrMr?r@ZfromlocoutZtolocr%r%r&r\sD  zAnalysisCycle.deliver_productsc Cs||jkr|j|S|}|d}|dk s0ttj|tj|}|j.}t|j|j ||d}| |d<||d<W5QRX|r||_ ||j|<|S)a)Converts a WRFOutput to a Product. Returns a Product for a WRFOutput. The Product is cached in self.__prodcache and as_product() will return that cached value if one is available. Otherwise, a new one is created. Call clear_cached_products() to clear the cache..r{N)categoryprodnamer7rTr7) rur<r6r;rr+rrqr rzrTr7)r$Zwrfoutrelocaterelr{r@rwZufr%r%r& as_product|s"       zAnalysisCycle.as_productc #s|dk r|D]p}|dkr:||D]}|j||dVq"q |dkrd||D]}|j||dVqLq |j||||dVq nZ|dkr|jVn,||kr|jVdS|dkr|jVtt| |D] }|VqdS)au!iterates over all selected products Iterates over all products for the specified domain, or all products if the domain is unspecified or None. @param domain the requested domain (an integer, string name or WRFDomain) @param domains a list of domains of interest @param time the analysis time @param stream the output streamN)r) Zget_all_outputsrr~rYrrrr\rtrs) r$rMrrTrnrrErrNr]r%r&rss(   zAnalysisCycle.products)NF)NN)F)NNNNF)rGrHrIrJr'rrryrzrwrrUr!rrrsrfr%r%r]r&rts"    7  rt)OrJ__all__r;r|rrr4 fractionsrsysrnr0 produtil.cdr3produtil.fileopprodutil.datastore produtil.run produtil.prog hwrf.hwrftaskrahwrf.exceptions hwrf.numericsZhwrf.wrfZ hwrf.wrfbaserrrrrrr r r r r rrrrrrrrrrrrrrrrrrr r!objectr"rKrPrRrXr[rgrjhwrftaskHWRFTaskrrrFractionr6r+rHrQrRrMrarcrtr%r%r%r&sP$P   $, K ..}$L ]".