U  g% ,@s"dZddlZddlZddlZddlZddlZddlZddlm Z m Z m Z m Z m Z ddlmZddlmZmZmZmZmZmZddlmZddlTddlTddlTddlTdd d d gZd dZed dddddddgdZeddddddddddd d!d"d#d$d%d&d'd(d)d*d+d,d-d.d/d0d1d2d3d4d5d6d7d8d9d:d;dd?d@g+edAdBdCdDdEdFdGdHdIdJdKdLdMdNdOdPdQdRdSdTdUdVdWdXdYdZd[d\d]d^d_ged`dadbdcdddedfdgdhdidjdkdldmdndodpdqdrdsdtdudvdwdxdydzd{ged|d}d~dddgeddgedddgedddgdZ Gdd d e!Z"Gdd d e#Z$Gdd d eZ%dS)a!@brief Create namelists, monitor wrf simulations, generate filenames. @details This module contains classes that manipulate WRF namelists in complex ways, and predict the resulting output and input filenames regardless of whatever crazy timesteps are requested. This module also contains a class, ExternalWRFTask, that can monitor a running WRF simulation, providing a list of output and input files, and check whether the simulation has completed, failed or is still running. @see hwrf.wrfbaseN) COMPLETED UpstreamFile UNSTARTEDRUNNINGFAILED) isnonempty)bigexecheckrunmpirunmpirunstrbatchexe)HWRFTask)*default_wrf_outname WRFDomain WRFSimulationExternalWRFTaskcCs||dkr dS|dkrd|fS|dkr*dS|dks:|dkrDd |fS|d krVd |fS|d krbd S|dkrndSd|fSdS)a!default wrf output filename patterns Generate a reasonable default wrf outname input value for the specified stream. Presently, these match the WRF defaults. These do not have to match the WRF defaults since we always specify the outname for all streams. @param stream the stream @returns the wrf output filename pattern, including and if relevanthistoryzwrfout_d_Zanlzwrf%s_d_restartzwrfrst_d_bdyinputzwrf%s_dZgeo_nmmz %s_dinputoutzwrfinput_d_Zbdyoutzwrfbdy_dz%s_d_N)streamrr5/lfs/h1/ops/prod/packages/hmon.v3.2.7/ush/hwrf/wrf.pyr s     time_controlfddadomainsphysicsdynamics bdy_controlnamelist_quiltloggingZ start_yearZ start_monthZ start_dayZ start_hourZ start_minuteZ start_secondZend_yearZ end_monthZend_dayZend_hourZ end_minuteZ end_secondinterval_secondsZhistory_intervalZauxhist1_intervalZauxhist2_intervalZauxhist3_intervalZ history_endZ auxhist2_endZauxhist1_outnameZauxhist2_outnameZauxhist3_outnameframes_per_outfileZframes_per_auxhist1Zframes_per_auxhist2Zframes_per_auxhist3analysis anl_outnamerrestart_intervalreset_simulation_startZ io_form_inputZio_form_historyio_form_restartZio_form_boundaryZio_form_auxinput1Zio_form_auxhist1Zio_form_auxhist2Zio_form_auxhist3auxinput1_innameZ debug_levelZtg_reset_streamoverride_restart_timers time_steptime_step_fract_numtime_step_fract_denZmax_doms_wee_wes_sne_sns_verte_vertdxdygrid_id tile_sz_x tile_sz_ynumtilesnproc_xnproc_y parent_idparent_grid_ratioparent_time_step_ratioi_parent_startj_parent_startZfeedbackZ num_movesnum_metgrid_levelsp_top_requestedptsgm eta_levelsuse_prep_hybridnum_metgrid_soil_levelsZnum_soil_layersZ mp_physicsZ ra_lw_physicsZ ra_sw_physicsZsf_sfclay_physicsZsf_surface_physicsZbl_pbl_physicsZ cu_physicsZmommixZvar_ricZ coef_ric_lZ coef_ric_sZh_diffZgwd_optZsfenthZnradsZnradlnphsZncnvcntrackZ gfs_alphaZ sas_pgconZ sas_mass_fluxZco2tfZvortex_trackerZ nomove_freqZ tg_optionZntornadoZnon_hydrostaticZ euler_advwpZcoacZcodampZterrain_smoothingZspec_bdy_widthZ specified poll_serversnio_tasks_per_group nio_groupsZcompute_tasks_silentZio_servers_silentZstderr_logging)rrrr r!r"r#c@s$eZdZdZd@ddZddZddZd d Zeeddd Z d d Z ddZ ddZ ddZ ddZddZeddZeddZeddZddZd d!Zd"d#Zd$d%Zd&d'Zd(d)Zd*d+Zd,d-Zd.d/ZdAd0d1ZdBd2d3Zd4d5ZdCd6d7Zd8d9Z d:d;Z!dDdd?Z#dS)Era!A domain in a WRF simulation This subclass of WRFDomainsBase adds needed details that let it provide information about a domain in a WRF simulation. It can predict output and input filenames based on timesteps and start/end times. It can do complex manipulations of the WRF namelist. Most functionality should be accessed via the WRFSimulation, after a WRFDomain is created. Note that after you provide a WRFDomain to a WRFSimulation, the WRFSimulation makes its own copy of that WRFDomain. The original is unmodified. That means that if you want details on the namelist and output files added by the WRFSimultion, you must obtain its copy of the WRFDomain like so: @code{.py} moad=WRFDomain(conf,'moad') storm1outer=WRFDomain(conf,'storm1outer') storm1inner=WRFDomain(conf,'storm1inner') wrf=WRFSimulation(conf,'wrf',moad,conf.cycle, conf.cycle+hwrf.numerics.to_timedelta(fcstlen*3600)) wrf.add(storm1outer,moad) wrf.add(storm1inner,storm1outer) wrf.add_output('history',step=3600*3,end=9*3600) sim_moad=wrf[moad] sim_storm1outer=wrf[storm1outer] sim_storm1inner=wrf[storm1inner] @endcode In this example, the sim_moad, sim_storm1inner and sim_storm1outer will be new objects, contained within the WRFSimulation named "wrf". They will contain additional information about the WRFDomain that is not in the original moad, storm1inner and storm1outer.NcCsd|_d|_d|_d|_d|_d|_i|_d|_d|_|dk r|j |_ t |j |_ |j|j|j|j|jf\|_|_|_|_|_|j|j|j|_|_|_|j D]\}}t||j|<qnt|||_ t ||_ dS)a!WRFDomain constructor Creates a new WRFDomain based on the information in a section of the HWRFConfig object conf. The domain's name is in "name." The "copy" argument should never be specified: it is used by self.copy for deep copies of a WRFDomain.NT) nestlevelparentnocolons_start_end_dt_outputr7r8nlcopystrnameitemsdict Conf2Namelist)selfconfsectionrZrXnorrr__init__s(   zWRFDomain.__init__cCs.|jddd|jdd|jdddS)Nhifreqhifreq_d.htcf)outnamerZframes_per_hifreqZhifreq_interval) add_outputrWnl_delr^rrr add_hifreqszWRFDomain.add_hifreqcCs*|jdkrdS|j|jddSdS)z)rYrZrirrr__repr__szWRFDomain.__repr__cCsdS)z!Returns True. This will be used in the future to ensure users do not specify namelist options that should be automatically configured.Tr)r^svrrr _nl_subsetters zWRFDomain._nl_subsettercCstdd|dS)z!Returns a deep copy of this object Returns a deep copy of this object. The copy has its own data structures, so modifying the copy will not modify the original.N)rX)rrirrrrXszWRFDomain.copycCsft||||||||\}}}dD]8}|jdd|t|||jdd|t||q(dS)a!Sets start and end times and timestep Sets this WRFDomain's idea of the simulation start and end times and timestep. @param start the start time of the domain @param end the end time of the domain @param timestep the domain's timestep)yearmonthdayhourminutesecondrZstart_Zend_N) WRFDomainBase set_timingZ_validate_timespanrWnl_setgetattr)r^startendtimesteptsZtedtrarrrrzs zWRFDomain.set_timingcCs|jddS)z!Returns the WRF grid id.rr9rWrmrirrr get_grid_idszWRFDomain.get_grid_idcCs|}t|}|dkS)z!Is this the outermost domain? Returns True if this is the WRF Mother of All Domains (MOAD) and False otherwise. The MOAD is the outermost domain.rk)rint)r^gidrrris_moad szWRFDomain.is_moadcCs|jddS)z*!The number of grid cells the X direction.rr2rrirrrnxsz WRFDomain.nxcCs|jddS)z*!The number of grid cells the Y direction.rr4rrirrrnysz WRFDomain.nycCs|jddS)z*!The number of grid cells the Z direction.rr6rrirrrnzsz WRFDomain.nzcCsh|jj}|jj}|ddd|ddd|ddd|dd|d|dd|d |dd t|d S) z!Internal helper function that initializes variables common to all domains Initializes domain variables that are needed by all domains. This is called as a helper function to the other domain initialization variables.rr1rkr3r5r2rr4rr9N)rWr{ trait_getr)r^r9rptrrr init_domains   zWRFDomain.init_domainc Cst||||||jj}|jj}d|_t|\}}} |dt|} t|| krhtdt | t|f|dd| |d|_ |dd|j |d|_ |dd|j |ddd|dd d |dd d |dd d|dd d|j dk st dS)a!Called by WRFSimulation to initialize this as the outermost domain Do not call this function directly. It is called by the WRFSimulation to initialize the domain as the Mother Of All Domains (MOAD). @param simstart the simulation start time @param simend the simulation end time @param simdt the outermost domain timestep @param eta_levels the NMM eta levelsrrz)nz and len(eta_levels) mismatch: %d vs %drr6r7r8r?r@rkrArBrCN) ry init_as_moadrWr{rrPsplit_fractionlenWRFErrorrr7r8AssertionError) r^simstartsimendZsimdtrGrpriradrrrrr+s*          zWRFDomain.init_as_moadcCst|||||t|j||js8td|j||jf|jj}|jj }|j d|_ |jj }|dd|dd|ddd|ddd|j d |_ |j d |_ |dd |dd d|dd |dd d|dd |dd t|d d}|dkr"|ddt|d|ddt|dnb|dkrF|ddd|dddn>|dkrj|ddd|dddntd|jt|f|j|j dk stdS)a_!Called by WRFSimulation to initialize this domain as a nest Do not call this function directly. It is called by the WRFSimulation to initialize the domain as a nest @param parent the parent WRFDomain @param grid_id the integer grid_id @param start the domain start time @param end the domain end timez9Start time %s for domain %d is not at parent %d timestep.rkrr?r9rAr@g@r7r8r6r}autoZfixedrBistartrCjstartZcenteredz **CENTERED**z**AUTO**zV%s: Invalid value for start. It must be "fixed" "centered" or "auto", but you gave %sN)ry init_as_nestis_at_timesteprSrUStartNotAtParentTimestepr9rWr{rrPrmr7r8rYlowerrInvalidDomainStartrZreprr)r^rQr9r}r~rprprrrrIsH             zWRFDomain.init_as_nestcCs |jS)z>!Creates the WRF namelist contents and returns it as a string.)rW make_namelistrirrrryszWRFDomain.make_namelistcCs ||jkS)at!Returns True if the domain will output to the specified stream. Checks the internal data structures maintained by add_output to see if output was requested for the specified stream. @param stream the stream to check @return True if the domain will output to the specified stream, and False if it won't (as far as we know).rVr^rrrr has_output|s zWRFDomain.has_outputcCs||std|t|f|j}d|j|krX|j|ddk rXt|j|d|}|j}d|j|kr|j|ddk rt|j|d|}t|j|d}|||fS)a!Returns the range of times that has output for the given stream. @return a tuple containing the first output time, last output time and output interval for this domain and the specified stream. @param stream the stream for whom output is requestedz#Stream %s is disabled for domain %sr}Nr~step)rOutputStreamDisabledrrSrVto_datetime_relrT to_timedelta)r^rr}r~intervalrrrget_output_ranges  zWRFDomain.get_output_rangecCstd||jdS)z-!Returns the hifreq filename for this domain.reTparse_wrf_outnamerrSrirrr hifreq_files zWRFDomain.hifreq_filecCstd||jdS)z.Returns the track patcf file for this domain. ztrack_d.patcfTrrirrrtrackpatcf_files zWRFDomain.trackpatcf_filecCst|j||jS)a!Internal function that determines the time output would actually be generated This is an internal implementation function. You should not call it directly. It returns the nearest datetime.datetime to the specified time that lies on a model timestep, without going over. Uses hwrf.numerics.nearest_datetime() @param when the desired time @returns a datetime.datetime for the actual time that will appear)nearest_datetimerSrU)r^whenrrr_get_output_times zWRFDomain._get_output_timecCsn|dkr||}|dkr8t|||||dSt||||j}|dk sXtt|||||dS)a\!Internal function that generates WRFOutput objects This is an internal implementation function. You should not call it directly. It returns a WRFOutput object for the specified output time (when) and stream. The outname is the WRF output filename syntax, with and in it. If actual_time is specified, it is used as the output time, otherwise, "when" is passed into self._get_output_time to get the actual time WRF will output it. @param stream the desired stream @param outname the output filename format @param when the desired output time @param actual_time the actual time from _get_output_time. If missing or None, then _get_output_time will be called. @param logger if specified, the logging.Logger to use for log messagesNrd) validtime)rZ WRFOutputZ get_anl_timerrrrRr)r^rrfr actual_timeloggerpathrrr _get_outputs   zWRFDomain._get_outputccsF|jD]:}|dkr*||D] }|Vqq||t||jVqdS)a!Iterate over all output files for a specified time. Iterates over all output files as WRFOutput objects. If a time is specified, then only outputs for that time are yielded. @param time the time of interest, or None (the default)N)rV get_outputs get_outputrrS)r^timerobjrrrget_all_outputss   zWRFDomain.get_all_outputsc cs||\}}}td}|}|}d}|j|d} |||kr||} |dksX|| krr||| || } | }| V||}||kr2tdt|t|t|t|fq2dS)z!Iterates over all outputs for a specified stream Iterates over all output files for the specified stream, as WRFOutput objects. @param stream the string name of the output streamz0+1/500Nrfz)Zero output interval %s: somehow %s+%s=%s)rrrVrrInvalidTimespanr) r^rr}r~repsilonrZ firstwhenZprevwhenrfrrrrrrs&  zWRFDomain.get_outputsc CsX||\}}}t||j}|j|d}t|||}||krBdS|j||||d} | S)a!Get output for a specified time and stream Returns a WRFOutput object for the output file for the specified stream and time, or None if no such file exists. Will return the first output time not before the given time. @param stream the desired stream @param time the desired time @param logger if specified, a logging.Logger to log torfNr)rrrSrVrr) r^rrrr}r~rrfZnearresultrrrrs   zWRFDomain.get_outputcCs |j|=dS)a(!Forget that output was requested for a given stream Removes the specified stream from the internal data structures. Its output will revert to the WRF default, and will be unavailable via this WRFDomain. @param stream the string name of the stream of disinterestNrrrrr no_outputszWRFDomain.no_outputcCs|j|sdSd}|jj}|jj}|dkr6|s6dS||j|d<|d|j|d<||j|d<|dkr|d d |f||d d |f|dn(|d d |f||d d |f|d|d d|f|d d|f|j|=dS)aH!Disable output for a specified stream, if the stream was requested before If output is enabled for the specified stream, moves the output start time to after the end of the simulation. That way any WRF code relying on the output frequency will still work, but no output will be generated. Will not work for restart stream since that is not controlled on a per-domain basis. Note that code that queries the output times will break if this is called. @param stream the string name of the stream of disinterestNi?Brr}rkr~rrr %s_begin_m%s_end_m%s_begin%s_end %s_begin_s%s_end_s)rVrWr{rhr)r^rZforeverrprrrr hide_output s$ zWRFDomain.hide_outputc Cs|dkr,|jdd} | dkr$d}nt| }t|ts:t|jj} |dkrRt|}d\} } } |dk sl|dk r||dkr|td|dk rt||}||} t | \}}}|dkrt d|dk r|dkr|j n|}t||} | |}t |\}}}|dkrt d |dk rt |} | dkr"t d } t | \}}}|dkrBt d d }|d krZd|f}|dkrx| dd|f|n| dd|f||dkr| dd|f||dkr| snV|dk r|dkr| dd|f|n| dd|f||dkr| dd|f||dk rr|dkrr|dkrD| dd|f|n| dd|f||dkrr| dd|f||dkr|dkr|dk r| d|t|n | d|d| | | |d|j|<|jddddS)a!Adds output to the specified stream. Other arguments are optional: @param stream the stream: "history" or "auxhistN" for an integer N>0 @param start output start time (anything accepted by to_datetime) Default: simulation start time. @param end output end time (anything accepted by to_datetime. Default: simulation end time. @param step output interval, sent into to_out_interval(). Default: trait stream+"_interval" or 6hrs @param outname output name or array of output names (one per domain). Can only specify for all domains, not for only one. Default: leave unspecified, and let WRF use its defaults. @param frames_per_outfile how many output times per output file @param io_form WRF IO form. Simply calls self.set_io_form(stream,io_form) @param simstart the simulation start time which must be provided fi start or end times are givenNio_formmissing)NNNzOWRFDomain.add_output: simstart must be provided if start or end times are givenrzVOutput start time must be an integer multiple of a second after simulation start time.zTOutput end time must be an integer multiple of a second after simulation start time.i`Tz9Output frequency must be an integer multiple of a second.r&rz frames_per_%srrz %s_interval_mz %s_intervalz %s_interval_srrrrrrrrk)r}r~rrfrRT)rWrr isinstancerr{r TypeErrorrminutes_seconds_restPrecisionTooHighrS to_fractionrrVnl_set_if_unset)r^rr}r~rrfr&rrZiofrpdstartZdendZfstepZsminZssecZsrestZstartrelZfendZeminZesecZerestminutessecondsrestZ frames_perrrrrg)s                  zWRFDomain.add_outputcCs||jkr|j|dSdS)z!Return the output interval for the stream Returns the output interval for the specified stream, or None if the stream is disabled. @param stream the stream of interestrNrrrrr interval_fors zWRFDomain.interval_for)NN)NN)N)N)NNNNNNN)$__name__ __module__ __qualname____doc__rcrjrlrnpropertyrrorrrXrzrrrrrrrrrrrrrrrrrrrrrgrrrrrrvsN8      0     cc@seZdZdZddZd=ddZd>dd Zd d Zd?d d ZddZ d@ddZ e ddZ e ddZ ddZddZddZddZdd Zd!d"ZdAd#d$ZdBd%d&ZdCd'd(ZdDd)d*Zd+d,Zd-d.Zd/d0Zd1d2ZdEd3d4ZdFd5d6ZdGd7d8ZdHd9d:ZdId;d<Z dS)Jra/!generate and manipulate wrf namelists, predict output filenames The WRFSimulation class is at the core of the HWRF scripting system. It stores information about every aspect of the WRF namelist, and can manipulate it in complex ways. It automatically generates I/O information, and can predict output and input filenames no matter what crazy timesteps and start/end times you select. There are a number of safeguards that will raise exceptions in Python if you try to set up a simulation that is not possible in WRF. cCstdddddd|S)a!Makes a deep copy of this object Returns a deep copy of this object, providing new data structures so modifying the copy will not modify the original. The underlying WRFDomain objects and their data structures are also duplicated.N)rrirrrrXszWRFSimulation.copyNc Cs(|dk r@tj|dddddd|d|j|_|j|_|D]}q6dSt|||||||d|_|jj} |jj} |jj} |jj} |jj } |jj }|jj }|j ddddddd d t |d }t|\}}}| dd || dd || dd|| dd|d| ddt|d| ddt|d| ddt|d| ddt|dd| ddt|dd| d ddg| d dddD]D}d |}| d|s||r| d|||n| d||jq| dd!d"|jd#r:| d#d$d%g| d#d&d%g| d#d'd%g| dd(rR| dd)rR| dd(| dd)n| dd(d%| dd)d%| d d}| d d}d}t|tr|D]}|t|7}qzt|tr|d*}d*|kr|d*}n|}|D]}|t|7}q||}| d d+|dkd|_| dd,d| dd-d| dd.ddS)/a!Creates a new WRFSimulation object: Creates a new WRFSimulation object. @param conf the HWRFConfig to provide configuration information @param section the section to use in that config object @param moad the Mother of All Domains, as a WRFDomain @param simstart,simend - simulation start and end times @param timestep the simulation timestep @param dup do not use. This is used by the self.copy() do do a deep copy of a WRFDomains.N)duprrrrr r!r"r#rr.r/r0r%bdysteprFrEZptoprHZ prep_hybridrIZmetgrid_soil_levelsrDZmetgrid_levelsrNrrOrk) rboundaryZ auxinput1auxhist1auxhist2auxhist3Zauxhist4Zauxhist5rZauxhist6Z auxinput2Zio_form_r,zmet_nmm.d. dm_task_split comm_start nest_pes_x nest_pes_yr=r>,rMr<r:r;) WRFDomainsrcZ_wps_tilingrWrhrmr{Znl_haverrZ trait_haveZ nl_sectionrrfloatboolrr nl_have_sectrlistrYstripsplit)r^r_r`moadrrrrdomainZndngrpshZsiurthrrrarrZnio_tpgZnio_g total_nio_tpgnumZ nio_tpg_str nio_tpg_splitniorrrrcs                     zWRFSimulation.__init__rcCs4t|}t|}|jj}|dd||dd||S)a,!Sets nproc_x and nproc_y in the namelist Sets the WRF namelist values of nproc_x and nproc_y, which configure task geometry. Default values are -1, which tells WRF to automatically decide the task geometry. @param nproc_x,nproc_y the new values, which default to -1rr=r>)rrWr{)r^r=r>rprrr set_nprocss   zWRFSimulation.set_nprocscCs |D]}|j|kr|qdS)zmAdds the WRF hifreq_d.htcf product to the specified domains of nestlevel in this simulation.N)rPrj)r^rPrrrrrj"s zWRFSimulation.add_hifreqc Cs|dkr4|dkr4|dkr4|jdr0|jdnt|trt|ddkrndd|dD}qt|dkrdd|D}qt|g}nt|}t|tr,t|ddkrdd|dD}nFsz3WRFSimulation.set_dm_task_split..cSsg|] }t|qSrrrrrrrHscSsg|] }t|qSrrrrrrrPscSsg|] }t|qSrrrrrrrRscSsg|] }t|qSrrrrrrrZscSsg|] }t|qSrrrrrrr\srrr) rWrZ nl_del_sectrrYrrrrr{) r^rrrcomm_start_d01nest_pes_x_d01nest_pes_y_d01Zcomm_start_intsZnest_pes_x_intsZnest_pes_y_intsrrrset_dm_task_split+s8    zWRFSimulation.set_dm_task_splitcCs|D]}||rdSqdS)a!!Does this stream have any outputs? Determines if the specified stream has output. @returns True if the stream if add_output() has been called for this stream, for any domain, and False otherwise. @param stream the string name of the stream (lower-case).TF)r)r^rrrrrrhs zWRFSimulation.has_outputTcCst|tr`|d}d|kr0|d}n|}t|dkrTdd|D}qht|g}nt|}t|}t|}|jj}|dd||dd||dd||S) a.!Sets the I/O server configuration in WRF. Sets the WRF I/O server configuration in the &namelist_quilt setting. @return self @param tasks_per_group the nio_tasks_per_group setting, which specifies the number of I/O server tasks in each I/O server group. @param groups the nio_groups setting, an integer which specifies the number of I/O server groups. @param poll_servers the poll_servers setting, a logical that specifies whether I/O server polling should be enabled.rrkcSsg|] }t|qSrrrrrrrsz0WRFSimulation.set_io_servers..r"rNrOrM) rrYrrrrrrWr{)r^Ztasks_per_groupgroupsrMZtasks_per_group_strZtasks_per_group_splitZ nio_tpg_intsrprrrset_io_serversss        zWRFSimulation.set_io_serverscCs|jddd}|S)z)!The number of I/O server tasks per groupr"rN1r)r^ZiopgrrrrNsz!WRFSimulation.nio_tasks_per_groupcCs|jddd}t|}|S)z !The number of I/O server groupsr"rO0)rWrmr)r^ngroupsrrrrOszWRFSimulation.nio_groupscCsBt|}t|}ttt|}|jd||jdd|dS)aF!Sets the boundary input interval (interval_seconds) Sets the interval at which this WRF simulation expects boundary conditions. Accepts anything that can be passed to to_timedelta. @param step boundary input interval. Can be anything accepted by to_timedelta. @return selfrrr%N)rrrrroundrW trait_setr{)r^rrrr set_bdysteps zWRFSimulation.set_bdystepcCst|jdS)a@!Returns the boundary input interval (interval_seconds) Computes the interval at which this WRF simulation expects boundary conditions as a datetime.timedelta. This is done using the "bdystep" trait. @return the boundary input interval (interval_seconds) as a datetime.timedeltar)rrWrrirrrrszWRFSimulation.bdystepcCst|jddS)a!Returns the epsilon for boundary time equality comparison Returns the largest difference between two times such that they are considered identical. This is used in the context of WRF boundary input times. This is equal to bdystep()/10 @return a fractions.Fraction with the suggested epsilon for equality comparisons of boundary output time.r )rrWrrirrr bdyepsilonszWRFSimulation.bdyepsilonccsD|}|}|tt|d}||kr@|V||7}q(dS)z!Iterates over boundary times Iterates over times at which this WRF simulation expects boundary conditions. Yields datetime objects for each time.r N)rrrrr)r^rnowr~rrrbdytimess zWRFSimulation.bdytimescCs&|jdkrdS|jd|jdSdS)z!Returns the number of OpenMP tiles per MPI patch Gets the number of WRF tiles in each WRF patch, returning 1 if tiling is not in use.Nrkr)rrirrr num_tiless zWRFSimulation.num_tilescCsNt|t|g|_tddt|t|tddt|tddt|dS)z!Sets the OpenMP tiling information. Sets the number of WRF OpenMP tiles in each WRF MPI patch to x by y. Don't use this: OpenMP is not supported by WRF-NMMrr<r:r;N)rrrp)r^xyrrr set_tilingszWRFSimulation.set_tilingcCsF|j}dd|D}|d|dd}|j|dttS)a!Generates a Conf2Namelist for this simulation Generates a Conf2Namelist object for the namelist that should be input to wrf.exe @param section_sorter the section_sorter argument to hwrf.namelist.Conf2Namelist.__init__ @param var_sorters the var_sorters argument to hwrf.namelist.Conf2Namelist.__init__ @return an hwrf.namelist.Conf2Namelist object that can generate namelists for this simulationcSsg|] }|jqSr)rWrrrrrrsz.WRFSimulation.wrf_namelist..rrkN)other)rWrXjoinZ remove_traitsZ set_sorters_wrf_namelist_order_wrf_nl_var_order)r^Zsection_sorterZ var_sortersrWZdomain_nl_listZ domain_nlrrr wrf_namelists zWRFSimulation.wrf_namelistcCs|d|_|dkr(|jd|jd|j}|jdd||jddd|jddd|jdd d |jdd ddS) aC!Internal function for configuring wrfanl file generation Sets several namelist settings related to reading and writing wrfanl files. This does not enable wrfanl reading or writing though - that is done by analysis_out() and analysis_in(). @param io_form the restart file io_form TNr+rrr-rFr)ir*)Z _domains_donerWrZ_io_formr{r)r^rrrr_analysis_setupszWRFSimulation._analysis_setupcCs(|D]}|jdddq|||S)z!Requests that this WRF simulation write analysis files. Sets up the namelist settings for writing analysis files @param io_form the io_form for restart (wrfanl) files @return selfrr'F)rWr{r)r^rrrrr analysis_outs zWRFSimulation.analysis_outcCsB|||}|D]}||k}|jdd|q|||S)z!Requests that this WRF simulation read an analysis file. Sets up the namelist settings for reading analysis files @param io_form the io_form for restart (wrfanl) files @return selfrr')rget_moadrWr{)r^rrrvalrrr analysis_ins  zWRFSimulation.analysis_incCs|D]}|jdddqdS)a!Sets the WRF wrfanl output file pattern for all domains. Sets the output file pattern for the wrfanl file. It sets this for ALL domains. @param pattern the pattern for all wrfout files. Make sure you include at least one in the pattern @bug this function ignores the pattern argument. It always sets the pattern to "wrfanl_d_" rr(wrfanl_d_N)rWr{)r^patternrrrrset_wrfanl_outnames  z WRFSimulation.set_wrfanl_outnamecCs>||}||}|jddd}t||||S)aj!Returns the wrfanl name for the specified domain Produces an analysis filename for the specified domain. NOTE: this function assumes all domains have the same wrfanl filename format. @param domain the wrf domain of interest (integer grid_id, string name or a WRFDomain object) @return the filename as a stringrr(r)getrrWrmrr get_nocolons)r^rZdomidrrrr analysis_name"s   zWRFSimulation.analysis_namecCs td|S)a !Requests reading of a restart file Raises NotImplementedError(). This would request that this WRF simulation read a restart file. This is not implemented since the restart capability was broken as of the writing of this function.z/Restart capability is presently broken in HWRF.)NotImplementedError)r^Z restartsourcerrr restart_in1szWRFSimulation.restart_incCsZt|}t}|jdD] \}}|ddkr||q|D]}|jd||q@dS)z!Sets the io_form for all active streams. Changes the io_form for all streams to the specified io_form @param io_form the io_form as an integerrrrN)rsetrWZnl_eachfindaddr{)r^rZformsvarvaluerrrset_active_io_form_to;s z#WRFSimulation.set_active_io_form_tocCspd|f}|jj}|jj}|jj}|dk rJ|d|t|||t|n"|d|||||||dS)a!Sets the io_form for the given stream Changes the io_form for the specified stream @param stream the string name of the stream, lower-case @param io_form the io_form to use. If unspecified or None, then self.io_form_for(stream) is calledz io_form_%sNr)rWr{rrr io_form_for)r^rrZio_form_streamrprrrrr set_io_formGs zWRFSimulation.set_io_formcCsz|dkr|j}|dkr|j}|dkr*|j}t|}t||}t|}|D]}|||||qH||_||_||_dS)a!sets the simulation start and tend times, and timestep Sets the simulation start and end times, and timestep. The start may be anything accepted by to_datetime. The end is passed through to_datetime_rel, relative to start. The timestep must be accepted by to_fraction. @param start,end simulation start and end times @param timestep the outermost domain timestepN)Z _simstartZ_simend _timestep to_datetimerrrzrl)r^r}r~rrrrrrzXs  zWRFSimulation.set_timingc Cst|}t|}tt|||d}tt||df|d}tt||df|d}t|}t|}t|}t||} |dk r|d|| ||f|jdd||jdd| dS) a!Sets the num_metgrid_levels and num_metgrid_soil_levels Overrides the num_metgrid_levels in &domains to equal the value in the specified metgrid file. Does this by analyzing the output of metgrid. @param exepath path to the hwrf_metgrid_levels program @param metgrid_out_file path to the metgrid out file to read @param logger optional logging.Logger for loggingrZ num_sm_levelsZ num_st_levelsNz+Have %d levels, %d soil levels (m=%d t=%d).rrDrI)rYr r rmininforWr{) r^exepathZmetgrid_out_filerstrexeZstrmetZnummetZnumsmZnumstZnumsoilrrrset_metgrid_levels_fromls   z%WRFSimulation.set_metgrid_levels_fromc Cs|}|d|}tdd}||W5QRXtdd }|dt|t|fW5QRXtdd}|d|j|jfW5QRXt |} t t | t |j >|dt dstd | fzd \} } tdd ~}|D]r} | } td | } | r@| \}}|d kr,t|} |dkr^t|} q| r|r|d| | fqW5QRX| dkr~| dkstd| | | f|| | |WStk r}z&|r|jd| t |fddW5d}~XYnXdS)aV!Suns swcorner_dynamic to set domain start locations Runs the swcorner_dynamic program to fill in this WRFSimulation's domain start locations. Returns the resulting namelist as a multi-line string. @param exepath full path to the hwrf_swcorner_dynamic @param storminfo an hwrf.storminfo.StormInfo object for the storm of interest @param domlat,domlon the outermost domain center lat & lon, which is also the projection center @param logger optional logger.Logger object for loggingfort.12wt domain.center%f %f storm.centerrset_nest3%s could not find the nest south-west corner point.r=rt([IJ])START=([0-9.+-]+)IJ!%s: ignoring unrecognized line %sD%s: could not find the south-west corner point (istart=%d jstart=%d)%s unexpected exception: %sTexc_infoN)rXfill_auto_startsrropenwriterlatlonrYr r hwrfbasin2r SetNestFailedupperrstripresearchrrwarning Exception)r^r1 storminfodomlatdomlonrjunkwrfjunknmlfr2rrlinemijrerrrswcorner_dynamics^    $           zWRFSimulation.swcorner_dynamicc Csv|}|d|}tdd}||W5QRXtdd }|dt|t|fW5QRXg} g} t|D]\} } tdd}|d| j| j fW5QRXt |} t t | t | j >|dtdstd | fzd \}}tdd }|D]}|}td |}|rn|\}}|d krPt|}| ||dkrt|}| |n"|r|r|d| |fqW5QRX|dkr|dkstd| ||fWq~tk r }z&|r|jd| t |fddW5d}~XYq~Xq~z|| | |WStk rp}z&|r^|jd| t |fddW5d}~XYnXdS)a#Runs the swcorner_dynamic program to fill in this WRFSimulation's domain start locations. Returns the resulting namelist as a multi-line string. Inputs: exepath = full path to the hwrf_swcorner_dynamic all_storminfo = a list of hwrf.storminfo.StormInfo objects for all the real storms in the multistorm run. domlat, domlon = the outermost domain center lat & lon, which is also the projection center logger = optional: a logger.Logger object for loggingr4r5r6r7r8r9rr:r;r<r>r?r@rArBrCrDrETrFN)rXrHrrrIrJr enumeraterKrLrYr rrMrrNrOrPrQrRrrappendrSrTZfill_auto_starts_multistorm)r^r1Z all_storminforVrWrrXrYrZZistartsZjstartsindexrUr2rrr[r\r]rr^rrrswcorner_dynamic_multistormsz    $               z)WRFSimulation.swcorner_dynamic_multistorm)NN)rr)rrr)T)NN)N)N)N)N)NNN)N)N)N)!rrrrrXrcrrjrrrrrNrOr rr r rrrrrrrr"r$r*r,rzr3r_rcrrrrrsD  e  = #              ;c@szeZdZdZdddZddZddZd d Zd d Zd dZ ddZ dddZ dddZ dddZ d ddZddZdS)!ra!monitors a running wrf simulation This class represents a WRF simulation that is running in an external workflow. It reads the WRF configuration to internally generate namelist information, as if it was going to run the WRF itself. It then monitors the running or completed WRF simulation for simulation output, making it available as Product objects. All WRF outputs are available as UpstreamProduct objects. The WRFSimulation object is available as the public "wrf" member variable and is initialized by the ExternalWRFTask constructor using arguments similar to the WRFSimulation constructor. The simulation start, end and timestep, if unspecified, are taken from the specified conf section's variables by the same name.Fc Kst|_||_d|ks|ds2tj||||f||d|krvd|krftj| d|j |d<nt |d|d<|j }|j|dD]}qW5QRXdS)a[!ExternalWRFTask constructor Creates an ExternalWRFTask, as a wrapper around a WRFSimulation. The conf, section, moad, simstart, simend and timestep are passed on to the WRFSimulation. If simstart, simend, or timestep are None or missing, then they are taken from the configuration section for this task.Z skip_parentoutdirWORKhwrfrelocateN)r\_ExternalWRFTask__prodcache_ExternalWRFTask__wrfrrcchange_locationosrrgetdirtasknamerYdstore transactionproducts) r^rnr_r`wrfrgkwargsrrrrrrc s  zExternalWRFTask.__init__cCsdS)z_Allows subclasses to change self.location in the constructor before product generation.Nrrirrrrj=szExternalWRFTask.change_locationcCs|jS)z!returns the WRFSimulation object that describes the simulation Returns the underlying WRFSimulation object that describes the simulation that is being run.)ririrrrrq@szExternalWRFTask.wrfcCs|D] }|qdS)z!marks products as not having been delivered Marks all products as not having been delivered. Does not delete anything.N)rp undeliver)r^productrrrunrunFs zExternalWRFTask.unruncCs t|_dS)z!clears cached Product objects Clears the cache of WRFOutput -> Product mappings, used to speed up as_product. Calling this will ensure that any later calls to as_product will generate new Product objects.N)r\rhrirrrclear_cached_productsMsz%ExternalWRFTask.clear_cached_productscCs||jkr|j|SdS)z!Returns the product cached for the specified wrfout or None if not found. @protected @param wrfout the hwrf.wrfbase.WRFOutputNrh)r^wrfoutrrr _get_cacheTs  zExternalWRFTask._get_cachecCs||j|<|S)z!Sets the cached produtil.datastore.UpstreamFile for the given wrfout @param wrfout the hwrf.wrfbase.WRFOutput for which uf is the cached product @param uf the product to return from _get_cache() and as_product() @returns ufrw)r^rxufrrr _set_cache[s zExternalWRFTask._set_cachec Cs||jkr|j|S|}|d}|dk s0ttj|tj|}|jz}t|j|j ||d}| }| dd} | d|| } | dd} | d || } ||d <||d <| |d<| |d<W5QRX|r||_ ||j|<|S) a)!Converts a WRFOutput to a Product. Returns a Product for a WRFOutput. The Product is cached in self.__prodcache and as_product() will return that cached value if one is available. Otherwise, a new one is created. Call clear_cached_products() to clear the cache.rdN)categoryprodnamelocationminsizerZminsize_minageZminage_rr~) rhrrrkrbasenamernrorrmrconfintr~) r^rxrgrelrdlocrrzrZ minsize_defrZ minage_defrrrr as_productbs0        zExternalWRFTask.as_productNc#s|dkrddjD}n@t|ts8t|ts8t|trHj|g}nfdd|D}|D]p}|dkr||D]}j||dVqtq^|dkr||D]}j||dVqq^j| |||dVq^dS)a!Iterate over products Iterates over all Products subject to the given constraints, or all Products if no constraints are given: @param domains only these WRFDomains @param stream only these streams (strings) @param time only these times. The earliest output time that is not before the target time is yielded @param relocate passed to self.as_product. Forces an update of the product location NcSsg|]}|qSrrrrrrrsz,ExternalWRFTask.products..csg|]}j|qSrrir rrirrrsrf) rirryrrYr rrrr)r^rrrrgdomlistroutrrirrps" zExternalWRFTask.productsc st}t}fddjD}|D]\}|D]R}||D]B}|} |} || | js:|rl| | js:|| q:q,q$d} t|D]} | |krq| } q| S)a!Determines the last output time at which all streams have completed their output. Determines the last time at which all streams have completed their output. If check=True, then all products are checked, otherwise cached information is used. @param streams a list of WRF stream names @param check if True, call produtil.datastore.Datum.check() on any products that are unavailable. Otherwise, cached information is used. @returns None if no times are complete, or a datetime.datetime of the last forecast time at which all streams are complete. csg|]}j|qSrrrrirrrsz7ExternalWRFTask.last_completed_time..N) r%rirrrr' availablechecksorted) r^streamsrtimesZ incompleterrrrprodrlastrrrirlast_completed_times(   z#ExternalWRFTask.last_completed_timecCs0|dk r|n|||D] }|qdS)a!Update file availability information Calls produtil.datastore.UpstreamFile.check() to update the availability information on all products() that match the given constraints. @param product only this product is checked @param stream only these streams (strings) @param time only these times. The earliest output time that is not before the target time is yieldedN)rrp)r^rtrrrrrr wrf_checks  zExternalWRFTask.wrf_checkc Csh|}|dtj|jd}tj|sF|dt |t |_ zt |d}z| dtjWn,tk r}z|dW5d}~XYnX|D]t}td|r|d |ft|_ W5QRWdStd |r|d |ft|_ W5QRWdSqW5QRXWnBtk r\}z"|jd t|fd dt |_ W5d}~XYnXt|_ dS)a!Is the WRF running, completed or failed? Scans the rsl.out.0000 file to automatically determine the state of the WRF simulation. Looks for "SUCCESS COMPLETE" and "FATAL CALLED" to detect successful completion, or calling of wrf_error_fatal. Sets self.state to produtil.datastore.COMPLETED, produtil.datastore.FAILED, produtil.datastore.RUNNING or produtil.datastore.UNSTARTED based on the contents of rsl.out.0000zCheck on status of WRF...z rsl.out.0000zNo RSL file here: r>iz0Cannot seek -10000 bytes. Will read whole file.NzSUCCESS COMPLETEzWRF is complete: %sz FATAL CALLEDzWRF failed: %sz!Unexpected error checking WRF: %sTrF)logr0rkrrr~produtilfileoprrrstaterIseekSEEK_ENDEnvironmentErrorrQrRrPrrrSrYr)r^rZrsl0rZr^r[rrr update_states<      "zExternalWRFTask.update_state)F)F)NNNF)F)NNN)rrrrrcrjrqrurvryr{rrprrrrrrrrs    # )&r fractionsmathrQdatetimerkprodutil.fileoprprodutil.datastorerrrrrr produtil.runrr r r r r hwrf.hwrftaskr hwrf.numericsZ hwrf.namelisthwrf.exceptions hwrf.wrfbase__all__rpartial_orderingrrryrrrrrrrrs" (    #   ,&}