U $g9,@s"dZddlZddlZddlZddlZddlZddlZddlm Z m Z m Z m Z m Z ddlmZddlmZmZmZmZmZmZddlmZddlTddlTddlTddlTdd d d gZd dZed dddddddgdZeddddddddddd d!d"d#d$d%d&d'd(d)d*d+d,d-d.d/d0d1d2d3d4d5d6d7d8d9d:d;dd?d@g+edAdBdCdDdEdFdGdHdIdJdKdLdMdNdOdPdQdRdSdTdUdVdWdXdYdZd[d\d]d^d_ged`dadbdcdddedfdgdhdidjdkdldmdndodpdqdrdsdtdudvdwdxdydzd{ged|d}d~dddgeddgedddgedddgdZ Gdd d e!Z"Gdd d e#Z$Gdd d eZ%dS)a!@brief Create namelists, monitor wrf simulations, generate filenames. @details This module contains classes that manipulate WRF namelists in complex ways, and predict the resulting output and input filenames regardless of whatever crazy timesteps are requested. This module also contains a class, ExternalWRFTask, that can monitor a running WRF simulation, providing a list of output and input files, and check whether the simulation has completed, failed or is still running. @see hwrf.wrfbaseN) COMPLETED UpstreamFile UNSTARTEDRUNNINGFAILED) isnonempty)bigexecheckrunmpirunmpirunstrbatchexe)HWRFTask)*default_wrf_outname WRFDomain WRFSimulationExternalWRFTaskcCs||dkr dS|dkrd|fS|dkr*dS|dks:|dkrDd |fS|d krVd |fS|d krbd S|dkrndSd|fSdS)a!default wrf output filename patterns Generate a reasonable default wrf outname input value for the specified stream. Presently, these match the WRF defaults. These do not have to match the WRF defaults since we always specify the outname for all streams. @param stream the stream @returns the wrf output filename pattern, including and if relevanthistoryzwrfout_d_Zanlzwrf%s_d_restartzwrfrst_d_Zbdyinputzwrf%s_dZgeo_nmmz %s_dinputoutzwrfinput_d_Zbdyoutzwrfbdy_dz%s_d_N)streamrr6/lfs/h1/ops/prod/packages/hwrf.v13.2.9/ush/hwrf/wrf.pyrs     time_controlfddadomainsphysicsdynamics bdy_controlnamelist_quiltloggingZ start_yearZ start_monthZ start_dayZ start_hourZ start_minuteZ start_secondZend_yearZ end_monthZend_dayZend_hourZ end_minuteZ end_secondinterval_secondsZhistory_intervalZauxhist1_intervalZauxhist2_intervalZauxhist3_intervalZ history_endZ auxhist2_endZauxhist1_outnameZauxhist2_outnameZauxhist3_outnameframes_per_outfileZframes_per_auxhist1Zframes_per_auxhist2Zframes_per_auxhist3analysis anl_outnamerrestart_intervalreset_simulation_startZ io_form_inputZio_form_historyio_form_restartZio_form_boundaryZio_form_auxinput1Zio_form_auxhist1Zio_form_auxhist2Zio_form_auxhist3auxinput1_innameZ debug_levelZtg_reset_streamoverride_restart_timers time_steptime_step_fract_numtime_step_fract_denZmax_doms_wee_wes_sne_sns_verte_vertdxdygrid_id tile_sz_x tile_sz_ynumtilesnproc_xnproc_y parent_idparent_grid_ratioparent_time_step_ratioi_parent_startj_parent_startZfeedbackZ num_movesnum_metgrid_levelsp_top_requestedptsgm eta_levelsuse_prep_hybridnum_metgrid_soil_levelsZnum_soil_layersZ mp_physicsZ ra_lw_physicsZ ra_sw_physicsZsf_sfclay_physicsZsf_surface_physicsZbl_pbl_physicsZ cu_physicsZmommixZvar_ricZ coef_ric_lZ coef_ric_sZh_diffZgwd_optZsfenthZnradsZnradlnphsZncnvcntrackZ gfs_alphaZ sas_pgconZ sas_mass_fluxZco2tfZvortex_trackerZ nomove_freqZ tg_optionZntornadoZnon_hydrostaticZ euler_advwpZcoacZcodampZterrain_smoothingZspec_bdy_widthZ specified poll_serversnio_tasks_per_group nio_groupsZcompute_tasks_silentZio_servers_silentZstderr_logging)rrrrr r!r"c@s$eZdZdZd@ddZddZddZd d Zeeddd Z d d Z ddZ ddZ ddZ ddZddZeddZeddZeddZddZd d!Zd"d#Zd$d%Zd&d'Zd(d)Zd*d+Zd,d-Zd.d/ZdAd0d1ZdBd2d3Zd4d5ZdCd6d7Zd8d9Z d:d;Z!dDdd?Z#dS)Era!A domain in a WRF simulation This subclass of WRFDomainsBase adds needed details that let it provide information about a domain in a WRF simulation. It can predict output and input filenames based on timesteps and start/end times. It can do complex manipulations of the WRF namelist. Most functionality should be accessed via the WRFSimulation, after a WRFDomain is created. Note that after you provide a WRFDomain to a WRFSimulation, the WRFSimulation makes its own copy of that WRFDomain. The original is unmodified. That means that if you want details on the namelist and output files added by the WRFSimultion, you must obtain its copy of the WRFDomain like so: @code{.py} moad=WRFDomain(conf,'moad') storm1outer=WRFDomain(conf,'storm1outer') storm1inner=WRFDomain(conf,'storm1inner') wrf=WRFSimulation(conf,'wrf',moad,conf.cycle, conf.cycle+hwrf.numerics.to_timedelta(fcstlen*3600)) wrf.add(storm1outer,moad) wrf.add(storm1inner,storm1outer) wrf.add_output('history',step=3600*3,end=9*3600) sim_moad=wrf[moad] sim_storm1outer=wrf[storm1outer] sim_storm1inner=wrf[storm1inner] @endcode In this example, the sim_moad, sim_storm1inner and sim_storm1outer will be new objects, contained within the WRFSimulation named "wrf". They will contain additional information about the WRFDomain that is not in the original moad, storm1inner and storm1outer.NcCsd|_d|_d|_d|_d|_d|_i|_d|_d|_|dk r|j |_ t |j |_ |j|j|j|j|jf\|_|_|_|_|_|j|j|j|_|_|_|j D]\}}t||j|<qnt|||_ t ||_ dS)a!WRFDomain constructor Creates a new WRFDomain based on the information in a section of the HWRFConfig object conf. The domain's name is in "name." The "copy" argument should never be specified: it is used by self.copy for deep copies of a WRFDomain.NT) nestlevelparentnocolons_start_end_dt_outputr6r7nlcopystrnameitemsdictZ Conf2Namelist)selfconfsectionrYrWnorrr__init__s(   zWRFDomain.__init__cCs.|jddd|jdd|jdddS)Nhifreqhifreq_d.htcf)outnamerZframes_per_hifreqZhifreq_interval) add_outputrVnl_delr\rrr add_hifreqszWRFDomain.add_hifreqcCs*|jdkrdS|j|jddSdS)z)rXrYrgrrr__repr__szWRFDomain.__repr__cCsdS)z!Returns True. This will be used in the future to ensure users do not specify namelist options that should be automatically configured.Tr)r\svrrr _nl_subsetters zWRFDomain._nl_subsettercCstdd|dS)z!Returns a deep copy of this object Returns a deep copy of this object. The copy has its own data structures, so modifying the copy will not modify the original.N)rW)rrgrrrrWszWRFDomain.copycCsft||||||||\}}}dD]8}|jdd|t|||jdd|t||q(dS)a!Sets start and end times and timestep Sets this WRFDomain's idea of the simulation start and end times and timestep. @param start the start time of the domain @param end the end time of the domain @param timestep the domain's timestep)yearmonthdayhourminutesecondrZstart_Zend_N) WRFDomainBase set_timingZ_validate_timespanrVnl_setgetattr)r\startendtimesteptstedtr_rrrrxs zWRFDomain.set_timingcCs|jddS)z!Returns the WRF grid id.rr8rVrkrgrrr get_grid_idszWRFDomain.get_grid_idcCs|}t|}|dkS)z!Is this the outermost domain? Returns True if this is the WRF Mother of All Domains (MOAD) and False otherwise. The MOAD is the outermost domain.ri)rint)r\gidrrris_moadszWRFDomain.is_moadcCs|jddS)z*!The number of grid cells the X direction.rr1rrgrrrnxsz WRFDomain.nxcCs|jddS)z*!The number of grid cells the Y direction.rr3rrgrrrnysz WRFDomain.nycCs|jddS)z*!The number of grid cells the Z direction.rr5rrgrrrnzsz WRFDomain.nzcCsh|jj}|jj}|ddd|ddd|ddd|dd|d|dd|d |dd t|d S) z!Internal helper function that initializes variables common to all domains Initializes domain variables that are needed by all domains. This is called as a helper function to the other domain initialization variables.rr0rir2r4r1rr3rr8N)rVry trait_getr)r\r8rntrrr init_domains   zWRFDomain.init_domainc Cst||||||jj}|jj}d|_t|\}}} |dt|} t|| krhtdt | t|f|dd| |d|_ |dd|j |d|_ |dd|j |ddd|dd d |dd d |dd d|dd d|j dk st dS)a!Called by WRFSimulation to initialize this as the outermost domain Do not call this function directly. It is called by the WRFSimulation to initialize the domain as the Mother Of All Domains (MOAD). @param simstart the simulation start time @param simend the simulation end time @param simdt the outermost domain timestep @param eta_levels the NMM eta levelsrrz)nz and len(eta_levels) mismatch: %d vs %drr5r6r7r>r?rir@rArBN) rw init_as_moadrVryrrOsplit_fractionlenWRFErrorrr6r7AssertionError) r\simstartsimendZsimdtrFrnrir_drrrrr*s*          zWRFDomain.init_as_moadcCst|||||t|j||js8td|j||jf|jj}|jj }|j d|_ |jj }|dd|dd|ddd|dddt |j d d |_ t |jd d |_|dd t |dd dd |dd t |dd dd |dd |dd t|dd}|dkr:|ddt|d|ddt|dnb|dkr^|ddd|dddn>|dkr|ddd|dddntd|jt|f|j|jdk stdS)a_!Called by WRFSimulation to initialize this domain as a nest Do not call this function directly. It is called by the WRFSimulation to initialize the domain as a nest @param parent the parent WRFDomain @param grid_id the integer grid_id @param start the domain start time @param end the domain end timez9Start time %s for domain %d is not at parent %d timestep.rirr>r8r@r?g@r6r7r5r{autoZfixedrAistartrBjstartZcenteredz **CENTERED**z**AUTO**zV%s: Invalid value for start. It must be "fixed" "centered" or "auto", but you gave %sN)rw init_as_nestis_at_timesteprRrTStartNotAtParentTimestepr8rVryrrOrkroundr6r7rXlowerrInvalidDomainStartrYreprr)r\rPr8r{r|rnrprrrrHsH           zWRFDomain.init_as_nestcCs |jS)z>!Creates the WRF namelist contents and returns it as a string.)rV make_namelistrgrrrrxszWRFDomain.make_namelistcCs ||jkS)at!Returns True if the domain will output to the specified stream. Checks the internal data structures maintained by add_output to see if output was requested for the specified stream. @param stream the stream to check @return True if the domain will output to the specified stream, and False if it won't (as far as we know).rUr\rrrr has_output{s zWRFDomain.has_outputcCs||std|t|f|j}d|j|krX|j|ddk rXt|j|d|}|j}d|j|kr|j|ddk rt|j|d|}t|j|d}|||fS)a!Returns the range of times that has output for the given stream. @return a tuple containing the first output time, last output time and output interval for this domain and the specified stream. @param stream the stream for whom output is requestedz#Stream %s is disabled for domain %sr{Nr|step)rOutputStreamDisabledrrRrUto_datetime_relrS to_timedelta)r\rr{r|intervalrrrget_output_ranges  zWRFDomain.get_output_rangecCstd||jdS)z-!Returns the hifreq filename for this domain.rcTparse_wrf_outnamerrRrgrrr hifreq_files zWRFDomain.hifreq_filecCstd||jdS)z.Returns the track patcf file for this domain. ztrack_d.patcfTrrgrrrtrackpatcf_files zWRFDomain.trackpatcf_filecCst|j||jS)a!Internal function that determines the time output would actually be generated This is an internal implementation function. You should not call it directly. It returns the nearest datetime.datetime to the specified time that lies on a model timestep, without going over. Uses hwrf.numerics.nearest_datetime() @param when the desired time @returns a datetime.datetime for the actual time that will appear)nearest_datetimerRrT)r\whenrrr_get_output_times zWRFDomain._get_output_timecCsn|dkr||}|dkr8t|||||dSt||||j}|dk sXtt|||||dS)a\!Internal function that generates WRFOutput objects This is an internal implementation function. You should not call it directly. It returns a WRFOutput object for the specified output time (when) and stream. The outname is the WRF output filename syntax, with and in it. If actual_time is specified, it is used as the output time, otherwise, "when" is passed into self._get_output_time to get the actual time WRF will output it. @param stream the desired stream @param outname the output filename format @param when the desired output time @param actual_time the actual time from _get_output_time. If missing or None, then _get_output_time will be called. @param logger if specified, the logging.Logger to use for log messagesNrb) validtime)rZ WRFOutputZ get_anl_timerrrrQr)r\rrdr actual_timeloggerpathrrr _get_outputs   zWRFDomain._get_outputccsF|jD]:}|dkr*||D] }|Vqq||t||jVqdS)a!Iterate over all output files for a specified time. Iterates over all output files as WRFOutput objects. If a time is specified, then only outputs for that time are yielded. @param time the time of interest, or None (the default)N)rU get_outputs get_outputrrR)r\timerobjrrrget_all_outputss   zWRFDomain.get_all_outputsc cs||\}}}td}|}|}d}|j|d} |||kr||} |dksX|| krr||| || } | }| V||}||kr2tdt|t|t|t|fq2dS)z!Iterates over all outputs for a specified stream Iterates over all output files for the specified stream, as WRFOutput objects. @param stream the string name of the output streamz0+1/500Nrdz)Zero output interval %s: somehow %s+%s=%s)rrrUrrInvalidTimespanr) r\rr{r|repsilonrZ firstwhenZprevwhenrdrrrrrrs&  zWRFDomain.get_outputsc CsX||\}}}t||j}|j|d}t|||}||krBdS|j||||d} | S)a!Get output for a specified time and stream Returns a WRFOutput object for the output file for the specified stream and time, or None if no such file exists. Will return the first output time not before the given time. @param stream the desired stream @param time the desired time @param logger if specified, a logging.Logger to log tordNr)rrrRrUrr) r\rrrr{r|rrdZnearresultrrrrs   zWRFDomain.get_outputcCs |j|=dS)a(!Forget that output was requested for a given stream Removes the specified stream from the internal data structures. Its output will revert to the WRF default, and will be unavailable via this WRFDomain. @param stream the string name of the stream of disinterestNrrrrr no_outputszWRFDomain.no_outputcCs|j|sdSd}|jj}|jj}|dkr6|s6dS||j|d<|d|j|d<||j|d<|dkr|d d |f||d d |f|dn(|d d |f||d d |f|d|d d|f|d d|f|j|=dS)aH!Disable output for a specified stream, if the stream was requested before If output is enabled for the specified stream, moves the output start time to after the end of the simulation. That way any WRF code relying on the output frequency will still work, but no output will be generated. Will not work for restart stream since that is not controlled on a per-domain basis. Note that code that queries the output times will break if this is called. @param stream the string name of the stream of disinterestNi?Brr{rir|rrr %s_begin_m%s_end_m%s_begin%s_end %s_begin_s%s_end_s)rUrVryrfr)r\rZforeverrnrrrr hide_output s$ zWRFDomain.hide_outputc Cs|dkr,|jdd} | dkr$d}nt| }t|ts:t|jj} |dkrRt|}d\} } } |dk sl|dk r||dkr|td|dk rt||}||} t | \}}}|dkrt d|dk r|dkr|j n|}t||} | |}t |\}}}|dkrt d |dk rt |} | dkr"t d } t | \}}}|dkrBt d d }|d krZd|f}|dkrx| dd|f|n| dd|f||dkr| dd|f||dkr| snV|dk r|dkr| dd|f|n| dd|f||dkr| dd|f||dk rr|dkrr|dkrD| dd|f|n| dd|f||dkrr| dd|f||dkr|dkr|dk r| d|t|n | d|d| | | |d|j|<|jddddS)a!Adds output to the specified stream. Other arguments are optional: @param stream the stream: "history" or "auxhistN" for an integer N>0 @param start output start time (anything accepted by to_datetime) Default: simulation start time. @param end output end time (anything accepted by to_datetime. Default: simulation end time. @param step output interval, sent into to_out_interval(). Default: trait stream+"_interval" or 6hrs @param outname output name or array of output names (one per domain). Can only specify for all domains, not for only one. Default: leave unspecified, and let WRF use its defaults. @param frames_per_outfile how many output times per output file @param io_form WRF IO form. Simply calls self.set_io_form(stream,io_form) @param simstart the simulation start time which must be provided fi start or end times are givenNio_formmissing)NNNzOWRFDomain.add_output: simstart must be provided if start or end times are givenrzVOutput start time must be an integer multiple of a second after simulation start time.zTOutput end time must be an integer multiple of a second after simulation start time.i`Tz9Output frequency must be an integer multiple of a second.r%rz frames_per_%srrz %s_interval_mz %s_intervalz %s_interval_srrrrrrrri)r{r|rrdrQT)rVrr isinstancerryr TypeErrorrminutes_seconds_restPrecisionTooHighrR to_fractionrrUnl_set_if_unset)r\rr{r|rrdr%rrZiofrndstartZdendZfstepZsminZssecZsrestZstartrelZfendZeminZesecZerestminutessecondsrestZ frames_perrrrre(s                  zWRFDomain.add_outputcCs||jkr|j|dSdS)z!Return the output interval for the stream Returns the output interval for the specified stream, or None if the stream is disabled. @param stream the stream of interestrNrrrrr interval_fors zWRFDomain.interval_for)NN)NN)N)N)NNNNNNN)$__name__ __module__ __qualname____doc__rarhrjrlpropertyrrmrprWrxrrrrrrrrrrrrrrrrrrrrrerrrrrrusN8      0     cc@seZdZdZddZd;ddZdddZ e ddZ e ddZ ddZddZddZddZdd Zd!d"Zd?d#d$Zd@d%d&ZdAd'd(ZdBd)d*Zd+d,Zd-d.Zd/d0Zd1d2ZdCd3d4ZdDd5d6ZdEd7d8ZdFd9d:ZdS)Gra/!generate and manipulate wrf namelists, predict output filenames The WRFSimulation class is at the core of the HWRF scripting system. It stores information about every aspect of the WRF namelist, and can manipulate it in complex ways. It automatically generates I/O information, and can predict output and input filenames no matter what crazy timesteps and start/end times you select. There are a number of safeguards that will raise exceptions in Python if you try to set up a simulation that is not possible in WRF. cCstdddddd|S)a!Makes a deep copy of this object Returns a deep copy of this object, providing new data structures so modifying the copy will not modify the original. The underlying WRFDomain objects and their data structures are also duplicated.N)rrgrrrrWszWRFSimulation.copyNc Cs(|dk r@tj|dddddd|d|j|_|j|_|D]}q6dSt|||||||d|_|jj} |jj} |jj} |jj} |jj } |jj }|jj }|j ddddddd d t |d }t|\}}}| dd || dd || dd|| dd|d| ddt|d| ddt|d| ddt|d| ddt|dd| ddt|dd| d ddg| d dddD]D}d |}| d|s||r| d|||n| d||jq| dd!d"|jd#r:| d#d$d%g| d#d&d%g| d#d'd%g| dd(rR| dd)rR| dd(| dd)n| dd(d%| dd)d%| d d}| d d}d}t|tr|D]}|t|7}qzt|tr|d*}d*|kr|d*}n|}|D]}|t|7}q||}| d d+|dkd|_| dd,d| dd-d| dd.ddS)/a!Creates a new WRFSimulation object: Creates a new WRFSimulation object. @param conf the HWRFConfig to provide configuration information @param section the section to use in that config object @param moad the Mother of All Domains, as a WRFDomain @param simstart,simend - simulation start and end times @param timestep the simulation timestep @param dup do not use. This is used by the self.copy() do do a deep copy of a WRFDomains.N)duprrrrrr r!r"rr-r.r/r$bdysteprErDZptoprGZ prep_hybridrHZmetgrid_soil_levelsrCZmetgrid_levelsrMrrNri) rboundaryZ auxinput1auxhist1auxhist2auxhist3Zauxhist4Zauxhist5rZauxhist6Z auxinput2Zio_form_r+zmet_nmm.d. dm_task_split comm_start nest_pes_x nest_pes_yr<r=,rLr;r9r:) WRFDomainsraZ_wps_tilingrVrfrkryZnl_haverrZ trait_haveZ nl_sectionrrfloatboolrr nl_have_sectrlistrXstripsplit)r\r]r^moadrrr}rdomainZndngrnshZsiurthrrr_rrZnio_tpgZnio_gZ total_nio_tpgnumZ nio_tpg_strZ nio_tpg_splitZniorrrras                     zWRFSimulation.__init__rcCs4t|}t|}|jj}|dd||dd||S)a,!Sets nproc_x and nproc_y in the namelist Sets the WRF namelist values of nproc_x and nproc_y, which configure task geometry. Default values are -1, which tells WRF to automatically decide the task geometry. @param nproc_x,nproc_y the new values, which default to -1rr<r=)rrVry)r\r<r=rnrrr set_nprocss   zWRFSimulation.set_nprocscCs |D]}|j|kr|qdS)zmAdds the WRF hifreq_d.htcf product to the specified domains of nestlevel in this simulation.N)rOrh)r\rOrrrrrh!s zWRFSimulation.add_hifreqc Cs|dkr4|dkr4|dkr4|jdr0|jdnt|trt|ddkrndd|dD}qt|dkrdd|D}qt|g}nt|}t|tr,t|ddkrdd|dD}nEsz3WRFSimulation.set_dm_task_split..cSsg|] }t|qSrrrrrrrGscSsg|] }t|qSrrrrrrrOscSsg|] }t|qSrrrrrrrQscSsg|] }t|qSrrrrrrrYscSsg|] }t|qSrrrrrrr[srrr) rVrZ nl_del_sectrrXrrrrry) r\rrrZcomm_start_d01Znest_pes_x_d01Znest_pes_y_d01Zcomm_start_intsZnest_pes_x_intsZnest_pes_y_intsrrrset_dm_task_split*s8    zWRFSimulation.set_dm_task_splitcCs|D]}||rdSqdS)a!!Does this stream have any outputs? Determines if the specified stream has output. @returns True if the stream if add_output() has been called for this stream, for any domain, and False otherwise. @param stream the string name of the stream (lower-case).TF)r)r\rrrrrrgs zWRFSimulation.has_outputTcCst|tr`|d}d|kr0|d}n|}t|dkrTdd|D}qht|g}nt|}t|}t|}|jj}|dd||dd||dd||S) a.!Sets the I/O server configuration in WRF. Sets the WRF I/O server configuration in the &namelist_quilt setting. @return self @param tasks_per_group the nio_tasks_per_group setting, which specifies the number of I/O server tasks in each I/O server group. @param groups the nio_groups setting, an integer which specifies the number of I/O server groups. @param poll_servers the poll_servers setting, a logical that specifies whether I/O server polling should be enabled.rricSsg|] }t|qSrrrrrrrsz0WRFSimulation.set_io_servers..r!rMrNrL) rrXrrrrrrVry)r\Ztasks_per_groupgroupsrLZtasks_per_group_strZtasks_per_group_splitZ nio_tpg_intsrnrrrset_io_serversrs        zWRFSimulation.set_io_serverscCs|jddd}|S)z)!The number of I/O server tasks per groupr!rM1r)r\ZiopgrrrrMsz!WRFSimulation.nio_tasks_per_groupcCs|jddd}t|}|S)z !The number of I/O server groupsr!rN0)rVrkr)r\ngroupsrrrrNszWRFSimulation.nio_groupscCsBt|}t|}ttt|}|jd||jdd|dS)aF!Sets the boundary input interval (interval_seconds) Sets the interval at which this WRF simulation expects boundary conditions. Accepts anything that can be passed to to_timedelta. @param step boundary input interval. Can be anything accepted by to_timedelta. @return selfrrr$N)rrrrrrV trait_setry)r\rrrr set_bdysteps zWRFSimulation.set_bdystepcCst|jdS)a@!Returns the boundary input interval (interval_seconds) Computes the interval at which this WRF simulation expects boundary conditions as a datetime.timedelta. This is done using the "bdystep" trait. @return the boundary input interval (interval_seconds) as a datetime.timedeltar)rrVrrgrrrrszWRFSimulation.bdystepcCst|jddS)a!Returns the epsilon for boundary time equality comparison Returns the largest difference between two times such that they are considered identical. This is used in the context of WRF boundary input times. This is equal to bdystep()/10 @return a fractions.Fraction with the suggested epsilon for equality comparisons of boundary output time.r )rrVrrgrrr bdyepsilonszWRFSimulation.bdyepsilonccsD|}|}|tt|d}||kr@|V||7}q(dS)z!Iterates over boundary times Iterates over times at which this WRF simulation expects boundary conditions. Yields datetime objects for each time.rN)rrrrr)r\rnowr|rrrbdytimess zWRFSimulation.bdytimescCs&|jdkrdS|jd|jdSdS)z!Returns the number of OpenMP tiles per MPI patch Gets the number of WRF tiles in each WRF patch, returning 1 if tiling is not in use.Nrir)rrgrrr num_tiless zWRFSimulation.num_tilescCsNt|t|g|_tddt|t|tddt|tddt|dS)z!Sets the OpenMP tiling information. Sets the number of WRF OpenMP tiles in each WRF MPI patch to x by y. Don't use this: OpenMP is not supported by WRF-NMMrr;r9r:N)rrrn)r\xyrrr set_tilingszWRFSimulation.set_tilingcCsF|j}dd|D}|d|dd}|j|dttS)a!Generates a Conf2Namelist for this simulation Generates a Conf2Namelist object for the namelist that should be input to wrf.exe @param section_sorter the section_sorter argument to hwrf.namelist.Conf2Namelist.__init__ @param var_sorters the var_sorters argument to hwrf.namelist.Conf2Namelist.__init__ @return an hwrf.namelist.Conf2Namelist object that can generate namelists for this simulationcSsg|] }|jqSr)rVrrrrrrsz.WRFSimulation.wrf_namelist..rriN)other)rVrWjoinZ remove_traitsZ set_sorters_wrf_namelist_order_wrf_nl_var_order)r\Zsection_sorterZ var_sortersrVZdomain_nl_listZ domain_nlrrr wrf_namelists zWRFSimulation.wrf_namelistcCs|d|_|dkr(|jd|jd|j}|jdd||jddd|jddd|jdd d |jdd ddS) aC!Internal function for configuring wrfanl file generation Sets several namelist settings related to reading and writing wrfanl files. This does not enable wrfanl reading or writing though - that is done by analysis_out() and analysis_in(). @param io_form the restart file io_form TNr*rrr,rFr(ir))Z _domains_donerVrZ_io_formryr)r\rrrr_analysis_setupszWRFSimulation._analysis_setupcCs(|D]}|jdddq|||S)z!Requests that this WRF simulation write analysis files. Sets up the namelist settings for writing analysis files @param io_form the io_form for restart (wrfanl) files @return selfrr&F)rVryr)r\rrrrr analysis_outs zWRFSimulation.analysis_outcCsB|||}|D]}||k}|jdd|q|||S)z!Requests that this WRF simulation read an analysis file. Sets up the namelist settings for reading analysis files @param io_form the io_form for restart (wrfanl) files @return selfrr&)rZget_moadrVry)r\rrrvalrrr analysis_ins  zWRFSimulation.analysis_incCs|D]}|jdddqdS)a!Sets the WRF wrfanl output file pattern for all domains. Sets the output file pattern for the wrfanl file. It sets this for ALL domains. @param pattern the pattern for all wrfout files. Make sure you include at least one in the pattern @bug this function ignores the pattern argument. It always sets the pattern to "wrfanl_d_" rr'wrfanl_d_N)rVry)r\patternrrrrset_wrfanl_outnames  z WRFSimulation.set_wrfanl_outnamecCs>||}||}|jddd}t||||S)aj!Returns the wrfanl name for the specified domain Produces an analysis filename for the specified domain. NOTE: this function assumes all domains have the same wrfanl filename format. @param domain the wrf domain of interest (integer grid_id, string name or a WRFDomain object) @return the filename as a stringrr'r)getrrVrkrrZ get_nocolons)r\rZdomidrrrr analysis_name!s   zWRFSimulation.analysis_namecCs td|S)a !Requests reading of a restart file Raises NotImplementedError(). This would request that this WRF simulation read a restart file. This is not implemented since the restart capability was broken as of the writing of this function.z/Restart capability is presently broken in HWRF.)NotImplementedError)r\Z restartsourcerrr restart_in0szWRFSimulation.restart_incCsZt|}t}|jdD] \}}|ddkr||q|D]}|jd||q@dS)z!Sets the io_form for all active streams. Changes the io_form for all streams to the specified io_form @param io_form the io_form as an integerrrrN)rsetrVZnl_eachfindaddry)r\rZformsvarvaluerrrset_active_io_form_to:s z#WRFSimulation.set_active_io_form_tocCspd|f}|jj}|jj}|jj}|dk rJ|d|t|||t|n"|d|||||||dS)a!Sets the io_form for the given stream Changes the io_form for the specified stream @param stream the string name of the stream, lower-case @param io_form the io_form to use. If unspecified or None, then self.io_form_for(stream) is calledz io_form_%sNr)rVryrrrZ io_form_for)r\rrZio_form_streamrnr~rrrr set_io_formFs zWRFSimulation.set_io_formcCsz|dkr|j}|dkr|j}|dkr*|j}t|}t||}t|}|D]}|||||qH||_||_||_dS)a!sets the simulation start and tend times, and timestep Sets the simulation start and end times, and timestep. The start may be anything accepted by to_datetime. The end is passed through to_datetime_rel, relative to start. The timestep must be accepted by to_fraction. @param start,end simulation start and end times @param timestep the outermost domain timestepN)Z _simstartZ_simend _timestep to_datetimerrrxrj)r\r{r|r}rrrrrxWs  zWRFSimulation.set_timingc Cst|}t|}tt|||d}tt||df|d}tt||df|d}t|}t|}t|}t||} |dk r|d|| ||f|jdd||jdd| dS) a!Sets the num_metgrid_levels and num_metgrid_soil_levels Overrides the num_metgrid_levels in &domains to equal the value in the specified metgrid file. Does this by analyzing the output of metgrid. @param exepath path to the hwrf_metgrid_levels program @param metgrid_out_file path to the metgrid out file to read @param logger optional logging.Logger for loggingrZ num_sm_levelsZ num_st_levelsNz+Have %d levels, %d soil levels (m=%d t=%d).rrCrH)rXr r rmininforVry) r\exepathZmetgrid_out_filerstrexeZstrmetZnummetZnumsmZnumstZnumsoilrrrset_metgrid_levels_fromks   z%WRFSimulation.set_metgrid_levels_fromc Cst|ts|g}|}|dg|}tdd}||W5QRXtdd }|dt|t|fW5QRXg} g} t |D]\} }tdd}|d|j |j fW5QRXd} | dt |j f7} | d 7} t |} tt| | >|d td std | fzd \}}td d}|D]}|}td|}|r|\}}|dkr~t|}| ||dkrt|}| |n"|r4|r4|d| |fq4W5QRX|dkr|dkstd| ||fWqtk r:}z&|r(|jd| t |fddW5d}~XYqXqz|| | |WStk r}z&|r|jd| t |fddW5d}~XYnXdS)ac!Suns swcorner_dynamic to set domain start locations Runs the swcorner_dynamic program to fill in this WRFSimulation's domain start locations. Returns the resulting namelist as a multi-line string. @param exepath full path to the hwrf_swcorner_dynamic @param storminfo a list of hwrf.storminfo.StormInfo objects for the storm(s) of interest. @param domlat,domlon the outermost domain center lat & lon, which is also the projection center @param logger optional logger.Logger object for loggingzfort.12wtz domain.centerz%f %f z storm.centerz%s  rZset_nestz3%s could not find the nest south-west corner point.)r/rtz([IJ])START=([0-9.+-]+)IJz!%s: ignoring unrecognized line %srzD%s: could not find the south-west corner point (istart=%d jstart=%d)z%s unexpected exception: %sTexc_infoN)rrrWZfill_auto_startsrropenwriter enumeratelatlonrX hwrfbasin2r rr SetNestFailedupperrstripresearchrrappendwarning Exception)r\r( storminfodomlatdomlonrZjunkwrfZjunknmlfZistartsZjstartsindexZechostrr)rrlinemZijrerrrswcorner_dynamics     $                zWRFSimulation.swcorner_dynamic)NN)rr)rrr)T)NN)N)N)N)N)NNN)N)N) rrrrrWrarrhrrrrrMrNrrrrrr rrrrrrrr"r#rxr*rKrrrrrsB  e  = #             c@szeZdZdZdddZddZddZd d Zd d Zd dZ ddZ dddZ dddZ dddZ d ddZddZdS)!ra!monitors a running wrf simulation This class represents a WRF simulation that is running in an external workflow. It reads the WRF configuration to internally generate namelist information, as if it was going to run the WRF itself. It then monitors the running or completed WRF simulation for simulation output, making it available as Product objects. All WRF outputs are available as UpstreamProduct objects. The WRFSimulation object is available as the public "wrf" member variable and is initialized by the ExternalWRFTask constructor using arguments similar to the WRFSimulation constructor. The simulation start, end and timestep, if unspecified, are taken from the specified conf section's variables by the same name.Fc Kst|_||_d|ks|ds2tj||||f||d|krvd|krftj| d|j |d<nt |d|d<|j }|j|dD]}qW5QRXdS)a[!ExternalWRFTask constructor Creates an ExternalWRFTask, as a wrapper around a WRFSimulation. The conf, section, moad, simstart, simend and timestep are passed on to the WRFSimulation. If simstart, simend, or timestep are None or missing, then they are taken from the configuration section for this task.Z skip_parentoutdirWORKhwrfrelocateN)r[_ExternalWRFTask__prodcache_ExternalWRFTask__wrfrrachange_locationosrrgetdirtasknamerXdstore transactionproducts) r\rVr]r^wrfrOkwargsrrrrrras  zExternalWRFTask.__init__cCsdS)z_Allows subclasses to change self.location in the constructor before product generation.NrrgrrrrRszExternalWRFTask.change_locationcCs|jS)z!returns the WRFSimulation object that describes the simulation Returns the underlying WRFSimulation object that describes the simulation that is being run.)rQrgrrrrYszExternalWRFTask.wrfcCs|D] }|qdS)z!marks products as not having been delivered Marks all products as not having been delivered. Does not delete anything.N)rX undeliver)r\productrrrunruns zExternalWRFTask.unruncCs t|_dS)z!clears cached Product objects Clears the cache of WRFOutput -> Product mappings, used to speed up as_product. Calling this will ensure that any later calls to as_product will generate new Product objects.N)r[rPrgrrrclear_cached_products#sz%ExternalWRFTask.clear_cached_productscCs||jkr|j|SdS)z!Returns the product cached for the specified wrfout or None if not found. @protected @param wrfout the hwrf.wrfbase.WRFOutputNrP)r\wrfoutrrr _get_cache*s  zExternalWRFTask._get_cachecCs||j|<|S)z!Sets the cached produtil.datastore.UpstreamFile for the given wrfout @param wrfout the hwrf.wrfbase.WRFOutput for which uf is the cached product @param uf the product to return from _get_cache() and as_product() @returns ufr_)r\r`ufrrr _set_cache1s zExternalWRFTask._set_cachec Cs||jkr|j|S|}|d}|dk s0ttj|tj|}|jz}t|j|j ||d}| }| dd} | d|| } | dd} | d || } ||d <||d <| |d<| |d<W5QRX|r||_ ||j|<|S) a)!Converts a WRFOutput to a Product. Returns a Product for a WRFOutput. The Product is cached in self.__prodcache and as_product() will return that cached value if one is available. Otherwise, a new one is created. Call clear_cached_products() to clear the cache.rLN)categoryprodnamelocationminsizerZminsize_minageZminage_rrf) rPrrrSrbasenamerVrWrrUrconfintrf) r\r`rOrelrLlocrrbrZ minsize_defrgZ minage_defrhrrr as_product8s0        zExternalWRFTask.as_productNc#s|dkrddjD}n@t|ts8t|ts8t|trHj|g}nfdd|D}|D]p}|dkr||D]}j||dVqtq^|dkr||D]}j||dVqq^j| |||dVq^dS)a!Iterate over products Iterates over all Products subject to the given constraints, or all Products if no constraints are given: @param domains only these WRFDomains @param stream only these streams (strings) @param time only these times. The earliest output time that is not before the target time is yielded @param relocate passed to self.as_product. Forces an update of the product location NcSsg|]}|qSrrr rrrrbsz,ExternalWRFTask.products..csg|]}j|qSrrQrr rgrrrgsrN) rQrrwrrXrrrnrr)r\rrrrOdomlistroutrrgrrXUs" zExternalWRFTask.productsc st}t}fddjD}|D]\}|D]R}||D]B}|} |} || | js:|rl| | js:|| q:q,q$d} t|D]} | |krq| } q| S)a!Determines the last output time at which all streams have completed their output. Determines the last time at which all streams have completed their output. If check=True, then all products are checked, otherwise cached information is used. @param streams a list of WRF stream names @param check if True, call produtil.datastore.Datum.check() on any products that are unavailable. Otherwise, cached information is used. @returns None if no times are complete, or a datetime.datetime of the last forecast time at which all streams are complete. csg|]}j|qSrror rgrrrsz7ExternalWRFTask.last_completed_time..N) rrQrrnrr availablechecksorted) r\streamsrstimesZ incompleterprrrqprodrlastrrrgrlast_completed_timers(   z#ExternalWRFTask.last_completed_timecCs0|dk r|n|||D] }|qdS)a!Update file availability information Calls produtil.datastore.UpstreamFile.check() to update the availability information on all products() that match the given constraints. @param product only this product is checked @param stream only these streams (strings) @param time only these times. The earliest output time that is not before the target time is yieldedN)rsrX)r\r\rrrwrrr wrf_checks  zExternalWRFTask.wrf_checkc Csh|}|dtj|jd}tj|sF|dt |t |_ zt |d}z| dtjWn,tk r}z|dW5d}~XYnX|D]t}td|r|d |ft|_ W5QRWdStd |r|d |ft|_ W5QRWdSqW5QRXWnBtk r\}z"|jd t|fd dt |_ W5d}~XYnXt|_ dS)a!Is the WRF running, completed or failed? Scans the rsl.out.0000 file to automatically determine the state of the WRF simulation. Looks for "SUCCESS COMPLETE" and "FATAL CALLED" to detect successful completion, or calling of wrf_error_fatal. Sets self.state to produtil.datastore.COMPLETED, produtil.datastore.FAILED, produtil.datastore.RUNNING or produtil.datastore.UNSTARTED based on the contents of rsl.out.0000zCheck on status of WRF...z rsl.out.0000zNo RSL file here: r0iz0Cannot seek -10000 bytes. Will read whole file.NzSUCCESS COMPLETEzWRF is complete: %sz FATAL CALLEDzWRF failed: %sz!Unexpected error checking WRF: %sTr3)logr'rSrrrfprodutilfileoprrrstater5seekSEEK_ENDEnvironmentErrorr>r?r=rrrArXr)r\rZrsl0rFrJrHrrr update_states<      "zExternalWRFTask.update_state)F)F)NNNF)F)NNN)rrrrrarRrYr]r^rarcrnrXryrzrrrrrrs    # )&r fractionsmathr>datetimerSprodutil.fileopr|produtil.datastorerrrrrr produtil.runrr r r r r Z hwrf.hwrftaskr hwrf.numericsZ hwrf.namelisthwrf.exceptionsZ hwrf.wrfbase__all__rpartial_orderingrrrwrrrrrrrrs" (    #   ,&T