U $g* @sldZddddddddd g Zd d lZd d lZd d lZd d lZd d lZd d lZd d l Zd d l Z d d l Z d d l Z d d lZ d d lZ d d lZ d d lZ d d lZ d d l mZd d lmZmZmZd dlmZmZmZmZd dl mZmZmZmZmZd dl m!Z!d dlm"Z"d dl#m$Z$d dl%m&Z&m'Z'm(Z(m)Z)m*Z*m+Z+d dlm,Z,d dl-m.Z.d dl/m0Z0d dl%m1Z1m+Z+Gddde j2j3Z4Gddde j2j3Z5Gddde j2j3Z6Gddde j2j3Z7Gddde j2j3Z8ddZ9d dZ:Gd!d"d"e j2j3Z;Gd#d d e j2j3ZGd&d'd'e=Z?d(dZ@Gd)d*d*e j2j3ZAd1d+d,ZBd2d-d.ZCd3d/d0ZDd S)4a!Utilities for ensemble-based data assimilation. This module contains utilities for doing various forms of ensemble-based data assimilation. It manages a two-dimension ensemble-time array of Task objects that know how to run each analysis cycle for each member. This module also provides a way of presenting the previous forecast cycle's ensda objects such that this cycle can use them. This is all built on top of the classes in the hwrf.fcsttask module. DAEnsembleFromPriorCycle FromGFSENKFwrite_ensda_flag_fileread_ensda_flag_file CycleTDRCheckAlwaysRunENSDAenada_pre_object_forGFSEnKFInputCheckN)jlogger) COMPLETED UpstreamFile FileProduct) isnonemptywait_for_files make_symlink deliver_file)to_datetime_rel to_timedelta TimeArray to_datetimewithin_dt_epsilon) PostManyWRF)NamedDir) WRFDomain)aliasexempirunmpicheckrunbatchexe)clatlon)get_enkfstatus)set_ecflow_event)runr cseZdZdZd!fdd ZeddZeddZed d Zed d Z ed dZ eddZ ddZ ddZ ddZddZddZddZddZdd ZZS)"rz[!Represents a two-dimensional ensemble-vs-time array of hwrf.hwrftask.HWRFTask objects.Nc  stt|j||||f||dkr(|j}t||j}||_t|_|ddd}t | d||_ |j t dksvt t||j}t |j||j |j t|_|jj|_dS)a!Constructor for DAEnsemble @param dstore the produtil.datastore.Datastore database to use @param conf the hwrf.config.HWRFConfig for configuration info @param section the section to use in conf @param anlintime the analysis input time @param taskname the name of this task in the database @param kwargs passed to the superclass constructorNconfigcycling_intervalg @Zda_cyclei)superr__init__cycler_DAEnsemble__anlintimeset_DAEnsemble__memberidsgetfloatrconfstrZ_DAEnsemble__tstepAssertionErrorrdict_DAEnsemble__memberslasttime_DAEnsemble__anlouttime) selfdstoreconfsection anlintimetasknamekwargsr&endtime __class__8/lfs/h1/ops/prod/packages/hwrf.v13.2.9/ush/hwrf/ensda.pyr((s"   zDAEnsemble.__init__cCs|jS)z6!The time at the beginning of the first ensemble step.)r*r4r>r>r?r8DszDAEnsemble.anlintimecCs|jS)z/!The time at the end of the last ensemble step.)r3r@r>r>r? anlouttimeIszDAEnsemble.anlouttimecCs t|jS)z'!The number of members of the ensemble.)lenr,r@r>r>r?nmembersNszDAEnsemble.nmemberscCs t|jS)z&!The number of ensemble DA time steps.)rBr1r@r>r>r?nstepsSszDAEnsemble.nstepsccs|jD] }|Vq dS)z1!Iterates over all ensemble analysis input times.N)r1times)r4tr>r>r? anlintimesXszDAEnsemble.anlintimesccs0d}|jD]}|rd}q|Vq|jVdS)z2!Iterates over all ensemble analysis output times.TFN)r1rEr3)r4firstrFr>r>r? anlouttimes^s zDAEnsemble.anlouttimesccs|jD] }|VqdS)z!Iterates over all member ids.N)r,)r4Zmemberidr>r>r? member_idsis zDAEnsemble.member_idscCs|j|||j||<dS)aH!sets the HWRFTask to use to use for one cycle of one member Tells member enkfmem to use the specified task to produce output whose input analysis time is atime. @param atime the analysis time, a datetime.datetime @param enkfmem the enkf member id @param task the HWRFTask to useN)r,addr1)r4atimeenkfmemtaskr>r>r? set_memberns zDAEnsemble.set_memberccs2t||j}|j|D]\}}||fVqdS)z!iterate over members for a specified analysis input time Iterates over all members at the specified anlintime, yielding (id,member) tuples. @param atime the analysis time, a datetime.datetimeN)rr*r1items)r4rLtimerMZmemstepr>r>r?members_at_timeys zDAEnsemble.members_at_timeccs&||jjD]\}}||fVqdS)z!iterate over members at the final analysis output time. Iterates over all members at the final analysis output time, yielding (id,member) tuples.N)rRr1r2)r4emr>r>r?members_at_anlouttimesz DAEnsemble.members_at_anlouttimeccs&|jD]\}}|||fVq dS)z!iterate over analysis cycles for a specific member Iterates over (time,EnsembleDAMemberStep) pairs for the specified member.N)r1rP)r4rMrFar>r>r?steps_for_memberszDAEnsemble.steps_for_membercCs|j||S)z!get the analysis cycle for the specified member and time Returns the da cycle for member enkfmem at analysis input time atime. @param atime the analysis time, a datetime.datetime @param enkfmem the ensemble id)r1)r4rLrMr>r>r?memberszDAEnsemble.memberccs:|jD]*\}}|D]}|D] }|Vq&qq dS)z!iterate over all needed input data, for hwrf.input Calls inputiter for all steps of all ENKF members. This is for use by the hwrf.input to figure out what input data is required for the DA ensemble.N)r1rPvalues inputiter)r4rFrSrTdr>r>r?rZs  zDAEnsemble.inputitercCs|jD]8\}}|D]&\}}tdt|t|t|fqq |jj}tdt|f||D] \}}tdt|t|fqhdS)z!print detailed diagnostics Sends detailed diagnostics about all members to the print() statement. This is intended for debugging only.zself.__members[%s][%s]=%szlast time t is %szself.__members[t][%s]=%sN)r1rPprintreprr2rR)r4rQstuffenkfidZ membersteprFr>r>r?dumps zDAEnsemble.dump)NN)__name__ __module__ __qualname____doc__r(propertyr8rArCrDrGrIrJrOrRrUrWrXrZr` __classcell__r>r>r<r?r%s0          csJeZdZdZfddZdddZddZd d Zd d Zd dZ Z S)rz!Represents an ensemble member from the previous forecast cycle. This is used to generate UpstreamFile objects for the previous cycle's ensemble of FromGFSENKF simulations.c sDtt|j|||f|dd|D|_t||_tj||_ dS)ai!FromPriorCycle constructor Makes a new FromPriorCycle object. @param dstore the produtil.datastore.Datastore database object @param conf the hwrf.config.HWRFConfig with configuration info @param section the section to use within conf @param domains the list or tuple of hwrf.wrf.WRFDomain objects to obtain from the prior cycle, in order of grid ID. @param enkfmem the enkf member ID @param anlouttime the output time at the end of the analysis @param kwargs passed to the superclass constructorcSsg|]}|qSr>r>.0r[r>r>r? sz+FromPriorCycle.__init__..N) r'rr(_FromPriorCycle__domainsint_FromPriorCycle__enkfmemhwrfnumericsr_FromPriorCycle__anlouttime)r4r5r6r7domainsrMrAr:r<r>r?r(s zFromPriorCycle.__init__Ncksf|dkr|j}|st|D]F}||jks,t||jkr|rP|||jVq|||jVqdS)a!Iterates over all products Iterates over products produced by the prior forecast cycle's analysis cycle. @param domains if provided and non-None, only products from these domains are yielded @param kwargs ignored N)rjr/is_moad get_wrfinputro get_wrfanl)r4rpr:domainr>r>r?productss  zFromPriorCycle.productscCs |||S)aN!return the product for the specified domain and analysis time Returns the product for the wrfinput_d01 file for the specified domain and time. This is simply a wrapper around get_product(domain,atime) @param domain the domain of interest @param atime the analysis time as a datetime.datetime get_productr4rtrLr>r>r?rrszFromPriorCycle.get_wrfinputcCs |||S)z"!returns the wrfanl file's productrvrxr>r>r?rsszFromPriorCycle.get_wrfanlcCs|}tj|}||jks>|dt|t|jfdS||jkrt|dt|ddd|jDfdS|j d|j t | d}|d t|t|t |j t |ft|j|jtj||d }||S) a!Returns a product for the specified domain and time. Creates a new produtil.datastore.UpstreamFile for the wrfinput or wrfanl file for the given domain, at the given analysis time. The analysis time must be the anlouttime. @return a newly created UpstreamFile for the file of interest @param domain the domain of interest @param atime the analysis timeWrong atime: %s vs %sNzInvalid domain: %s not in %sz, cSsg|] }t|qSr>)str)rhxr>r>r?risz.FromPriorCycle.get_product..zj{oldcom}/{oldvit[stormnamelc]}{oldvit[stormid3lc]}.{oldvit[YMDH]}.ensda_{enkfid:03d}.wrfinput_d{domid:02d})r_domidz$Domain %s atime %s enkfmem %s loc %scategoryprodnamelocation)logrmrnrroinforzrjjoin confstrinterprlrk get_grid_idr]r r5r9ospathbasenamecheck)r4rtrLloggerlocufr>r>r?rws<       zFromPriorCycle.get_productcCs|}tj|}||jks>|dt|t|jfdSdt|j}| d|}|dt|t |jt |ft |j |j tj||d}||S)Nry%03dzX{oldcom}/{oldvit[stormnamelc]}{oldvit[stormid3lc]}.{oldvit[YMDH]}.trak.hwrf.atcfunix.memzatime %s enkfmem %s loc %sr})rrmrnrrorrzrkrlrr]r r5r9rrrr)r4rLrr_rrr>r>r? get_track s0    zFromPriorCycle.get_track)N) rarbrcrdr(rurrrsrwrrfr>r>r<r?rs   cszeZdZdZdfdd ZeddZddZd d Zed d Z d dZ ddZ ddZ ddZ ddZddZZS)ra! Forecast ensemble member based on the GFS ENKF. Runs one member of an ensemble DA forecast ensemble, using a member of the GFS ENKF ensemble as initial and boundary conditions. Some data from the earlier deterministic forecast jobs is reused to simplify the process.Nc  stt|j||||f| t|tjjs,t|dkrr?r(*s$    zFromGFSENKF.__init__cCs |jS)z!The analysis input time.)_FromGFSENKF__wrfsimstartr@r>r>r?r8fszFromGFSENKF.anlintimec Csftjj|j|j|d||jd|jd|j ddd|_ ||_ |j ddddrb|j j d d S) a~!Create the wrf() and fcst This function, called from the constructor, creates the hwrf.wrf.WRFSimulation and hwrf.fcsttask.AnalysisCycle used for this forecast. @param detinit the deterministic model initialization, an hwrf.init.HWRFInit or hwrf.init.InitBeforeGSI. @param sim the hwrf.wrf.WRFSimulation passed to the constructor.fcsttaskz.fcstz/fcstT)r9outdirworkdirkeeprun io_netcdf ensda_runwrfr7N)rmr AnalysisCycler5r6r.copyr9rrfcstrconfboolrset_active_io_form_to)r4rrr>r>r?rks zFromGFSENKF.make_wrfcCs|j|j|j|j|j|js<|j |jn|j dt j |j|jD]b}|rhqZ|js|j|j|n&d|jf}|j |t j |j||jjjdddddqZdS)a!Adds input sources to the forecast object. Adds metgrid, geogrid, wrfinput, wrfbdy, wrfanl, and coupler fort.65 input to the fcst member variable. @param delinit the deterministic model initialization, an hwrf.init.HWRFInit or hwrf.init.InitBeforeGSI.wrfinputz wrfanl-%shistoryr `T)startstependN)r add_geogridgeogrid add_metgridmetgrid add_fort65realinit add_wrfbdyr add_wrfinput add_inputrmr WRFInput2WRF enkfmergerrq add_wrfanlwrfanlname WRFAnl2WRF add_output)r4rrtrr>r>r?r~s0    zFromGFSENKF.make_fcstcCs|jS)z&!The wrf simulation made by make_wrf())rr@r>r>r?rszFromGFSENKF.simccs.|jdddd}|jj|dD] }|VqdS)z!Passes control to the hwrf.prep.PrepHybrid.inputiter(). Iterates over the prep member's inputiter, a hwrf.prep.PrepHybrid.inputiter(). Yields all input data information needed by the hwrf.input module to pull input data.Zensda_fallbackFr%r)optionalN)rpreprZ)r4Z inoptionalr[r>r>r?rZszFromGFSENKF.inputitercCs|j|_|j|_|j}|}|j|}|d}|d}|d}tjj|j |j ||j||j |j d|j d|jdd |_tjj|j |j ||j|j dd|jd |j d d d |j|j|j|_|jd d d dr|jjd|jd|jtjj|j |j ||jd|j |j d|j d|jdd |j|j|j|j|j|_|jdk rtjj |j |j ||j|j|j|j|j|j!|j |j d|j d|jdd |_"dS)a!Creates initialization tasks. Called from the constructor. Creates the initialization tasks, prep, realinit and wrfanl; and links to the deterministic init geogrid and metgrid tasks. This is called by the constructor, to create the needed inputs to the fcst member. @param detinit the deterministic model initialization, an hwrf.init.HWRFInit or hwrf.init.InitBeforeGSI.r prep_hybridZ enkf_mergez.prepz/prep)rLr9rrz.realFz/realT)r9rrris_ensda_realnmmrrrrrMz.wrfanlz/wrfanlNz.mergez/merge)#rrrget_moadgeodatr.rmrZ PrepHybridr5r6r8r9rrrRealNMMrradd_prep_hybridrrrtvsetrMWRFGhostrrrrrensdaMergeGFSHWRFEnKFrr)r4rrmoadrZ realsectionZ prepsectionZ mergesectionr>r>r?rs       zFromGFSENKF.make_initc CsDdd|jjD}|d}|g}t|j||j|ddddg|jd|jd |jd d |_t j j |j|d }t j d t j jd }t j j|jd|jdd}t j j|jj|jjd}|jj}|dks|dkr|jdkr|jd7_t j jtt|jdtt|jdd} | dt|ddgddgdddgd| d|| d| dt j | d||jd} d t|j } | !d!| dt j"j#|j$|j|d"| |j%dd|jd#|j%|jdd$ |_&t j j'|j$|j|d%|jd%|j%dd|jd&|jd&d' |_ |j (|j|j&d|j )d(d)| d S)*z/Makes the gribber and tracker member variables.cSsg|]}|qSr>r>rgr>r>r?risz+FromGFSENKF._make_track..postrFrz.postz/post)needcrtmstreamsr9rrrtpartN)r%domlat)r%domlon)latlonALEPgv@copygbwgrib)rrd2g ףp= ?g>@i)ressizescannZtrkgridhwrftrkz /regribberrz'{out_prefix}.hwrftrk.grbf{fahr:02d}.mem regribberz .regribber)rrrr9rLrtrackerz/tracker)r9rrrrrtrack0z){com}/{out_prefix}.trak.hwrf.atcfunix.mem)*rrrr6r.r9rrrrmregribigrb1 GRIBSubsetterrtracker_subset FixedLocation storminforr pubbasin2upperewcenter RegribManyrrgetexerKr!gridvinttaveGRIBrkrM to_intercomgribtaskGRIBTaskr5r8gribber TrackerTaskadd_moving_grid send_atcfunix) r4 ensdadomsrZ postdomainsgrid2ZtrksubZdomlocZstormlocbasinrbasedirZenkf000r>r>r?rs   zFromGFSENKF._make_trackcks|jjf|D] }|VqdS)z!Iterates over all forecast products. Passes control to hwrf.fcsttask.AnalysisCycle.products() to iterate over all products that match the specified arguments. @param kwargs passed to hwrf.fcsttask.AnalysisCycle.products()N)rru)r4r:pr>r>r?ruszFromGFSENKF.productsc Cs|j}|}|d|ftjj|dd|d|j|j|j |j rb|j |j |j r|j}|j|jW5QRX|j|j|jr|jnd}||tj|t|_W5QRXdS)z!Runs the initialization and forecast for this ensemble member. Runs the prep, realinit, wrfanl and fcst member tasks, using input from the GFS ENKF, and the deterministic initialization.Run ensemble member in %sFTkeep keep_on_errorrzJError regridding inputs to tracker. See earlier log messages for details.N)rrrprodutilcdrrr$rrrrrrr5 transactionrunrunr is_completedrerrorrm exceptions GribberErrorr state)r4whererrFmsgr>r>r?r$"s0              zFromGFSENKF.runc Cs|j}|}|d|ftjj|dd|dT|j}|j |j W5QRX|j |j |j t |_W5QRXdS)zV!Runs tracker for the initialization and forecast for this ensemble member.rFTrN)rrrr r rr5r rr rr$rr r)r4rrrFr>r>r? run_track@s      zFromGFSENKF.run_track)NNNN)rarbrcrdr(rer8rrrrZrrrur$rrfr>r>r<r?r"s$< !  =. csJeZdZdZdfdd ZddZddZd d Zd d Zd dZ Z S) ForHWRFEnKFz!Represents an ensemble member from the previous forecast cycle. Runs EnKF pre-processing (gsihx) for one member , using a member from previous cycle ensemble forecast with or without vortex relocationNc  sztt|j|||| f| ||_||_t||_tj ||_ ||_ t | |_ |j rb||j d||dk svtdS)aG!FromHWRFEnKF constructor Makes a new FromHWRFEnKF object. @param dstore the produtil.datastore.Datastore database object @param conf the hwrf.config.HWRFConfig with configuration info @param section the section to use within conf @param ensdadom the ensda domain object @param ensdasim the WRFSimulation object @param enkfmem the enkf member ID @param atime the output time at the end of the analysis @prior_ensda the priorcycle ensda object @param kwargs passed to the superclass constructorENKFN)r'rr(rtrrkrMrmrnrrL prior_ensdarr_make_relocate _make_gsihxr/) r4r5r6r7ensdadomensdasimrMrLrrr9r:r<r>r?r(Xs  zForHWRFEnKF.__init__c Csldd|jD}t|j||||jd|jd|jdd}|dk rN|j|d|rh|j||jdd|S) zMakes a dict containing the keyword arguments to send in to the constructor for the hwrf.relocate task(s). modin - the modin argument to the constructor dest_dir - the directory in which to run the relocatecSsg|]}|qSr>r>rgr>r>r?rixsz5ForHWRFEnKF._make_relocate_kwargs..z /relocate)rrpmodindest_dirrrrN)rr)Z parentTrackZ trackName)rr0rrupdaterrL)r4rrr priorcyclerr:r>r>r?_make_relocate_kwargsrs" z!ForHWRFEnKF._make_relocate_kwargscsvtjjd}|||j}fdd}tjjj j dfd|ddi|_ j j _ j j_dS)zMakes the relocation, rstage1, rstage2 and rstage3 member variables. track - the track argument to the constructor modin - the modin argument to the constructorrcsdj|fS)Nz%s.%s)r9)sr@r>r?rFz%ForHWRFEnKF._make_relocate..tZtaskname_patternzrelocate.stagez%dN)rrrrr rrmrZ Relocationr5r6r.Z relocationrstage1rstage3)r4rrrr:rFr>r@r?rs   zForHWRFEnKF._make_relocatecCsPd|jkrL|j|jd|jkr2|j|jjrL|jjrL|jdS)#Runs the relocate jobs, if present.r#r$N)__dict__r#Z delete_tempr$r$scrubr@r>r>r? run_relocates     zForHWRFEnKF.run_relocatecCst|j|jd|jdd}|jr4|jj|jd}n|jj |j|jd}t j j |j |j|d|j||j|jfd|jdi||_dS) z!Makes the gsihx member variables.z/gsihx)rLrrr)rtrLgsihxr9z.gsihxN)r0rLrrrr$ get_wrfoutrtrrsrmgsiZGSIHxr5r6r.rrMr9r))r4r:Z wrf_in_prodr>r>r?rs&zForHWRFEnKF._make_gsihxcCs|jdS)r%N)r)r$r@r>r>r? run_gsihxszForHWRFEnKF.run_gsihx)N) rarbrcrdr(r rr(rr,rfr>r>r<r?rQs rcseZdZdZd!fdd Zd"ddZdd Zd d Zd#d dZddZ ddZ ddZ ddZ ddZ ddZddZddZd$dd ZZS)%rz> Merge HWRF EnKF analysis to GDAS EnKF analysis for one memberNc  stt|j|||| f| |_fddD|_t||_tj | |_ |_ |dk sbt |dk snt |dk szt ||_||_||_|j|_|jD]} | rq|j| |_q|j|j|_| dk r| |_|dS)aN!MergeGFSHWRFEnKF constructor Makes a new FromHWRFEnKF object. @param dstore the produtil.datastore.Datastore database object @param conf the hwrf.config.HWRFConfig with configuration info @param section the section to use within conf @param ensdasim the WRFSimulation object @param atime the output time at the end of the analysis @realinit the hwrf.fcsttask.RealNMM object @wrfanl the hwrf.fcsttask.WRFGhost object @hwrfenkf the hwrf.enkf.ENKF object @param kwargs passed to the superclass constructorcsg|] }|qSr>r>)rhrtrr>r?risz-MergeGFSHWRFEnKF.__init__..N)r'rr(rrprkrMrmrnrrL bdyepsilon dt_epsilonr/rrenkfrr _wrfinput_d01rq get_ghost _wrfanl_d02Zget_enkfanl_member_enkfanlZget_storm_radius_input_storm_radius make_products)r4r5r6r7rrMrrrrrLr9r:rtr<r-r?r(s*        zMergeGFSHWRFEnKF.__init__hwrf_interpolate.nmlc Cs|}t}|r"|jdd|d}|d|j}|j}tj |j |}t |d>} t |d(} | |j | f||d|d|W5QRXW5QRXd S) z!Creates the hwrf_interpolate namelist in the specified file. @param domain the domain to be interpolated @param filename the destination filename wrfinput_d01)ZINTRP_TO_FILENAMEZinterpolate_nml_fileZinterpolate_nml_sectionrtwtT)rsource raise_allrLN)rr0rqrr.r7rLrmnamelistNamelistInserterr6openwriteparse) r4rtfilenamerZinvarsZnml_fileZ nml_sectionrLZniZnfZofr>r>r? make_namelists     zMergeGFSHWRFEnKF.make_namelistcCsT|}|j}t|jd|dd|j}t|jd|dd|j}t|jd|dddS)z!Grab input filesZ hwrf_enkf_anlT)rforcer8rr wrfanl_d02N)rr4rrr1rr3)r4rrr>r>r? grab_inputszMergeGFSHWRFEnKF.grab_inputcCs|}|jdkrLtj|dd}|d|dt|dd|dn|j}t|jdd|dt dsd }| |t |dS) z!If no relocate was given, gets the storm radius file from a fix file. Also checks to see if the storm_radius file is present and non-empty, regardless of whether it came from the fix or relocate.NFIXhwrfZhwrf_storm_radiusz2Could not get storm_radius from the relocate jobs.z9Will use the fix file $FIXhwrf/hwrf_storm_radius instead. storm_radiusTrDrzstorm_radius file is missing) rr5rrrgetdirwarningrrrrStormRadiusError)r4rrIrrr>r>r?check_storm_radiuss$  z#MergeGFSHWRFEnKF.check_storm_radius tcvitals.asc Csb|dkr|}|dt|ft|d}||jdW5QRXtj |s^t dS)z!Writes the tcvitals (from self.storminfo) to the specified file. @param filename the file to write @param logger a logging.Logger for log messagesNzWriting tcvitals to %sr: ) rrr]r?r@r as_tcvitalsrrexistsr/)r4rBrfr>r>r? write_vitalss  zMergeGFSHWRFEnKF.write_vitalsc Cs||}tt|ddddk}|dk}|d|zt||dWn.tk rv}z|dW5d }~XYnXd S) zLRuns interpolation to merge HWRF EnKF analysis to GDAS EnKF analysisZhwrf_interpolateT)allranksr7Zstdout_interpolatez'Starting hwrf_interpolate for domain:%srzhwrf_interpolate failedN)rrrrrr Exceptioncritical)r4rtrcmdrSr>r>r? run_intrp s zMergeGFSHWRFEnKF.run_intrpcCsHt|jd|jtj|jdd|_t|jd|jtj|jdd|_dS)z!prepares products for deliver_products() and products() Generates produtil.datastore.FileProduct objects for the files that should be delivered in deliver_products() and listed by products().r8)rrFN) rr5r9rrrr_prod_wrfinput_prod_wrfanl_d02r@r>r>r?r60szMergeGFSHWRFEnKF.make_productscCs<tjtj|jj|jjddd|j jddddS)z:!delivers products that were identified by make_products()r8F)frominforrFN) r fileopmakedirsrrdirnamerZrdeliverr[r@r>r>r?deliver_products;s  z!MergeGFSHWRFEnKF.deliver_productsccs|jV|jVdS)z!iterates over all products Iterates over all of this Task's products (produtil.datastore.FileProduct objects) created by make_products()N)rZr[r@r>r>r?ruBszMergeGFSHWRFEnKF.productsc Cs|}|j}|d|ftj|rD|d|ft|t||j dH| | | d|j D]}||||qv|W5QRXdS)z,!Merge EnKF analysis and deliver the resultsz$Run hwrf_interpolate in directory %sDelete old data in %srrON)rrrrrrRshutilrmtreerr'rGrNrTrrCrYra)r4rr_rtr>r>r?r$Js      zMergeGFSHWRFEnKF.runcCs ||S)z!Returns the wrfanl output product for the specified domain and time or None if no such data is available. @param atime the time of interest @param domain the domain of interest)rsr4rLrtr>r>r?wrfanl_at_time[szMergeGFSHWRFEnKF.wrfanl_at_timecCs>|dkr dS||jkrdS|jd|jdjdkr:|jSdS)z!Returns the wrfanl product for the specified domain or None if no such data is available. @param domain the domain of interestNr)rr&rpr[r4rtr>r>r?rsbs  zMergeGFSHWRFEnKF.get_wrfanlcCsn|r"|jd|jjdkr"dS|dk rdt||j|jsd|d|d|dfdS| |S)z!Returns a Product object for the wrfinput output file for the specified domain if the atime matches this object's self.sim.simstart() @param atime the time of interest @param domain the domain of interestrNz1wrfinput_at_time: atime=%s is not near my time %s%Y%m%d%H) r&rrrrr/rrstrftimerrrfr>r>r?wrfinput_at_timejs z!MergeGFSHWRFEnKF.wrfinput_at_timecCs,|dk r&|jd|jdjdkr&dS|jS)z!Returns a Product object for the wrfinput output file. If a domain is specified, and is not the correct MOAD, then None is returned. @param domain the domain of interestNrr )r&rprZrir>r>r?rrzs"zMergeGFSHWRFEnKF.get_wrfinput)N)r7)rON)N)rarbrcrdr(rCrGrNrTrYr6rarur$rgrsrlrrrfr>r>r<r?rs $    rc Cs\|d}|r4t|d}||dW5QRXn$t|d}||dW5QRXdS)a!Writes the stormX.run_ensda* flag file. Writs the storm*.run_ensda flag file for this cycle. The purpose of the file is to tell the workflow and scripting layers whether the ensemble needs to be run, whether it is a cycled ensemble, and whether the relocation need to be run for the ensemble The file will contain lines: RUN_ENSDA=YES or RUN_ENSDA=NO RUN_CYCLED_ENSDA=YES or RUN_CYCLED_ENSDA=NO RUN_ENSDA_RELOCATE=YES or RUN_ENSDA_RELOCATE=NO Will also log a message to the jlogfile at INFO level telling which was written. @param flag_file the full path to the flag file @param run_mode True or False: should the ENSDA or relocation job be run? @param flag_name the name of the flag 'RUN_ENSDA' or 'RUN_ENSDA_RELOCATE'ZRUN_atz=YES z=NO N)stripr?r@) flag_fileZrun_modeZ flag_nameZlog_namerSr>r>r?rs    c CsFd}t|rBt|d&}|D]}||ddkrd}qW5QRX|S)aI!Reads the stormX.run_ensda or stormX.run_ensda_relocate* flag file This function is used by the scripting and workflow layers to determine if the data assimilation ensemble should be run, if cycled data assimilation ensemble should be run, and if relocation should be run for the ensemble. Reads the storm*.run_ensda* flag file line by line, searching for a single line RUN_ENSDA=YES/NO or RUN_CYCLED_ENSDA=YES/NO or RUN_ENSDA_RELOCATE=YES/NO. Returns True for YES or False for No, based on the last such line seen. Returns None otherwise.Fr9z=YESr T)rr?find)roZ whichflag run_ensdarSliner>r>r?rs  cs8eZdZdZfddZddZddZdd ZZS) Storm1EnsdazC!Tells HWRF to run the ENSDA for storm 1, but not any other storms.c s4tt|j|||f||d|_|d|_dS)a!Storm1Ensda constructor. Create a new Storm1Ensda which will instruct HWRF to run ensda only for a specific storm priority, by default storm 1. This is intended only for testing purposes. @param dstore the produtil.datastore.Datastore database object @param conf the hwrf.config.HWRFConfig with configuration info @param section the section to use within conf @param kwargs passed to the superclass constructor tdr_flag_filetdr_flag_file2N)r'rsr(r._Storm1Ensda__ensda_flag_file_Storm1Ensda__ensda_flag_file2)r4r5r6r7r:r<r>r?r(s  zStorm1Ensda.__init__cCs|jdd}t|}|dkS)z[!Should ENSDA be run? Returns true if the storm is priority 1, and false otherwise.r% storm_numrh)r6getrk)r4rxr>r>r?should_run_ensdaszStorm1Ensda.should_run_ensdacCs.t|j|t|j||r*td|dS)!Write the ensda flag file. Calls hwrf.ensda.write_ensda_flag_file to write the flag file. @param run_ensda True means the ensemble should be run, False if it should not be run.EnsdaN)rrvrwr#r)r4rqr>r>r?write_flag_files  zStorm1Ensda.write_flag_filecCs||dS)z!creates the storm*.run_ensda file Creates the storm1.run_ensda flag file with RUN_ENSDA=YES for storm 1, and RUN_ENSDA=NO otherwise.N)r}rzr@r>r>r?r$szStorm1Ensda.run) rarbrcrdr(rzr}r$rfr>r>r<r?rss   rscs(eZdZdZfddZddZZS)r z!Determines if GFS EnKF analysis data are available. This class checks to see if GFS EnKF analysis files are available. This is the condition used to decide whether to run the DA ensemble in HWRF.c stt|j|||f|t|jj|_t||jj|_| d}t j |j||j|_ | dd|_| dd|_| dd|_dS) a!GFSEnKFInputCheck constructor. Create a new GFSEnKFInputCheck which will look for GFS EnKF analysis files for specified cycle. @param dstore the produtil.datastore.Datastore database object @param conf the hwrf.config.HWRFConfig with configuration info @param section the section to use within conf @param kwargs passed to the superclass constructorcatalogdatasetr0Zanl_itemZ enkf_siganlitemenkf_sfgN)r'r r(rr6r)rLrftimer.rminput DataCataloginputs enkf_dataset enkf_anl_item enkf_item)r4r5r6r7 cycle_relr: incat_namer<r>r?r(s  zGFSEnKFInputCheck.__init__c Cs|}|ddd}d}tdD]}|d}|jj|j|j|j|||jd}t |sl| |dd}q|jj|j|j |j|||jd}t |s"| |dd}qq"|S) z!Check if GFS EnKF analysis data are available Checks the on-disk input data to see if the GFS EnKF data are available. hwrf_da_ens ensda_size(Trh)rLrrMrz: does not existF) rconfintrangerlocaterrrLrrrLr)r4rZens_sizeZ gfsenkfthereZimemZimembtherer>r>r?r$s4 zGFSEnKFInputCheck.run)rarbrcrdr(r$rfr>r>r<r?r s cspeZdZdZfddZddZddZdd Zd d Zd d Z ddZ ddZ ddZ ddZ ddZZS)rz!Determines if Tail Doppler Radar (TDR) data is available. This class checks to see if a specified cycle has Tail Doppler Radar data available. This is the condition used to decide whether to run the DA ensemble in the 2015 Operational HWRF.c srtt|j|||f||d}|d|_|d|_d|_t||j|_ t j |j ||j |_|dd|_|dd|_|d d |_t|j|j|j|j|j |j d d |_|j|_|d d|_|j ddd}|jj}t|dkrd|jj|_nl|dkr$dd|jjf|_nJ|dkrFdd|jjf|_n(|dkrhdd|jjf|_nd|_dS)a!CycleTDRCheck constructor. Create a new CycleTDRCheck which will look for TDR for the specified cycle. The cycle_rel is anything accepted by to_datetime_rel's second argument. @param dstore the produtil.datastore.Datastore database object @param conf the hwrf.config.HWRFConfig with configuration info @param section the section to use within conf @param cycle_rel specifies the cycle. This must be a number of hours relative to the current cycle (conf.cycle) analysis time. For example, -6*3600 would be the prior cycle and 48*3600 would be two days from now. @param kwargs passed to the superclass constructorr~rtruNrZtdrrZ gdas1_bufrobstypetldplrTrrrrLrr ensda_optr rjiz%02drz%s%02d1r2CP3Z999) r'rr(r._CycleTDRCheck__ensda_flag_file _CycleTDRCheck__ensda_flag_file2_CycleTDRCheck__run_ensdarr)tgtcyclermrrr6_CycleTDRCheck__in_catalogrrrr0taskvars_CycleTDRCheck__tdrdictr_dirnamerrrkrrrkstnumstormidr) r4r5r6r7rr:rZyyyyrr<r>r?r(sB   zCycleTDRCheck.__init__cCs|jdkrt|jd|_|jS)z!Should ENSDA be run? If self.run was called in this process, returns the cached result of that. Otherwise, reads the run_ensda flag file from COM. Uses hwrf.ensda.read_ensda_flag_file()N RUN_ENSDA)rrrr@r>r>r?rzFs  zCycleTDRCheck.should_run_ensdaccs |jVdS)a;!Iterates over needed files from upstream workflows. This iterator's purpose is to provide "all files external to this workflow" that are needed by the task. In this case, of course, it is only the TDR bufr_d file. Hence, there is a single "yield" statement that requests the TDR.N)rr@r>r>r?rZQszCycleTDRCheck.inputiterc Cs|}zzn|d|jftj|jrJ|d|jft|jt |j|j d| \}}}W5QRXWn:t k r}z|j dt|ddW5d}~XYnXW5|||XdS)z!creates the storm*.run_ensda file Creates the storm1.run_ensda flag file with RUN_ENSDA=YES if the TDR data is available, and RUN_ENSDA=NO otherwise.zRun ensda_pre in directory %srbrcz1unhandled exception running self._actually_run() Texc_infoN)rr}rrrrrRrdrerr' _actually_runrVrLrz)r4rrqrun_ensda_cycled switch_stormrSr>r>r?r$Zs  zCycleTDRCheck.runcCsNt|j|dt|j|dt|j|dt|j|d|rJtd|dS)r{rRUN_CYCLED_ENSDAr|N)rrrr#r)r4rqrun_cycled_ensdar>r>r?r}os zCycleTDRCheck.write_flag_filec Cs|}t|jj}tj|j|d||_|j dd}t j | dd}|jj|||dd}t j ||}|d|t|r|d d S|d }|jsht|j|j|j|j||d d |_|jjf|j}|dkr|dttfdSt|s |d||jd fdSt |dd |d|!|j"|tdrV|dd S|ddSn<|j dd} t j# dd} |j | d| } d} t j $| rxt%d|} t j | |dd }t j | | dd }|d!||ft|st|rrt&t'd"t j# d#d$f|d d%dfj(| |j)d&d'}t|rld } t |dd |d|!|j"|nd} nd} n|d(| dStdr| d fS| dfSdS))z!Check if TDR data is available for this cycle Checks the on-disk input data to see if the TDR data is available.r~Ztdr_new_obstyperintercombufrprepr)rLrrzTDR bufrprep could be at %szFind TDR data for this storm)TTrjTrNzGConfiguration error: DataCatalog %s does not know how to find TDR data.FF?%s: %s Tail Doppler Radar bufr_d file is empty or non-existant. tldplrbufrrJrunensdazKFind TDR data for this storm.The data may be a trigger file or a small dumpz#Find TDR data for a different storm)TFr% fcst_catalogDCOMROOT/dcomdcomF逮%Y%m%d b006/xx070"Locations: tank1 at %s tank2 at %s %s/dumpjbUSHobsproc_dump"/you/forgot/to/set/USHobsproc_dump3.00ZTANKDATA tldplr.ibmU%s does not exist. This is not wcoss.real-time ensda trigger can only be run on wcoss)*rrr6r)rmrrr.Z _in_catalogryrrrrKrArrrkrealtimer0rrrrZthiscycletdrdictrrrr]icrLrrreadensdatriggerrenvironisdirrr$r envr)r4rrLrdsitrymdh there_it_is input_catalogdcomenvr tdrpresentbtimetank1tank2tdrbufrr>r>r?tdr_this_cycle|s           zCycleTDRCheck.tdr_this_cyclecCs,|}|jdd}|jrB|j|sB|d|ftj| \}}|r\|}|}nh|jrt j dd}|j|d|}t j |rd} qd} nd} || \}}|r|d |jd d} |jd kr|\} } | s||r@| sd} d}|d |jjd |d n d} d}|d|jjd n:| sZd} d}|dn d} d}|d|jjd n|rd} d}|dnf|rtdrd} d}d} |d|jjd |dnd} d}|dnd} d}|dn$|} d}|r|dn |d| || fS)a!Search on disk for TDR or a trigger file. Do not call this routine directly; it is an internal implementation routine. This routine contains the code that actually determines if TDR is present or not. The "run" routine calls _actually_run in a try-finally block to ensure the run_ensda flag file is created no matter what. This function returns a tuple (tdrpresent, is4thisstorm)r%rzFThis is a realtime run. Forecast catalog section [%s] does not exist. rz/you/forgot/to/set/DCOMROOTrTFz'There will be P3 TDR data for cycle %s!rjr zFP3 TDR data found for this storm. Enabling cycled ENSDA from cycle %s!rhzKP3 TDR data found for this storm. Will continue cycled ENSDA from cycle %s!zBNo P3 TDR data found for this storm, Will continue without ENSDA.zyNo P3 TDR data found for this storm. ENSDA has been enabled in previouse cycle. Will continue cycled ENSDA from cycle %s!z=P3 TDR data found for this storm. Will continue cycled ENSDA.ZP3TDRzTFind P3 TDR data for a different storm, Will discontinue cycled ENSDA from cycle %s!rz?Find G-IV TDR data. Will continue cycled ENSDA for this storm.z.No TDR data found. Will continue cycled ENSDA.z.TDR data found for this storm. Enabling ENSDA.zVTail Doppler Radar bufr_d file is empty or non-existant. Will continue without ENSDA.)rr6ryr has_sectionrrmrUnexpectedFailureTestrrrrrread_trigger_filerrrkrreadpriorensdar)write_cycled_ensda_alert_filerrL)r4rrZtdrpresent_this_cycleZis4thisstorm_this_cyclerZ is4thisstormrrZrealtimeonwcossrZrun_ensda_priorZrun_ensda_cycled_priorrqrr>r>r?rs                   zCycleTDRCheck._actually_runc Csv|}t|j}|d}d}|s|j}|jf|j}|dkrX|dt|fdSt |s~| d||jdfdSd}| d||jdft |d d|d | |j|n|jd d }tjd d} |j|d| } tj| rBtd|} tj| |dd} tj| | dd} | d| | f|dd}|dd}|d}d}d|_||krXt | st | rttdtjddf|dddfj| |jdd }t |rd}t |d d|d | |j|nd}nd}||kr4td!|||dd"ddd#d$ r4| d%|d}|d7}qjn| d&| |dfSt d!rj|dfS|dfSdS)'a)!Read TDR trigger file for operational run Reads the TDR trigger file sent by the Aircraft Operations Center (AOC) before a NOAA P3 Orion flight. This is used in NCEP Operations when running the operational HWRF, to determine if TDR is going to be available soon.rjFNzeConfiguration error: DataCatalog %s does not know how to find TDR data. Will continue without ENSDA.rrTz %s: TDR data found for cycle %s!rrJr%rrrrrrrrZ numofcheckrhZ checksecinvi,rr rrrrrrrrg?)maxwait sleeptimemin_size min_mtime_age min_atime_age min_ctime_age min_fractionzfound trigger filer)rrrrkrrrrr]rrLrrrrr6ryrrrrrrrZnumtiredr$r rrr)r4rrrLrrrrrrrrrrZnumtryZtimeinvstimerrr>r>r?r;s          zCycleTDRCheck.read_trigger_filecCs^|d|}d}||}tj||d|>}|jrJ|tk}tjj||ddS)z!Runs the hwrf_readtdrtrigger program. Runs the hwrf_readtdrtrigger program to decide if the TDR trigger file is for this storm or not. @param stmidin the storm @param tgtcycle the cycle of interestrZhwrf_readtdrtrigger rN)rrrr r$r redirectr)r4ZstmidinrrZfprogprogrXr>r>r?rs zCycleTDRCheck.readensdatriggerc Cs&|}d}d}tj|dr |d}t||ddddd}|sZd |}||nzzHt |d 4}|D](}| d dkrd}| ddkrnd}qnW5QRXWn>t k r} z |jd|t| ddW5d} ~ XYnXW5|d t|d t|Xn||dd||fS): !Read RUN_CYCLED_ENSDA from prior cycle storm1.run_ensda F{oldcom}G{oldcom}/run_ensda.{oldvit[stnum]:02d}{oldvit[basin1lc]}.{oldvit[YMDH]} rhrrrrr4%s: did not exist or was too small after 150 secondspreviouse cycle RUN_ENSDA=z RUN_CYCLED_ENSDA=r9 RUN_ENSDA=YESr TzRUN_CYCLED_ENSDA=YESunhandled exception reading rN does not exist.) rrrricstrrrLrrzr?rprV r4rrqrroZwaitedrrSrrrSr>r>r?rsR    zCycleTDRCheck.readpriorensdac Csl|dkrd}nd}|jdd|d}|d}t|d&}||tjjd |d W5QRXd S) z!Writs the alert_cycled_ensda file for this cycle. The purpose of the file is to tell a cron job to alert HWRF team that cycled ensemble will be launched or shut down @param alert_mode the alert mode - 1: launch, 2: shut downrhz5Find HWRF ensemble trigger for this storm, will startz:Find HWRF ensemble trigger for a different storm, will endr%zATTN HWRF team: aM cycled HWRF ensemble hybrid DA HWRF storm slot {storm_num} Storm = {vit[basinname]} Tropical System {vit[stormname]} {vit[stormid3]} R.S.M.C. = {vit[center]} Storm slot = hwrf{storm_num} Fcst. cycle = {ayear}-{amonth}-{aday} {aHH} Sincerely, HWRF z%{com}/{out_prefix}.alert_cycled_ensdarmzWrite z for this cycleN) r6 timestrinterprr?r@r rr r)r4Z alert_modeZ alert_msgcontentZ alert_filerSr>r>r?rs    z+CycleTDRCheck.write_cycled_ensda_alert_file)rarbrcrdr(rzrZr$r}rrrrrrrfr>r>r<r?rs .   RmQ#cs0eZdZdZddZfddZddZZS)rz!Used in place of CycleTDRCheck to force ENSDA to always run. This subclass of CycleTDRCheck instructs ENSDA to run whether TDR is available or not.cCsdS)z!Returns True. Always returns True, indicating that ENSDA should always be run even if the world is imploding and pigs are raining from clouds of cotton candy. @returns TrueTr>r@r>r>r?rzszAlwaysRunENSDA.should_run_ensdacsJtt|\}}}|s4d}||t||\}}|||fS)!Runs the TDR check and ignores its results. Calls the superclass _actually_run, so that the TDR check is done, and then returns True regardless of the TDR availability. Logs a warning about this to the produtil.log.jlogger. @returns Truez>OVERRIDE: Will run ENSDA anyway due to configuration settings.)r'rrrrLr r)r4rqrrrr<r>r?rs  zAlwaysRunENSDA._actually_runc Cs|}d}d}|jdkr tj|dr|d}t||dddd d }|sdd |}||nzzz6t |d "}|D]}| ddkrxd}qxW5QRXWn>t k r} z |jd|t | ddW5d} ~ XYnXW5|d t |Xn||dd||fS)rTFr rrrrrhrrrrr9rrrNr) rrrrrrrrLrrzr?rprVrr>r>r?rsL     zAlwaysRunENSDA.readpriorensda)rarbrcrdrzrrrfr>r>r<r?rs cs eZdZdZfddZZS)CheckVmaxEnsdazThis subclass of CycleTDRCheck instructs ENSDA to run when Vmax exceeds certain threshold, whether TDR is available or not.cstt|\}}}|dd}|jj|kr|sd}d|}||t||j dkrd}d|}||t||||fS)rZrun_ensda_wmaxg1@Tz5Will run ENSDA due to Vmax exceeding threshold %s m/sr ) r'rr conffloatrwmaxrrLr r)r4rqrrZ vmaxthresholdrr<r>r?r%s"    zCheckVmaxEnsda._actually_run)rarbrcrdrrfr>r>r<r?r!srcCs|ddd}|dkr,tj||d|S|dkrDtj||dS|dkr^tj||d|S|dkrxtj||d|Stdt |fd S) a!Generates a CycleTDRCheck or AlwaysRunENSDA based on configuration settings. Reads the [config] section ensda_when option to decide what TDR check class should be used. * ensda_when="tdr_next_cycle" - create a CycleTDRCheck * ensda_when="always" - create an AlwaysRunENSDA * ensda_when=anything else - raise an exception @param ds,conf,section,next_cycle - passed to the constructorr% ensda_whenZtdr_next_cycle ensda_prestorm1alwaysZ tdr_and_wmaxzXThe ensda_when option must be set to tdr_next_cycle or always (case-insensitive) not %s.N) getstrlowerrmrrrsrr ValueErrorr])rr6r7 next_cyclerr>r>r?r?s2 cs:eZdZdZd fdd ZddZddZd d ZZS) EnsRelocateCheckz!Determines if relocation for ensemble should be run This class checks if TCvital Vmax is greater than 14 m/s and if the ensemble atcf have expected forecast hours (6 hour forecast track for 3D application).Nc sFtt|j|||f||d|_||_|dkr8|j}t||_dS)an EnsRelocateCheck constructor @param dstore the produtil.datastore.Datastore database to use @param conf the hwrf.config.HWRFConfig that provides configuration data @param section the section in conf to use @param atime the analysis time @param ensda the hwrf.ensda.DAEnsemble that provides ensemble atcf filesens_rlct_flag_fileN) r'rr(r.%_EnsRelocateCheck__ens_rlct_flag_filerr)rrL)r4r5r6r7rLrr:r<r>r?r(_s  zEnsRelocateCheck.__init__cCs|}|dt}|j|D]8\}}t|}||}tj |j } | | |j fq$| d} |D]0\} }d|} t | } | sn|dd} qqn| S)z!Check if the atcf of ensemble members have 6 hour forecast track @atime analysis time @param ensda the hwrf.ensda.DAEnsemble that provides the atcf fileszin check_ensda_atcfTzgrep "HWRF, 006," %sz6hr forecast not in atcfF)rrlistrrRrkrrrgetsizerappendsort subprocessZ getoutput)r4rLrrplistensrXiensprodrrun_ens_relocate_rXZf6hrr>r>r?check_ensda_atcfns$     z!EnsRelocateCheck.check_ensda_atcfc Csd|}|jdddd}z:|jj|ddkr<|dd}||j|j sPd}W5||XdS) z!create the storm*.run_ens_relocate file Creates the storm1.run_ens_relocate file with RUN_ENS_RELOCATE=YES, if TCvital vmax is greater than 14 m/s and ensemble atcf files have expected forecast hoursrun_ens_relocationFr%rZ intensityming,@zwmax is smaller than 14 m/sN) rrr}rrrrrrLr)r4rr r>r>r?r$s zEnsRelocateCheck.runcCst|j|ddS)z!Write the ensemble relocate flag file. Calls hwrf.ensda.write_ensda_flag_file to write the flag file. @param run_ens_relocate True means relocation for the ensemble should be run, False if it should not be run.RUN_ENSDA_RELOCATEN)rr)r4r r>r>r?r}sz EnsRelocateCheck.write_flag_file)NN) rarbrcrdr(rr$r}rfr>r>r<r?rYs rcCs0|dkr|d}t|r,tjj|d|ddS)aT!Delete the enkf status file. Deletes all EnKF status files, whose paths are determined from the given config object. If the logger is not specified, the enkfstatus subdomain of the conf default logging domain is used. @param conf the hwrf.config.HWRFConfig object @param logger Optional: a logging.Logger for logging.N ensdastatusT)rr)rrr r] remove_file)r6rorr>r>r?unset_ensdastatuss rcCs|dkr|d}tj|d|}t||dt||d|dd}tjj |||dd |dd }tjj |||dd dS) am!reset ensda flag file due to job failure. The flag file is used to fall back to the run without ensda. @param conf the hwrf.config.HWRFConfig object @param flag_file name of flag file @param run_ensda True means run ensemble forecast @param run_cycled_ensda True means run cycle ensda @param logger Optional: a logging.Logger for logging.NrWORKhwrfrrrrtTrEru rrrrrKrrr r]r)r6rorqrrflag_tmprr>r>r?reset_ensda_flag_files      rcCsV|dkr|d}tj|d|}t||d|dd}tjj |||dddS) aA!reset ensda flag file due to job failure. The flag file is used to fall back to the run without ensda. @param conf the hwrf.config.HWRFConfig object @param flag_file name of flag file @param run_ens_relocate True means run relocate for ensemble @param logger Optional: a logging.Logger for logging.NZensda_relocate_statusrrensda_relocate_prerTrEr)r6ror rrrr>r>r?reset_ensda_relocate_flag_files    r)N)N)N)Erd__all__rrrdprodutil.datastorer  produtil.cdprodutil.fileop produtil.log hwrf.hwrftaskrm hwrf.numerics hwrf.prep hwrf.fcsttask hwrf.input hwrf.regrib hwrf.trackerhwrf.exceptionsr r r rrrrrrrrrr hwrf.postrrhwrf.wrfr produtil.runrrrrrr r! hwrf.enkfr"produtil.ecflowr#r$hwrftaskHWRFTaskrrrrrrrrsr rrrrrrrrr>r>r>r?s`            j1iJ(8PBK