U  gI@sdZddddddddgZd d lZd d lZd d lZd d lZd d lZd d lZ d d l Z d d l Z d d l Z d d l Z d d lZ d d lZ d d lZ d d lmZd d lmZmZd d lmZmZmZd dl mZmZmZmZd dlmZd dlmZd dlmZd dl m!Z!m"Z"d dlm#Z#d dl$m%Z%Gddde j&j'Z(Gddde j&j'Z)Gddde j&j'Z*ddZ+ddZ,Gddde j&j'Z-Gddde j&j'Z.Gddde.Z/ddZ0d S)a!Utilities for ensemble-based data assimilation. This module contains utilities for doing various forms of ensemble-based data assimilation. It manages a two-dimension ensemble-time array of Task objects that know how to run each analysis cycle for each member. This module also provides a way of presenting the previous forecast cycle's ensda objects such that this cycle can use them. This is all built on top of the classes in the hwrf.fcsttask module. DAEnsembleFromPriorCycle FromGFSENKFwrite_ensda_flag_fileread_ensda_flag_file CycleTDRCheckAlwaysRunENSDAenada_pre_object_forN)jlogger) COMPLETED UpstreamFile) isnonemptywait_for_files make_symlink)to_datetime_rel to_timedelta TimeArray to_datetime) PostManyWRF)NamedDir) WRFDomain)aliasexe)clatlon)set_ecflow_eventcseZdZdZd!fdd ZeddZeddZed d Zed d Z ed dZ eddZ ddZ ddZ ddZddZddZddZddZdd ZZS)"rz[!Represents a two-dimensional ensemble-vs-time array of hwrf.hwrftask.HWRFTask objects.Nc  stt|j||||f||dkr(|j}t||j}||_t|_|ddd}t | d||_ |j t dksvt t||j}t |j||j |j t|_|jj|_dS)a!Constructor for DAEnsemble @param dstore the produtil.datastore.Datastore database to use @param conf the hwrf.config.HWRFConfig for configuration info @param section the section to use in conf @param anlintime the analysis input time @param taskname the name of this task in the database @param kwargs passed to the superclass constructorNconfigcycling_intervalg @Zda_cyclei)superr__init__cycler_DAEnsemble__anlintimeset_DAEnsemble__memberidsgetfloatrconfstrZ_DAEnsemble__tstepAssertionErrorrdict_DAEnsemble__memberslasttime_DAEnsemble__anlouttime) selfdstoreconfsection anlintimetasknamekwargsrendtime __class__7/lfs/h1/ops/prod/packages/hmon.v3.2.7/ush/hwrf/ensda.pyr&s"   zDAEnsemble.__init__cCs|jS)z6!The time at the beginning of the first ensemble step.)r r*r4r4r5r.BszDAEnsemble.anlintimecCs|jS)z/!The time at the end of the last ensemble step.)r)r6r4r4r5 anlouttimeGszDAEnsemble.anlouttimecCs t|jS)z'!The number of members of the ensemble.)lenr"r6r4r4r5nmembersLszDAEnsemble.nmemberscCs t|jS)z&!The number of ensemble DA time steps.)r8r'r6r4r4r5nstepsQszDAEnsemble.nstepsccs|jD] }|Vq dS)z1!Iterates over all ensemble analysis input times.N)r'times)r*tr4r4r5 anlintimesVszDAEnsemble.anlintimesccs0d}|jD]}|rd}q|Vq|jVdS)z2!Iterates over all ensemble analysis output times.TFN)r'r;r))r*firstr<r4r4r5 anlouttimes\s zDAEnsemble.anlouttimesccs|jD] }|VqdS)z!Iterates over all member ids.N)r")r*Zmemberidr4r4r5 member_idsgs zDAEnsemble.member_idscCs|j|||j||<dS)aH!sets the HWRFTask to use to use for one cycle of one member Tells member enkfmem to use the specified task to produce output whose input analysis time is atime. @param atime the analysis time, a datetime.datetime @param enkfmem the enkf member id @param task the HWRFTask to useN)r"addr')r*atimeenkfmemtaskr4r4r5 set_memberls zDAEnsemble.set_memberccs2t||j}|j|D]\}}||fVqdS)z!iterate over members for a specified analysis input time Iterates over all members at the specified anlintime, yielding (id,member) tuples. @param atime the analysis time, a datetime.datetimeN)rr r'items)r*rBtimerCZmemstepr4r4r5members_at_timews zDAEnsemble.members_at_timeccs&||jjD]\}}||fVqdS)z!iterate over members at the final analysis output time. Iterates over all members at the final analysis output time, yielding (id,member) tuples.N)rHr'r()r*emr4r4r5members_at_anlouttimesz DAEnsemble.members_at_anlouttimeccs&|jD]\}}|||fVq dS)z!iterate over analysis cycles for a specific member Iterates over (time,EnsembleDAMemberStep) pairs for the specified member.N)r'rF)r*rCr<ar4r4r5steps_for_memberszDAEnsemble.steps_for_membercCs|j||S)z!get the analysis cycle for the specified member and time Returns the da cycle for member enkfmem at analysis input time atime. @param atime the analysis time, a datetime.datetime @param enkfmem the ensemble id)r')r*rBrCr4r4r5memberszDAEnsemble.memberccs:|jD]*\}}|D]}|D] }|Vq&qq dS)z!iterate over all needed input data, for hwrf.input Calls inputiter for all steps of all ENKF members. This is for use by the hwrf.input to figure out what input data is required for the DA ensemble.N)r'rFvalues inputiter)r*r<rIrJdr4r4r5rPs  zDAEnsemble.inputitercCs|jD]8\}}|D]&\}}tdt|t|t|fqq |jj}tdt|f||D] \}}tdt|t|fqhdS)z!print detailed diagnostics Sends detailed diagnostics about all members to the print() statement. This is intended for debugging only.zself.__members[%s][%s]=%szlast time t is %szself.__members[t][%s]=%sN)r'rFprintreprr(rH)r*rGstuffenkfidZ memberstepr<r4r4r5dumps zDAEnsemble.dump)NN)__name__ __module__ __qualname____doc__rpropertyr.r7r9r:r=r?r@rErHrKrMrNrPrV __classcell__r4r4r2r5r#s0          csJeZdZdZfddZdddZddZd d Zd d Zd dZ Z S)rz!Represents an ensemble member from the previous forecast cycle. This is used to generate UpstreamFile objects for the previous cycle's ensemble of FromGFSENKF simulations.c sDtt|j|||f|dd|D|_t||_tj||_ dS)ai!FromPriorCycle constructor Makes a new FromPriorCycle object. @param dstore the produtil.datastore.Datastore database object @param conf the hwrf.config.HWRFConfig with configuration info @param section the section to use within conf @param domains the list or tuple of hwrf.wrf.WRFDomain objects to obtain from the prior cycle, in order of grid ID. @param enkfmem the enkf member ID @param anlouttime the output time at the end of the analysis @param kwargs passed to the superclass constructorcSsg|]}|qSr4r4.0rQr4r4r5 sz+FromPriorCycle.__init__..N) rrr_FromPriorCycle__domainsint_FromPriorCycle__enkfmemhwrfnumericsr_FromPriorCycle__anlouttime)r*r+r,r-domainsrCr7r0r2r4r5rs zFromPriorCycle.__init__Ncksf|dkr|j}|st|D]F}||jks,t||jkr|rP|||jVq|||jVqdS)a!Iterates over all products Iterates over products produced by the prior forecast cycle's analysis cycle. @param domains if provided and non-None, only products from these domains are yielded @param kwargs ignored N)r`r%is_moad get_wrfinputre get_wrfanl)r*rfr0domainr4r4r5productss  zFromPriorCycle.productscCs |||S)aN!return the product for the specified domain and analysis time Returns the product for the wrfinput_d01 file for the specified domain and time. This is simply a wrapper around get_product(domain,atime) @param domain the domain of interest @param atime the analysis time as a datetime.datetime get_productr*rjrBr4r4r5rhszFromPriorCycle.get_wrfinputcCs |||S)z"!returns the wrfanl file's productrlrnr4r4r5riszFromPriorCycle.get_wrfanlcCs|}tj|}||jks>|dt|t|jfdS||jkrt|dt|ddd|jDfdS|j d|j t | d}|d t|t|t |j t |ft|j|jtj||d }||S) a!Returns a product for the specified domain and time. Creates a new produtil.datastore.UpstreamFile for the wrfinput or wrfanl file for the given domain, at the given analysis time. The analysis time must be the anlouttime. @return a newly created UpstreamFile for the file of interest @param domain the domain of interest @param atime the analysis timeWrong atime: %s vs %sNzInvalid domain: %s not in %sz, cSsg|] }t|qSr4)str)r^xr4r4r5r_sz.FromPriorCycle.get_product..zj{oldcom}/{oldvit[stormnamelc]}{oldvit[stormid3lc]}.{oldvit[YMDH]}.ensda_{enkfid:03d}.wrfinput_d{domid:02d})rUdomidz$Domain %s atime %s enkfmem %s loc %scategoryprodnamelocation)logrcrdrreinforpr`join confstrinterprbra get_grid_idrSr r+r/ospathbasenamecheck)r*rjrBloggerlocufr4r4r5rms<       zFromPriorCycle.get_productcCs|}tj|}||jks>|dt|t|jfdSdt|j}| d|}|dt|t |jt |ft |j |j tj||d}||S)Nro%03dzX{oldcom}/{oldvit[stormnamelc]}{oldvit[stormid3lc]}.{oldvit[YMDH]}.trak.hwrf.atcfunix.memzatime %s enkfmem %s loc %srs)rwrcrdrrerxrprarbrzrSr r+r/r|r}r~r)r*rBrrUrrr4r4r5 get_track s0    zFromPriorCycle.get_track)N) rWrXrYrZrrkrhrirmrr\r4r4r2r5rs   cseZdZdZdfdd ZeddZddZd d Zed d Z d dZ ddZ ddZ ddZ ddZddZddZddZZS)ra! Forecast ensemble member based on the GFS ENKF. Runs one member of an ensemble DA forecast ensemble, using a member of the GFS ENKF ensemble as initial and boundary conditions. Some data from the earlier deterministic forecast jobs is reused to simplify the process.Nc  stt|j||||f| t|tjjs,t|dks<| dkrDtd||_ t ||_ t | |_ | |||||||j r||j r||d| dS)aO!Constructor for FromGFSENKF @param dstore the produtil.datastore.Datastore database to use @param conf the hwrf.config.HWRFConfig that provides configuration data @param section the section in conf to use @param detinit the deterministic initialization, an hwrf.init.HWRFInit or hwrf.init.InitBeforeGSI object. @param enkfmem the ensemble member id @param sim the hwrf.wrf.WRFSimulation object @param taskname the name of the task within the database @param kwargs passed to the parent class constructorNzYou must explicitly specify the track and relocate arguments to FromGFSENKF.__init__ and they must be boolean values, not None.ZENKF)rrr isinstancercwrf WRFSimulationr% TypeErrorrCbooltrackrelocatemake_wrf make_init make_fcst _make_track_make_relocate) r*r+r,r-detinitrCsimr/rr priorcycler0r2r4r5r(s$     zFromGFSENKF.__init__cCs |jS)z!The analysis input time.)_FromGFSENKF__wrfsimstartr6r4r4r5r.oszFromGFSENKF.anlintimec CsVtjj|j|j|d||jd|jd|j ddd|_ ||_ |j j ddS)a~!Create the wrf() and fcst This function, called from the constructor, creates the hwrf.wrf.WRFSimulation and hwrf.fcsttask.AnalysisCycle used for this forecast. @param detinit the deterministic model initialization, an hwrf.init.HWRFInit or hwrf.init.InitBeforeGSI. @param sim the hwrf.wrf.WRFSimulation passed to the constructor.fcsttaskz.fcstz/fcstT)r/outdirworkdirkeeprunN)rcr AnalysisCycler+r,r$copyr/rrfcstrrset_active_io_form_to)r*rrr4r4r5rts zFromGFSENKF.make_wrfcCsn|j|j|j|j|j|j|j D]4}| rBq4|j |j ||jj j dddddq4dS)a!Adds input sources to the forecast object. Adds metgrid, geogrid, wrfinput, wrfbdy, wrfanl, and coupler fort.65 input to the fcst member variable. @param delinit the deterministic model initialization, an hwrf.init.HWRFInit or hwrf.init.InitBeforeGSI.historyr `T)startstependN)r add_geogridgeogrid add_metgridmetgrid add_fort65realinit add_wrfinput add_wrfbdyrrg add_wrfanlwrfanl add_output)r*rrjr4r4r5rs  zFromGFSENKF.make_fcstcCs|jS)z&!The wrf simulation made by make_wrf())rr6r4r4r5rszFromGFSENKF.simccs|jD] }|Vq dS)z!Passes control to the hwrf.prep.PrepHybrid.inputiter(). Iterates over the prep member's inputiter, a hwrf.prep.PrepHybrid.inputiter(). Yields all input data information needed by the hwrf.input module to pull input data.N)preprP)r*rQr4r4r5rPszFromGFSENKF.inputiterc CsJ|j|_|j|_|j}|}|j|}|d}|d}tjj|j |j ||j||j |j d|j d|jdd |_tjj|j |j ||j|j dd|jd|j dd |j|j|j|_|jjd |jd |jtjj|j |j ||jd|j |j d |j d |jd d |j|j|j|j|j|_dS)a!Creates initialization tasks. Called from the constructor. Creates the initialization tasks, prep, realinit and wrfanl; and links to the deterministic init geogrid and metgrid tasks. This is called by the constructor, to create the needed inputs to the fcst member. @param detinit the deterministic model initialization, an hwrf.init.HWRFInit or hwrf.init.InitBeforeGSI.r prep_hybridz.prepz/prep)rBr/rrz.realFz/real)r/rrrrrCz.wrfanlz/wrfanlN)rrrget_moadgeodatr$rcrZ PrepHybridr+r,r.r/rrrRealNMMrradd_prep_hybridrrtvsetrCWRFGhostrrrr)r*rrmoadrZ realsectionZ prepsectionr4r4r5rsr     zFromGFSENKF.make_initc CsDdd|jjD}|d}|g}t|j||j|ddddg|jd|jd |jd d |_t j j |j|d }t j d t j jd }t j j|jd|jdd}t j j|jj|jjd}|jj}|dks|dkr|jdkr|jd7_t j jtt|jdtt|jdd} | dt|ddgddgdddgd| d|| d| dt j | d||jd} d t|j } | !d!| dt j"j#|j$|j|d"| |j%dd|jd#|j%|jdd$ |_&t j j'|j$|j|d%|jd%|j%dd|jd&|jd&d' |_ |j (|j|j&d|j )d(d)| d S)*z/Makes the gribber and tracker member variables.cSsg|]}|qSr4r4r]r4r4r5r_sz+FromGFSENKF._make_track..postrFrz.postz/post)needcrtmstreamsr/rr)rjpartN)rdomlat)rdomlon)latlonALEPgv@copygbwgrib)rrd2g ףp= ?g>@i)ressizescannZtrkgridhwrftrkz /regribberrz'{out_prefix}.hwrftrk.grbf{fahr:02d}.mem regribberz .regribber)rrrr/rBrtrackerz/tracker)r/rrrrrtrack0z){com}/{out_prefix}.trak.hwrf.atcfunix.mem)*rrrr,r$r/rrrrcregribigrb1 GRIBSubsetterrtracker_subset FixedLocation storminforr pubbasin2upperewcenter RegribManyrrgetexerArgridvinttaveGRIBrarC to_intercomgribtaskGRIBTaskr+r.gribber TrackerTaskadd_moving_grid send_atcfunix) r* ensdadomsrZ postdomainsgrid2ZtrksubZdomlocZstormlocbasinrbasedirZenkf000r4r4r5rs   zFromGFSENKF._make_trackcks|jjf|D] }|VqdS)z!Iterates over all forecast products. Passes control to hwrf.fcsttask.AnalysisCycle.products() to iterate over all products that match the specified arguments. @param kwargs passed to hwrf.fcsttask.AnalysisCycle.products()N)rrk)r*r0pr4r4r5rk szFromGFSENKF.productsc Cs|j}|}|d|ftjj|dd|d|j|j|j |j |j r|j }|j|jW5QRX|j|j|jr|jnd}||tj|t|_W5QRXdS)z!Runs the initialization and forecast for this ensemble member. Runs the prep, realinit, wrfanl and fcst member tasks, using input from the GFS ENKF, and the deterministic initialization.zRun ensemble member in %sFT)keep keep_on_errorrzJError regridding inputs to tracker. See earlier log messages for details.N)rrwrxprodutilcdrrrunrrrrr+ transactionrunrunr is_completedrerrorrc exceptions GribberErrorr state)r*whererr<msgr4r4r5rs,             zFromGFSENKF.runcCsbdd|jjD}t|jj||||jd|jdd}|dk rJ|j|d|r^|j|jdd|S) zMakes a dict containing the keyword arguments to send in to the constructor for the hwrf.relocate task(s). modin - the modin argument to the constructor dest_dir - the directory in which to run the relocatecSsg|]}|qSr4r4r]r4r4r5r_8sz5FromGFSENKF._make_relocate_kwargs..z /relocate)rrfmodindest_dirrrN)ensdar)Z parentTrackZ trackName)rrr&rrupdater)r*rrrrrr0r4r4r5_make_relocate_kwargs2s z!FromGFSENKF._make_relocate_kwargscCsjtj|jd}|||||}tjj|j|j | dfd|j ddi||_ |j j |_ |j j|_dS)zMakes the relocation, rstage1, rstage2 and rstage3 member variables. track - the track argument to the constructor modin - the modin argument to the constructorrZtaskname_patternzrelocate.stagez%dN)r|r}ryrrrcrZ Relocationr+r,r$r/Z relocationrstage1rstage3)r*rrrrr0r4r4r5rCs  zFromGFSENKF._make_relocatecCsPd|jkrL|j|jd|jkr2|j|jjrL|jjrL|jdS)z#Runs the relocate jobs, if present.rrN)__dict__rZ delete_temprrscrubr6r4r4r5 run_relocateTs     zFromGFSENKF.run_relocate)NNNN)rWrXrYrZrr[r.rrrrPrrrkrrrrr\r4r4r2r5r s(G   3. c Csj|dkr8t|d}|dW5QRXtjjdn.t|d}|dW5QRXtjjddS)a!Writes the stormX.run_ensda flag file. Writs the storm*.run_ensda flag file for this cycle. The purpose of the file is to tell the workflow and scripting layers whether the ensemble needs to be run. The file will contain a single line: RUN_ENSDA=YES or RUN_ENSDA=NO. Will also log a message to the jlogfile at INFO level telling which was written. @param flag_file the full path to the flag file @param run_ensda True or False: should the ENSDA be run? TwtzRUN_ENSDA=YES z#Will run HWRF ENSDA for this cycle.z RUN_ENSDA=NO z#Disabled HWRF ENSDA for this cycle.N)openwriterrwr rx) flag_file run_ensdafr4r4r5r`s   c CsNd}t|d6}|D]*}|ddkr,d}q|ddkrd}qW5QRX|S)a!Reads the stormX.run_ensda flag file This function is used by the scripting and workflow layers to determine if the data assimilation ensemble should be run. Reads the storm*.run_ensda flag file line by line, searching for a single line RUN_ENSDA=YES or RUN_ENSDA=NO. Returns True for YES or False for No, based on the last such line seen. Returns None otherwise.Nrtz RUN_ENSDA=YESr Tz RUN_ENSDA=NOF)rfind)rrrliner4r4r5rss  cs8eZdZdZfddZddZddZdd ZZS) Storm1EnsdazC!Tells HWRF to run the ENSDA for storm 1, but not any other storms.c s(tt|j|||f||d|_dS)a!Storm1Ensda constructor. Create a new Storm1Ensda which will instruct HWRF to run ensda only for a specific storm priority, by default storm 1. This is intended only for testing purposes. @param dstore the produtil.datastore.Datastore database object @param conf the hwrf.config.HWRFConfig with configuration info @param section the section to use within conf @param kwargs passed to the superclass constructor tdr_flag_fileN)rr rr$_Storm1Ensda__ensda_flag_file)r*r+r,r-r0r2r4r5rs zStorm1Ensda.__init__cCs|jdd}t|}|dkS)z[!Should ENSDA be run? Returns true if the storm is priority 1, and false otherwise.r storm_num)r,getra)r*rr4r4r5should_run_ensdaszStorm1Ensda.should_run_ensdacCs"t|j||rtd|dSz!Write the ensda flag file. Calls hwrf.ensda.write_ensda_flag_file to write the flag file. @param run_ensda True means the ensemble should be run, False if it should not be run.ZEnsdaN)rr rrwr*rr4r4r5write_flag_files zStorm1Ensda.write_flag_filecCs||dS)z!creates the storm*.run_ensda file Creates the storm1.run_ensda flag file with RUN_ENSDA=YES for storm 1, and RUN_ENSDA=NO otherwise.N)rrr6r4r4r5rszStorm1Ensda.run) rWrXrYrZrrrrr\r4r4r2r5r s   r cs`eZdZdZfddZddZddZdd Zd d Zd d Z ddZ ddZ ddZ Z S)rz!Determines if Tail Doppler Radar (TDR) data is available. This class checks to see if a specified cycle has Tail Doppler Radar data available. This is the condition used to decide whether to run the DA ensemble in the 2015 Operational HWRF.c stt|j|||f||d}|d|_d|_t||j|_t j |j ||j|_ |dd}|dd}|dd } t|j||| |j|jd d |_|j|_d |_dS) a!CycleTDRCheck constructor. Create a new CycleTDRCheck which will look for TDR for the specified cycle. The cycle_rel is anything accepted by to_datetime_rel's second argument. @param dstore the produtil.datastore.Datastore database object @param conf the hwrf.config.HWRFConfig with configuration info @param section the section to use within conf @param cycle_rel specifies the cycle. This must be a number of hours relative to the current cycle (conf.cycle) analysis time. For example, -6*3600 would be the prior cycle and 48*3600 would be two days from now. @param kwargs passed to the superclass constructorcatalogr NdatasetZtdritemZ gdas1_bufrobstypetldplrT)rrrrBftimeoptional999)rrrr$_CycleTDRCheck__ensda_flag_file_CycleTDRCheck__run_ensdarrtgtcyclercinput DataCatalogr,_CycleTDRCheck__in_catalogr&taskvars_CycleTDRCheck__tdrdictr_dirname_stormid) r*r+r,r-Z cycle_relr0Z incat_namerrrr2r4r5rs*      zCycleTDRCheck.__init__cCs|jdkrt|j|_|jS)z!Should ENSDA be run? If self.run was called in this process, returns the cached result of that. Otherwise, reads the run_ensda flag file from COM. Uses hwrf.ensda.read_ensda_flag_file()N)rrrr6r4r4r5rs  zCycleTDRCheck.should_run_ensdaccs |jVdS)a;!Iterates over needed files from upstream workflows. This iterator's purpose is to provide "all files external to this workflow" that are needed by the task. In this case, of course, it is only the TDR bufr_d file. Hence, there is a single "yield" statement that requests the TDR.N)r$r6r4r4r5rPszCycleTDRCheck.inputiterc Cs&d}z|rd}W5||XdS)z!creates the storm*.run_ensda file Creates the storm1.run_ensda flag file with RUN_ENSDA=YES if the TDR data is available, and RUN_ENSDA=NO otherwise.FTN)r _actually_runrr4r4r5rs zCycleTDRCheck.runcCs"t|j||rtd|dSr)rrrrwrr4r4r5rs zCycleTDRCheck.write_flag_filecCs|}t|jj}tj|j|d||_|j dd}t j | dd}|jj|||dd}t j ||}|d|t|rd Sd Sd S) z!Check if TDR data is available for this cycle Checks the on-disk input data to see if the TDR data is available.rZtdr_new_obstyperintercombufrprepr)rBrrzTDR bufrprep should be at %sTFN)rwrr,rrcr r!r$Z _in_catalogrr|r}rygetdirparserxr )r*rrBrdsittherer4r4r5tdr_this_cycles$  zCycleTDRCheck.tdr_this_cyclecCsJ|}|jdd}|j|s6|d|fdStjtjddd}|j|d|}|j r|rtj |r| r| d |jj d d S|r| d |jd d S|d dSn||j}|jf|j}|dkr|dt|fdSt|s(|d||jd fdS| d||jd fd SdS)an!Search on disk for TDR or a trigger file. Do not call this routine directly; it is an internal implementation routine. This routine contains the code that actually determines if TDR is present or not. The "run" routine calls _actually_run in a try-finally block to ensure the run_ensda flag file is created no matter what.r fcst_catalogz>Forecast catalog section [%s] does not exist. Disabling ENSDA.FZDCOMROOTz/you/forgot/to/set/DCOMROOTZus007003dcomz:TDR data is available for current cycle %s!Enabling ENSDA.%Y%m%d%HTz3There will be TDR data for cycle %s!Enabling ENSDA.zjensda trigger file is not found. TDR data is not available for current cycle. Will continue without ENSDA.NzeConfiguration error: DataCatalog %s does not know how to find TDR data. Will continue without ENSDA.z]%s: %s Tail Doppler Radar bufr_d file is empty or non-existant. Will continue without ENSDA.z1%s: TDR data found for cycle %s! Enabling ENSDA.)rwr,r has_sectionrr|r}ryenvironrealtimeisdirr/rxrstrftimeread_trigger_filerwarningr"locater$rSr )r*r input_catalogZdcomenvr1icZ there_it_isr4r4r5r's^       zCycleTDRCheck._actually_runc Cs|}t|j}|d}|jj}t|dddkrD|jj|_nf| dkrddd|jjf|_nF| dkrdd |jjf|_n&| d krdd |jjf|_nd |_|j d d}|j |d}t j |rptd|}t j ||dd}t j ||dd} |d|| ft|j|j |d6|dd} |dd} | d} d} | | krt|r|dt|dd|d||j|tds|dd d!krt| rt| dd|d||j|| | krtd|| | dd"d#d#d$d% r|d&| d} | d7} qXtdr4W5QRdS| | dd'}|d(t|W5QRd)SW5QRXn|d*|d)Sd#S)+a)!Read TDR trigger file for operational run Reads the TDR trigger file sent by the Aircraft Operations Center (AOC) before a NOAA P3 Orion flight. This is used in NCEP Operations when running the operational HWRF, to determine if TDR is going to be available soon.r2r irz%s%02d1r2CP3rrr0r1iz%Y%m%dz b006/xx070z"Locations: tank1 at %s tank2 at %s)rrZ numofcheckrZ checksecinvi,rz tank1 existZ tldplrbufrT)forcerZrunensda Z00Ng?)maxwait sleeptimemin_size min_mtime_age min_atime_age min_ctime_age min_fractionzfound trigger filegN@z1waited for %s minutes, ensda trigger is not foundFzU%s does not exist. This is not wcoss.real-time ensda trigger can only be run on wcoss)rwrrr7rrrastnumr&rr,rr|r}r6rryrxrr%rconfintr rreadensdatriggerrrpr9)r*rrBymdhrr;r1ZbtimeZtank1Ztank2ZnumtryZtimeinvstimerZ totalwaitr4r4r5r8Gsz                zCycleTDRCheck.read_trigger_filecCsP|d|}d}||}tj||d|>}tjj||ddS)z!Runs the hwrf_readtdrtrigger program. Runs the hwrf_readtdrtrigger program to decide if the TDR trigger file is for this storm or not. @param stmidin the storm @param tgtcycle the cycle of interestrOZhwrf_readtdrtrigger )rN)rwrxrrrrcheckrun)r*ZstmidinrrZfprogprogcmdr4r4r5rOs  zCycleTDRCheck.readensdatrigger)rWrXrYrZrrrPrrr/r'r8rOr\r4r4r2r5rs !   4@cs(eZdZdZddZfddZZS)rz!Used in place of CycleTDRCheck to force ENSDA to always run. This subclass of CycleTDRCheck instructs ENSDA to run whether TDR is available or not.cCsdS)z!Returns True. Always returns True, indicating that ENSDA should always be run even if the world is imploding and pigs are raining from clouds of cotton candy. @returns TrueTr4r6r4r4r5rszAlwaysRunENSDA.should_run_ensdacs.tt|s*d}||t|dS)a!Runs the TDR check and ignores its results. Calls the superclass _actually_run, so that the TDR check is done, and then returns True regardless of the TDR availability. Logs a warning about this to the produtil.log.jlogger. @returns Truez>OVERRIDE: Will run ENSDA anyway due to configuration settings.T)rrr'rwr9r )r*rr2r4r5r's  zAlwaysRunENSDA._actually_run)rWrXrYrZrr'r\r4r4r2r5rscCst|ddd}|dkr,tj||d|S|dkrDtj||dS|dkr^tj||d|Stdt|fdS) a!Generates a CycleTDRCheck or AlwaysRunENSDA based on configuration settings. Reads the [config] section ensda_when option to decide what TDR check class should be used. * ensda_when="tdr_next_cycle" - create a CycleTDRCheck * ensda_when="always" - create an AlwaysRunENSDA * ensda_when=anything else - raise an exception @param ds,conf,section,next_cycle - passed to the constructorr ensda_whenZtdr_next_cycletdrcheckstorm1alwayszXThe ensda_when option must be set to tdr_next_cycle or always (case-insensitive) not %s.N) getstrlowerrcrrr r ValueErrorrS)r,r,r- next_cyclerVr4r4r5rs$ )1rZ__all__r|produtil.datastorer produtil.cdprodutil.fileop produtil.log hwrf.hwrftaskrc hwrf.numerics hwrf.prep hwrf.fcsttask hwrf.input hwrf.regrib hwrf.trackerhwrf.exceptionsr r r r rrrrrr hwrf.postrrhwrf.wrfr produtil.runrrrprodutil.ecflowrhwrftaskHWRFTaskrrrrrr rrrr4r4r4r5sD          jB&j