U

����� g������������������������@���sH��d�Z�dddddgZddlZddlZddlZddlZddlZddlZddlZddl	Z
ddlZ
ddlZ
ddl
Z
ddlZ
ddlZddlZddlZddlZddlmZ�dd	lmZmZ�dd
l
mZmZmZmZmZ�ddl	mZ�ddlT�ddlT�dd
lmZ�ddlT�dd��Zddd�Ze Z!G�dd��dej"j�Z#G�dd��de�Z$G�dd��de�Z%dS�)zu!Runs the Unified Post Processor on outputs from the WRF-NMM,
producing E grid GRIB files as EGRIB1Product objects.  �
PostOneWRF�PostManyWRF�
EGRIB1Product�
check_post�
link_post_fix�����N)�
PostFailed)�	GRIB1File�
UpstreamGRIB1)�FileProduct�	COMPLETED�FAILED�RUNNING�	UNSTARTED)�TempDir)�*)�HWRFTaskc��������������
���C���s���|�dkr�|��d||�f���tdd���}|�dtj��|���}td|�}|�|�tj��|�|�}d|�d�kr�|��d|||f���|�	��}|d	d
��D�]}|��d|f���q�W�5�Q�R���dS�W�5�Q�R�X�tj
�d�r�tdd
��}W�5�Q�R�X�d
}	d
}
d}tj
�d��s4|�d|f���ttd�d���ttd�d���dS�tdd
���}|�d�D�]�}zFt�d|�}
|
�r�|
���\}}|dk�r�t|�}	|dk�r�t|�}
W�nJ�ttttfk
�r��}�z"|�s�d}|j�d|f�dd��W�5�d
}~X�Y�nX��qJW�5�Q�R�X�|	d
k�r|��d|f���|
d
k�r"|��d|f���|	d
k	�o2|
d
k	}t�d�D�](}|�d||	|
|f���||	|
|f��S�|��d|f���ttd�d���ttd�d���dS�) a��!Did the post run successfully in the current working directory?

    Checks the current working directory and the specified return
    value retval from the post to determine if the post succeeded.
    Returns a four-element tuple (ok,cenla,cenlo,filename) where "ok"
    is True if the post succeded, (cenla,cenlo) is the domain center
    and filename is the name of the post output file.
    @param retval the post return value
    @param what for log messages: what was posted?
    @param logger the logging.Logger for log messagesr���z%%s:Non-zero exit status %d from post.�	vpost.log�rbi��������zALL GRIDS PROCESSEDzj%s: Did not find "ALL GRIDS PROCESSED" in last %d bytes of vpost.log (file size %d).  Post probably failedi����Nz
vpost.log: %s)FNNN�rtF�this-domain-center.ksh.inczG%s: this-domain-center.ksh.inc is empty or does not exist.  WRF failed.Zlsz-ltr�tail)z-20r���z (clat|clon)\s*=\s*([+-]?[.0-9]+)ZclatZclonTz6%s: Exception while parsing this-domain-center.ksh.inc��exc_infoz5%s: Could not get clat from this-domain-center.ksh.inz5%s: Could not get clon from this-domain-center.ksh.inzWRFPRS*z'%s: success: cenla=%s cenlo=%s file=%s.z%s: No WRFPRS* files found.)�warning�open�seek�os�SEEK_END�tell�min�read�find�
splitlines�path�exists�produtil�fileop�
isnonempty�error�run�exe�	readlines�re�search�groups�float�KeyError�
ValueError�	TypeError�AttributeError�glob�info)�retval�what�logger�fZfilesize�readsizeZvpostdatZlinsZlin�cenla�cenloZsentexc�line�mZvarr�where�e�ok�filename��rD����6/lfs/h1/ops/prod/packages/hmon.v3.2.7/ush/hwrf/post.pyr������s�����

��
�
�
���&
�
�
�Fc�����������������C���s����|dk	r&|��dtt����t|�f���dD�]*}tj�|�dd�}tjj	|||ddd��q*|r�tj�|�d	�}g�}d
D�](}|�
d||f���|�
d||f���qpd
D�]}|�
d|||f���q�t|d|d|d��dS�)z�!Links or copies all fix files for the post to the current
    working directory.
    @param fixd the fix directory
    @param needcrtm flag: is the CRTM data needed?
    @param logger the logging.Logger for log messagesNz*%s: link post fix files here.  Needcrtm=%s)zeta_micro_lookup.datznam_micro_lookup.datzhires_micro_lookup.datzhwrf-wrfzETAMPNEW_DATA.expanded_rainTF)r9����keep�preserve_permszhwrf-crtm-2.0.6)Z
amsre_aquaZimgr_g11Zimgr_g12Zimgr_g13Zimgr_g15Z	imgr_mt1rZimgr_mt2Z
seviri_m10Zssmi_f13Zssmi_f14Zssmi_f15Z	ssmis_f16Z	ssmis_f17Z	ssmis_f18Z	ssmis_f19Z	ssmis_f20Ztmi_trmmzv.seviri_m10Zimgr_insat3dZ
ahi_himawari8z&%s/SpcCoeff/Big_Endian/%s.SpcCoeff.binz&%s/TauCoeff/Big_Endian/%s.TauCoeff.bin)ZAerosolZEmisZCloudz!%s/%sCoeff/Big_Endian/%sCoeff.bin�.)r9����force�copy)r6����strr����getcwd�reprr$����joinr&���r'����deliver_file�append�make_symlinks_in)�fixd�needcrtmr9���rJ����tgt�srcZcrtmdZlinksrD���rD���rE���r���n���s*����
���
�c�������������������@���sV���e�Zd�ZdZddd�Zdd��Zdd��Zd	d
��Zdd��Ze	eddd
�Z
e	eddd�ZdS�)r���z�!Represents an E grid WRF-NMM GRIB1 file, and stores two
    metadata values: CENLA and CENLO which contain the domain center
    location.Nc��������������	���C���s����|dk	st��tj�|�}tj�|�s.t�|��t|d�|d|d��|d�}|d�}|dk	s^t��|dk	sjt��|�j����B}||�d<�||�d<�|��	|d��|��
���d|�ks�t��d|�ks�t��W�5�Q�R�X�|�j|d��dS�)	at��!Copies the file to its destination, and sets the CENLA and
        CENLO metadata values to the domain center.
        @param location the destination location
        @param fileinfo a dict containing fromloc, the location;
        CENLA, the domain center latitude; and CENLO, the domain
        center longitude.
        @param logger a logging.Logger for log messagesN�fromlocF�rF���r9����CENLA�CENLOT�r9���)�AssertionErrorr���r$����dirnamer%����makedirsrO����dstore�transaction�
set_loc_avail�update�call_callbacks)�self�locationZfileinfor9���r\���r<���r=����trD���rD���rE����deliver����s"����
zEGRIB1Product.deliverc�����������������C���s���t�j�t�j�||�j�|�j�S�)zy!Decides a filename which is of the format
        "outdir/category/prodname".
        @param outdir the output directory)r���r$���rN����category�prodname)rc����outdirrD���rD���rE����
make_location����s����zEGRIB1Product.make_locationc�����������������O���sL���|�j�}|�dtj�|���\}}tjj|||jd��t	|dd|�d�|�d��S�)z�!Delivers the  file.
        @returns a hwrf.regrib.GRIB1File for the file.
        @param regrib the hwrf.regrib.Regrib with input information
        @param args,kwargs ignoredzprod.rZ���NrX���rY���)
rd���Zgribtempr���r$����basenamer&���r'���rO���r9���r���)rc����regrib�args�kwargs�locrC����indexrD���rD���rE����make����s����zEGRIB1Product.makec�����������������C���s���|�d�S�)z!The domain center latitude.rX���rD����rc���rD���rD���rE����getnscenter����s����zEGRIB1Product.getnscenterc�����������������C���s���|�d�S�)z!The domain center longitude.rY���rD���rr���rD���rD���rE����getewcenter����s����zEGRIB1Product.getewcenterz7Returns None or the center latitude of this GRIB1 file.z8Returns None or the center longitude of this GRIB1 file.)N)�__name__�
__module__�__qualname__�__doc__rf���rj���rq���rs���rt����propertyZnscenterZewcenterrD���rD���rD���rE���r�������s���
	��c�����������������������s����e�Zd�ZdZd#��fdd�	Zdd	��Zd
d��Zdd
��Zdd��Zdd��Z	dd��Z
dd��Zdd��Zdd��Z
dd��Zdd��Zdd��Zd$d!d"�Z���ZS�)%r���z}!This is an HWRFTask that post-processes output data for a
    single WRF stream, from several WRF domains at a a given time.�historyT����Nc��������������	������s@��t�t|��j|j|j|fd|
i|���||�_||�_t||����	���|�_
||�_||�_i�|�_
i�|�_|dk|�_|snt�|dkrzdnd}|dkr�td��d}
|�jD�]�}d}|dk	s�t�|dk	s�t�|j|g||�j
d	�D�]b}d}
|s�t�d}||�j
|<�|�����d
|�jt|�t|�f���|��|�}t|�j|�j|d�|�j|<�q�q�|
�s<t�dS�)a���!PostOneWRF constructor
        @param wrf the hwrf.fcsttask.WRFTaskBase or subclass, whose
          data is being posted
        @param domains list of domains to post, as hwrf.wrf.WRFDomain objects 
        @param conf the hwrf.config.HWRFConfig that provides configuration ifnormation
        @param section the config section in conf
        @param time the forecast time being posted
        @param stream the WRF stream to post
        @param needcrtm do we need CRTM fix files?
        @param grib GRIB version: 1 or 2
        @param faketime unused
        @param taskname the task name in the database
        @param kwargs additional keyword arguments passed to hwrf.hwrftask.HWRFTask.__init__�taskname�����egrb2�egrbz.GRIB2 support not yet implemented in hwrf.postFTN)�domains�stream�timez%s added %s => %s)rg���rh���)�superr����__init__r^����conf�_PostOneWRF__needcrtm�_PostOneWRF__wrf�to_datetime_rel�wrf�simstart�_PostOneWRF__time�_PostOneWRF__domainsZ_PostOneWRF__stream�_PostOneWRF__wrfproducts�_PostOneWRF__myproducts�_PostOneWRF__grib2r[����NotImplementedError�products�log�debugr|���rM����product_namer���)rc���r����r����r�����sectionr����r����rS����gribZfaketimer|���rn���ZprodextZadded�domain�first�productrh�����	__class__rD���rE���r��������sN������
�

�

��
�zPostOneWRF.__init__c�����������������C���sJ���|�j�r
dnd}|dkr&d|�j|f�}nd|�j|j|f�}t�dd|�}|S�)z�!Returns a human readable string representation of the
        product name for the given domain.  This is used for filenames
        and product ids.
        @param domain the domain of interesr~���r���Nz%s.%sz%s-%s.%sz[^a-zA-Z0-9_.-]�_)r����r|����namer-����sub)rc���r�����ext�resultrD���rD���rE���r������s����zPostOneWRF.product_namec�����������������C���s
���|�j����S�)z"!Returns the WRFSimulation object.)r����r����rr���rD���rD���rE���r������s����zPostOneWRF.wrfc�����������������C���s���|�j�S�)z=!Returns the Task that represents the running WRF simulation.)r����rr���rD���rD���rE����wrftask��s����zPostOneWRF.wrftaskc�����������������o���sr���d|kr.|d�D�]}||�j�kr|�j�|�V��qn@d|krV|d�|�j�krV|�j�|d��V��n|�j�D�]}|�j�|�V��q\dS�)z�!Iterates over products.  

        If the domain keyword is in kwargs, then only the specified
        domain is iterated.  If the domains keyword is in kwargs, then
        that list of domains is iterated.
        @param args,kwargs argument listr����r����N)r����)rc���rm���rn���r����rD���rD���rE���r������s����

zPostOneWRF.productsc�����������������C���sx���d}|�����}|��d|f�|�}||kr0|��d�}|������d|f���tj�d�r`tjj	d|d��tjj
|dd|d	��d
S�)a���!Makes the post control file for the specified stream as
        the file fort.14 in the local directory

        Figure out what control file to use from the conf section for
        this task.  We look for a stream-specific one first (ie.:
        "auxhist2_control").  If that is missing we use a default
        control file from the "control" variable.  The control file is
        then copied to fort.14.
        @bug this function should use deliver_file with loggingz**MISSING**z
%s_control�controlzUse control file %sz	./fort.14zfort.14rZ���TrW���N)r�����confget�confstrr����r���r$���r%���r&���r'����remove_filerO���)rc���r����ZMissingr9���r����rD���rD���rE����make_control*��s����
�
zPostOneWRF.make_controlc�����������������C���s6���|���d�}|�j}|��dd�}t|||����|�d��dS�)z)!Calls link_post_fix() to link fix files.�FIXhwrfZlink_wrf_fixT)r9���rJ���N)�getdirr�����confboolr���r����)rc���rR���rS���Z
link_filesrD���rD���rE����link_fix=��s����
zPostOneWRF.link_fixc�����������������C���s���|�j�S�)z3!Returns the forecast time that is being processed.)r����rr���rD���rD���rE����requested_timeD��s����zPostOneWRF.requested_timec�����������������c���s���|�j�D�]
}|V��qdS�)z2!Iterates over all domains that will be processed.N)r�����rc���r����rD���rD���rE���r����G��s����
�zPostOneWRF.domainsc�����������������c���s(���|�j�D�]}||�j|�|�j|�fV��qdS�)a��!Iterates over wrf domains and products

        Iterates over tuples (domain,wrfproduct,myproduct) where
        domain is the WRF domain, wrfproduct is the WRF output Product
        from that domain for the chosen time and myproduct is the
        output EGRIB1Product for that time.N)r����r����r����r����rD���rD���rE����domprodJ��s����
zPostOneWRF.domprodc�����������������C���s4���t���d�D�]}t�|��q
tj�d�r0t�d��dS�)zJ!Deletes any post output files from the current working
        directory.z	./WRFPRS*r���N)r5���r����unlinkr$���r%���)rc���rC���rD���rD���rE����del_post_outputS��s����zPostOneWRF.del_post_outputc�����������������C���s���t�|||�����S�)z�!Calls check_post() to see if the post succeeded
        @param retval the post return value
        @param what String description of what the post was run on (for log messages)
        @returns True if the post succeeded, False otherwise)r���r����)rc���r7���r8���rD���rD���rE���r���Z��s����zPostOneWRF.check_postc��������������	���C���sd���|�����D�]V\}}}|jrq|js|����|js|�����d|�j|jt|j�t|j�f����dS�qdS�)za!Returns True if the needed inputs are available for the
        post, and False if they are not.z2%s: cannot run: %s not available (loc=%s avail=%s)FT)	r�����	available�checkr����r����r|����didrM���rd���)rc���r����ZwrfprodZmyprodrD���rD���rE����can_run`��s ���������zPostOneWRF.can_runFc�����������������C���s��|�����}|����}d}|tks$|tkr(dS�td|��d�f�dd|����d����}|�����dt������t	�
dt����rvt�|�����t
��}t
��}t��}	t|�j�}
|�j}t|�t|��|
k��r�d	}|����D��]�\}
}}|
j}|
|ks�|����r|�|
��|d
|
jf�7�}n`|����sbtt����}||d�k}t�rZ|�d��ttd
�|d��|�d��tt����}|����|
|k�rnq�|
|	k�r�|	|
�dk�r�|�|
��q�|����r�z|
|	k�r�|	|
��d7��<�nd|	|
<�|�����t|jdd|d��|��|d���tj �!|j�}t	�"d|�}|�r,t|�#���dk�r,|�#��d�}n |�j$�%d�}|�&d|||f���|dd��d�|dd���d�|dd���d�|dd ���d!�|d"d#���d!�|d$d%���}t't(|��)d&��dd'�d(k}|�d)t*|�f���t+d*d+��}|�,d,|f���W�5�Q�R�X�t|�}|��-||�\}}}}|�rh|�d-|f���|j.|�/|�|||d.�|d��|�|
��|d/|
jf�7�}nJ|�r�d0|f�}|�&|��t0|��n&|�d0|f���|d1|
j|	|
�f�7�}W�nP�t1k
�r�}�z0|j&d2|t2|�f�dd3��|d4|
jf�7�}��W�5�d}~X�Y�nX�q�|j�rH|d5t2|j�|
jf�7�}|�rpt3d6|
j|j4|jf���q�|d7|
jf�7�}|r�t3d8|
j|j4f���q�|�|��t|�t|��|
k��r�|�d9��t�5d��|�d:��|�r�W�5�Q�R���dS�q�t|�dk�r�|�6d;��|��7t��q�|�d<��|��7t��q�W�5�Q�R�X�dS�)=z�!Runs the post for one forecast time and all WRF domains.
        @param nosleep If True, disable sleep calls.
        @param raiseall If True, do not catch exceptions.r���Nzpost.%s.rH���z.work)�prefix�dir�suffixr9���zcwd: z\A/tmpzstatus: z[%s: done] ����zCalling sync...�syncrZ���zReturned from sync.����r{���z./INFILET)rI���r9���r����z#(\d\d\d\d.\d\d.\d\d.\d\d.\d\d.\d\d)z%Y-%m-%d_%H:%M:%Sz:%s: %s: cannot get datestamp from this name; will guess %s�����-���������
���r���������
����:�����������������post)�allranksr���zPost command: %s�itag�wtzINFILE
netcdf
%s
NMM NEST
z%s: deliver)rX���rY���rV���z[%s: just posted] z
%s: failedz[%s: post failed %d times] z %s: Exception caught in post: %sr���z[%s: exception] z[%s (%s) unavailable] z;%s: %s: not available (should be at %s but available=False)z[%s: unavailable] z(%s: %s: not available, and location=NonezSleep 30...zDone sleeping.zstate=FAILEDzstate=COMPLETED)8�getstater����r���r���r���r����r6���r���rL���r-����matchr[���r�����set�dict�lenr����ri���r����r�����is_available�add�intr����Zshould_syncr*����batchexer����r�����make_symlinkrd���r����r$���rk���r.���r/���r�����strftimer����mpirun�mpi�getexerM���r����writer���rf���rj���r����	ExceptionrK����PostHasNoInputr�����sleep�critical�setstate)rc����nosleepZraiseall�stater9���Zlastsync�tempdir�doneZgaveupZtries�countri����messager����Z
wrfproductZ	myproductr8����nowZ	shoudsyncZwrf_baser�����	datestamp�cmdr�����retrB���r<���r=���rC����msgrA���rD���rD���rE���r*���n��s>�����







�������
��
��
��
���������



�������������



�

zPostOneWRF.run)rz���Tr{���NN)FF)ru���rv���rw���rx���r����r����r����r����r����r����r����r����r����r����r����r���r����r*����
__classcell__rD���rD���r����rE���r�������s(������������/	c�����������������������s����e�Zd�ZdZedddgdddf��fdd�	Zdd	��Zd
d��Zdd
��Zdd��Z	dd��Z
dd��Zdd��Zdd��Z
dd��Zdd��Zdd��Zdd��Zd d!��Zd"d#��Zd$d%��Zd(d&d'�Z���ZS�))r���z>!A wrapper around PostOneWRF that posts many WRF output times.Nrz���Tr{���c
��������������	������sF��t�t���j|j|j|fd|i|
���|
��_|��_g���_t����_	|��_
�����|d���|	d��\}}}t
��}|	D�]f}zH|D�]4}�����|��|�\}}}||k�r�|}||krx|}qx|�|��W�qn�tjjk
r����Y�qnX�qn|}	|dkr�|n|}|dkr�|n|}|}t||�}t||�}t|�}|��_��fdd�|D����_���|	||��dS�)a0��!PostManyWRF constructor

        @param wrf the hwrf.fcsttask.WRFTaskBase or subclass, whose
          data is being posted
        @param domains list of domains to post, as hwrf.wrf.WRFDomain objects 
        @param conf the hwrf.config.HWRFConfig that provides configuration ifnormation
        @param section the config section in conf
        @param step time step between post input times
        @param postclass should be PostOneWRF
        @param start the first time to post
        @param end the last time to post
        @param streams the streams to consider posting
        @param needcrtm do we need CRTM fix files?
        @param grib GRIB version: 1 or 2
        @param taskname the task name in the database
        @param kwargs additional keyword arguments passed to PostOneWRF.__init__        r|���r���Nc��������������������s���g�|�]}������|��qS�rD���)r����)�.0r����rr���rD���rE����
<listcomp>=��s�����z(PostManyWRF.__init__.<locals>.<listcomp>)r����r���r����r^���r�����	_needcrtm�_PostManyWRF__wrf�	_subtasksr����Z_PostManyWRF__done�
_postclassr�����get_output_range�listrP����hwrf�
exceptions�OutputStreamDisabledr�����to_timedelta�_step�_PostManyWRF__mydomains�
_add_subtasks)rc���r����r����r����r�����stepZ	postclass�start�end�streamsrS���r����r|���rn����istartZiendZ	iintervalZkeepstreamsr����r�����jstartZjendZ	jinterval�intervalr����rr���rE���r�������sN��������
�
��

zPostManyWRF.__init__c�����������������C���s���|�j�}|}t|�d�}t��}|����}t||�}	||	k��r�t��}
|D�]�}|�d|���d}|����D�]�}
|
|f}|
j|||����d�|
|<�|
|�dkr�q`|�dt|
�t|�t|�t|
|��f���|
|��	��}t
t||�dd��}||k�r`|�d	t|�t|�t|�t|�f���|}d}��qq`|rB��q&qB|�r�|�d
|��|�t|�t|�f���|�j�
||�j|����dd��|����D��|�j|�j|||�j|��|�d
�f��|
}n|�d|�d�f���||7�}q.|�����dtt|�j�����dS�)ao��!Generate subtasks

        Fills the self._subtasks array.  It figures out which times
        have data for all domains and creates a PostOneWRF object for
        each of those times.  The subtasks are created in temporal
        order.
        @param streams the list of streams to consider
        @param start,end start and end of the range of times to considerr����z  check stream %sFrZ���Nz'  domain %s stream %s time %s result %sT��negokz  dt=%s-%s=%s > epsilon=%sz%Adding post %s with stream %s time %sc�����������������S���s���g�|�]}|�qS�rD���rD���)r�����xrD���rD���rE���r����k��s����z-PostManyWRF._add_subtasks.<locals>.<listcomp>)r����rS���r|���z>%s: ignoring duplicate output time due to WRF output frequencyz%Y-%m-%d %H:%M:%Szlen(self._subtasks)=%s)r�����to_fractionr����r����r����r����r�����
get_outputrM����	validtime�abs�taskname_forr����rP���r����r�����_conf�_sectionr����r����r����)rc���r����r����r����r�����when�epsilon�lastr9���Zende�thisr����rB���r�����keyZattime�dtZokstreamrD���rD���rE���r����A��sv����	

���������������
zPostManyWRF._add_subtasksc�����������������c���s���|�j�D�]
}|V��qdS�)z%!Iterates over all WRFDomain objects.N)r�����rc���r��rD���rD���rE���r����w��s����
zPostManyWRF.domainsc�����������������c���s8���t�t|�j��D�]$}||�j|�d�|�j|�d�fV��qdS�)a4��!Iterate over all subtasks

        Iterator that loops over all subtasks yielding a tuple:

        @code
          (itask,rtime,subtask)
        @endcode

        Where:
        *  itask = task index from 0 to ntasks-1
        *  rtime = output time this task processes
        *  subtask = the Task objectr���r{���N)�ranger����r����r��rD���rD���rE����subtasks{��s����
zPostManyWRF.subtasksc�����������������C���s"���|������|����D�]}|����qdS�)z1!Calls uncomplete, and then deletes all products.N)�
uncompleter�����	undeliver)rc���r����rD���rD���rE����unrun���s�����zPostManyWRF.unrunc�����������������C���s$���t�|�_|����D�]\}}}t�|_qdS�)z�!Marks this task and all subtasks as incomplete so that all
        post-processing will be rerun.  Does not undeliver any
        delivered products.N)r���r����r��)rc����itask�rtime�subtaskrD���rD���rE���r�����s����zPostManyWRF.uncompletec�����������������C���s&���t�||��������\}}d|�j||f�S�)zo!Returns a human-readable taskname for the given subtask
        time.
        @param time the time of interestz%s-f%02dh%02dm)�
wrf_hr_minr����r����r|���)rc���r�����ihours�iminutesrD���rD���rE���r�����s����zPostManyWRF.taskname_forc�����������������C���s
���t�|�j�S�)z !Returns the number of subtasks.)r����r����rr���rD���rD���rE����
subtask_count���s����zPostManyWRF.subtask_countc�����������������C���s���|�j�d�d�S�)z'!Returns the first time to be processedr���r{���)r����rr���rD���rD���rE����	starttime���s����zPostManyWRF.starttimec�����������������C���s���|�j�t|�j��d��d�S�)z&!Returns the last time to be processedr{���)r����r����rr���rD���rD���rE����endtime���s����zPostManyWRF.endtimec�����������������C���s
���|�j����S�)z$!Returns the WRF object being posted)r����r����rr���rD���rD���rE���r�������s����zPostManyWRF.wrfc�����������������C���s���|�j�S�)z!Returns the Task that ran WRF)r����rr���rD���rD���rE���r�������s����zPostManyWRF.wrftaskc�����������������C���s@���d�}|�����D�].\}}}|jr6|d�kr(|}||kr:|}q�q<q|S�)N)r���	completed)rc���r��r��r��r��rD���rD���rE����
last_complete���s������zPostManyWRF.last_completec�����������������C���s���|�����}|����tkrdS�d}d}|����}d}tj�|��d�|�j�}t	j
�|��|�sN||�|k��rNd}d}d}d}	|����D��]�\}
}}|���}
|
tkr�|d7�}qznN|
t
kr�|d7�}qzn:|���s�|d7�}|�d|jf���||	krz|r�d}��qqztj�|d	|�d
�|
f��}t	jj|dd�}zX|�J�|�d|jf���|jdd
��|����rh|�rhd}W�5�Q�R���W���qW�5�Q�R�X�W�qz�t	jjk
�r��}�z|�d|jf���W�5�d}~X�Y�qz�tk
�r�}�zN|��dd��s�|jd|jt|�f�dd����n|jd|jt|�f�dd��W�5�d}~X�Y�qzX�qz|sN||�|k�rN|�d��t�d��|�d��qN||k�rv|��t��|��d||f���n||k�r�|��t
��|� d��dS�)z�!Internal implementation function: this implements run and
        runone.  Do not call directly.
        @param one True = runone(), False=run()Nr���F�lockdirr����r{���zsubstask %s cannot run yet.Tz	%s.task%dz
%Y%m%d.%H%M%S)rC����	max_trieszrun subtask %s)r����z!subtask %s: lock held, moving on.�
ignore_errorsz,aborting: %s raised unexpected exception: %sr���z"%s raised unexpected exception: %szSleep 20...����zdone sleeping.z All %d of %d subtasks completed.zMULTI-TASK WORKSTREAM FAILED.)!r����r����r���r��r���r$���rN���r����r|���r&���r'���r]���r��r���r����r����r�����locking�LockFiler6���r*����is_completed�LockHeldr����r����r)���rK���r���r����r����r�����postmsgr����)rc���Zoner9���r���failedr����Zbreak_outerr!��Zn_unrunableZ
max_unrunabler��r��r��r�����lockfileZlockerZlhrA���rD���rD���rE����_run_helper���s�������
��� �����





zPostManyWRF._run_helperc�����������������C���s���|���d��dS�)z!Post all inputs.FN�r,��rr���rD���rD���rE���r*������s����zPostManyWRF.runc�����������������C���s���|���d��dS�)z !Post one input time and return.TNr-��rr���rD���rD���rE����runpart���s����zPostManyWRF.runpartc�����������
������k���s����|dkr6|�����D�]"\}}}|jf�|�D�]
}|V��q&qntt||��������}t|�jd��}|�����D�]J\}}}tt||�dd��}	||kr^|	|k�r^|jf�|�D�]
}|V��q��q�q^dS�)aT��!Iterate over products.
        
        @param time Only iterate over this time's products.  The time
        that is actually iterated is the last time that is not before
        the requested time.

        @param kwargs passed to the subtask to further limit the
        products iterated over.  Typically, that is PostOneWRF.products()Nr����Tr���)r��r����r����r����r����r��r����r��)
rc���r����rn���r��Zxtimer��r����Zreltimer��r��rD���rD���rE���r������s����	zPostManyWRF.products)N)ru���rv���rw���rx���r���r����r����r����r��r��r��r��r��r��r��r����r����r ��r,��r*���r.��r����r����rD���rD���r����rE���r������s2��������E6	H)NF)&rx����__all__r5���r����os.path�statr�����mathr-����produtil.cdr&����produtil.fileop�produtil.run�produtil.datastore�produtil.locking�
hwrf.hwrftaskr�����
hwrf.numerics�hwrf.regrib�hwrf.exceptionsr���r���r	���r
���r���r���r
���r���r���r���r���r����fcst_hr_minr��rl���r���r���r���rD���rD���rD���rE����<module>���s.����8  P
,:��&