U

����� g������������������������@���s��d�Z�dddgZddlZddlZddlZddlZddlZddlZ	ddl
Z	ddlZ	ddlZ	ddl
Z	ddlZ	ddlZ	ddlZ	ddlZ	ddlZ	ddlmZ�ddlmZmZ�ddlmZ�dd	lmZmZmZmZmZ�dd
l
mZ�ddlmZm Z �G�dd��de�Z!G�d
d��de!�Z"G�dd��de�Z#dS�)a(��!Combines Tasks from multiple modules into a single initialization object.

This class combines Tasks from several hwrf modules to make two
different initialization systems: the InitBeforeGSI and the HWRFInit.
Also, the FGATInit is provided, which is an array of InitBeforeGSI,
one per FGAT time.�HWRFInit�
InitBeforeGSI�FGATInit�����N)�set_ecflow_label)�alias�exe)�HWRFTask)�to_datetime_rel�to_timedelta�to_datetime�TimeMapping�to_fraction)�clatlon)�NoSuchDomain�GribberErrorc�����������������������s����e�Zd�ZdZdd��Zdd��Zd+��fd	d
�	Zdd��Zd
d��Zdd��Z	dd��Z
dd��Zdd��Zdd��Z
dd��Zdd��Zdd��Zeeddd�Zeeddd �Zd!d"��Zd#d$��Zd%d&��Zd'd(��Zd)d*��Z���ZS�),r���a���!Runs the deterministic model initialization.

    This class runs the WPS, real, prep_hybrid (if enabled), WRF
    wrfanl and ghost jobs, and runs the tracker to find the parent
    vortex.  Passes that into the relocation (if enabled) to produce
    the final vortex.  It encapsulates that logic within one top-level
    wrapper object, and provides a way to run several parts of the
    initialization from a single function call.c�����������������C���s���t�j�|�j|�}|S�)a
��!Helper function for constructor workdir arguments.

        Generates a workdir location for input to an HWRFTask
        constructor by appending the child name as a new directory
        path component after self.workdir.
        @param childname the child task name)�os�path�join�workdir��selfZ	childname�dir��r����6/lfs/h1/ops/prod/packages/hmon.v3.2.7/ush/hwrf/init.py�_subworkdir<���s����zHWRFInit._subworkdirc�����������������C���s���t�j�|�j|�}|S�)a
��!Helper function for constructor outdir arguments.

        Generates a outdir location for input to an HWRFTask
        constructor by appending the child name as a new directory
        path component after self.outdir.
        @param childname the child task name)r���r���r����outdirr���r���r���r����
_suboutdirF���s����zHWRFInit._suboutdirNFTc�����������&���������s(��t�t���j|||f|��t|���_|r2|
s2td��|dkr@|�}|��_��j}��j}�����}|��_	|��_
|dkr�tj�
���d���j���_
��j}��j}t||������_t||������_|dk	s�t�|dkrވ��dd�}|dk�r�d}|��_|�����_��jj��jd��dd	����jD����_|�����_��jj��jd����j}��j}|dk	�rL��j�|��|���}d
d	��|� ��D��}dd	��|� ��D����_ dd	��|� ��D����_!|�"��|�"��k�r�|
�r�|�r�td
��|���}|d�}|�#��jd���j��dk��s�t���fdd�}��fdd�} t$��}!t$��}"d|k�r<|d�}#|#|!d<�|#|"d<�|dk�r<|#�%��}d|k�rVt|d��|!d<�|dk�rdd}|dk	�rv|��_&n4t'j(j)|||d�|��j| d�|||d�d�	��_*��j*}|
�r|�"��|�"��k�rʈ�j!}$n��j }$t'j+j,|||d�||�-|�f|$| d�||
||d�|d�d�|"����_+|}%n|dk�r,��j}%n|}%���.|
|||||%||
||!||| ||��|�r����/|
|||||||
||!||| ||��t'j0j1|||d�|| d�|d�|d�d��2��j3��4|��5��j6���_7|�r���8|||||| ||��|	dk	�r���9|||	||| ||��|�r$���:|||	|||| ||�	�dS�)ap��!HWRFInit constructor.

        Creates the deterministic HWRF initialization.  Which pieces
        are selected, and how they are plugged in to one another, is
        dependent on the input arguments.

        @param ds passed to Datum: the Datastore object for this Task
        @param conf the conf object for this task
        @param section the conf section for this task
        @param wrf (required) the WRFSimulation representing the full WRF
          forecast
        @param initlen (required) the length of the real_nmm run for the
          initialization 
        @param fcstlen (required) the length of the real_nmm run for the
          full forecast
        @param outdir The directory in intercom to receive output.
          Default: intercom/{taskname}.  Individual initialization
          components will be in subdirectories.
        @param realfcst Flag: do we need to run real for the full forecast
          length?
        @param wrfghost hwrf.wrf.WRFSimulation object for the wrfghost
          files.  If absent, no wrfghost-related jobs will be run
        @param prep if True, use prep_hybrid input.  This changes the way
          in which real_nmm, ungrib and metgrid are run.
        @param prepfcst If True AND prep=True, use prep_hybrid for
          boundary conditions.  This changes how realfcst, ungribfcst
          and metgridfcst are run.
        @param track Run a post, regribber and tracker on the wrfanl
          history file for domain 1, to find the location of the
          parent vortex.
        @param in_ftime parent model forecast hour to treat as "time 0."
          This is used for the FGAT to use GDAS forecasts as input
        @param in_atime parent model analysis time.  Default: start time
          of wrf
        @param geogrid Should be an hwrf.wps.Geogrid, or None.  This is
          used to avoid running expensive geogrid jobs multiple times
          for multiple FGAT hours.  (This is especially important for
          hourly FGAT.)  If None or missing, a Geogrid is created.
        @param prep_one_time Flag.  If True, pretend hour 0 is also
          hour 1, 2, 3, and so on.  This is only used by prep_hybrid,
          and is intended to prevent expensive spectral to grid
          transformations of hour 3 when only the analysis time is
          needed from WRF.
        @param relocate If True, vortex relocation is performed.
        @param modin The modin argument to hwrf.relocate task constructors.
          Default: in_dataset.upper()
        @param ibdystep Optional: the boundary input frequency for the 
          ~1 minute forecasts.  Default: bdystep.
        @param fgat_times: the times at which FGAT is run.
        @param kwargs Passed to hwrf.hwrftask.HWRFTask.__init__

        @note If "in_dataset" is in kwargs, it is the option sent to
          hwrf.input.DataCatalog.locate used for obtaining input files
          for Ungrib and PrepHybrid.

          ----------------------------------------------------------

        This constructor creates the following public member variables
        or accessors.  These are intended to be used to query various
        components of the initialization.  The scripting layer can run
        the sub-objects via their run methods, or it can simply call
        the self.run_* subroutines to run them in groups.

        Reference variables:
          initlen -- timedelta for length of initialization
          fcstlen -- timedelta for length of forecast
          wrffcst -- Task for running or monitoring the WRF forecast
          wrfdoms -- WRFDomain objects from that wrffcst object
          wrfinit -- Task for running the wrfanl run of WRF
          bdytimes -- an array of boundary times needed by the forecast run
          ibdytimes -- an array of boundary times needed by the init
            run of wrf

        Initialization-Length WRF Init:
          geogrid -- the hwrf.wps.Geogrid object for running geogrid.exe
          external_geogrid -- if a geogrid argument was sent to the
            constructor, it is stored here
          ungrib -- the hwrf.wps.Ungrib object for running ungrib
          metgrid -- the hwrf.metgrid.Metgrid object for running metgrid
          realinit -- the hwrf.fcsttask.RealNMM object for running real 
            for the WRF init jobs
          runwrfanl -- the hwrf.fcsttask.WRFAnl4Trak object used to run 
            the WRF for to produce the wrfanl files and outer domain
            time 0 wrfout file
          wrfghost -- the WRFSimulation representing the ghost if
            requested.  This is the same as the wrfghost argument to
            the constructor
          runghost -- the hwrf.fcsttask.WRFGhost to run the ghost
            if requested.

        Prep Hybrid:
          prep -- the hwrf.prep.PrepHybrid object to run prep_hybrid.
            The same prep object is used for both the init-length and
            fcst-length initialization.

        Forecast-Length WRF Init:
          ungribfcst -- the hwrf.wps.Ungrib object for running ungrib for
            the full WRF forecast job, if prep=False
          metgridfcst -- the hwrf.wps.Metgrid object for running metgrid
            for the full WRF forecast job if prep=False
          realfcst -- the hwrf.fcsttask.RealNMM object

        Parent domain vortex detection:
          post -- if track=True, this is the hwrf.post.PostOneWRF
            object used to post-process the wrfanl job's time 0 outer
            domain wrfout file
          gribber -- the hwrf.gribtask.GRIBTask object used to
            regrib the output of self.post
          tracker -- the hwrf.tracker.TrackerTask object used to 
            run the tracker on the gribber
          �^When running prep_hybrid for boundary conditions, you must also run it for initial conditions.N�intercom�ibdystepr�������)�endc�����������������S���s���g�|�]}|�qS�r���r�����.0�dr���r���r����
<listcomp>����s�����z%HWRFInit.__init__.<locals>.<listcomp>c�����������������S���s���g�|�]}|�qS�r���r����r#����xr���r���r���r%�������s�����c�����������������S���s���g�|�]}|�qS�r���r���r&���r���r���r���r%�������s�����c�����������������S���s���g�|�]}|�qS�r���r���r&���r���r���r���r%�������s�����ag��You have asked me to run prep_hybrid to produce full simulation length inputs AND you specified different boundary input timesteps for the initialization-length and forecast-length jobs.  I cannot do that.  You need to either disable realfcst or disable prep_hybrid, or use the same boundary input timesteps for forecast-length and initialization-length jobs.�wps�/c��������������������s�������|�|��S�)N)�confstr��s�r���r���r����c�������zHWRFInit.__init__.<locals>.cc��������������������s���d��j�|�f�S�)Nz%s/%s)�tasknamer+���r-���r���r����t��r/���zHWRFInit.__init__.<locals>.t�
in_dataset�ungrib_one_time�one_time�GFS�geogrid)r0����locationr���r����prep_hybrid)�timesr0����ftime�atimer4���r���r����gfsinit�wrfanl�r0���r���r���);�superr����__init__�bool�	_prepfcst�NotImplementedErrorr���r����log�_fgat_timesr���r���r���r����getdirr0����dstore�confr	����simstart�initlen�fcstlen�AssertionError�confintr����copyZwrffcst�
set_timing�wrfdoms�wrfinit�set_bdystep�get_moad�bdytimes�	ibdytimes�bdystep�find�dict�upperZexternal_geogrid�hwrfr(����Geogridr6����prep�
PrepHybrid�geodat�_make_realinit�_make_realfcst�fcsttask�WRFAnl4Trak�add_metgrid�metgrid�add_geogrid�add_real�realinit�	runwrfanl�_make_track�_make_ghost�_make_relocate)&r����dsrH����section�wrfrJ���rK���r����realfcst�wrfghostr\����track�in_ftime�in_atimer6����
prep_one_time�relocate�modinr����
fgat_times�prepfcst�kwargs�so�sw�logger�fwrf�iwrf�moadrT����	starttime�wpslocr.���r1����ungrib_optsZ	prep_opts�dstZ	pbdytimes�endtime��	__class__r-���r���r@���P���s����t
���


�	 

�

�
�����������
��������������������
zHWRFInit.__init__c��������������
���C���s����t�jj|||d�||�jf||
d�|||	||d�d�|
��|�_t�jj|||d�||�j|�j||
d�||d�|d�|�_t�jj	|||d�||
d�|d�|d�d��
|�j��|�|�_|r�|�j�
|�j��dS�)	a<��!Makes the ungrib, metgrid and realinit member variables.
          @param prep True if prep_hybrid is in use
          @param ds the produtil.datastore.Datastore object
          @param conf the hwrf.config.HWRFConfig
          @param fwrf the hwrf.wrf.WRFSimulation for the forecast-length run
          @param iwrf the hwrf.wrf.WRFSimulation for the init-length run
          @param endtime the end time of the iwrf
          @param wpsloc directory in which to run WPS
          @param in_atime the analysis time for the parent model
          @param in_ftime the forecast time for the parent model that 
            corresponds to the analysis time of this HWRFInit
          @param ungrib_opts - dict of keyword arguments to send to the hwrf.wps.Ungrib.__init__
          @param geogrid the hwrf.wps.Geogrid
          @param c - function to generate config section names for subtasks
          @param t - function to generate task names for subtasks
          @param so - function to generate outdirs for subtasks
          @param sw - function to generate workdirs for subtasks�ungribr(���)r����r0���r7���rs���rr���r���r���rd���)�	incrementr����r0���r���r���r7���rg���r>���N)rZ���r(����UngribrP���r�����Metgridr���rd���ra����RealNMMrc���re���rg����add_prep_hybridr\���)r���r\���rl���rH���r}���r~���r����r����rs���rr���r����r6���r.���r1���rz���r{���r���r���r���r_���;��sD����������������������zHWRFInit._make_realinitc��������������
���C���s����|stt�jj|||d�||�jf||
d�|||	|d�|d�|
��|�_t�jj|||d�||�j||
d�|d�|||�jd�|�_t�jj	|||d�||d�|d�|
d�d	��
|�|�_|r�|�j�|�j
��|�j��n|�j�|�j��d
S�)a:��!Makes the ungribfcst, metgridfcst and realfcst member variables:
        @param prep True if prep_hybrid is in use
        @param prepfcst True if prep_hybrid is in use for boundary conditions
        @param ds the produtil.datastore.Datastore object
        @param conf the hwrf.config.HWRFConfig
        @param fwrf the WRFSimulation for the forecast-length run
        @param fcstlen the length of the forecast as a datetime.time_delta
        @param wpsloc directory in which to run WPS
        @param in_atime the analysis time for the parent model
        @param in_ftime the forecast time for the parent model that 
          corresponds to the analysis time of this HWRFInit
        @param ungrib_opts dict of keyword arguments to send to 
          hwrf.wps.Ungrib.__init__
        @param geogrid the hwrf.wps.Geogrid
        @param c function to generate config section names for subtasks
        @param t function to generate task names for subtasks
        @param so function to generate outdirs for subtasks
        @param sw function to generate workdirs for subtasksr�����
ungribfcstZwpsfcst)r����r0���r7���rs���rr���r���r���rd����metgridfcst)r����r0���r���r���r7����geogrid_fromro���)r���r���r0���N)rZ���r(����SteppingUngribrP���r����r����r6���r����ra���r����re���ro���r����r\���rc���rd���)r���r\���rx���rl���rH���r}���rK���r����rs���rr���r����r6���r.���r1���rz���r{���r���r���r���r`���\��sF��������������������zHWRFInit._make_realfcstc	�����������������C���s\��t�jj|�j|g||d�|d|d�|d�|d�d�	|�_t�jj|�j|d�}	t�j�dt�jjd�}
t�jj	|d�|d�d	�}t�jj	|�j
j|�j
jd	�}|�j
j
}
|
���d
ks�|
���dkr�|jdk�r�|�jd7��_t�jjtt|�d
���tt|�d���d�}|�dt|ddgddgdddgd���|�dt|ddgddgdddgd���|�d|	|�d����|�d|�d�|
���|�dt�j�|�d�|�d�����tj�|��d�|d ��}|�|d!�d��|�|d"�d��|�|d#�d��t�jj|||d �||d$||d �||d �d%�
|�_ t�jj!|||d&�|d&�|d$||d&�|d&�d'�	|�_|�j�"|�j
|�j d��|�j�#d(|�j$d)���dS�)*a'��!Makes the gribber and tracker member variables.
        @param ds the produtil.datastore.Datastore object
        @param conf the hwrf.config.HWRFConfig
        @param moad the hwrf.wrf.WRFDomain for the outermost domain
        @param starttime the start time of this HWRFInit's WRF simulations
        @param c function to generate config section names for subtasks
        @param t function to generate task names for subtasks
        @param so function to generate outdirs for subtasks
        @param sw function to generate workdirs for subtasks�postF)�time�needcrtmr0���r���r���)�domain�partN)�config�domlat)r�����domlon)�lat�lon�AL�EPg������v@�copygb�wgrib)r����r����Zp25gridg�������?g������V@g������[@����i���ii��)�res�size�scan�n�trkgridg������4@�����Q���Zquarter_degree�subset�hwrftrkr����	regribberz/quarter_degree.grbz/subset.grbz$/{out_prefix}.hwrftrk.grbf{fahr:02d}���)�start�stepr!���r0���r;���r����tracker)r0���r����r����r!���r���r����track0z/gfs.track0.atcfunix)%rZ���r�����
PostOneWRFrh����regrib�igrb1�
GRIBSubsetterr�����tracker_subset�
FixedLocation�	storminfor����r�����	pubbasin2rY����ewcenter�
RegribManyr���r����getexe�addr����grid�GRIB�vinttaver���r���r���rF����to_intercom�gribtask�GRIBTask�gribber�TrackerTask�add_moving_grid�
send_atcfunixr���)r���rl���rH���r���r����r.���r1���rz���r{����grid1�trksub�domloc�stormloc�basin�r�basedirr���r���r���ri������s������������������������������������������zHWRFInit._make_trackc	�����������	���	���C���s`���||�_�tjj|||d�|�j�|d�|d�|d�d��|�j��|��|�j�|�_	dd��|�j�D��|�_
dS�)a��!Makes the wrfghost, runghost and ghost_domains member variables.
        @param ds the produtil.datastore.Datastore object
        @param conf the hwrf.config.HWRFConfig
        @param wrfghost the WRFSimulation for the ghost run of wrf
        @param geogrid the hwrf.wps.Geogrid
        @param c function to generate config section names for subtasks
        @param t function to generate task names for subtasks
        @param so function to generate outdirs for subtasks
        @param sw function to generate workdirs for subtasksrp����ghostr>���c�����������������S���s���g�|�]}|�qS�r���r���r"���r���r���r���r%������s�����z(HWRFInit._make_ghost.<locals>.<listcomp>N)rp���rZ���ra����WRFGhostrc���rd���re���rf���rg����runghost�
ghost_domains�	r���rl���rH���rp���r6���r.���r1���rz���r{���r���r���r���rj������s"����
���������zHWRFInit._make_ghostc	�����������
������C���sZ���t�|�j|�j|�j|�j|�j|||d�|d�d�	}	|dk	rB|	j|�jd��|rV|	j|�jdd��|	S�)aO��!Makes a dict containing the keyword arguments to send in to
        the constructor for the hwrf.relocate task(s).
        @param ds the produtil.datastore.Datastore object
        @param conf the hwrf.config.HWRFConfig
        @param wrfghost the wrfghost argument to the constructor
        @param track the track argument to the constructor
        @param modin the modin argument to the constructor
        @param dest_dir the directory in which to run the relocate
        @param so function to generate outdirs for subtasks
        @param sw function to generate workdirs for subtasksru���)	r=����wrfinput�sim�domainsr����rv����dest_dirr���r���N)rp���r����)�parentTrack�	trackName)	rX���rh���rg���rQ���rP���r�����updater����r�����
r���rl���rH���rp���rq���rv���r����rz���r{���ry���r���r���r����_make_relocate_kwargs���s ������������zHWRFInit._make_relocate_kwargsc
��������������
���C���s����|dkrt�d��tj�|�jd�}
|��||||||
||	�}tjj|||d�fd|d�d�i|��|�_	|�j	j
|�_
|�j	j|�_|�j	j|�_dS�)al��!Makes the relocation, rstage1, rstage2 and rstage3 member
        variables.
        @param ds the produtil.datastore.Datastore object
        @param conf the hwrf.config.HWRFConfig
        @param wrfghost the wrfghost argument to the constructor
        @param track the track argument to the constructor
        @param modin the modin argument to the constructor
        @param c function to generate config section names for subtasks
        @param t function to generate task names for subtasks
        @param so function to generate outdirs for subtasks
        @param sw function to generate workdirs for subtasksNzHWhen in HWRFInit.__init__, when relocate=True, wrfghost must not be Noneru����taskname_patternzrelocate.stagez%d)
Z
ArgumentErrorr���r���r���r���r����rZ���ru����
Relocation�
relocation�rstage1�rstage2�rstage3)r���rl���rH���rp���rq���rv���r.���r1���rz���r{���r����ry���r���r���r���rk������s �������
��

zHWRFInit._make_relocatec�����������������c���s ���|�j�dk	r|�j�D�]
}|V��qdS�)z(!Iterates over all fgat times, if known.N)rw���)r���r1���r���r���r���rw������s����

zHWRFInit.fgat_timesc�����������������C���sl���d|�j�krh|�j����|�j����d|�j�krF|�j����d|�j�krF|�j����|�jjrh|�jjrh|�jjrh|�j����dS�)z$!Runs the relocate jobs, if present.r����r����r����N)�__dict__r�����delete_temp�runr����r�����scrubr-���r���r���r����run_relocate���s����






�zHWRFInit.run_relocatec�����������������C���s���|�j�S�)z`!Returns the parent model forecast hour treated as this
        HWRFInit object's analysis time.)Z	_in_ftimer-���r���r���r����	get_ftime��s����zHWRFInit.get_ftimec�����������������C���s���|�j�S�)zy!Returns the parent model analysis time.  The value of that
        may differ from this HWRFInit object's analysis time.)�	_in_atimer-���r���r���r����	get_atime��s����zHWRFInit.get_atimez:!Forecast time of parent model used for initializing HWRF.zAnalysis time of parent model.c�����������������C���s$���|������|�����|�����|�����dS�)z2!Runs all steps of the initialization in sequence.N)�run_through_anl�run_init_after_anlr�����run_real_bdyr-���r���r���r���r������s����zHWRFInit.runc�����������������C���s����|������d��d|�jkr,tdd��|�j����tdd��|�j����tdd��|�j����d|�jkr�tdd��tt	|�j
��D�](}|�j
|�}||�jkr��q�|�j�
|��qvtdd	��|�j����tdd
��|�j����tdd��dS�)
z�!Runs the following jobs, if they are enabled: geogrid,
        ungrib, metgrid, init-length prep_hybrid, init-length real_nmm
        and wrfanl.r����r6����statusz[geogrid] process terrainz[ungrib] process parent GRIBz"[metgrid] interpolate horizontallyr\���z[prep] process parent spectralz[realinit] make wrfinput filez[runwrfanl] make wrfanl fileszPassed through anl.N)rD����warningr����r���r6���r����r����rd����range�lenrU���rJ���r\����
run_ipiecerg���rh����r����ir1���r���r���r���r����#��s(����










�



zHWRFInit.run_through_anlc��������������	���C���s���|�����}|�d��d|�jkrXtjj|d��tjj|d���tdd��|�j����W�5�Q�R�X�d|�jkr�|�j	�
���}|�j����|�j
����W�5�Q�R�X�tdd��|�jjdd	��tdd
��|�j
jdd	��|�j
���r�tdd��|�j����nd}|�|��t|��tdd
��dS�)zT!Runs the following jobs if they are enabled: ghost, post,
        gribber, tracker.r����r����)r|���r����z[runghost] make wrfghost filesr����z [post] post wrfghost for trackerT)�raiseallz%[gribber] regrid wrfghost for trackerz[tracker] find parent vortexzJError regridding inputs to tracker.  See earlier log messages for details.zPassed full init except bdy.N)rD���r����r�����produtil�rusage�	getrlimitr���r����r����rG����transactionr�����unrunr�����is_completedr�����errorr���)r���r|���r1����msgr���r���r���r����:��s,����









zHWRFInit.run_init_after_anlc�����������������C���s����|������d��d|�jkr,tdd��|�j����d|�jkrJtdd��|�j����d|�jkr�d|�jkr�|�jr�tt	|�j
��D�]@}|�j
|�}||�jkr�|������d	|�
d
�f���qr|�j�|��qrtdd��|�j����tdd��d
S�)a���!Runs boundary processing jobs.

        Runs the forecast-length versions of the following jobs if
        they are enabled: ungribfcst, metgridfcst, prep_hybrid and
        real.  Note that ungribfcst and metgridfcst should be disabled
        if prep_hybrid is enabled.  That is done in the constructor
        though; the run_real_bdy will happily run all four jobs if
        requested.r����r����r����z&[ungribfcst] process parent GRIB (all)r����z+[metgridfcst] interpolate parent GRIB (all)ro���r\���z=%s: skipping: is an initial bdy time (should already be done)z%Y%m%d%Hz([realfcst] interpolate parent GRIB (all)zEntire init complete.N)rD���r����r����r���r����r����r����rB���r����r����rT���rU����info�strftimer\���r����ro���r����r���r���r���r����T��s,����	









��

zHWRFInit.run_real_bdyc�����������������c���sd���d|�j�kr |�j���D�]
}|V��qd|�j�kr@|�j���D�]
}|V��q4d|�j�kr`|�j���D�]
}|V��qTdS�)z�!Iterates over all needed input data 

        Calls the inputiter for the ungrib, ungribfcst and prep
        members, iterating over all data they need to run the
        initialization.r����r����r\���N)r����r�����	inputiterr����r\���)r���r$���r���r���r���r����q��s����
�
�
�zHWRFInit.inputiter)NFNTTNNNNTNNNN)�__name__�
__module__�__qualname__�__doc__r���r���r@���r_���r`���ri���rj���r����rk���rw���r����r����r�����propertyr:���r;���r����r����r����r����r�����
__classcell__r���r���r����r���r������sL���

�������������������������l!(4�c�����������������������s6���e�Zd�ZdZd	��fdd�	Z��fdd�Zdd��Z���ZS�)
r���z�!Runs the FGAT initialization for one FGAT member

    This class serves a similar purpose to its superclass, HWRFInit,
    but is intended for initialization that runs before the GSI.  It
    has a modified list of arguments to the hwrf.relocate classes.Nc�����������	���������st���|dk	st��t|t�s0tdt|�jt|�f���||�_||�_||�_	t
t|��j|||fd|i|���|�j
dk	spt��dS�)a���!Creates a new InitBeforeGSI object.  See the
        HWRFInit.__init__ for details.  
        @param ds passed to Datum: the Datastore object for this Task
        @param conf the conf object for this task
        @param section the conf section for this task
        @param gsi_d01 passed to the hwrf.relocate constructors' 
                gsi_d01 argument
        @param gsi_d02 passed to the hwrf.relocate constructors' 
                gsi_d02 argument
        @param gsi_d03 passed to the hwrf.relocate constructors' 
                gsi_d03 argument
        @param fgat_times array of FGAT times
        @param kwargs All other keyword arguments are passed to
          HWRFInit.__init__
            NzKIn InitBeforeGSI.__init__, fgat_times must be a list or tuple, not a %s %s.rw���)rL����
isinstance�list�	TypeError�typer�����repr�_gsi_d01�_gsi_d02�_gsi_d03r?���r���r@���rE���)	r���rl���rH���rm����gsi_d01�gsi_d02�gsi_d03rw���ry���r����r���r���r@������s$����
�
�����zInitBeforeGSI.__init__c	�����������
���
������sJ���|�j�dk	st�tt|���||||||||�}	|	j|�j|�j|�j|�j�d��|	S�)z�!Makes the FGAT relocation.

        This serves the same purpose as the
        HWRFInit._make_relocate_kwargs(), but it adds to those
        arguments the gsi_d01, d02 and d03 sent to the constructor,
        plus the fgat_times.N)r��r
��r��rw���)	rE���rL���r?���r���r����r����r	��r
��r��r����r����r���r���r�������s ����
����������z#InitBeforeGSI._make_relocate_kwargsc	�����������	���	���C���s`���||�_�tjj|||d�|�j�|d�|d�|d�d��|�j��|��|�j�|�_	dd��|�j�D��|�_
dS�)a���!Makes the wrfghost, runghost and ghost_domains member
        variables.  

        Overrides the HWRFInit._make_ghost() to use the
        WRFGhostForPost class instead of WRFGhost in order to get the
        wrfout "ghout_d0X" files for each domain for post-processing.

        @param ds the produtil.datastore.Datastore object
        @param conf the hwrf.config.HWRFConfig
        @param wrfghost the WRFSimulation for the ghost run of wrf
        @param geogrid the hwrf.wps.Geogrid
        @param c function to generate config section names for subtasks
        @param t function to generate task names for subtasks
        @param so function to generate outdirs for subtasks
        @param sw function to generate workdirs for subtasksrp���r����r>���c�����������������S���s���g�|�]}|�qS�r���r���r"���r���r���r���r%�����s�����z-InitBeforeGSI._make_ghost.<locals>.<listcomp>N)rp���rZ���ra����WRFGhostForPostrc���rd���re���rf���rg���r����r����r����r���r���r���rj�����s"�������������zInitBeforeGSI._make_ghost)NNNN)r����r����r���r��r@���r����rj���r��r���r���r����r���r������s������"c����������������
�������s����e�Zd�ZdZd$��fdd�	Zdd	��Zd
d��Zdd
��Zd%dd�Zdd��Z	dd��Z
dd��Zdd��Zdd��Z
dd��Zdd��Zdd��Zd d!��Zd"d#��Z���ZS�)&r���z�!The FGATInit represents an array of InitBeforeGSI objects, each run
    for one forecast hour of some parent model (usually GDAS).NFT�`T��c��������������������s���d|kr|��d�|d<�tt|��j|||f|��|����}||�_||�_||�_||�_|r^|
s^t	d��|
|�_
||�_||�_||�_
|	|�_||�_||�_t|�|�_|dkr�|��dd�}|dk�r�d}||�_|dkr�tj�|���d�|�j�|�_t|�|�_��dkr�|���|�j�����|�_|dk�r��fd	d
�|����D��}dd
��|D��|�_t��|�_ t!|�jdd
���|�_"d|�j#k�sXt$�t%��}d|k�rt|d�|d<�d|k�s�t$�|d�dk�s�t$�|�j&f�|��|�j �s�t$�|��'|j(�}|j)�*��}|�+d|���|��,��D�]\}}||j)_-�q�dS�)a�
��!Creates an FGATInit, passing most arguments to the child
        InitBeforeGSI objects' constructors.  The cycling_interval is the
        time to subtract from this model's analysis time to get the
        parent model's analysis time.  The default of six hours
        (6*3600 as an int) is correct for the operational HWRF and
        GDAS.

        @param ds passed to Datum: the Datastore object for this Task
        @param conf the conf object for this task
        @param section the conf section for this task
        @param wrf (required) the WRFSimulation representing the full WRF
          forecast
        @param initlen (required) the length of the real_nmm run for the
          initialization 
        @param fcstlen (required) the length of the real_nmm run for the
          full forecast
        @param outdir The directory in intercom to receive output.
          Default: intercom/{taskname}.  Individual initialization
          components will be in subdirectories.
        @param outdir The directory in intercom to receive output.
          Default: intercom/{taskname}.  Individual initialization
          components will be in subdirectories.
        @param realfcst Flag: do we need to run real for the full forecast
          length?
        @param wrfghost hwrf.wrf.WRFSimulation object for the wrfghost
          files.  If absent, no wrfghost-related jobs will be run
        @param prep if True, use prep_hybrid input.  This changes the way
          in which real_nmm, ungrib and metgrid are run.
        @param track Run a post, regribber and tracker on the wrfanl
          history file for domain 1, to find the location of the
          parent vortex.
        @param in_ftimes array of parent model forecast hours to treat
          as "time 0" for each FGAT time. This is used for the FGAT to
          use GDAS forecasts as input
        @param in_atime parent model analysis time.  Default: start time
          of wrf
        @param cycling_interval HWRF cycling interval in seconds.
        @param gsi_d01 passed to the hwrf.relocate constructors' 
                gsi_d01 argument
        @param gsi_d02 passed to the hwrf.relocate constructors' 
                gsi_d02 argument
        @param gsi_d03 passed to the hwrf.relocate constructors' 
                gsi_d03 argument
        @param ibdystep Optional: the boundary input frequency for the 
          ~1 minute forecasts.  Default: bdystep.
        @param prepfcst If True AND prep=True, use prep_hybrid for
          boundary conditions.  This changes how realfcst, ungribfcst
          and metgridfcst are run.
        @param kwargs Other keyword arguments are passed to HWRFTask.__init__
        r����WORKhwrfr���Nr���r���r ���r���c��������������������s���g�|�]}t�|����qS�r���)r	����r#���r1����rs���r���r���r%������s����z%FGATInit.__init__.<locals>.<listcomp>c�����������������S���s���g�|�]}|�qS�r���r���r��r���r���r���r%������s�����c�������������������S���s���dS�)Nr ���r���r���r���r���r����<lambda>���r/���z#FGATInit.__init__.<locals>.<lambda>�
ftime2initr2����gdas1zcentrack %s).rF���r?���r���r@���rD���r���r	��r
��r��rC����_preprB����_track�_wrf�	_wrfghost�_initlen�_fcstlenrA����	_realfcstrM����	_ibdystepr���r���r���r0���r
����_cycling_intervalrI���r�����_default_ftimes�
_in_ftimesr���initsr���r��r����rL���rX����_make_inits�init_at_time�cycler�����	get_track�debug�fhr_and_init�centrack)r���rl���rH���rm���rn���rJ���rK���r���ro���rp���r\���rq���Z	in_ftimesrs����cycling_intervalr��r
��r��r���rx���ry���rD���Zdsopt�ceninitr)���fhr�initr����r��r���r@���*��sj����7�
���


�
�
zFGATInit.__init__c�����������������C���s���t�||�j�}|�j|�S�)a���!Get the subtask for one time.

        Returns the initialization InitBeforeGSI object for the
        specified time (the earliest time not before the specified
        time).  The time can be an absolute time (2015091418) or a
        forecast hour (12), as long as it is accepted by the first
        argument of to_datetime_rel.  Raises an exception if the time
        has no InitBeforeGSI objects.)r	���r����r��)r����whenr���r���r���r$�����s����	zFGATInit.init_at_timec�����������������c���s,���|�j����D�]\}}|���D�]
}|V��qq
dS�)z�!Iterates over needed input data.

        Calls all inputiter functions for all child objects, hence
        iterating over all input sources required for all FGAT hours.N)r���itemsr����)r���r1����pr$���r���r���r���r�������s����zFGATInit.inputiterc�����������������C���sN���t�|�j�}|�j���D�]4\}}|��|�}d|jkr|j}|�|�}|||<�q|S�)z�!Get all ghost output products.
        @param domain The domain of interest, an hwrf.wrf.WRFDomain

        Returns a TimeMapping, mapping from init time to the ghost
        product output from the runghost member of each init time.r����)r���r!��r��r/��r$��r����r�����	get_ghost)r���r�����retr1���r0��r�����rgZggr���r���r����
get_ghosts���s����




zFGATInit.get_ghostsc�����������	������C���s����t�|�j�}|�j���D�]�\}}|��|�}|j}|dkr>|||<�qt|tjj	�sPt
�|j|�}|���rp|�
||�}n
|�|�}|dkr�tdt|�t|�f���|||<�q|S�)a���!Get hwrf.relocate.Stage3 objects or output products for
        each FGAT time.

        Returns a TimeMapping of objects, one per init time.  If a
        domain is specified, the objects are the wrfinput or ghost
        product output by the relocation for that domain.  If a domain
        is NOT specified, then the objects are the relocate stage 3
        tasks.
        @param domain The domain of interest.Nz#Error: %s does not exist at time %s)r���r!��r��r/��r$��r����r��rZ���ru����Stage3rL���rp����is_moad�wrfinput_at_timer1��r����str)	r���r����r2��r1���r0��r����ZrrZrdomr3��r���r���r����
get_relocates���s(����




��
zFGATInit.get_relocatesc�����������������c���s*���|�j����D�]}t|tj�st�|V��q
dS�)z;!Iterates over all fgat times as datetime.datetime objects.N)r���keysr���datetimerL���)r���r:���r���r���r����parent_fhrs���s����zFGATInit.parent_fhrsc�����������������c���s>���|�j�s
t�|�j���D�]$\}}t||�j��d�}||fV��qdS�)z�!Iterates over pairs of (fhr,init) where fhr is a
        fraction.Fraction FGAT hour, and init is an InitBeforeGSI
        object for that time.g����� �@N)r"��rL���r��r/��r
���r����)r���r:���r-��r,��r���r���r���r(�����s����
zFGATInit.fhr_and_initc�����������������c���s����|���dd�d�}|���dd�d�}|���dd�d�}|dk�rFtd|f���|}|d	�|kr�|�jd
k	sdt�|d
k	spt�t||�j�}|d
k	s�t�t|�j|�V��||7�}qJd
S�)z�!Internal function that implements other iterators.

        This is an iterator that generates the default FGAT hours from
        the configuration file.  The iterator yields the forecast
        times for the input model as datetime.datetime objects.�FGATSTRg�������r�����FGATINV�����FGATEND�<���zaERROR: in hwrf.init.FGATInit the FGATINV must be at least one minute, but you provided %f seconds����N)�	conffloat�
ValueErrorr����rL���r	���r��)r���r����r����r!����now�relr���r���r���r �����s�����zFGATInit._default_ftimesc�����������������K���s����|�j����}t||�j�}|���|����}||�}|j||d��|�j���}|j||d��||�j�}dd��|����D��}	|	dk	s|t	�t
f�|�j|�j|�j
||�j|�jd|�j||�j|�j||�j|d|�j|�j|�j|�j|	|�jdd�|��}
|
jjs�t	�|
S�)a^��!Internal implementation function that generates the
        InitBeforeGSI object for one FGAT time.

        @param in_ftime The forecast time of interest
        @param taskname The name of the child task
        @param kwargs Additional keyword arguments passed to
        InitBeforeGSI.__init__
        @returns the resulting InitBeforeGSI object)r����r!���c�����������������S���s���g�|�]}|�qS�r���r���r��r���r���r���r%�����s�����z'FGATInit._make_init.<locals>.<listcomp>NT)rl���rH���rm���rn���rJ���rK���r���ro���rp���r\���rq���rr���rs���r0���rt���r���r��r
��r��rw���rx���r3���)r��rN���r	���r�����simendrI���rO���r��r<��rL���r���rG���rH���rm���r��r��r��r��r��r��r	��r
��r��rB���r����r4���)r���rr���r0���ry���rn���Ztstartr:����tendrp���rw����childr���r���r����
_make_init���sD����	


�������������zFGATInit._make_initc�����������������K���s����d|�j�kst�|�jst�|�jD�]d}|�����dt|�f���d|�j|�d�f��}}|�j||f|�}|�j	�
|��|�j	sxt�||�j|<�qdS�)a���!Creates all child InitBeforeGSI objects.

        Creates InitBeforeGSI objects for all input times in
        self.ftime2init and self.inits.  The self._make_init is used
        to make each individual object.  The self.inits will be a list
        of InitBeforeGSI objects, and self.ftime2init will be a
        mapping from time to InitBeforeGSI.
        @param kwargs Passed to _make_init()r��zprocess ftime %sz%s.t%sz
%Y%m%d%H%MN)r����rL���r!��rD���r'��r��r0���r����rJ��r"���appendr��)r���ry���rr���r0���rI��r���r���r���r#����s����	


zFGATInit._make_initsc�����������������C���s���|�j�D�]}|����qdS�)z3!Runs all InitBeforeGSI.run() commands in sequence.N)r"��r�����r���r-��r���r���r���r����*��s����
�zFGATInit.runc�����������������C���s���|�j�D�]}|����qdS�)z?!Runs all InitBeforeGSI.run_through_anl() commands in sequence.N)r"��r����rL��r���r���r���r����-��s����
�zFGATInit.run_through_anlc�����������������C���s���|�j�D�]}|����qdS�)z=!Runs all InitBeforeGSI.run_after_anl() commands in sequence.N)r"��r����rL��r���r���r���r����0��s����
�zFGATInit.run_init_after_anlc�����������������C���s���|�j�D�]}|����qdS�)z<!Runs all InitBeforeGSI.run_real_bdy() commands in sequence.N)r"��r����rL��r���r���r���r����3��s����
�zFGATInit.run_real_bdyc�����������������C���s���|�j�D�]}|����qdS�)z<!Runs all InitBeforeGSI.run_relocate() commands in sequence.N)r"��r����rL��r���r���r���r����6��s����
�zFGATInit.run_relocate)
NFNTTNNr��NNNNT)N)r����r����r���r��r@���r$��r����r4��r9��r<��r(��r ��rJ��r#��r����r����r����r����r����r��r���r���r����r���r�����s:���
����������������������q
)$r���__all__r���r;���pdb�produtil.runr�����produtil.rusage�
hwrf.hwrftaskrZ����hwrf.wps�
hwrf.fcsttask�	hwrf.post�hwrf.regrib�
hwrf.gribtask�hwrf.tracker�	hwrf.prep�
hwrf.relocate�hwrf.exceptions�produtil.ecflowr���r���r���r����
hwrf.numericsr	���r
���r���r���r
���r���r���r���r���r���r���r���r���r���r����<module>���s$���	
P�����4Q