U

����� g%�����������������������@���s��g�Z�d�dlZd�dlZd�dlZd�dlZd�dlZd�dlZd�dl	Zd�dl
Zd�dlZd�dlZd�dl
Zd�dlZd�dlZd�dlZd�dlZd�dlZd�dlZd�dlZd�dlZd�dlZd�dlZd�dlZd�dlZd�dlZd�dlZd�dlZd�dlZd�dlZd�dlZd�dl Zd�dl!Zd�dl"Zd�dlm#Z#m$Z$�d�dl%m&Z&�d�dl	m'Z'm(Z(m)Z)�d�dl
m*Z*�d�dl
m+Z+m,Z,m-Z-m.Z.m/Z/�dd��Z0d	d
��Z1dd��Z2da3da4da5da6da7da8da9da:da;da<da=da>da?da@daAdaBdaCdaDdaEdaFdaGdaHdaIdaJdaKdaLdaMdaNdaOdaPdaQdaRdaSdaTdaUdaVdaWdaXddd�ZYdS�)�����N)�exe�alias)�
isnonempty)�	WRFDomain�
WRFSimulation�ExternalWRFTask)�PostManyWRF)�
RegribMany�igrb1�clatlon�GRIB2�SATGRIB2c�����������������C���sN���t�j�|�|��|dkrdS�t�j�|�||��t�j�|�||��t�j�|�||��dS�)aI��!This function makes per-cycle modifications to the
    configuration file storm1.conf.  

    This is called in scripts.exhwrf_launch and run_hwrf.py by
    hwrf.launcher.launch() on the configuration object
    (hwrf.launcher.HWRFLauncher, a subclass of
    hwrf.config.HWRFConfig), before the per-cycle storm1.conf
    configuration file is written.  Any changes made to the conf
    object will be stored in storm1.conf file and used in later jobs.
    This allows modifications to the configuration on a per-cycle
    basis.  Note that cycle=None and conf.cycle is unavailable when
    run_hwrf.py calls prelaunch.
    @param conf the hwrf.launcher.HWRFLauncher to modify
    @param logger a logging.Logger for log messages
    @param cycle the cycle to run, or None if this is being
       run from run_hwrf.py or the ush.psychoanalystN)�hwrf�	prelaunchZprelaunch_ensidZprelaunch_ungribZprelaunch_rsmcZprelaunch_basin)�conf�logger�cycle��r����6/lfs/h1/ops/prod/packages/hmon.v3.2.7/ush/hwrf_expt.pyr���8���s�����r���c�����������������C���s���|���dtt�f���|���dtt�f���|���dtt�f���|���dtt�f���t�dd�}t�dd�}t�dd�}t�dd	�}t�dd
�}t�dd�}t�dd�}t�dd
�o�|}t�dd�}	t�ddd�}
t�	ddd�}|�rF|dk�r|���dtt
�f���|���dtt�f���n0|dk�r6|���dtt�f���nt
j�d|���|dk��r^t
j�d��n*|dk�r�dd|�k�r�t
j�d|f���|�r�|dk�r�|���d tt�f���|���d!tt�f���nt
j�d"|���|�r�|�s�|��d#��t
j�d#��d$S�)%a���!Runs a sanity check on this module's contents.  This should be
    called after init_module.  

    This sanity check routine is called automatically by
    hwrf.launcher.HWRFLauncher.sanity_check_expt() as part of the
    standard sanity checks in the scripts.exhwrf_launch job.  It
    checks to see if all expected module-scope variables are present
    and initialized correctly.

    @param logger a logging.Logger for log messageszThe runwrf object = %szThe gribber object = %szThe datastore (ds) object = %szThe wrf object = %s�config�run_gsi�	run_ocean�ocean_model�run_wave�
wave_model�run_relocation�use_spectral�spectral_bdy�allow_fallbacks�extra_trackersF�forecast_products�wrf_output_step�0*���HYCOMzThe hycominit object = %szThe hycompost object = %s�POMzThe pominit object = %sz2[config] ocean_model=%s but should be POM or HYCOM����z-The wrf_output_step must be at least 1 secondr������zXOne hour (3600 seconds) must be divisable by the WRF output timestep.  You specified %d.�WW3zThe ww3init object = %szThe ww3post object = %sz([config] wave_model=%s but should be WW3z"Cannot use GSI without relocation.N)�info�repr�runwrf�gribber�ds�wrfr����getbool�getstr�getint�	hycominit�	hycompost�pominitr����
exceptions�HWRFConfigUnsupported�PrecisionTooHigh�TimestepModularityError�ww3init�ww3post�error)r����gsi_flag�
ocean_flag�ocean�	wave_flag�wave�
reloc_flag�
spectral_flagr����fallbacks_flagr���r!���r���r���r����sanity_checkT���sd����

��
���
��
�rC���c������������
������c���s���t��dd�}�t��ddd�}t��dd�}t��dd�o4|}t��dd�}t��ddd	�}t��dd
�}t��ddd�}t��dd
d�}t��dd�}	t��dd�}
|r�|r�t|	�r�d}nd}|r�|dkr�t���D�]
}|V��q�n|d	kr�t���D�]
}|V��q�|�r|dk�rt���D�]}|V���q�t	���D�]}|V���q|��r�t
���D�]}|V���q2|�r\t���D�]}|V���qNt���D�]}|V���qd|�r�t
���D�]}|V���q�|
�r�tj�d��s�t���D�]}|V���q�dS�)a6��!Iterates over all inputs required by this configuration.  

    Calls the inputiter() function on all tasks in the module scope
    that are expected to have input data.  The result can be passed
    into the "data" argument of hwrf.input.InputSource.get.  Iterates
    over dicts that contain the following:
    * dataset --- string name of the dataset (gfs, gdas1, gefs,
        enkf, etc.)
    * item --- string name of the object (ie.: gfs_sf, gfs_sfcanl, bufr)
    * atime --- self.conf.cycle
    * ftime --- only present when relevant: the forecast time, in a 
        format accepted by to_datetime_rel
    * enkfmem --- only present when relevant: the ENKF member ID
    * obstype --- only present when relevant: the bufr data type.
    * optional --- True if the absence of this data is not considered a failure.r���r����run_ensemble_daFr���r���r���r���r$���r���r���r'����conditional_gsid03�dir�{com}/{stormlabel}.tdr�realtimeTr#���z/dcom/us007003N)r���r.���r/����	strinterpr���r1����	inputiterr3���r8����gfs_init�gsi_d02�gsi_d03�	fgat_init�ensda�os�path�isdir�	ensda_pre)
r;���rD���rA���r���r<���r=���r>���r?���rE����tdrflagfilerH����gsid03_flag�dr���r���r���rJ�������sR�����
�
�
�
�
�
�
�
�rJ���Tc�����������?���������s�
��|�dkrt�jd�}�tj�|��at���}|�d��t�ddd�a	da
t	r�t�ddd�}t�dd	d
�}|dkrv||krvda
t�ddd
����}|d�d
kr�tj
�d��t|�}tjat�d�at�d�at�dd�}t�ddd�}	|o�|	}t�ddd�}
t�dd�}t�ddd�}t�dd�}
t�ddd�}t�dd�}t�dd�}t�dd��oN|}t�dd�}t�ddd�}t�dd d�}t�dd!d�}t�dd"d�}t�dd#d$�at�d%d&d'�}t�d%d(d)�}t�d%d*d)�}t�d+d,�}tj�td-�atj�td.�at�d/d0d1�}t�d/d2d1�}t�d/d3d1�}t�dd4d���|�rB|�rHt|��rHdandatjat�����h�t td5�a!t td6�a"t td7�a#t td8�}t td9�}t td:�} t td;�}!t!t"t#g}"t$td<t!tjtjtj%�&td=����a't'�(t"t!��t'�(t#t"��t
�rzi�}#i�}$t)d>|d?��D�]b}%t td@|%��|#d@|%�<�t tdA|%��|$dA|%�<�t'�(|$dA|%��t!��t'�(|#d@|%��|$dA|%�����qt#|#d7<�t"|$d6<�t'�*d��t�d/dBdC�}&t�d/dDdE�}'|&dCk��r�t'j+dF|&dG��n2t'j+dFdCdHdI��t'j+dJd=dHdKdL��t'j+dMdCdKdN��t'j+dOd=dPdQdRdS��t'�,d>��dTdU��t'D��}(t
�r2tj-j.ttdVdVdW�a/t$td<t!tjtjtj%�&d1���})|)�(|t!��|)�(||��t$td<t!tjtjtj%�&d1���}*|*�(| t!��|*�(|!| ��t
�r�tj0j1ttdXt'd1td=�d1dd||ddY�a2n*tj0j1ttdXt'd1td=�d1|*|dd||dZ�
a2tjtj%�&d1��}+tj3�4ttd[|+�a5|�r|�r�t td\�},t$td]t!tj|+����(|,t!���j+dFdCdHdI���t!��|,�g}-|-D�]}.|.�6��dk	�sxt7��qxd����fd_d`�	}/|/tj3j8dadb�a9tj3�:ttdctjdd�a;t;�<dedf�a=t)t=�D�]B}%t;�>tj|%d?�tj3j?ttdc|-|%d?�tjdgdd|%d?�f�dW����q�tj3�:ttdctjd^�a3t)t=�D�]J}%���r\t;�@tj|%d?��}0nd}0t3�>tj|%d?�|/tj3j8|%d?�d^|0����q>t�r�d}1nd}1tj0jAttd%t'dCtd=�|)|dddCdh|d|1|di�aBtB�Ctj�}2tjDjEttdjdjdW�aDtB�F|�}3tjG�Httdk||2jI�J|�|3|)�aKt�rFtB�F|�}4tjG�Httdl||2jI�J|�|4|)�aLndaL|�r����rdtK�Mt3|-��ntK�Mt;|-��t�r����r�tL�Mt3|-��ntL�Mt;|-��tjNjOttdm|2jIt2jPt2jQdntKtLdo�	aR|2jI�J|�}5t�r�|2jI�J|�}6n|2jI�St#�}6tR�T|5|6��tjU�Vttdp�}7|7�WtKtL|||2t�\aXaYt�ddqdr�aZ|�rn|dsk�rNtj[j\ttdttdu�a]n |dk�rntj^j_ttdvt|'dw�a`dr}8|
�r�|dk�r�tjajbttdxt|||dy�actc�d��D�]}9�q�t�ddzd{�}:t	�s�tjU�ettt't`tct]t�};|;�f���|;jgag|;jhah|;jiait
�	r tjU�jttt'|t/|$|#�};|;�f���|;jgagnJt	�	rftjk�ltt||:t'd{�agtjkjmtt|:t'dd|d}�antn}<|<�ot2jP��ntg}<t
�
s2|<�pt2jq��rt2js��tt2jP��ut2jv��|�	r�t
�	s�|<�wtR��xtRt"��xtRt#��|�	r�|�	s�|�
r2|�
r2|�
r2t
�
s2|�	s�|�
r|�
r|<�wt2jI��xt2jIt"��xt2jIt#��n |<�wt2jP��xt2jQt"��xt2jQt#��|�
sP|�d~��W�5�Q�R���dS�tjU�yttd/tgt'|"|(t!t#t"�
}=|=�z��a{|=�|��a}|=j~t	d�atdk	�
s�t7�|=��||
|�\a�a�}>a�|�
r�|=����\a�a�W�5�Q�R�X�|�d���dS�)�a���!Initializes the HWRF object structure.

    This function is called from all scripts and the
    hwrf.launcher.HWRFLauncher to initialize the HWRF object
    structure.  The created objects know how to run various parts of
    the system, and this function connects those objects to one
    another.
    @param CONFhwrf the configuration filename
    @param make_ensemble_da if True, the ensemble data assimilation objects
      are created
    @param make_post if True, the post-processing objects are createdN�CONFhwrfz Initializing hwrf_expt module...r����run_multistormF�fakestormid�nofakeid�STID�nosidT�multistorm_sidsZnosidsr���z%No sids  provided for multistorm run.�WORKhwrf�HOMEhwrfr���rD���Zrun_satpostr���r���r$���r���r���r'���r���r���r���r���Zmake_gofiler���rE����conditional_gsid02�forecast_length�~���ZfgatZFGATSTR�����ZFGATEND����ZFGATINVrF���rG����ocstatus�wvstatusr ����ww3_output_stepi`T���ww3_pntout_step�ww3_restart_step�run_ens_relocation�moad�storm1outer�storm1inner�storm1ghost_parent�storm1ghost�storm1ghost_parent_big�storm1ghost_bigr-���r&�������r%���zstorm%sinnerzstorm%souterr!���r"����pom_output_stepi�Q��history)�stepi�~��)ru����endZauxhist2zwrfout_d<domain>_<date>)ru���rv����outnameZauxhist3)ru���rw���Zauxhist1zwrfdiag_d<domain>i��������)ru���rw���Zframes_per_outfileZio_formc�����������������S���s���g�|�]}|�qS�r���r���)�.0rV���r���r���r����
<listcomp>��s�����zinit_module.<locals>.<listcomp>�
finalmerge)�tasknameZgfsinit)�ibdystep�realfcst�relocate�prep�prepfcst�track)r}����wrfghostr����r����r~���r���r����Ztdrcheck�ensdadomZenswrfrO���c��������������������sh���t�|t�st�t�|t�st�|�ttdt|�d||f���|��t�d�d||f��t�d�d||f��d�S�)N�hwrf_da_ens�%s.%03dr^���z/%s/%03d�intercom)r����
priorcycler�����workdir�outdir)�
isinstance�str�AssertionError�intr,���r���rK����getdir)ZclazzZienkfZtopnamer�����Zens_reloc_flagZensdawrfr���r����makemembS��s,���������
��
��
���zinit_module.<locals>.makememb�����entestr�����prior_ensda�
ensda_size�(���r����Zgdas1)
r����r����r����r~���r}����
in_datasetr���rL���rM���r�����bufrpreprL���rM����merge�
gdas_merge)ZwrfinputZwrfanlr|���rL���rM���Zgsi_products�non_ocean_basins�Qr#���r1���)�fcstlenZpom)r�����outstepr8���)r����r����ZpntstepZrststepZforecast_sectionr*���Zmultistorm_init)Zkeeprunr|���z?make_post=False, will not make post.  Done in hwrf_expt module.)�
multistormzDone in hwrf_expt module.)rO���N)�rP����environr����launcher�loadr����logr(���r.���rX����run_multistorm_00flagr/����splitr4����HWRFConfigInsane�lenr���r����r^���r_���r0���r����rI���ZcouplingZCouplingStatusre���rf���r���rU����	datastorer,����transactionr���rk���rl���rm���r����numerics�to_timedeltar-����add�rangeZanalysis_inZ
add_outputZ
add_hifreqZfinalmergetaskZFinalMergeTaskr{����initZHWRFInitrK���rO���Zenada_pre_object_forrS���Zget_grid_idr����ZFromGFSENKFr����Z
DAEnsembler�����confint�da_ensemble_sizeZ
set_memberZFromPriorCycle�memberZFGATInitrN���Zinit_at_timer����ZBufrprepZ
get_relocates�gsiZFGATGSIZrstage3Z	get_ghostrL���rM���Z	set_ensdar���ZMergeZrealinitZ	runwrfanlr����Z
get_wrfanlZset_gesZ
hwrfsystemZHWRFGSIPostProcessingZ
make_gsi_post�gsipost�
gsigribberr�����hycomZ	HYCOMInitr1���ZmpipomtcZPOMInitr3���Zww3ZWW3Initr8����productsZForecastBaseZ
make_forecastr*���r2���r9���ZFakeStormForecastr����ZForecastWatcherZRealInitZmultistorminZadd_orig_wrfinputZadd_metgridZmetgridZadd_geogridZgeogridZ
add_fort65Z
add_wrfbdyr~���Zadd_wrfinputZ
add_wrfanlZHWRFForecastPostProcessingZmake_nonsatpost�
nonsatpostZmake_satpost�satpostZmake_wrfcopier�	wrfcopierZmake_gribber_trackerr+����tracker�nhcpZmake_extra_trackers�
trackerd01�
trackerd02)?rW���Zmake_ensemble_daZ	make_postr���rY����this_stormidr]���Znum_realstormsr;���rD���Zsatpost_flagr<���r=���r>���r?���r@���rA���r���rB���Zgofile_flagr���rE���r`���ZfgatstrZfgatendZfgatinvrT���rg���rh���ri���rn���ro���rp���rq���ZpostdomsZstormNinnerZstormNouter�ir!���rs���Zwrfdomsr����Zwrfghost_bigZ
next_cycler����Z	ensdadomsZdomr����r����Zgsid03ZceninitZ	ingsi_d02Z	ingsi_d03Zges_d02Zges_d03ZhgppZnon_wave_basins�pZfcst_secZhfcstZrealwrfZhfppr����r���r����r����init_module���sV���

����









�� 


��
��

������������������������������
������������������������������
�����������������
���
��
���
���������������

��������������������������������
�����������������
����������
�r����)NTT)Z�__all__rP���Zos.path�pdb�produtil.datastore�produtil�produtil.run�hwrf.configr���Zhwrf.wrfZ	hwrf.post�
hwrf.numerics�
hwrf.launcherZhwrf.regribZ
hwrf.gribtaskZhwrf.tracker�hwrf.storminfoZhwrf.wpsZhwrf.nhc_productsZhwrf.copywrfZ
hwrf.fcsttaskZ
hwrf.ensdaZ
hwrf.relocateZ	hwrf.initZ	hwrf.prepZhwrf.gsiZ
hwrf.bufrprepZhwrf.gsipostZhwrf.prelaunchZhwrf.multistormZ
hwrf.couplingZhwrf.hwrfsystemZhwrf.finalmergetask�
hwrf.hycomZhwrf.ww3r���r����produtil.fileopr���r���r���r���r���r	���r
���r���r���r
���r���rC���rJ���r8���r1���r9���r2���r���r,���rk���rl���rm���r-���r*���r����r����rK���r+���r����r3���r����r����r^���r_���rN���r����r����rL���rM���rU���r����r���rO���r����r����r����r����r����r����r����rS���r����r���r���r���r����<module>#���sr���( (   A6