U

����� g�e����������������������@���s��d�dl�Z�d�dlZd�dlZd�dlZd�dlZd�dlZd�dlZd�dlZd�dl	Zd�dl
Zd�dlZd�dl
Zd�dlZd�dl	mZmZmZ�d�dlmZmZmZ�d�dlmZmZ�d�dlmZmZmZ�d�dlmZ�G�dd��de�ZG�d	d
��d
e�ZG�dd��de�ZG�d
d��dej j!�Z"G�dd��de�Z#dS�)�����N)�	setrlimit�rusage�	getrlimit)�
isnonempty�make_symlink�deliver_file)�mpirun�mpi)�to_fraction�to_datetime_rel�TimeMapping)�jloggerc�������������������@���s0���e�Zd�ZdZdd��Zedd���Zedd���ZdS�)	�	Componentaf��This is a simple utility class that stores information about
    coupler components (wrf, pom, hycom, wavewatch3, coupler, etc.)
    for the CoupledWRF class.  You should never instantiate a
    Component -- only CoupledWRF should do that.  It will return one
    of these when you run CoupledWRF's "component" function.

    You can safely modify the exe, rankname, rankdefault and initer
    elements of this class, and it will modify the corresponding
    information in the CoupledWRF.  Do not modify the name and order.
    Modifying the name or order will break the internal data
    structures in CoupledWRFc�����������������C���s,���|||||f\|�_�|�_|�_|�_|�_d|�_dS�)z�Creates a Component with the given characteristics.
        Initializes the order to None.  The order must be overwritten
        externally, and is done in CoupledWRF.couple.N)�_name�exe�rankname�rankdefault�initer�_order)�self�namer���r���r���r�����r����:/lfs/h1/ops/prod/packages/hmon.v3.2.7/ush/hwrf/coupling.py�__init__���s�����zComponent.__init__c�����������������C���s���|�j�S��N)r����r���r���r���r���r���"���s����zComponent.namec�����������������C���s���|�j�S�r���)r���r���r���r���r����order$���s����zComponent.orderN)�__name__�
__module__�__qualname__�__doc__r����propertyr���r���r���r���r���r���r������s���
r���c�������������������@���s(���e�Zd�ZdZdd��Zdd��Zdd��ZdS�)	�ComponentInitera���This class should be subclassed to make coupling component
    initializers for CoupledWRF.  That is, you should implement a
    subclass and send it to the CoupledWRF.couple(...,initer=).  Its
    job is to provide input data and initialization checks for the
    various coupling components such as ocean and wave.  Note that the
    CoupledWRF itself initializes the WRF and coupler, so you do not
    need a ComponentIniter for those components.c�����������������C���s���t�d��dS�)z�This function implements a very fast (<1 second) check to
        see if the components initialization was complete.  Returns
        True on success or False on failure.z2Subclass forgot to implement check_coupled_inputs.N)�NotImplementedError�r����loggerr���r���r����check_coupled_inputs0���s�����z$ComponentIniter.check_coupled_inputsc�����������������C���s���|r|���|�S�dS�)a���If just_check is True, then this function implements a more
        expensive check of whether the initialization for this
        component succeeded.  If just_check is False, then this
        component should link or copy all files needed to run a
        coupled forecast.  The default implementation calls
        check_coupled_inputs if just_check is True, and simply returns
        True if just_check is False.  Subclasses should re-implement
        this function.T)r&���)r����
just_checkr%���r���r���r����link_coupled_inputs6���s����	
z#ComponentIniter.link_coupled_inputsc�����������������C���s���t�|�|��|�S�)a)��Called when it is time to actually run the coupled forecast
        executables.  Returns a produtil.mpiprog.MPIRanksBase for the
        MPI ranks for this component of the coupling.  By default,
        this returns mpi(task.getexe(exe))*ranks.  However, the caller
        can re-implement this function to replace the executable
        depending on the results of the initialization.  If that is
        desired, the task.getexe (and other task.get*) should be used
        so the executable locations can be overridden in the config
        files.)r	����getexe)r����taskr����ranksr���r���r����make_exeB���s����
zComponentIniter.make_exeN)r���r���r���r ���r&���r(���r,���r���r���r���r���r"���(���s���r"���c�����������������������s$���e�Zd�Zdd��Z��fdd�Z���ZS�)�TurboIniterc�����������������C���s���dS��NTr���r$���r���r���r���r&���O���s����z TurboIniter.check_coupled_inputsc��������������������sB���t�t|���|||�}d|_t�d��dt|�f���|js>t�|S�)zk!Turns on Intel turbo mode for the specified executable, if the
        local MPI implementation knows how.Tzcoupling.pyzexe for wrf compute is %s)	�superr-���r,����	turbomode�logging�	getLogger�info�repr�AssertionError)r���r*���r���r+����m��	__class__r���r���r,���Q���s����
�
zTurboIniter.make_exe)r���r���r���r&���r,����
__classcell__r���r���r7���r���r-���N���s���r-���c�����������������������s����e�Zd�ZdZd+��fdd�	Zd,��fdd�	Zd-��fd
d�	Zd.��fdd
�	Z��fdd�Zd/dd�Z	dd��Z
d0dd�Zdd��Zdd��Z
dd��Zd1dd�Zdd ��Zd!d"��Zd#d$��Zd%d&��Zd2d'd(�Zd3��fd)d*�	Z���ZS�)4�
CoupledWRFz�Runs the NCEP coupler and WRF, coupled to at least one ocean or
    wave model.  This class is not thread-safe due to how self.couple
    is implemented.T�auxhist1c��������������������sF���t�t|��j||||||f|��t��|�_t��|�_t��|�_d�|�_d|�_	d�S�r.���)
r/���r:���r����dict�_CoupledWRF__components�list�_CoupledWRF__order�_coupled_products�_CoupledWRF__coupled�_default_coupling)r����dstore�conf�section�wrf�keeprun�wrfdiag_stream�kwargsr7���r���r���r���`���s������zCoupledWRF.__init__Nc��������������������s\���|dk	r|�j�|�_|����}tt|�����}|rX|rX|����D�] }|jdk	r6|jj|����d�}q6|S�)z�Returns True if all inputs needed to run the forecast are
        present, and false otherwise.  If coupled=True (the default),
        then coupled components inputs are also checked, otherwise
        only WRF inputs are checked.N�r%���)	rB���rA����logr/���r:����check_all_inputs�
coupleiterr���r&���)r����coupledr%����okay�cr7���r���r���rL���m���s�����
zCoupledWRF.check_all_inputsFc��������������������s����|�����}tt|��j|d�}|r8|r.|�d��n
|�d��|dkrF|�j}|dkrT|�j}|sf|�d��|S�|st|��d��|��	��D�]d}|j
dk	r||o�|j
j||d�}|s�d|jf�}|�|��|s�t
j�|���d	S�|�d
|jf���q||S�)a���If just_check=True, links or copies inputs required by all
        components of the coupled simulation.  If just_check=False,
        runs through all steps required to link or copy inputs without
        actually linking or copying.  That mode is intended to be an
        expensive but more thurough check than check_all_inputs.

        Returns True if all inputs were linked, and False otherwise.)r'���zWRF inputs are present.zFAIL: WRF inputs are missing.NzYNot copying ocean, wave and coupler inputs because an uncoupled simulation was requested.�cpl_nmlrJ���z%s inputs are missing.Fz%s inputs are all present.)rK���r/���r:����link_all_inputsr3����errorrA���rB����make_coupler_namelistrM���r���r(���r����hwrf�
exceptions�OceanInitFailed)r���r'���rN���r%���rO���rP����msgr7���r���r���rR���{���s@����
�
���

�
zCoupledWRF.link_all_inputsc��������������������s&���|dkr|�j�}||�_tt|������dS�)z�Runs the coupled simulation.  Sets the internal coupled
        vs. uncoupled flag to the specified value.  Default: True
        (coupled).N)rB���rA���r/���r:����run)r���rN���r7���r���r���rY�������s�����zCoupledWRF.runc��������������
������s*��|�����}t|ddd��|�jdkr(|�j|�_|�j�r|�d��d}t��}t��}|����D�]�}|�d|j|j	|j
|jt|j
�f���|j
}|��|j|j
�}t|t�s�tdt|�jt|�f���|dks�td	|���|jdk	r�|j�|�||�}nt|��|��|�}|�d
t|�t|j�f���|||j	<�|�|j	��|dk�r8|n||�}qR|dk�rXtj�d��|��d�}	t�d
|	f���|j dd�D�]B\}
}|�dt|
�t|�t|
j�f���|dk�r~|
j�s~t!��q~t"|�|	k}|��#|||�}t$d��s�tj�%d��t&t'|���(dd|��n|�d��t&t'|���(ddd��dS�)zrRuns the MPI command for the coupled coupled simulation.
        Do not call this directly: call self.run instead.g����Z�AT)r%����stack�ignoreNz#Starting the coupled wrf simulationz6Component #%d is %s: exe=%s ranks from %s (default %s)zbSomehow ended up with a non-int for ranks in CoupledWRF.run_exe.  Check inputs.  Ranks is a %s %s.r���zLRanks (%d) is not >0 in CoupledWRF.run_exe.  Check config files and scripts.z Appending ranks %s with turbo=%szTNo coupled components specified in CoupledWRF.  You must call CoupledWRF.couple(...)z
{coupled_log}z)%s: will log coupled forecast stdout hereF)�expandz#Will run rank %s * %s with turbo=%si���rQ���z5Logic error: somehow the cpl_nml is empty or missing.rF���z%Starting the uncoupled wrf simulation))rK���r���rA���rB���r3���r<���r>���rM���r���r���r���r���r4���r����confint�
isinstance�int�	TypeError�typer����
ValueErrorr���r,���r	���r)���r0����appendrU���rV����NoCoupledComponents�
confstrinterpr
����expand_iterr5���r���Zcall_run_exe_callbacksr����EmptyCouplerNamelistr/���r:����run_exe)r���r%����cmdZ
callback_argsZcallback_componentsrP���r���r+���Zmpiified�logfile�rank�countr7���r���r���rh�������s�����
�
��
����
��

�
����
���
�
zCoupledWRF.run_exerQ���c��������������	���C���s����|�����}|�d|�jf���|��d�}|��d�}t|�}t|�j���|�j�����}t	t
�t||����}t
||d�}|�dt|�t|�t|�t|�f���tjj|�j||d�}	t|d��}
|	j|d�}|
�|��W�5�Q�R�X�d	S�)
z'Makes the namelist for the NCEP Couplerz
section is %srQ����dt_c)�cstepmaxrm���z'dt_c=%s f_dt_c=%s simlen=%s cstepmax=%s)rD���rE����morevars�wt)ro���N)rK���r3���rE����confstrr]���r
����simZsimend�simstartr_����math�ceil�floatr<���r4���rU����namelistZ
Conf2NamelistrD����openZ
make_namelist�write)r����filenamer%���Zcplsecrm���Zf_dt_cZsimlenrn���ro���ZcplnmlZcnZnmlstrr���r���r���rT�������s,����

�������z CoupledWRF.make_coupler_namelistc�����������������c���s,���d}|�j�D�]}|d7�}|�j|�}|V��q
dS�)zNIterates over all Component objects that describe coupling
        components.���������N)r?���r=���)r����ir���rP���r���r���r���rM�������s
����

zCoupledWRF.coupleiterc�����������������C���s��t�|t�s$tdt|�jt|�f���|dksH|�d�dksH|�d�dkrPtd��|dk	r|t�|t�s|tdt|�jt|�f���t�|t�s�td	t|�jt|�f���t�|t�s�td
t|�jt|�f���|dkr�dnt|�}||�j	kr�|�j
�|��t|||||�}|��
|�|_||�j	|<�|�S�)aW��Adds the specified coupling component.  Returns self.
           name -- a user-defined name of this component, must be a string.
           exe -- option in the [exe] section with the executable path
           rankname -- name of the option in this task's section that
              has the number of ranks
           rankdefault -- number of ranks if the rankname option is
              missing or empty.  Can be None, in which case the
              rankname option must be set.
           initer -- the object that will link inputs ot the working
              directory before the forecast begins.  Can be None.
        Note that the superclass, WRFAtmos, initializes the WRF 
        component and this class initializes the coupler, so you can
        pass None for those components' "initer" objects.

        This subroutine is not thread-safe.zOThe "name" argument to CoupledWRF.couple must be a string.  You passed a %s %s.��output�historyZrestartZ	restartin�inputZinputout�auxhistr����auxinputz�Component name cannot be the same as any WRF stream name: output, history, restart, restartin, input, inputout, auxhist* or auxinput*.Nz\The "rankdefault" argument to CoupledWRF.couple must be None or an int.  You passed a %s %s.zOThe "exe" argument to CoupledWRF.couple must be a string.  You plassed a %s %s.zTThe "rankname" argument to CoupledWRF.couple must be a string.  You plassed a %s %s.)r^����strr`���ra���r���r4����findrb���r_���r=���r?���rc���r���r���r���)r���r���r���r���r���r���ZrdrP���r���r���r����couple��sN����
�������
��
��

zCoupledWRF.couplec�����������������C���s����|�����}|j}|j}|��dd�}t|t�s4t|t�rB|t|��}n|���}||�|�}|dkr�|dkrp||�}nd}|��	ddd|t
����|��	ddd|��n|dkr�|��	ddd|t
����dS�)	a}��!Wrapper around one or two calls to couple() to add the WRF.

        Attempts to add the WRF to the coupling in two different sets of
        MPI ranks: one for WRF compute, and one for WRF I/O.  This is
        done to allow the local MPI implementation to set up the 
        compute nodes differently for the two blocks of processors, 
        thereby speeding up the forecast.�	wrf_ranksr���NZwrf_computerF����wrf_compute_ranksZwrf_ioZwrf_io_ranks)rF���Znio_tasks_per_group�
nio_groupsr]���r^���r>����tuple�sum�maxdomr����r-���)r���rF���Znio_pergroupr����r����Znior����r����r���r���r����	couplewrf8��s ����
zCoupledWRF.couplewrfc�����������������C���s���d�S�r���r���r���r���r���r����remove_wavea�������zCoupledWRF.remove_wavec�����������������C���s���d�S�r���r���r���r���r���r����remove_oceanb��r����zCoupledWRF.remove_oceanc�����������������C���sx���|dkrd|�_�|�S�|�j|=�t��}|�jD�]}||kr&|�|��q&||�_|����D�]}|��|j�|_qLt	|�dkrt|��
���|�S�)z�!Removes a component, or all components, from the coupling.
        @param which the name of the component to remove.  If None or 
           unspecified, run totally uncoupled.
        @returns selfNF����)rB���r=���r>���r?���rc���rM���r���r���r����len�uncouple)r����whichr����wrP���r���r���r���r����d��s����
zCoupledWRF.uncouplec�����������������C���sP���t�|t�r|�j|�S�t�|t�r2|�j|�}|�j|�S�tdt|�jt|�f���dS�)z�Returns information about the component with the specified
        name (a string) or at the specified zero-based index (an
        int).  Returns a mutable object of class Component.zMCoupledWRF.component requires an int or string argument.  You passed a %s %s.N)	r^���r����r=���r_���r?���r`���ra���r���r4���)r���r����r���r���r���r����	componentz��s����




�zCoupledWRF.componentc�����������������C���s4���d}|�j�D�]}|d7�}||kr
|��S�q
t|��dS�)a���Components of coupling are ordered by where they show up in
        the MPI execution.  The NCEP coupler must be first.  This
        function returns the zero-based order.  Hence, if one does:

            self.couple("coupler",...).couple("hycom",...).couple("wrf",...)

        then:

            self.order("coupler") = 0
            self.order("hycom")   = 1
            self.order("wrf")     = 2 

        Other values will raise KeyError.r{���r|���N)r?����KeyError)r���r���r}����or���r���r���r������s����
�
zCoupledWRF.orderc��������������������sl���|dks$|��d�dks$|��d�dkr2td|f���||�jkrh|�j�������fdd�|D��}t|t�|�j|<�|�S�)z!Adds a new non-WRF output stream.r~���r����r���r����z�Component stream name (%s) cannot be the same as any WRF stream name: output, history, restart, restartin, input, inputout, auxhist* or auxinput*.c��������������������s���g�|�]}t�|����qS�r���)r���)�.0�t��ar���r����
<listcomp>���s�����z1CoupledWRF.add_coupled_stream.<locals>.<listcomp>)r����rb���r@���rr���rs���r���r>���)r����stream�timesZtimerelr���r����r����add_coupled_stream���s��������

zCoupledWRF.add_coupled_streamc�����������������C���s���|�j�|�|��|��|�S�)z;Adds a product for a non-WRF output stream at a given time.)r@���rc���)r���r�����time�productr���r���r����add_coupled_product���s����zCoupledWRF.add_coupled_productc�����������������c���s����|dkr�|dkrH|�j����D�]*\}}|���D�]\}}|D�]
}|V��q6q*qq�|�j����D�].\}}|�|d�}|dkrpqR|D�]
}|V��qtqRnd||�j�kr�|�j�|�}|dkr�|���D�]\}}|D�]
}|V��q�q�n$|�|d�}|dk	r�|D�]
}|V��q�dS�)zIterates over non-WRF products.N)r@����items�get)r���r����r�����sZtprodr�����prods�pr���r���r����coupled_products���s,�������

��zCoupledWRF.coupled_productsc�����������������+���sp���|dkrd}d}n||�j�k}|�}|rPtt|��jf�|||d�|��D�]
}|V��qD|rl|��||�D�]
}|V��q`dS�)z1Iterates over all products, both WRF and non-WRF.NT)�domainsr����r����)r@���r/���r:����productsr����)r���r����r����r����rI���rN���rF���r����r7���r���r���r�������s"����
����
zCoupledWRF.products)Tr;���)N)FN)N)rQ���)NN)N)NN)NNN)r���r���r���r ���r���rL���rR���rY���rh���rT���rM���r����r����r����r����r����r����r���r����r����r����r����r9���r���r���r7���r���r:���\���s*������
'>

1)


r:���c�������������������@���sn���e�Zd�ZdZddd�Zedd���Zedd���Zed	d
���Zddd�Z	d
d��Z
ddd�Zddd�Zddd�Z
dS�)�CouplingStatusz?Maintains the ocean and wave status files in the COM directory.Nc�����������������C���sL���||�_�||�_t|t�s0tdt|�jt|�f���|d�krB|�|�}||�_	d�S�)NzNThe section argument to CouplingStatus.__init__ must be a string, not a %s %s.)
�_CouplingStatus__conf�_CouplingStatus__sectionr^���r����r`���ra���r���r4���rK����_CouplingStatus__logger)r���rD���rE���r%���r���r���r���r������s����

zCouplingStatus.__init__c�����������������C���s���|�j�S�)z/The logging.Logger to use for logging messages.)r����r���r���r���r���r%������s����zCouplingStatus.loggerc�����������������C���s���|�j�S�)z>The configuration object, a subclass of hwrf.config.HWRFConfig)r����r���r���r���r���rD������s����zCouplingStatus.confc�����������������C���s���|�j�S�)z>The section in self.conf to use for configuration information.)r����r���r���r���r���rE������s����zCouplingStatus.sectionc�����������������C���s`���|dkr|�j�}|�j|�jd�fD�]<}|�j�|�j|�}tj�|�j�d�|�}tj	j
|d|d��qdS�)zzDelete the coupling status files.  If the logger is
        not specified, the section name is used for the logging domainN�2ZcomT)r3���r%���)r%���rE���rD����getstr�os�path�join�getdir�produtil�fileop�remove_file)r���r%���Zocstat�
ocstatfiler���r���r����unset���s�����zCouplingStatus.unsetc�����������������c���s<���|�j��|�jd|�j�d��V��|�j��|�jd|�j�d��V��d�S�)N�{�}z2})rD����	strinterprE���r���r���r���r����fileiter���s����zCouplingStatus.fileiterc��������������
���C���s����|dkr|�����}t|�}|����D�]|}|r,dnd}|�d||f���t|d��J}|�d|�d���|dk	r�|D�]$}|�d||f���|�|d���qlW�5�Q�R�X�q dS�)	z�Set RUN_COUPLED=YES (true) or =NO (false) depending on the
        value of coupled_flag.  If the logger is not specified, the
        section name is used as the logging domain.N�YES�NOz(Setting RUN_COUPLED=%s in status file %srp���zRUN_COUPLED=�
z%s: write %s)rK����boolr����r3���rx���ry���)r���Z
coupling_flagr%���Z	morelinesr����Zstrflag�f�liner���r���r����set���s�������zCouplingStatus.setc��������������	���C���s^���|dkr|�j�}|�j�|�jd|�j�d��}tj�|�s:t��S�t|d��}|�	d�}W�5�Q�R�X�|S�)zvReads the first coupling status file (identified by {section}=)
        and returns the contents as an array of lines.Nr����r�����rti���)
r%���rD���r����rE���r����r�����existsr>���rx����	readlines)r���r%���r����r�����linesr���r���r����read��s�����zCouplingStatus.readc��������������
���C���sR��|dkr|�j�}|�j�|�jd|�j�d��}d}|�d|���zft|d��R}|D�]F}|����d�dkrrd}|�d	��qL|����d
�dkrLd}|�d��qLW�5�Q�R�X�W�nv�t	k
r��}�z|j
d
t|�f�dd��W�5�d}~X�Y�n>�tk
�r�}�z|j
dt|�f�dd����W�5�d}~X�Y�nX�|dk�r2|�d��d}n|�rD|�d��n
|�d��|S�)a���Checks the coupling status file.  If the file does not exist or
        cannot be opened or read, then False is returned.  Otherwise, the
        file is scanned for RUN_COUPLED=YES or RUN_COUPLED=NO (case
        insensitive).  The last of those RUN_COUPLED lines is used:
        NO=return False, YES=return True.  If the logger is not specified,
        the section name is used as the logging domainNr����r����z.%s: scan status file for RUN_COUPLED=YES or NOr����zRUN_COUPLED=YESr���Tz!Status file says: RUN_COUPLED=YESzRUN_COUPLED=NOFz Status file says: RUN_COUPLED=NOzError checking status file: %s)�exc_infoz2Unhandled exception while checking status file: %szOCould not scan status file for RUN_COUPLED=YES or NO.  Assuming RUN_COUPLED=NO.zCRUN_COUPLED=YES: status file says coupled component init succeeded.z?RUN_COUPLED=NO: status file says coupled component init failed.)
r%���rD���r����rE���r3���rx����upperr�����warning�EnvironmentErrorrS���r�����	Exception)r���r%���r����Zsuccessr����r�����e�eer���r���r���r������s\������������

��zCouplingStatus.get)N)N)NN)N)N)r���r���r���r ���r���r!���r%���rD���rE���r����r����r����r����r����r���r���r���r���r�������s���






r����)$r�����shutilrt���r1����produtil.datastorer�����produtil.fileop�produtil.cd�produtil.runZprodutil.rusage�
hwrf.hwrftaskrU����
hwrf.numerics�hwrf.exceptionsZ
hwrf.fcsttaskr���r���r���r���r���r���r���r	���r
���r���r����produtil.logr
����objectr���r"���r-���ZfcsttaskZWRFAtmosr:���r����r���r���r���r����<module>���s���   &��|