U ����� g�e����������������������@���s��d�dl�Z�d�dlZd�dlZd�dlZd�dlZd�dlZd�dlZd�dlZd�dl Zd�dl Zd�dlZd�dl Zd�dlZd�dl mZmZmZ�d�dlmZmZmZ�d�dlmZmZ�d�dlmZmZmZ�d�dlmZ�G�dd��de�ZG�d d ��d e�ZG�dd��de�ZG�d d��dej j!�Z"G�dd��de�Z#dS�)�����N)� setrlimit�rusage� getrlimit)� isnonempty�make_symlink�deliver_file)�mpirun�mpi)�to_fraction�to_datetime_rel�TimeMapping)�jloggerc�������������������@���s0���e�Zd�ZdZdd��Zedd���Zedd���ZdS�) � Componentaf��This is a simple utility class that stores information about coupler components (wrf, pom, hycom, wavewatch3, coupler, etc.) for the CoupledWRF class. You should never instantiate a Component -- only CoupledWRF should do that. It will return one of these when you run CoupledWRF's "component" function. You can safely modify the exe, rankname, rankdefault and initer elements of this class, and it will modify the corresponding information in the CoupledWRF. Do not modify the name and order. Modifying the name or order will break the internal data structures in CoupledWRFc�����������������C���s,���|||||f\|�_�|�_|�_|�_|�_d|�_dS�)z�Creates a Component with the given characteristics. Initializes the order to None. The order must be overwritten externally, and is done in CoupledWRF.couple.N)�_name�exe�rankname�rankdefault�initer�_order)�self�namer���r���r���r�����r����:/lfs/h1/ops/prod/packages/hmon.v3.2.7/ush/hwrf/coupling.py�__init__���s�����zComponent.__init__c�����������������C���s���|�j�S��N)r����r���r���r���r���r���"���s����zComponent.namec�����������������C���s���|�j�S�r���)r���r���r���r���r����order$���s����zComponent.orderN)�__name__� __module__�__qualname__�__doc__r����propertyr���r���r���r���r���r���r������s��� r���c�������������������@���s(���e�Zd�ZdZdd��Zdd��Zdd��ZdS�) �ComponentInitera���This class should be subclassed to make coupling component initializers for CoupledWRF. That is, you should implement a subclass and send it to the CoupledWRF.couple(...,initer=). Its job is to provide input data and initialization checks for the various coupling components such as ocean and wave. Note that the CoupledWRF itself initializes the WRF and coupler, so you do not need a ComponentIniter for those components.c�����������������C���s���t�d��dS�)z�This function implements a very fast (<1 second) check to see if the components initialization was complete. Returns True on success or False on failure.z2Subclass forgot to implement check_coupled_inputs.N)�NotImplementedError�r����loggerr���r���r����check_coupled_inputs0���s�����z$ComponentIniter.check_coupled_inputsc�����������������C���s���|r|���|�S�dS�)a���If just_check is True, then this function implements a more expensive check of whether the initialization for this component succeeded. If just_check is False, then this component should link or copy all files needed to run a coupled forecast. The default implementation calls check_coupled_inputs if just_check is True, and simply returns True if just_check is False. Subclasses should re-implement this function.T)r&���)r���� just_checkr%���r���r���r����link_coupled_inputs6���s���� z#ComponentIniter.link_coupled_inputsc�����������������C���s���t�|�|��|�S�)a)��Called when it is time to actually run the coupled forecast executables. Returns a produtil.mpiprog.MPIRanksBase for the MPI ranks for this component of the coupling. By default, this returns mpi(task.getexe(exe))*ranks. However, the caller can re-implement this function to replace the executable depending on the results of the initialization. If that is desired, the task.getexe (and other task.get*) should be used so the executable locations can be overridden in the config files.)r ����getexe)r����taskr����ranksr���r���r����make_exeB���s���� zComponentIniter.make_exeN)r���r���r���r ���r&���r(���r,���r���r���r���r���r"���(���s���r"���c�����������������������s$���e�Zd�Zdd��Z��fdd�Z���ZS�)�TurboIniterc�����������������C���s���dS��NTr���r$���r���r���r���r&���O���s����z TurboIniter.check_coupled_inputsc��������������������sB���t�t|���|||�}d|_t�d��dt|�f���|js>t�|S�)zk!Turns on Intel turbo mode for the specified executable, if the local MPI implementation knows how.Tzcoupling.pyzexe for wrf compute is %s) �superr-���r,���� turbomode�logging� getLogger�info�repr�AssertionError)r���r*���r���r+����m�� __class__r���r���r,���Q���s���� � zTurboIniter.make_exe)r���r���r���r&���r,���� __classcell__r���r���r7���r���r-���N���s���r-���c�����������������������s����e�Zd�ZdZd+��fdd� Zd,��fdd� Zd-��fd d� Zd.��fdd � Z��fdd�Zd/dd�Z dd��Z d0dd�Zdd��Zdd��Z dd��Zd1dd�Zdd ��Zd!d"��Zd#d$��Zd%d&��Zd2d'd(�Zd3��fd)d*� Z���ZS�)4� CoupledWRFz�Runs the NCEP coupler and WRF, coupled to at least one ocean or wave model. This class is not thread-safe due to how self.couple is implemented.T�auxhist1c��������������������sF���t�t|��j||||||f|��t��|�_t��|�_t��|�_d�|�_d|�_ d�S�r.���) r/���r:���r����dict�_CoupledWRF__components�list�_CoupledWRF__order�_coupled_products�_CoupledWRF__coupled�_default_coupling)r����dstore�conf�section�wrf�keeprun�wrfdiag_stream�kwargsr7���r���r���r���`���s������zCoupledWRF.__init__Nc��������������������s\���|dk r|�j�|�_|����}tt|�����}|rX|rX|����D�] }|jdk r6|jj|����d�}q6|S�)z�Returns True if all inputs needed to run the forecast are present, and false otherwise. If coupled=True (the default), then coupled components inputs are also checked, otherwise only WRF inputs are checked.N�r%���) rB���rA����logr/���r:����check_all_inputs� coupleiterr���r&���)r����coupledr%����okay�cr7���r���r���rL���m���s����� zCoupledWRF.check_all_inputsFc��������������������s����|�����}tt|��j|d�}|r8|r.|�d��n |�d��|dkrF|�j}|dkrT|�j}|sf|�d��|S�|st|��d��|�� ��D�]d}|j dk r||o�|j j||d�}|s�d|jf�}|�|��|s�t j�|���d S�|�d |jf���q||S�)a���If just_check=True, links or copies inputs required by all components of the coupled simulation. If just_check=False, runs through all steps required to link or copy inputs without actually linking or copying. That mode is intended to be an expensive but more thurough check than check_all_inputs. Returns True if all inputs were linked, and False otherwise.)r'���zWRF inputs are present.zFAIL: WRF inputs are missing.NzYNot copying ocean, wave and coupler inputs because an uncoupled simulation was requested.�cpl_nmlrJ���z%s inputs are missing.Fz%s inputs are all present.)rK���r/���r:����link_all_inputsr3����errorrA���rB����make_coupler_namelistrM���r���r(���r����hwrf� exceptions�OceanInitFailed)r���r'���rN���r%���rO���rP����msgr7���r���r���rR���{���s@���� � ��� � zCoupledWRF.link_all_inputsc��������������������s&���|dkr|�j�}||�_tt|������dS�)z�Runs the coupled simulation. Sets the internal coupled vs. uncoupled flag to the specified value. Default: True (coupled).N)rB���rA���r/���r:����run)r���rN���r7���r���r���rY�������s�����zCoupledWRF.runc�������������� ������s*��|�����}t|ddd��|�jdkr(|�j|�_|�j�r|�d��d}t��}t��}|����D�]�}|�d|j|j |j |jt|j �f���|j }|��|j|j �}t|t�s�tdt|�jt|�f���|dks�td |���|jdk r�|j�|�||�}nt|��|��|�}|�d t|�t|j�f���|||j <�|�|j ��|dk�r8|n||�}qR|dk�rXtj�d��|��d�} t�d | f���|j dd�D�]B\} }|�dt| �t|�t| j�f���|dk�r~| j�s~t!��q~t"|�| k}|��#|||�}t$d��s�tj�%d��t&t'|���(dd|��n|�d��t&t'|���(ddd��dS�)zrRuns the MPI command for the coupled coupled simulation. Do not call this directly: call self.run instead.g����Z�AT)r%����stack�ignoreNz#Starting the coupled wrf simulationz6Component #%d is %s: exe=%s ranks from %s (default %s)zbSomehow ended up with a non-int for ranks in CoupledWRF.run_exe. Check inputs. Ranks is a %s %s.r���zLRanks (%d) is not >0 in CoupledWRF.run_exe. Check config files and scripts.z Appending ranks %s with turbo=%szTNo coupled components specified in CoupledWRF. You must call CoupledWRF.couple(...)z {coupled_log}z)%s: will log coupled forecast stdout hereF)�expandz#Will run rank %s * %s with turbo=%si���rQ���z5Logic error: somehow the cpl_nml is empty or missing.rF���z%Starting the uncoupled wrf simulation))rK���r���rA���rB���r3���r<���r>���rM���r���r���r���r���r4���r����confint� isinstance�int� TypeError�typer���� ValueErrorr���r,���r ���r)���r0����appendrU���rV����NoCoupledComponents� confstrinterpr ����expand_iterr5���r���Zcall_run_exe_callbacksr����EmptyCouplerNamelistr/���r:����run_exe)r���r%����cmdZ callback_argsZcallback_componentsrP���r���r+���Zmpiified�logfile�rank�countr7���r���r���rh�������s����� � �� ���� �� � ���� ��� � zCoupledWRF.run_exerQ���c�������������� ���C���s����|�����}|�d|�jf���|��d�}|��d�}t|�}t|�j���|�j�����}t t �t||����}t ||d�}|�dt|�t|�t|�t|�f���tjj|�j||d�} t|d��} | j|d�}| �|��W�5�Q�R�X�d S�) z'Makes the namelist for the NCEP Couplerz section is %srQ����dt_c)�cstepmaxrm���z'dt_c=%s f_dt_c=%s simlen=%s cstepmax=%s)rD���rE����morevars�wt)ro���N)rK���r3���rE����confstrr]���r ����simZsimend�simstartr_����math�ceil�floatr<���r4���rU����namelistZ Conf2NamelistrD����openZ make_namelist�write)r����filenamer%���Zcplsecrm���Zf_dt_cZsimlenrn���ro���ZcplnmlZcnZnmlstrr���r���r���rT�������s,���� �������z CoupledWRF.make_coupler_namelistc�����������������c���s,���d}|�j�D�]}|d7�}|�j|�}|V��q dS�)zNIterates over all Component objects that describe coupling components.���������N)r?���r=���)r����ir���rP���r���r���r���rM�������s ���� zCoupledWRF.coupleiterc�����������������C���s��t�|t�s$tdt|�jt|�f���|dksH|�d�dksH|�d�dkrPtd��|dk r|t�|t�s|tdt|�jt|�f���t�|t�s�td t|�jt|�f���t�|t�s�td t|�jt|�f���|dkr�dnt|�}||�j kr�|�j �|��t|||||�}|�� |�|_||�j |<�|�S�)aW��Adds the specified coupling component. Returns self. name -- a user-defined name of this component, must be a string. exe -- option in the [exe] section with the executable path rankname -- name of the option in this task's section that has the number of ranks rankdefault -- number of ranks if the rankname option is missing or empty. Can be None, in which case the rankname option must be set. initer -- the object that will link inputs ot the working directory before the forecast begins. Can be None. Note that the superclass, WRFAtmos, initializes the WRF component and this class initializes the coupler, so you can pass None for those components' "initer" objects. This subroutine is not thread-safe.zOThe "name" argument to CoupledWRF.couple must be a string. You passed a %s %s.��output�historyZrestartZ restartin�inputZinputout�auxhistr����auxinputz�Component name cannot be the same as any WRF stream name: output, history, restart, restartin, input, inputout, auxhist* or auxinput*.Nz\The "rankdefault" argument to CoupledWRF.couple must be None or an int. You passed a %s %s.zOThe "exe" argument to CoupledWRF.couple must be a string. You plassed a %s %s.zTThe "rankname" argument to CoupledWRF.couple must be a string. You plassed a %s %s.)r^����strr`���ra���r���r4����findrb���r_���r=���r?���rc���r���r���r���)r���r���r���r���r���r���ZrdrP���r���r���r����couple��sN���� ������� �� �� zCoupledWRF.couplec�����������������C���s����|�����}|j}|j}|��dd�}t|t�s4t|t�rB|t|��}n|���}||�|�}|dkr�|dkrp||�}nd}|�� ddd|t ����|�� ddd|��n|dkr�|�� ddd|t ����dS�) a}��!Wrapper around one or two calls to couple() to add the WRF. Attempts to add the WRF to the coupling in two different sets of MPI ranks: one for WRF compute, and one for WRF I/O. This is done to allow the local MPI implementation to set up the compute nodes differently for the two blocks of processors, thereby speeding up the forecast.� wrf_ranksr���NZwrf_computerF����wrf_compute_ranksZwrf_ioZwrf_io_ranks)rF���Znio_tasks_per_group� nio_groupsr]���r^���r>����tuple�sum�maxdomr����r-���)r���rF���Znio_pergroupr����r����Znior����r����r���r���r���� couplewrf8��s ���� zCoupledWRF.couplewrfc�����������������C���s���d�S�r���r���r���r���r���r����remove_wavea�������zCoupledWRF.remove_wavec�����������������C���s���d�S�r���r���r���r���r���r����remove_oceanb��r����zCoupledWRF.remove_oceanc�����������������C���sx���|dkrd|�_�|�S�|�j|=�t��}|�jD�]}||kr&|�|��q&||�_|����D�]}|��|j�|_qLt |�dkrt|�� ���|�S�)z�!Removes a component, or all components, from the coupling. @param which the name of the component to remove. If None or unspecified, run totally uncoupled. @returns selfNF����)rB���r=���r>���r?���rc���rM���r���r���r����len�uncouple)r����whichr����wrP���r���r���r���r����d��s���� zCoupledWRF.uncouplec�����������������C���sP���t�|t�r|�j|�S�t�|t�r2|�j|�}|�j|�S�tdt|�jt|�f���dS�)z�Returns information about the component with the specified name (a string) or at the specified zero-based index (an int). Returns a mutable object of class Component.zMCoupledWRF.component requires an int or string argument. You passed a %s %s.N) r^���r����r=���r_���r?���r`���ra���r���r4���)r���r����r���r���r���r���� componentz��s���� �zCoupledWRF.componentc�����������������C���s4���d}|�j�D�]}|d7�}||kr |��S�q t|��dS�)a���Components of coupling are ordered by where they show up in the MPI execution. The NCEP coupler must be first. This function returns the zero-based order. Hence, if one does: self.couple("coupler",...).couple("hycom",...).couple("wrf",...) then: self.order("coupler") = 0 self.order("hycom") = 1 self.order("wrf") = 2 Other values will raise KeyError.r{���r|���N)r?����KeyError)r���r���r}����or���r���r���r������s���� � zCoupledWRF.orderc��������������������sl���|dks$|��d�dks$|��d�dkr2td|f���||�jkrh|�j�������fdd�|D��}t|t�|�j|<�|�S�)z!Adds a new non-WRF output stream.r~���r����r���r����z�Component stream name (%s) cannot be the same as any WRF stream name: output, history, restart, restartin, input, inputout, auxhist* or auxinput*.c��������������������s���g�|�]}t�|����qS�r���)r���)�.0�t��ar���r���� <listcomp>���s�����z1CoupledWRF.add_coupled_stream.<locals>.<listcomp>)r����rb���r@���rr���rs���r���r>���)r����stream�timesZtimerelr���r����r����add_coupled_stream���s�������� zCoupledWRF.add_coupled_streamc�����������������C���s���|�j�|�|��|��|�S�)z;Adds a product for a non-WRF output stream at a given time.)r@���rc���)r���r�����time�productr���r���r����add_coupled_product���s����zCoupledWRF.add_coupled_productc�����������������c���s����|dkr�|dkrH|�j����D�]*\}}|���D�]\}}|D�] }|V��q6q*qq�|�j����D�].\}}|�|d�}|dkrpqR|D�] }|V��qtqRnd||�j�kr�|�j�|�}|dkr�|���D�]\}}|D�] }|V��q�q�n$|�|d�}|dk r�|D�] }|V��q�dS�)zIterates over non-WRF products.N)r@����items�get)r���r����r�����sZtprodr�����prods�pr���r���r����coupled_products���s,������� ��zCoupledWRF.coupled_productsc�����������������+���sp���|dkrd}d}n||�j�k}|�}|rPtt|��jf�|||d�|��D�] }|V��qD|rl|��||�D�] }|V��q`dS�)z1Iterates over all products, both WRF and non-WRF.NT)�domainsr����r����)r@���r/���r:����productsr����)r���r����r����r����rI���rN���rF���r����r7���r���r���r�������s"���� ���� zCoupledWRF.products)Tr;���)N)FN)N)rQ���)NN)N)NN)NNN)r���r���r���r ���r���rL���rR���rY���rh���rT���rM���r����r����r����r����r����r����r���r����r����r����r����r9���r���r���r7���r���r:���\���s*������ '> 1) r:���c�������������������@���sn���e�Zd�ZdZddd�Zedd���Zedd���Zed d ���Zddd�Z d d��Z ddd�Zddd�Zddd�Z dS�)�CouplingStatusz?Maintains the ocean and wave status files in the COM directory.Nc�����������������C���sL���||�_�||�_t|t�s0tdt|�jt|�f���|d�krB|�|�}||�_ d�S�)NzNThe section argument to CouplingStatus.__init__ must be a string, not a %s %s.) �_CouplingStatus__conf�_CouplingStatus__sectionr^���r����r`���ra���r���r4���rK����_CouplingStatus__logger)r���rD���rE���r%���r���r���r���r������s���� zCouplingStatus.__init__c�����������������C���s���|�j�S�)z/The logging.Logger to use for logging messages.)r����r���r���r���r���r%������s����zCouplingStatus.loggerc�����������������C���s���|�j�S�)z>The configuration object, a subclass of hwrf.config.HWRFConfig)r����r���r���r���r���rD������s����zCouplingStatus.confc�����������������C���s���|�j�S�)z>The section in self.conf to use for configuration information.)r����r���r���r���r���rE������s����zCouplingStatus.sectionc�����������������C���s`���|dkr|�j�}|�j|�jd�fD�]<}|�j�|�j|�}tj�|�j�d�|�}tj j |d|d��qdS�)zzDelete the coupling status files. If the logger is not specified, the section name is used for the logging domainN�2ZcomT)r3���r%���)r%���rE���rD����getstr�os�path�join�getdir�produtil�fileop�remove_file)r���r%���Zocstat� ocstatfiler���r���r����unset���s�����zCouplingStatus.unsetc�����������������c���s<���|�j��|�jd|�j�d��V��|�j��|�jd|�j�d��V��d�S�)N�{�}z2})rD���� strinterprE���r���r���r���r����fileiter���s����zCouplingStatus.fileiterc�������������� ���C���s����|dkr|�����}t|�}|����D�]|}|r,dnd}|�d||f���t|d��J}|�d|�d���|dk r�|D�]$}|�d||f���|�|d���qlW�5�Q�R�X�q dS�) z�Set RUN_COUPLED=YES (true) or =NO (false) depending on the value of coupled_flag. If the logger is not specified, the section name is used as the logging domain.N�YES�NOz(Setting RUN_COUPLED=%s in status file %srp���zRUN_COUPLED=� z%s: write %s)rK����boolr����r3���rx���ry���)r���Z coupling_flagr%���Z morelinesr����Zstrflag�f�liner���r���r����set���s�������zCouplingStatus.setc�������������� ���C���s^���|dkr|�j�}|�j�|�jd|�j�d��}tj�|�s:t��S�t|d��}|� d�}W�5�Q�R�X�|S�)zvReads the first coupling status file (identified by {section}=) and returns the contents as an array of lines.Nr����r�����rti���) r%���rD���r����rE���r����r�����existsr>���rx���� readlines)r���r%���r����r�����linesr���r���r����read��s�����zCouplingStatus.readc�������������� ���C���sR��|dkr|�j�}|�j�|�jd|�j�d��}d}|�d|���zft|d��R}|D�]F}|����d�dkrrd}|�d ��qL|����d �dkrLd}|�d��qLW�5�Q�R�X�W�nv�t k r��}�z|j d t|�f�dd��W�5�d}~X�Y�n>�tk �r�}�z|j dt|�f�dd����W�5�d}~X�Y�nX�|dk�r2|�d��d}n|�rD|�d��n |�d��|S�)a���Checks the coupling status file. If the file does not exist or cannot be opened or read, then False is returned. Otherwise, the file is scanned for RUN_COUPLED=YES or RUN_COUPLED=NO (case insensitive). The last of those RUN_COUPLED lines is used: NO=return False, YES=return True. If the logger is not specified, the section name is used as the logging domainNr����r����z.%s: scan status file for RUN_COUPLED=YES or NOr����zRUN_COUPLED=YESr���Tz!Status file says: RUN_COUPLED=YESzRUN_COUPLED=NOFz Status file says: RUN_COUPLED=NOzError checking status file: %s)�exc_infoz2Unhandled exception while checking status file: %szOCould not scan status file for RUN_COUPLED=YES or NO. Assuming RUN_COUPLED=NO.zCRUN_COUPLED=YES: status file says coupled component init succeeded.z?RUN_COUPLED=NO: status file says coupled component init failed.) r%���rD���r����rE���r3���rx����upperr�����warning�EnvironmentErrorrS���r����� Exception)r���r%���r����Zsuccessr����r�����e�eer���r���r���r������s\������������ ��zCouplingStatus.get)N)N)NN)N)N)r���r���r���r ���r���r!���r%���rD���rE���r����r����r����r����r����r���r���r���r���r�������s��� r����)$r�����shutilrt���r1����produtil.datastorer�����produtil.fileop�produtil.cd�produtil.runZprodutil.rusage� hwrf.hwrftaskrU���� hwrf.numerics�hwrf.exceptionsZ hwrf.fcsttaskr���r���r���r���r���r���r���r ���r ���r���r����produtil.logr ����objectr���r"���r-���ZfcsttaskZWRFAtmosr:���r����r���r���r���r����<module>���s��� &��|