ó ‹÷Îgc@spdZgZddlZddlZddlZddlZddlmZmZdefd„ƒYZ de fd„ƒYZ defd „ƒYZ d efd „ƒYZ d efd „ƒYZ defd„ƒYZde fd„ƒYZeƒZdefd„ƒYZdefd„ƒYZdefd„ƒYZdefd„ƒYZdefd„ƒYZd„ZdS(sp!Object structure for describing MPI programs. Do not load this module directly. It is meant to be loaded only by the produtil.run module. This module handles execution of MPI programs, and execution of groups of non-MPI programs through an MPI interface (which requires all sorts of tricks). This module is also the interface to the various produtil.mpi_impl.* modules that generate the shell command to run MPI programs. This module is built on top of the produtil.prog module and uses it to run the MPI-launching program for your local cluster (mpiexec, mpirun, poe, etc.) In addition, this module contains code to simplify adding new MPI implementations to the produtil.mpi_impl subpackage. High-level code, such as the HWRF scripts, use the produtil.run module to generate object trees of MPIRanksBase objects. The produtil.mpi_impl subpackages then implement an mpirunner function that turns those into a produtil.prog.Runner to be directly executed. The MPIRanksBase object, and its subclasses, implement a few utilites to automate that for you: * to_arglist --- converts the MPI ranks to an mpi launcher command as a produtil.prog.Runner, or to an array of strings for a command file. * nranks --- calculates the number of requested MPI ranks * expand_iter --- iterates over groups of identical MPI ranks * check_serial --- tells whether this program is running MPI programs, or running serial programs as if they were MPI (or both, which most MPI implementations don't support) For MPI implementations that require a command file, see the produtil.mpi_impl.mpi_impl_base CMDFGen class to have the produtil.prog module automatically write the command file before executing the program. The produtil.mpi_impl.mpirun_lsf shows an example of how to use it. See the produtil.run module for full documentation.iÿÿÿÿN(tProgSyntaxErrort shbackslashtMPIProgSyntaxErrorcBseZdZRS(s:!Base class of syntax errors in MPI program specifications(t__name__t __module__t__doc__(((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR8stComplexProgInputcBseZdZRS(sc!Raised when something that cannot be expressed as a pure MPI rank is given as a pure MPI rank.(RRR(((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR:st NotMPIProgcBseZdZRS(sJ!Raised when an MPI program was expected but something else was given.(RRR(((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR=st NotSerialProgcBseZdZRS(sM!Raised when a serial program was expected, but something else was given.(RRR(((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR@stInputsNotStringscBseZdZRS(s…!Raised when the validation scripts were expecting string arguments or string executable names, but something else was found.(RRR(((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR Cst MixedValuescBseZdZRS(s(!Special type for MIXED_VALUES constant.(RRR(((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR HstMixedValuesErrorcBseZdZRS(sIndicates that an iterator over information specific to an MPI rank cannot iterate over a group of ranks collectively because the information in each rank differs. This is used in MPMD mode for local options when the various MPI ranks have different local options.(RRR(((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR Kst MPIRanksBasec BsÊeZdZeed*gggggied„ Zd„Zd„Zd„Zd„Z d„Z d„Z d„Z d „Z d „Zd „Zee eed ƒZed „Zdd„Zd„Zd„Zd„Zd„Zd„Zd„Zd„ZeeeedƒZd„Zd„Zd„Zed„Z d„Z!ed„ƒZ"d„Z#d„Z$ee!e#e$d ƒZ%d!„Z&d"„Z'd#„Z(d$„Z)d%„Z*d&„Z+ed'„Z,d(„Z-d)„Z.RS(+sÚ!This is the abstract superclass of all classes that represent one or more MPI ranks, including MPI ranks that are actually serial programs. Subclasses of MPIRanksBase allow an MPI program to be represented as a tree of MPIRanksBase objects, in such a way that they can be easily converted to a produtil.prog.Runner object for execution. The actual conversion to a Runner is done in the produtil.mpi_impl package (see produtil/mpi_impl/__init__.py)c cs¹t| ƒ} |jƒ| dtloggingt getLoggertinfotrepr(Rttm((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyt setturbomode s   cCs d|_dS(s4!Removes the request for turbo mode to be on or off.N(RR>(RRD((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyt delturbomodess/Turbo mode setting for this group of MPI ranks.cCs ||_|S(N(t turbomode(Rtflag((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pytturbos icCs ||_|S(N(tranks_per_node(RR-((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pytrpns cGstdƒ‚dS(s7!Sets environment variables in the individual MPI rankss Subclass did not implement env()N(tNotImplementedError(Rtkwargs((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pytenv scCsdS(sš!Returns a copy of this object where all child produtil.prog.Runner objects have been replaced with produtil.prog.ImmutableRunner objects.N((R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pytmake_runners_immutable$tcCsdS(s«!Returns a logger.Logger object for this MPIRanksBase or one from its child MPIRanksBase objects (if it has any). If no logger is found, None is returned.N(R(R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyt get_logger(scCs ttfS(s0!Returns a tuple (s,p) where s=True if there are serial ranks in this part of the MPI program, and p=True if there are parallel ranks. Note that it is possible that both could be True, which is an error. It is also possible that neither are True if there are zero ranks.(R(R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyt check_serial-scCs|jS(sa!Returns the number of MPI ranks per node requsted by this MPI rank, or 0 if unspecified.(t_ranks_per_node(R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pytgetranks_per_node5scCs8t|ƒ}|dks+td|ƒ‚n||_dS(sI!Sets the number of MPI ranks per node requsted by this MPI rank.is!Ranks per node must be >=0 not %dN(Rt ValueErrorRS(RRK((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pytsetranks_per_node:s  cCs d|_dS(s/!Unsets the requested number of ranks per node.iN(RS(R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pytdelranks_per_nodeBssEThe number of MPI ranks per node or 0 if no specific request is made.cCsdS(sE!Returns the number of ranks in this part of the MPI program.i((R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRLscCsdS(sK!Iterates over all MPIRank objects in this part of the MPI program.N((R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pytranksPscCsdS(sO!Returns the number of groups of repeated MPI ranks in the MPI program.i((R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pytngroupsTscCsdS(s;!Iterates over all groups of repeating MPI ranks in the MPI program returning tuples (r,c) containing a rank r and the count (number) of that rank c. @param threads If True, then a three-element tuple is iterated, (r,c,t) where the third element is the number of threads.N((Rtthreads((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pytgroupsXscCs~d}xq|jdtƒD]]\}}}|dk rI|dkrI|}q|dk r|dk r||kr|}qqW|S(s!Returns the number of threads requested by this MPI rank, or by each MPI rank in this group of MPI ranks. If different ranks have different numbers of threads, returns the maximum requested. Returns None if no threads are requested.RZN(RR[R(RRtrtctt((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyt getthreadsas" $ cCs/|jƒ}|dkrdStdt|ƒƒS(sw!The number of threads requested, or 1 if no threads are requested. This is a simple wrapper around getthreadsiN(R_RtmaxR(RRZ((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pytnonzero_threadsms  cCs9x2|jƒD]$\}}||k r ||_q q W|S(s^!Sets the number of threads requested by each MPI rank within this group of MPI ranks.(R[RZ(RtnthreadsR\R]((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyt setthreadsus cCs'x |jƒD]\}}|`q WdS(s!!Removes the request for threads.N(R[RZ(RR\R]((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyt delthreads|ssThe number of threads per rank.cCstS(s–!Returns a new set of MPI ranks that consist of this group of ranks repeated "factor" times. @param factor how many times to duplicate(tNotImplemented(Rtfactor((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyt__mul__‚scCstS(s•!Returns a new set of MPI ranks that consist of this group of ranks repeated "factor" times. @param other how many times to duplicate(Re(RR:((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyt__rmul__‡scCstS(s!Returns a new set of MPI ranks that consist of this set of ranks with the "other" set appended. @param other the data to append(Re(RR:((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyt__add__ŒscCstS(s‘!Returns a new set of MPI ranks that consist of the "other" set of ranks with this set appended. @param other the data to prepend(Re(RR:((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyt__radd__‘scCstS(sê!Determines if this set of MPI ranks can be represented by a single serial executable with a single set of arguments run without MPI. Returns false by default: this function can only return true for MPISerial.(R(R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyt isplainexe–scCstdƒ‚dS(s!Returns a POSIX sh command that will execute the serial program, if possible, or raise a subclass of NotValidPosixSh otherwise. Works only on single MPI ranks that are actually MPI wrappers around a serial program (ie.: from mpiserial).sRThis is an MPI program, so it cannot be represented as a non-MPI POSIX sh command.N(R(R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRœsccsË|r\|r7xJ|jƒD]}|d|jfVqWqÇx|jƒD]}|dfVqDWnk|r™xb|jdtƒD]\}}}|||fVquWn.x+|jdtƒD]\}}||fVq¬WdS(sª!This is a wrapper around ranks() and groups() which will call self.groups() if expand=False. If expand=True, this will call ranks() returning a tuple (rank,1) for each rank. @param expand If True, expand groups of identical ranks into one rank of each member @param threads If True, then a third element will be in each tuple: the number of requested threads per MPI rank.iRZN(RXRZR[RR(RR RZR,R-((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR¢s"cCstdƒ‚dS(sG!Returns a string representation of this object intended for debugging.s&This class did not implement __repr__.N(RL(R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyt__repr__·sc Csët|jtƒƒ}t|jtƒƒ}tt}}x­|ræ|rætt}}yt|ƒ\}}t}Wntk r‚nXyt|ƒ\}} t}Wntk r²nXt|krÃtS||krÓtS|| kr:tSq:WtS(N(titerRRRtnextt StopIterationt have_rank( RR:tsitertoitert have_srankt have_oranktsranktscounttoranktocount((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyt__eq__ºs,         N(/RRRRRR0R1R3R5R7RR8R=R?RERFtpropertyRGRRIRKRNRORQRRRTRVRWRJRRXRYR[R_RaRcRdRZRgRhRiRjRkRRRlRy(((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR [sZ  R                                    t MPIRanksSPMDcBs eZdZd„Zd„Zd„Zd„Zd„ZeeeedƒZ d„Z d„Z d „Z ee e e d ƒZ d „Zd „Zd „Zd„Zd„Zd„Zed„Zd„Zd„Zd„Zd„Zd„Zd„Zd„Zd„ZRS(s9!Represents one MPI program duplicated across many ranks.cCsXt|tƒstdƒ‚n||_t|ƒ|_t|jƒ|_|j|_ dS(sq!MPIRanksSPMD constructor @param mpirank the program to run @param count how many times to run its)Input to MPIRanksSPMD must be an MPIRank.N( RtMPIRankRt_mpirankRt_countRRRGR>(RtmpirankR-((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyt__init__Ôs  cKs|jj||_|S(N(R}RN(RRM((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRNßscCs |jjS(sa!Returns the number of MPI ranks per node requsted by this MPI rank, or 0 if unspecified.(R}RJ(R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRTãscCs||j_dS(sI!Sets the number of MPI ranks per node requsted by this MPI rank.N(R}RJ(RRK((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRVèscCs |j`dS(s/!Unsets the requested number of ranks per node.N(R}RJ(R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRWíssEThe number of MPI ranks per node or 0 if no specific request is made.cCs%t|ƒ}||j_||_|S(N(RR}RGR>(RRDR^((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyREôs   cCs|jS(N(R>(R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR?ùscCs|j`d|_dS(N(R}RGRR>(R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRFûs  s/Turbo mode setting for this group of MPI ranks.cCs0g|D] }|^q|_|jj|ƒ|S(N(RR}R3(RR2R*((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR3scCs$|jj|ƒ|jj|ƒ|S(N(RR4R}R5(RR2((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR5scCs$|jj|ƒ|jj|ƒ|S(N(RR6R}R7(RR<((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR7 scCst|jjƒ|jƒS(sO!Returns a new MPIRanksSPMD with an immutable version of self._mpirank.(R{R}ROR~(R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRO scCs dt|jƒt|jƒfS(sO!Returns "X*N" where X is the MPI program and N is the number of ranks.s%s*%d(RCR}RR~(R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRlscCs|jdkrdSdSdS(s>!Returns 1 or 0: 1 if there are ranks and 0 if there are none.iiN(R~(R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRY!sccs8|r#|j|j|jjfVn|j|jfVdS(sV!Yields a tuple (X,N) where X is the mpi program and N is the number of ranks.N(R}R~RZ(RRZ((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR['scCsIt|jjƒ|jƒ}|j|_t|jƒ|_|j|_|S(s!Returns a deep copy of self.(R{R}tcopyR~R>RRRZ(RR]((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR.s   ccs8|jdkr4x"t|jƒD]}|jVqWndS(s%!Iterates over MPI ranks within self.iN(R~trangeR}(RR;((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRX5scCs|jdkr|jSdSdS(s3!Returns the number of ranks this program requests.iN(R~(R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR:scCs0t|tƒstSt|jjƒ|j|ƒS(s7!Multiply the number of requested ranks by some factor.(RRReR{R}RR~(RRf((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRg@scCs0t|tƒstSt|jjƒ|j|ƒS(s7!Multiply the number of requested ranks by some factor.(RRReR{R}RR~(RRf((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRhEscCst}t|dƒs=tdt|ƒjt|ƒfƒ‚n|jƒ}|j|ƒr»|j|jkr»|j |j kr»t }x6|j ƒD]%\}}||j kst}PqqWn|rát |j jƒ|jƒ|ƒSt|jƒ|jƒgƒSdS(s!Add some new ranks to self. If they are not identical to the MPI program presently requested, this returns a new MPIRanksMPMD.Rs%s %s: has no nranksN(Rthasattrt TypeErrorttypeRRCRR=R>RJRR[R}R{Rt MPIRanksMPMD(RR:RRxRR-((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRiJs "   cCs*|jdkr|jjƒSttfSdS(s6!Checks to see if this program contains serial (non-MPI) or MPI components. @returns a tuple (serial,parallel) where serial is True if there are serial components, and parallel is True if there are parallel components. If there are no components, returns (False,False)iN(R~R}RRR(R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRR_s cCs |jjƒS(s!!Returns my MPI program's logger.(R}RQ(R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRQjs(RRRR€RNRTRVRWRzRJRER?RFRGR3R5R7RORlRYRR[RRXRRgRhRiRRRQ(((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR{Òs8                         R†cBsLeZdZd„Zd„Zd„Zd„Zd„ZeeeedƒZ d„Z d„Z d „Z ee e e d ƒZ d „Zd „Zd „ZeeeedƒZd„Zd„Zd„Zd„Zd„Zd„Zd„Zd„Zd„Zed„Zd„Zd„Zd„Zd„Z d„Z!d„Z"d„Z#RS( sZ!Represents a group of MPI programs, each of which have some number of ranks assigned.cCs˜t|ƒ|_d|_d|_d|_|rvg|djD] }|^q>|_|dj|_|dj|_ ntƒ|_d|_d|_ dS(sR!MPIRanksMPMD constructor @param args an array of MPIRanksBase to execute.iN( Rt_elRt_ngcachet_nrcachet_threadsRR>RJRS(RRR*((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR€ss   #  cKs,g|jD]}|j|^q |_|S(N(R‡RN(RRMte((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRNƒs(cCs6t|ƒ}x|jD]}||_qW||_|S(N(RR‡RGR>(RRDR^R\((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRE†s    cCs?|jdj}x(|jdD]}||jkrtSqW|S(Nii(R‡RGt MIXED_VALUES(Rtresulttel((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR?Œs cCs'x|jD] }|`q Wd|_dS(N(R‡RGRR>(RR\((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRF’s  s/Turbo mode setting for this group of MPI ranks.cCs!x|jD]}||_q W|S(N(R‡RZ(RRZR\((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRcšs cCs?|jdj}x(|jdD]}||jkrtSqW|S(Nii(R‡RZRŒ(RRRŽ((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR_žs cCsx|jD] }|`q WdS(N(R‡RZ(RR\((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRd¤ssThe number of threads per rank.cCs6t|ƒ}x|jD]}||_qW||_|S(N(RR‡RJRS(RRDR^R\((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRV©s    cCs?|jdj}x(|jdD]}|j|krtSqW|S(Nii(R‡RSRJRŒ(RRR‹((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRT¯s cCs'x|jD] }|`q Wd|_dS(Ni(R‡RJRS(RR\((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRWµs  s@Ranks per node for this group of MPI ranks, or 0 if unspecified.cCsAg|D] }|^q|_x|jD]}|j|ƒq&W|S(N(RR‡R3(RR2R*R\((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR3¼scCs5|jj|ƒx|jD]}|j|ƒqW|S(N(RR4R‡R5(RR2R\((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR5ÁscCs5|jj|ƒx|jD]}|j|ƒqW|S(N(RR6R‡R7(RR<R\((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR7ÆscCs6x/|jdD] }|jdj|ƒstSqWtS(sS!Do the MPI ranks within this group contain mixed values for local options?ii(R‡R=RR(RR‹((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR8Ësccs?|jƒrtƒ‚nx |jdjƒD] }|Vq,WdS(Ni(R8R R‡R(RR;((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRÒs  cCs&tg|jD]}|jƒ^q ƒS(sª!Tells each containing element to make its produtil.prog.Runners into produtil.prog.ImmutableRunners so that changes to them will not change the original.(R†R‡RO(RRŽ((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRO×scCsOg}x9|jD].}|jƒdkr|jt|ƒƒqqWdj|ƒS(s/!Returns a pythonic description of this object.is + (R‡RR6RCtjoin(RtreprsRŽ((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRlÜs cCsL|jdkrEd}x!|jD]}||jƒ7}qW||_n|jS(sN!How many groups of identical repeated ranks are in this MPMD program?iN(RˆRR‡RY(Rtngtg((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRYãs  cCsL|jdkrEd}x!|jD]}||jƒ7}qW||_n|jS(s*!How many ranks does this program request?iN(R‰RR‡R(RtnrR’((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRìs  ccs‘|rQx„|jD]:}x1|jdtƒD]\}}}|||fVq)WqWn<x9|jD].}x%|jƒD]\}}||fVqnWq[WdS(sH!Iterates over tuples (rank,count) of groups of identical ranks.RZN(R‡R[R(RRZR[R,R-((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR[ôs"ccs4x-|jD]"}x|jƒD] }|VqWq WdS(s\!Iterates over groups of repeated ranks returning the number of ranks each requests.N(R‡RX(RRXR,((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRXÿscCsYt|tƒr#t|j|jƒSt|tƒsAt|tƒrUt|j|gƒStS(s]!Adds more ranks to this program. @param other an MPIRanksMPMD or MPIRanksSPMD to add(RR†R‡R|R Re(RR:((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRis cCsYt|tƒr#t|j|jƒSt|tƒsAt|tƒrUt|g|jƒStS(se!Prepends more ranks to this program. @param other an MPIRanksMPMD or MPIRanksSPMD to prepend(RR†R‡R|R Re(RR:((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRj s cCs$t|tƒr t|j|ƒStS(sm!Duplicates this MPMD program "factor" times. @param factor how many times to duplicate this program.(RRR†R‡Re(RRf((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRgscCs$t|tƒr t||jƒSdS(sm!Duplicates this MPMD program "factor" times. @param factor how many times to duplicate this program.N(RRR†R‡(RRf((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRhscCsTt}t}x;|jD]0}|jƒ\}}|p7|}|pC|}qW||fS(sú!Checks to see if this program contains serial (non-MPI) or MPI components. @returns a tuple (serial,parallel) where serial is True if there are serial components, and parallel is True if there are parallel components.(RR‡RR(RtserialtparallelRŽtstp((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRR s cCs4x-|jD]"}|jƒ}|dk r |Sq WdS(s:!Returns a logging.Logger for the first rank that has one.N(R‡RQR(RRŽtlogger((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRQ-s   ($RRRR€RNRER?RFRzRGRcR_RdRZRVRTRWRJR3R5R7R8RRORlRYRRR[RXRiRjRgRhRRRQ(((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR†psD                             R|cBséeZdZdd„Zd„Zd„Zd„Zd„Ze eeedƒZ d„Z d„Z d „Z d „Zdd „Zd „Zd „Zd„Zd„Zd„Zed„Zd„Zd„Zd„Zd„Zd„ZRS(s!Represents a single MPI rank.cCsr||_d|_tƒ|_d|_d|_tƒ|_t |t ƒr¨|jdkri|j|_nt|j ƒ|_ t|jƒ|_|j |_|j |_n¼t |tjjƒrý|jƒrîg|jƒD] }|^qÖ|_ qdtdƒ‚ngt |tƒr|g|_ nIt |tƒs9t |tƒrXg|D] }|^q@|_ n tdƒ‚|jƒdS(s!MPIRank constructor. @param arg What program to run. Can be a produtil.prog.Runner, or some way of creating one, such as a program name or list of program+arguments. @param logger a logging.Logger for log messages or None to have no logger.is„Tried to convert a Runner to an MPIRank directly, when the Runner had more than an executable and arguments. Use mpiserial instead.sInput to MPIRank.__init__ must be a string, a list of strings, or a Runner that contains only the executable and its arguments.N(t_loggerRRŠRRR>RSRt_envRR|t_argsRGRJtprodutiltprogtRunnerRkRRRttupleRtvalidate(RR/R˜R*((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR€9s2        %  cKs|jj||S(N(Rštupdate(RRM((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRN_scCs|jS(sr!Returns the number of threads requested by this MPI rank, or by each MPI rank in this group of MPI ranks.(RŠ(R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR_bscCs+|dkrd|_nt|ƒ|_|S(s^!Sets the number of threads requested by each MPI rank within this group of MPI ranks.N(RRŠR(RRb((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRcfs  cCs d|_dS(s!!Removes the request for threads.iN(RŠ(R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRdnssThe number of threads per rank.cCs/djg|jD]}tjj|ƒ^qƒS(sH!Return a POSIX sh representation of this MPI rank, if possible.t (RR›RœRR(RR*((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRrscCsB|jƒ}t|tƒr.|jj|ƒn|jj|ƒ|S(s+!Adds arguments to this MPI rank's program.(RRRR›R6R4(RRR]((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyt __getitem__vs  cCsAtjƒ}|jdt|jdƒƒt|jƒdkr€|jddjg|jdD]}t|ƒ^q\ƒdƒn|jƒr¬|jdt|jƒfƒn|j rÕ|jdt|j ƒfƒn|j rþ|jd t|j ƒfƒn|j r'|jd t|j ƒfƒn|j ƒ}|j ƒ|S( sH!Returns a Pythonic representation of this object for debugging.smpi(%s)iit[t,t]s.setlocalopts(%s)s .threads(%s)s.turbomode(%s)s.rpn(%s)(tiotStringIOtwriteRCR›R9RR1R2RZRGRJtgetvaluetclose(RtsioR*tret((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRl~s A          cCstS(s3!Returns a logging.Logger for this object, or None.(R˜(R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRQscCs‰x2|jƒD]$}t|tƒs tdƒ‚q q W|dk r…t|ƒdkr…x/|D]$}t|tƒsZtdƒ‚qZqZWndS(sÏ!Checks to see if this MPIRank is valid, or has errors. @param more Arguments to the executable to validate. @returns None if there are no errors, or raises a descriptive exception.s)Executable and arguments must be strings.iN(RRRR RR9(RtmoreR*((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR “s ccsc|jrFdVx5|jjƒD]!\}}d|t|ƒfVqWnx|jD] }|VqPWdS(s(!Iterates over the executable arguments.s/bin/envs%s=%sN(RštitemsRR›(RtktvR/((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR¡s  cCs:t|ƒ}|j|_t|jƒ|_|j|_|S(sk!Return a copy of self. This is a deep copy except for the logger which whose reference is copied.(R|R>RRRŠ(RR]((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR¨s    cCsdS(s$!Returns 1: the number of MPI ranks.i((R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR°scCsdS(s4!Returns 1: the number of groups of identical ranks.i((R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRY³sccs |VdS(s!!Yields self once: all MPI ranks.N((R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRX¶sccs)|r|d|jfVn |dfVdS(sQ!Yields (self,1): all groups of identical ranks and the number per group.iN(RŠ(RRZ((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR[¹scCsFt|tƒstS||kr2t|jƒdƒSt||gƒSdS(s}!Creates an MPIRanksSPMD or MPIRanksMPMD with this MPIRank and the other ranks. @param other The other ranks.iN(RR|ReR{RR†(RR:((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRiÀs  cCs t|tƒrt||ƒStS(sz!Creates an MPIRanksSPMD with this MPIRank duplicated factor times. @param factor the number of times to duplicate(RRR{Re(RRf((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRgÊs cCs t|tƒrt||ƒStS(sz!Creates an MPIRanksSPMD with this MPIRank duplicated factor times. @param factor the number of times to duplicate(RRR{Re(RRf((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRhÐs cCsRt|tƒoQ|j|jkoQ|j|ƒoQ|j|jkoQ|j|jkS(s;!Returns True if this MPIRank is equal to the other object.(RR|R›R=R>RS(RR:((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRyÖs cCs ttfS(s7!Returns (False,True): this is a pure parallel program.(RR(R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRRÝsN(RRRRR€RNR_RcRdRzRZRR£RlRQR RRRRYRXRR[RiRgRhRyRR(((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR|7s. &                  t MPISerialcBsŒeZdZdd„Zd„Zd„Zd„Zd„Zd„Z d„Z e d„ƒZ d „Z d „Zd „Zd „Zd „ZRS(sÅ!Represents a single rank of an MPI program that is actually running a serial program. This is supported directly by some MPI implementations while others require kludges to work properly.cCs=||_||_tƒ|_d|_d|_d|_dS(s!MPISerial constructor.iN(t_runnerR™RRRR>RŠRS(RtrunnerR˜((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR€çs      cCs?t|jtjjƒs7ttjj|jƒ|jƒS|SdS(sF!Creates a version of self with a produtil.prog.ImmutableRunner child.N(RR³RœRtImmutableRunnerR²R™(R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyROïscCs[t|j|jƒ}|j|_|j|_t|jƒ|_|j|_|j|_|S(s!Duplicates self.(R²R³R™R>RRRŠRS(RR]((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRõs    cCsdt|jƒfS(s@!Returns a pythonic string representation of self for debugging.s mpiserial(%s)(RCR³(R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRlþsccs#x|jjƒD] }|VqWdS(s=!Iterates over command arguments of the child serial program.N(R³R(RR/((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRscCsI||kr)t|jƒ|jƒtƒSt|jƒ|jƒgƒSdS(s!Add some new ranks to self. If they are not identical to the MPI program presently requested, this returns a new MPIRanksMPMD.N(R{RRRxR†(RR:((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRis cCs#|jdk r|jS|jjƒS(s7!Returns my logging.Logger that I use for log messages.N(R™RR³RQ(R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRQ scCs|jS(N(R³(R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR´scCsdS(s!Does nothing.N((R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR RPcCsgt|tƒof|j|jkof|j|jkof|j|jkof|j|jkof|j|jkS(s!Returns True if other is an MPISerial with the same Runner, False otherwise. @param other the other object to compare against.(RR²R³RSR>RŠR(RR:((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRys !cCs ttfS(sk!Returns (True,False) because this is a serial program (True,) and not a parallel program (,False).(RR(R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRR scCs |jjƒS(s•!Returns True if the child serial program is a plain executable, False otherwise. See produtil.prog.Runner.isplainexe() for details.(R³Rk(R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyRk$scCs |jjƒS(s8!Returns a POSIX sh version of the child serial program.(R³R(R((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR)sN(RRRRR€RORRlRRiRQRzR´R RyRRRkR(((s=/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/mpiprog.pyR²ãs         cCsñtƒ}tƒ}x€|jtƒD]o\}}t|ƒsS|j||gƒq"n|dd|kr~|ddc|7+s*     ÿxžÇ¬L