ó ‹÷Îgc@s dZddlZddlZddlZddlZddljZddlj Z ddl j Z ddlm Z m Z dddddd d d d d dddddddddgZejdƒZdaejjd„Zd„Zd„Zd„Zdd„Zd„Zdd„Zdd„Zed „Zddd!d"„Zddd#„Z ddd$„Z!dd%„Z"ddd&„Z#dd'„Z$d(„Z%d)„Z&dS(*s!A shell-like syntax for running serial, MPI and OpenMP programs. This module implements a shell-like syntax for launching MPI and non-MPI programs from Python. It recognizes three types of executables: mpi, "small serial" (safe for running on a batch node) and "big serial" (which should be run via aprun if applicable). There is no difference between "small serial" and "big serial" programs except on certain architectures (like Cray) where the job script runs on a heavily-loaded batch node and has compute nodes assigned for running other programs. @section progtype Program Types There are three types of programs: mpi, serial and "big non-MPI." A "big" executable is one that is either OpenMP, or is a serial program that cannot safely be run on heavily loaded batch nodes. On Cray architecture machines, the job script runs on a heavily-populated "batch" node, with some compute nodes assigned for "large" programs. In such environments, the "big" executables are run on compute nodes and the small ones on the batch node. * mpi('exename') = an executable "exename" that calls MPI_Init and MPI_Finalize exactly once each, in that order. * exe('exename') = a small non-MPI program safe to run on a batch node * bigexe('exename') = a big non-MPI program that must be run on a compute node it may or may not use other forms of parallelism You can also make reusable aliases to avoid having to call those functions over and over (more on that later). Examples: * Python: wrf=mpi('./wrf.exe') * Python: lsl=alias(exe('/bin/ls')['-l'].env(LANG='C',LS_COLORS='never')) Those can then be reused later on as if the code is pasted in, similar to a shell alias. @section serexs Serial Execution Syntax Select your serial programs by exe('name') for small serial programs and bigexe('name') for big serial programs. The return value of those functions can then be used with a shell-like syntax to specify redirection and piping. Example: * shell version: ls -l / | wc -l * Python version: run(exe('ls')['-l','/'] | exe('wc')['-l']) Redirection syntax similar to the shell (< > and << operators): @code run( ( exe('myprogram')['arg1','arg2','...'] < 'infile' ) > 'outfile') @endcode Note the extra set of parentheses: you cannot do "exe('prog') < infile > outfile" because of the order of precedence of Python operators Append also works: @code run(exe('myprogram')['arg1','arg2','...'] >> 'appendfile') @endcode You can also send strings as input with << @code run(exe('myprogram')['arg1','arg2','...'] << 'some input string') @endcode One difference from shells is that < and << always modify the beginning of the pipeline: * shell: cat < infile | wc -l * Python #1: ( exe('cat') < 'infile' ) | exe('wc')['-l'] * Python #2: exe('cat') | ( exe('wc')['-l'] < 'infile' ) Note that the last second one, equivalent to `cat|wc -ltr tStjjdƒatjjƒ}|r:|antS(s\!Called by functions inside produtil.run to automatically detect the available MPI implementation. Detects the available MPI implementation - but that isn't as simple as it sounds. The detection logic requires calling produtil.run. Hence, produtil.run must be usable during the call to detect_mpi(). The way this is handled is in three steps: 1. Set the MPI implementation to "no MPI." This will allow all serial (non-OpenMP, non-MPI) calls to succeed. 2. Run the MPI implementation detection functions. These will be able to use produtil.run to execute serial programs to detect the MPI implementation. 3. If an MPI implementation is detected, set the MPI implementation to that one. @note MPI implementation detection only happens during the first call to detect_mpi(). Any later calls will return the cached result from _detected_mpi. @returns a subclass of produtil.mpi_impl.mpi_impl_base.ImplementationBase for generation of produtil.prog.Runner objects from openmp or mpi program specifications. @see _detected_mpiN(t _detected_mpiRRRR(tdetected((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/run.pyRßs cCs\t|tjƒrtj|ƒSt|tjƒr?|jƒ|Stdt|ƒfƒ‚dS(s)!Attempts to generate an unmodifiable "copy on write" version of the argument. The returned copy will generate a modifiable duplicate of itself if you attempt to change it. @returns a produtil.prog.ImmutableRunner @param arg a produtil.prog.Runner or produtil.prog.ImmutableRunners†Arguments to alias() must be Runner objects (such as from exe()) or MPIRanksBase objects (such as from mpi() or mpiserial()). Got: %sN( t isinstanceR tRunnertImmutableRunnerR t MPIRanksBasetmake_runners_immutableRtrepr(targ((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/run.pyR s   cKstjt|ƒg|S(s!Returns a prog.ImmutableRunner object that represents a small serial program that can be safely run on a busy batch node. @param name the executable name or path @param kwargs passed to produtil.prog.Runner.__init__ @returns a new produtil.prog.ImmutableRunner(R Rtstr(tnameR((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/run.pyRscKs.|dkrtƒ}n|jt|ƒ|S(s÷!Returns a prog.ImmutableRunner object that represents a large serial program that must be run on a compute node. @note This function does NOT search $PATH on Cray. That ensures the $PATH will be expanded on the compute node instead. Use produtil.fileop.find_exe() if you want to explicitly search the PATH before execution. @param name the executable name or path @param kwargs passed to produtil.prog.Runner.__init__ @returns a new produtil.prog.ImmutableRunnerN(RRt make_bigexeR#(R$tmpiimplR((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/run.pyR#s  cKs t||S(s@!Alias for exe() for backward compatibility. Use exe() instead.(R(R$R((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/run.pyR1scKs(|dkrtƒ}n|j||S(sÍ!Converts an MPI program specification into a runnable shell program suitable for run(), runstr() or checkrun(). Options for kwargs: * allranks=True --- to run on all available MPI ranks. This cannot be used if a specific number of ranks (other than 1) was requested in the arg. * logger=L --- a logging.Logger for log messages * Other platform-specific arguments. See produtil.mpi_impl for details. @param arg the mpiprog.MPIRanksBase describing the MPI program to run. This is the output of the mpi() or mpiserial() function. @param kwargs additional arguments to control output. @returns a prog.Runner object for the specified mpiprog.MPIRanksBase object.N(RRt mpirunner(R"R&R((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/run.pyR5s  cKs=|dkrtƒ}nt|tjƒr3|}n@t|tjƒrZ|j||}ntdt |ƒfƒ‚d}d|kr’|d}n|dk rÑ|j dt |ƒfƒ|rÑ|j dƒqÑn|j dt ƒ}t |ƒ}tj|d|d|d|ƒ}|dk r9|jdt |ƒfƒn|S( s!This internal implementation function generates a prog.PopenCommand object for the specified input, which may be a prog.Runner or mpiprog.MPIRanksBase. @param arg the produtil.prog.Runner to convert. This is the output of exe(), bigexe() or mpirun() @param capture if True, capture the stdout into a string @param kwargs additional keyword arguments, same as for mpirun()syCan only run a Runner object (such as from exe()) or an MPIRanksBase object (such as from mpi() or mpiserial()). Got: %stloggers Starting: %ss - and will capture output.tbinarytcapturesPipeline is %sN(RRRR RR RR'RR!tinfotgettFalsetbooltpipelinetPipelinetdebug(R"R*R&RtrunnerR(tbtpl((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/run.pyt make_pipelineIs.        cKs t|||}|jƒ|S(sd!Not implemented: background execution Runs the specified process in the background. Specify capture=True to capture the command's output. Returns a produtil.prog.PopenCommand. Call poll() to determine process completion, and use the stdout_data property to get the output after completion, if capture=True was specified. @bug produtil.run.runbg() is not implemented @warning this is not implemented @param arg the produtil.prog.Runner to execute (output of exe(), bigexe() or mpirun() @param capture if True, capture output @param kwargs same as for mpirun()(R5t background(R"R*Rtp((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/run.pyR is ièc Cstƒ}t|tjjƒr.|j|ƒnx|D]}|j|ƒq5W|dk rq|jdt|ƒƒnx |r€tƒ}x|D]•}|j ƒ}|dk rÝ|dk r|jdt|ƒt|ƒfƒqqŠ|dk rŠ|dkrŠ|jdt|ƒfƒ|j|ƒqŠqŠW|}|s3Pn|dkrl|dk rl|jdt |dƒfƒnt j |dƒqtW|r‹t StS(sL!Not implemented: background process monitoring Waits for one or more backgrounded processes to complete. Logs to the specified logger while doing so. If a timeout is specified, returns False after the given time if some processes have not returned. The usleep argument is the number of microseconds to sleep between checks (can be a fraction). The first argument, procs specifies the processes to check. It must be a produtil.prog.Pipeline (return value from runbg) or an iterable (list or tuple) of such. @bug produtil.run.waitprocs() is untested @warning This is not tested and probably does not work. @param procs the processes to watch @param logger the logging.Logger for log messages @param timeout how long to wait before giving up @param usleep sleep time between checkss Wait for: %ss%s returned %sg€ÍSAs%s is still runnings... sleep %f ...g€„.AN(tsetRRR t PopenCommandtaddRR+R!tpolltfloatttimetsleepR-tTrue( tprocsR(ttimeouttusleepR7tpptp2tproctret((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/run.pyR ~s0         )!cCs(|dkrtƒ}n|jd|ƒS(s%!Runs the "sync" command as an exe().R(N(RRR (R(R&((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/run.pyR ®s  cKs^t|td|ƒ}|jd|ƒ|jƒ}|dk rZ|jdt|ƒfƒn|S(s!Executes the specified program and attempts to return its exit status. In the case of a pipeline, the highest exit status seen is returned. For MPI programs, exit statuses are unreliable and generally implementation-dependent, but it is usually safe to assume that a program that runs MPI_Finalize() and exits normally will return 0, and anything that runs MPI_Abort(MPI_COMM_WORLD) will return non-zero. Programs that exit due to a signal will return statuses >255 and can be interpreted with WTERMSIG, WIFSIGNALLED, etc. @param arg the produtil.prog.Runner to execute (output of exe(), bigexe() or mpirun() @param logger a logging.Logger to log messages @param sleeptime time to sleep between checks of child process @param kwargs ignoredR(t sleeptimes - exit status %dN(R5R-t communicateR;RR+tint(R"R(RGRR7tresult((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/run.pyR´s   cKs‹t|d|ƒ}|dk r\d|kr\||dkr‡tdt|ƒf|ƒ‚q‡n+|dks‡tdt|ƒf|ƒ‚n|S(sr!This is a simple wrapper round run that raises ExitStatusException if the program exit status is non-zero. @param arg the produtil.prog.Runner to execute (output of exe(), bigexe() or mpirun() @param logger a logging.Logger to log messages @param kwargs The optional run=[] argument can provide a different list of acceptable exit statuses.R(RFs%s: unexpected exit statusis%s: non-zero exit statusN(RRRR!(R"R(Rtr((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/run.pyRÊs " cCs(|dkrtƒ}n|j||ƒS(s×!Sets the number of OpenMP threads for the specified program. @warning Generally, when using MPI with OpenMP, the batch system must be configured correctly to handle this or unexpected errors will result. @param arg The "arg" argument must be from mpiserial, mpi, exe or bigexe. @param threads The optional "threads" argument is an integer number of threads. If it is not specified, the maximum possible number of threads will be used. Note that using threads=None with mpirun(...,allranks=True) will generally not work unless the batch system has already configured the environment correctly for an MPI+OpenMP task with default maximum threads and ranks. @returns see run()N(RRR(R"tthreadsR&((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/run.pyRÛs  cKs¦t|td|ƒ}|jƒ}|jƒ}|dk rwd|krw||dkr¢tdt|ƒf|ƒ‚q¢n+|dks¢tdt|ƒf|ƒ‚n|S(sç!Executes the specified program or pipeline, capturing its stdout and returning that as a string. If the exit status is non-zero, then NonZeroExit is thrown. Example: @code runstr(exe('false'),ret=(1)) @endcode succeeds if "false" returns 1, and raises ExitStatusError otherwise. @param arg The "arg" argument must be from mpiserial, mpi, exe or bigexe. @param logger a logging.Logger for logging messages @param kwargs You can specify an optional list or tuple "ret" that contains an alternative list of valid return codes. All return codes are zero or positive: negative values represent signal-terminated programs (ie.: SIGTERM produces -15, SIGKILL produces -9, etc.) R(RFs%s: unexpected exit statusis%s: non-zero exit statusN(R5R?t to_stringR;RRR!(R"R(RR7tsRK((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/run.pyRðs  " cKstj||S(s²!Returns an MPIRank object that represents the specified MPI executable. @param arg the MPI program to run @param kwargs logger=L for a logging.Logger to log messages(R tMPIRank(R"R((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/run.pyRscKstj|jƒ|S(s!Generates an mpiprog.MPISerial object that represents an MPI rank that executes a serial (non-MPI) program. The given value MUST be from bigexe() or exe(), NOT from mpi(). @param arg the MPI program to run @param kwargs logger=L for a logging.Logger to log messages(R t MPISerialt remove_prerun(R"R((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/run.pyRs('t__doc__R=tloggingtprodutil.mpi_implRtprodutil.sigsafetyt produtil.progR tprodutil.mpiprogR tprodutil.pipelineR/RRt__all__t getLoggert module_loggerRRRtNO_NAMERRRRRRRR5R-R R R RRRRRR(((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/run.pyts<   & .      0