ó ‹÷Îgc@s™dZddlZddlZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl Zddl m Z mZmZmZdefd„ƒYZdefd„ƒYZdefd „ƒYZd efd „ƒYZd efd „ƒYZdefd„ƒYZdefd„ƒYZdefd„ƒYZdefd„ƒYZdefd„ƒYZdefd„ƒYZdefd„ƒYZdefd„ƒYZdefd„ƒYZd „Z d!„Z!d"„Z"d#e#fd$„ƒYZ$d%e$fd&„ƒYZ%d'e$fd(„ƒYZ&d)e$fd*„ƒYZ'd+e$fd,„ƒYZ(d-e#fd.„ƒYZ)d/e)fd0„ƒYZ*dS(1sÙ!Implements the produtil.run: provides the object tree for representing shell commands. Do not load this module directly except for type checking (instanceof(o,produtil.prog.Runner)). It is meant to be used only by the produtil.run module. This module is part of the implementation of a shell-like syntax for running programs. The rest of the implementation is in the produtil.run and produtil.pipeline modules. MPI programs are implemented by the produtil.mpiprog and produtil.mpi_impl. This module implements a shell-like syntax of running shell programs from Python. This module should not be used directly: the produtil.run implements critical parts of the functionality. Specifically, this module implements the Runner and ImmutableRunner classes. It also knows how to convert them to produtil.pipeline.Pipeline objects for actual execution. * Runner --- This class represents a process that could be run. It keeps track of all possible aspects of running a process, including the command, arguments, environment variables, stdout stream, stderr stream, stdin stream, and a list of functions or callable objects to run before executing the problem. Provides public functions to modify the Runner. * ImmutableRunner --- A Runner that cannot be changed: when modifying the Runner, it returns a new object. This is to implement shell aliases. For example, one could make an ImmutableRunner for program to index GRIB2 files. All the user would have to do is add the GRIB2 file as an argument, and capture the output. Note that the actual work of creating the Runner or ImmutableRunner, or turning them into Pipeline objects done by the produtil.run module. Turning MPI programs into Runner objects is done by the produtil.mpiprog module and produtil.mpi_impl package, with the public interface in produtil.run. Hence, nobody would ever load this module directly, except for type checking (ie.: to see if your argument is a Runner before passing it to produtil.run.checkrun)..iÿÿÿÿN(tlaunchtmanagetPIPEtERR2OUTtProgSyntaxErrorcBseZdZRS(sY!Base class of exceptions raised when a Runner is given arguments that make no sense.(t__name__t __module__t__doc__(((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR0stOverspecifiedStreamcBseZdZRS(sp!Raised when one tries to specify the stdout, stderr or stdin to go to, or come from, more than one location(RRR(((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR3st MultipleStdincBseZdZRS(sT!Raised when the caller specifies more than one source for the stdin of a Runner(RRR(((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR 6stMultipleStdoutcBseZdZRS(sU!Raised when the caller specifies more than one destination for a Runner's stdout(RRR(((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR 9stMultipleStderrcBseZdZRS(sU!Raised when the caller specifies more than one destination for a Runner's stderr(RRR(((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR <stInvalidPipelinecBseZdZRS(st!Raised when the caller specifies an invalid input or output when piping a Runner into or out of another object.(RRR(((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR ?stNotValidPosixShcBseZdZRS(s§!Base class of exceptions that are raised when converting a Runner or pipeline of Runners to a POSIX sh command, if the Runner cannot be expressed as POSIX sh.(RRR(((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR CstPrerunNotValidPosixShcBseZdZRS(s•!Raised when trying to convert a pipeline of Runners to a POSIX sh string if the pipeline has a prerun object that lacks a to_shell function.(RRR(((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRGstNoSuchRedirectioncBseZdZRS(s”!Raised when trying to convert a pipeline of Runners to a POSIX sh string, if a redirection in the pipeline cannot be expressed in POSIX sh.(RRR(((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRKstNotValidPosixShStringcBseZdZRS(sÁ!Raised when converting a Runner or pipeline of Runners to a POSIX sh string. If a string is sent to a program's stdin, this is raised when that string cannot be expressed in POSIX sh.(RRR(((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyROstEqualInExecutablecBseZdZRS(s‹!Raised when converting a Runner or pipeline of Runners to a posix sh string if a Runner's executable contains an equal ("=") sign.(RRR(((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRSst EqualInEnvcBseZdZRS(s•!Raised when converting a Runner or pipeline of Runners to a POSIX sh string if there is an equal ("=") sign in an environment variable name.(RRR(((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRWstInvalidRunArgumentcBseZdZRS(s_!Raised to indicate that an invalid argument was sent into one of the run module functions.(RRR(((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR^stExitStatusExceptioncBs8eZdZd„Zed„ƒZd„Zd„ZRS(s !Raised to indicate that a program generated an invalid return code. Examine the "returncode" member variable for the returncode value. Negative values indicate the program was terminated by a signal while zero and positive values indicate the program exited. The highest exit status of the pipeline is returned when a pipeline is used. For MPI programs, the exit status is generally unreliable due to implementation-dependent issues, but this package attempts to return the highest exit status seen. Generally, you can count on MPI implementations to return zero if you call MPI_Finalize() and exit normally, and non-zero if you call MPI_Abort with a non-zero argument. Any other situation will produce unpredictable results.cCs||_||_dS(s~!ExitStatusException constructor @param message a description of what went wrong @param status the exit statusN(tmessaget returncode(tselfRtstatus((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyt__init__xs cCs|jS(s/!An alias for self.returncode: the exit status.(R(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRscCs dt|jƒt|jƒfS(s#!A string description of the error.s%s (returncode=%d)(tstrRtintR(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyt__str__„scCs dt|jƒt|jƒfS(s3!A pythonic description of the error for debugging.sNonZeroExit(%s,%s)(treprRR(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyt__repr__‡s(RRRRtpropertyRRR(((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRbs   cCstjd|ƒrtStSdS(sž!Returns True if the specified environment variable name is a valid POSIX sh variable name, and False otherwise. @param s an environment variable names\A[A-Za-z][A-Za-z0-9_]*\zN(tretsearchtTruetFalse(ts((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pytshvarokscCstjd|ƒrtStSdS(s{!Returns True if the specified string can be expressed as a POSIX sh string, and false otherwise. @param s a strings6\A[a-zA-Z0-9 !"#$%&?()*+,./:;<=>?@^_`{|}~\\\]\[\'-]*\ZN(R R!R"R#(R$((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pytshstrok–scCset|ƒ}t|ƒs4tdt|ƒfƒ‚ntjd|ƒradtjdd|ƒdS|S(sš!Given a Python str, returns a backslashed POSIX sh string, or raises NotValidPosixShString if that cannot be done. @param s a string to backslashs)String is not expressable in POSIX sh: %ss(?ms)[^a-zA-Z0-9_+.,/-]t"s (["\\\\$])s\\\1(RR&RRR R!tsub(R$((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyt shbackslash s   tStreamGeneratorcBs)eZdZd„Zd„Zd„ZRS(sä!This is part of the internal implementation of Runner, and is used to convert it to a produtil.pipeline.Pipeline for execution. This is an abstract class whose subclasses create the Popen's stdout, stdin and stderr.cCsdt|ƒfS(s0!Has no effect. This exists only for debugging.s(R(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyt for_input²scCsdt|ƒfS(s0!Has no effect. This exists only for debugging.s(R(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyt for_outputµscCs |jƒS(s’!Returns the stderr value. The default implementation returns repr_for_out(), causing stderr to receive whatever stdout receives.(t repr_for_out(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyt repr_for_err¸s(RRRR+R,R.(((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR*­s  t FileOpenercBsheZdZed„Zd„Zd„Zed„ƒZd„Z d„Z d„Z d„Z d „Z RS( s!This is part of the internal implementation of Runner, used to convert it to a produtil.pipeline.Pipeline for execution. It represents stdin, stdout or stderr being connected to an open file. It instructs the Runner to open the file before starting the process.cCs||_||_||_dS(s«!FileOpener constructor @param filename the name of the file being opened @param mode how it is being opened @param err if True, this is for stderrN(tfilenametmodeterr(RR0R1R2((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRÄs  cCst|j|j|jƒS(s'!Creates a shallow copy of this object.(R/R0R1R2(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pytcopyÕscCs¡d}|jrd}n|jdkr>d|t|jƒfS|jdkrdd|t|jƒfS|jdkr‡dt|jƒfStd |jfƒ‚d S( s`!Creates a POSIX sh representation of the part of the command that requests redirection.tt2tabs%s>> %stwbs%s> %strbs cat %s | s:Cannot convert file open mode %s to a POSIX sh redirectionN(R2R1R)R0R(Rtmore((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pytto_shellØs  cCs›d}ddkst‚d|jkr3tj}nRd|jkr\tjtjBtjB}n)d|jkr…tjtjBtjB}n|dk s—t‚|S(s8!Returns an integer version of mode suitable for os.opentrR8twtaN( tNonetAssertionErrorR1tostO_RDONLYtO_WRONLYtO_CREATtO_TRUNCtO_APPEND(Rtintmode((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRFãs cCs"dtj|j|jƒdtfS(sZ!Returns a tuple (None,stream,None,True) where "stream" is the opened file object.N(R>R@topenR0RFR"(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyt _gen_streamðscCs dt|jƒt|jƒfS(sM!Returns a string representation of this object as valid Python code.sFileOpener(%s,%s)(RR0R1(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRôscCst|jƒdS(sf!Part of the implementation of Runner.__repr__, this returns the filename and ",string=False".s ,string=False(RR0(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyt repr_for_inøscCs&dt|jƒt|jdkƒfS(sÅ!Part of the implementation of Runner.__repr__, this returns the filename and ",string=False". It also appends ",append=X" where X is the true/false flag for appending to the file.s %s,append=%sR6(RR0R1(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR-üscCs |jƒS(s!Same as repr_for_out.(R-(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR.s(RRRR#RR3R:RRFRHRRIR-R.(((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR/¾s       t StringInputcBsDeZdZd„Zd„Zd„Zd„Zd„Zd„ZRS(s2!Represents sending a string to a process's stdin.cCs ||_dS(sw!Creates a StringInput that sends the specified object to stdin. @param obj the object to send to stdinN(tobj(RRK((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRscCs t|jƒS(s'!Returns a shallow copy of this object.(RJRK(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR3scCs|jdddfS(sr!Returns a tuple containing (O,None,None,None) where O was the object sent to the StringInput constructor.N(RKR>(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRHscCsdt|jƒfS(sM!Returns a string representation of this object as valid Python code.sStringInput(%s)(RRK(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRscCsdt|jƒS(sX!Converts this object, if possible, to an echo command followed by a pipe ("|").s echo %s | (R)RK(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR:scCsLt|jƒdkr4dt|jdd!dƒfSdt|jƒfSdS(sí!Part of the implementation of Runner.__repr__. If possible, this creates valid Python code to represent specifying sending the given string to the stdin of a Runner. If the string is too long, it is abbreviated.i(s%s...,string=Trueii%s...s%s,string=TrueN(tlenRKR(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRIs( RRRRR3RHRR:RI(((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRJs     t StreamReusercBsDeZdZd„Zd„Zd„Zd„Zd„Zd„ZRS(s]!Arranges for a stream-like object to be sent to the stdout, stderr or stdin of a Runner.cCs ||_dS(sq!Creates a StreamReuser for the specified stream-like object. @param obj the stream-like object to reuse.N(RK(RRK((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR+scCs t|jƒS(sq!Returns a shallow copy of this object. Note that means that the underlying stream object is not copied.(RMRK(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR31scCstdƒ‚dS(s^!Raises NotValidPosixSh to indicate that the stream cannot be represented as POSIX sh.s=Python streams cannot be passed to remote POSIX sh processes.N(R (R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR:5scCsdd|jtfS(s\!Returns a tuple (None,None,obj,False) where obj is the provided stream-like object.N(R>RKR#(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRH:scCs t|jƒS(sE!Returns repr(obj) where obj is the given stream-like object.(RRK(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRI>scCs t|jƒS(sE!Returns repr(obj) where obj is the given stream-like object.(RRK(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR-Bs( RRRRR3R:RHRIR-(((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRM(s     t OutIsErrorcBsMeZdZd„Zd„Zd„Zd„Zd„Zd„Zd„Z RS(s,!Instructs a Runner to send stderr to stdoutcCsdS(s!OutIsError constructor.N((R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRIR4cCstƒS(s!!Returns a new OutIsError object.(RN(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR3KscCsdS(s!Returns "2>&1" s2>&1((R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR:NscCsddttfS(s>!Returns a tuple containing (None,None,pipeline.ERR2OUT,False)N(R>RR#(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRHQscCsdS(s7!This should never be called. It returns ".err2out()".s .err2out()((R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRITscCsdS(sˆ!Part of the representation of Runner.__repr__. Returns ".err2out()" which instructs a Runner to send stderr to stdout.s .err2out()((R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR-WscCs t|tƒS(sU!Is the other object an OutIsError? @param other the other object to analyze.(t isinstanceRN(Rtother((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyt__eq__\s( RRRRR3R:RHRIR-RQ(((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRNGs      tRunnercBs‘eZdZd„Zd„Zd„Zd„ZeeeedƒZed„ƒZ d„Z d„Z d „Z d „Z d „Zd „Zd „Zd„Zd„Zd„Zd„Zd„Zd„Zd„Zd„Zd„Zd„Zd„Zd„Zd(d„Zd„Zd„Z d„Z!d„Z"d„Z#d „Z$d!„Z%d"„Z&e'd#„Z(e'd$„Z)d%„Z*e'd&„Z+d(d(d'„Z,RS()s!Represents a single stage of a pipeline to execute. This is a linked list class used to store information about a program or pipeline of programs to be run. It has the capability of converting itself to a Pipeline object (run(Runner)), or converting itself to a POSIX sh command (Runner.to_shell()). Note that some commands cannot be represented in POSIX sh, such as commands with non-ASCII characters or commands that have Python streams as their stdout or stdin. Those commands can still be run with a Pipeline, but trying to convert them to a POSIX sh command will throw NotValidPosixSh or a subclass thereof.cKs8d|_|_|_|_|_d|_|_d|_t |t ƒrL|}t |j ƒ|_ |jdk r‰|jj ƒ|_n|jdk r­|jj ƒ|_n|jdk rÑ|jj ƒ|_n|jdk rõ|jj ƒ|_n|jdk rt|jƒ|_n|jdk r=t |jƒ|_n|j|_nˆt |t ƒsƒtdt|ƒjt|ƒfƒ‚nt |dtƒsÂtdt|dƒjt|ƒfƒ‚n||_ t|_d|kr÷|dr÷|jƒnd|krt|dƒ|_nd|kr6||dknd|krW|t|dƒ>nd|krt||dknd |kr||d ?nd |kr¨|jƒnd |krÎ|j|d d tƒnd |krô|j|d d tƒnd|kr|j|dƒnd|kr4|j|dƒndS(s!Creates a new Runner. The only non-keyword argument can be one of three things: 1. A Runner to copy. Every aspect of the Runner that can be copied will be. Note that if a stream-like object is connected to stdin, stdout or stderr, it will NOT be copied. 2. A list of strings. This will be used as the command path, and arguments. Many options can be set via keyword arguments: * clearenv=True - the environment should be cleared before running this command. Any arguments set by the env= keyword or the .env(...) member function ignore this. Also, PATH, USER, LOGNAME and HOME are retained since most programs cannot run without them. * env=dict(var=value,...) - a dict of environment variables to set before running the Runner. Does NOT affect this parent's process, only the child process. * in=filename - a file to send to stdin. * instr=str - a string to send to stdin * out=filename - a file to connect to stdout. Will truncate the file. * outa=filename - same as "out=filename," but appends to the file. * err2out - redirects stderr to stdout * err=filename - a file to connect to stderr. Will truncate the file. * erra=filename - same as "err=filename," but appends to the file. * prerun=[obj,anotherobj,...] - sent to self.prerun, this is a list of functions or callable objects to run before executing the process. The objects are not called until execution is requested via self._gen. @param args the arguments to the program @param kwargs other settings (see constructor description).s.The args argument must be a list, not a %s %s.is8The first element of args must be a string, not a %s %s.tclearenvtenvtintinstrtouttoutaterr2outR2tappendterratcdtprerunN(R>t_stdint_stdoutt_stderrt_prevt_envt_prerunt_cdt_threadsRORRtlistt_argsR3tdictt _copy_envt TypeErrorttypeRRRR"RSRYR2R#R\R](RtargstkwargsR;((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRosb.% "&             cCs|jS(s9!Returns the number of threads requested by this program.(Re(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyt getthreadsÇscCst|ƒ|_|jS(s6!Sets the number of threads requested by this program.(RRe(Rtnthreads((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyt setthreadsÊscCs d|_dS(s!!Removes the request for threads.N(R>Re(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyt delthreadsÎssThe number of threads per rank.cCs|jdkr|S|jjS(s+!Returns the first Runner in this pipeline.N(RaR>tfirst(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRrÔscCs|jrtƒ|_n|S(sG!Removes all prerun objects. @see prerun() @return self(RcRf(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyt remove_prerunÛs cCsdtƒ}x?|jD]4}|s%t‚|j|ƒ\}}|j|ƒqWtƒ|_dj|ƒS(s›!Internal implementation function - do not use. Applies all prerun objects using their to_shell functions, and then clears the prerun listR4(RfRcR?R:RZtjoin(RttextsR]ttexttrunner((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyt__prerun_to_shellâs   cCs2|jdkr|g|_n|jj|ƒ|S(s!Adds a function or callable object to be called before running the program. The callables should be very fast operations, and are executed by self._gen when creating the Pipeline. They take, as an argument, the Runner and an optional "logger" keyword argument that is either None, or a logging.Logger to use to log messages. @param arg a callable object that takes self as an argument, and an optional keyword argument "logger" with a logging.Logger for log messagesN(RcR>RZ(Rtarg((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR]ïs cCsUt|tƒrd|fSt|tƒr4d|fSt|tƒrG|St|ƒSdS(s€!Returns a string representation of the given argument to convert it to an input to produtil.pipeline.Pipeline. Conversions: * float --- converted via %g * int --- converted via %d * basestring --- no conversion; used directly * all others --- str(arg) @param arg the argument to convert @returns a string version of args%gs%dN(ROtfloatRR(RRy((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyt_stringify_args   cCsyt|tƒs-t|tƒs-t|tƒrI|jj|j|ƒƒn,|jjg|D]}|j|ƒ^qYƒ|S(sá!Add one or more arguments to the executable. Can ONLY accept strings, ints, floats or iterables (tuple, list). Strings, ints and floats are sent to _stringify_args, and the result is added to the end of the list of arguments to the command to run. For iterables (tuple, list), adds all elements to the list of arguments, passing each through _stringify_args. @param args one or more arguments to add @returns self(RORRzRRgRZR{textend(RRltx((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyt __getitem__s ,cCs |jƒS(s!Alias for __repr__()(R(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR#scCsI|jdk r(dt|jƒf}nd}t|jƒdkrP|d7}n|dt|jdƒ7}t|jƒdkr¾|ddjg|jdD]}t|ƒ^qšƒd 7}n|jdk rê|d |jjƒf7}n|jdk r|d |jj ƒf7}n|j dk rat |j t ƒrD|d 7}qa|d |j j ƒf7}n|jsw|d7}n|jdk rÐ|ddjg|jjƒD]\}}d||f^q¢ƒd7}n|jdk r|djg|jD]}dt|ƒf^qòƒ7}n|jdk rE|dt|jƒd7}n|S(s]!Attempts to produce valid Python code to represent this Runnable. Generally, that can be done, unless an input string is too long, no executable name is present, or a stream is connected to a Python object. In those cases, human-readable representations are given, which are not exactly Python code. s%s | R4isbatchexe()s batchexe(%s)it[t,t]s.in(%s)s.out(%s)s .err2out()s.err(%s)s .clearenv()s.env(s, s%s=%st)s .prerun(%s)s.cd(N(RaR>RRLRgRtR^RIR_R-R`RORNR.RiRbtitemsRcRd(RR$R}tktv((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR&s4 >   G9cCs“t|tƒr‹|j|jkoŠ|j|jkoŠ|j|jkoŠ|j|jkoŠ|j|jkoŠ|j|jkoŠ|j|jkSt SdS(s!Returns True if the other object is a Runner that is equal to this one, and False otherwise. @param other the object to compareN( RORRRgRiR^R_R`RbRctNotImplemented(RRP((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRQKscCsX|jdkoW|jdkoW|jdkoW|jdkoW|jdkoW|jdkS(s®!Returns true if this is simply an executable with arguments (no redirection, no prerun objects, no environment modification, no piping), and False otherwise.N(R^R>R_RcR`RbRa(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyt isplainexeZscCs ||_|S(sà!Requests that this process run in the specified directory. The directory must already exist before the program starts. @param dirpath the directory to cd into, which must already exist. @returns self(Rd(Rtdirpath((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR\bs cCs|j|dtƒS(s}!Connects the given object to stdin, via inp(stdin,string=False). @param stdin the stdin object @returns selftstring(tinpR#(Rtstdin((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyt__lt__iscCs|j|dtƒS(s­!Connects the given object to stdout, truncating it if it is a file. Same as out(stdout,append=False). @param stdout the stdout object @returns selfRZ(RWR#(Rtstdout((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyt__gt__nscCs|j|dtƒS(sŠ!Sends the specified string into stdin. Same as inp(stdin,string=True). @param stdin the stdin file @returns selfR‰(RŠR"(RR‹((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyt __lshift__tscCs|j|dtƒS(sŒ!Appends stdout to the specified file. Same as out(stdout,append=True). @param stdout the stdout file @returns selfRZ(RWR"(RR((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyt __rshift__zscCs |jƒS(sB!Sends stderr to stdout. Same as err2out(). @returns self(RY(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyt__pos__€scCs|jƒj|dtƒS(s¿!Redirects stderr and stdout to the specified file, truncating it. Same as err2out().out(filename,append=False) @param outerr the stdout and stderr file @returns selfRZ(RYRWR#(Rtouterr((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyt__ge__„scCs |j|ƒS(s‰!Pipes this Runner to the other Runner. Same as pipeto(other). @returns other @param other the other runner to pipe into(tpipeto(RRP((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyt__or__ŠscCs|jj||ƒ|S(s¯!Inserts the specified argument before the given index. This function is intended for internal use only. It is used to implement threading on Cray, where arguments relating to threading have to be added after the Runner is generated. @warning It is generally not safe to call this function outside the produtil.mpi_impl subpackage since its modules may generate completely different commands than you asked in order to execute your requested programs. @param arg a string argument to add @param index the index to insert before @note Index 0 is the executable, while later indices are arguments.(Rgtinsert(RtindexRy((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pytarginssccsx|jD] }|Vq WdS(s;!Iterates over the executable and arguments of this commandN(Rg(RRy((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRl£scCsi|dkrt}n|dk s't‚|t|jƒƒ}|j|_|jdk rl|jjƒ|_n|jdk r|jjƒ|_n|j dk r´|j jƒ|_ n|j dk rØt |j ƒ|_ n|j dk rü|j jƒ|_ n|j dk r>tƒ|_ x$|j D]}|j j|ƒq!Wn|dk sPt‚t||ƒset‚|S(s¾!Returns a deep copy of this object, almost. If stdin, stdout or stderr are connected to streams instead of files or strings, then the streams are not copied. Instead, the exact same stream objects are connected to the same unit in the new Runner. @param typeobj the type of the new object or None for Runner. Do not set this unless you know what you're doing. @returns the new objectN(R>RRR?RfRgRiR^R3R_R`RbRhRaRcRZRO(RttypeobjR;tp((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR3¨s,   cCs t|_|S(sp!Instructs this command to duplicate the parent process environment (the default). @returns self(R"Ri(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pytcopyenvÃs cCst|_i|_|S(s×!Instructs this command to start with an empty environment except for certain critical variables without which most programs cannot run. (Retains PATH, USER, LOGNAME and HOME.) @returns self(R#RiRb(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRSÊs  cCs²|jdkr|jrdSi}|jr=ttjƒ}n:i}x1dD])}|tjkrJtj|||RiRhR@tenviron(RRTtkey((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyt_impl_make_envÓs  cCs)|jdkrt|ƒ‚n|j|S(N(RbR>tKeyError(RRy((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pytgetenvêscKsM|jdkri|_nx+|D]#}t||ƒ|jt|ƒR(RRmR¡((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRTïs   !cCs¯|jrux=|jD]2}t|dƒstd|jdfƒ‚qqW|jƒ}|jƒ}||jƒ7}|S|jdk rš|jjƒd}n'|j dk r»|j jƒ}nd}|j dk rî|dt |j ƒd7}n|j dk s|j rtjd|jdƒr=td |jdfƒ‚n|d 7}|j s]|d 7}nx^|j D]S}tjd|ƒr•td |fƒ‚n|d t d||j |fƒ7}qgW|j sx7dD],}||j krÎ|d||f7}qÎqÎWn|d 7}n|d jg|jD]}t |ƒ^q!ƒ7}|jdk rf|d |jjƒ7}n|jdk r|d |jjƒ7}n|j dk r«|d7}n|S(s•!Returns a string that expresses this object as a POSIX sh shell command if possible, or raises a subclass of NotValidPosixSh if not.R:s<%s: has a prerun object that cannot be expressed as posix shis | R4s( set -e ; cd s ; exec t=sM%s: cannot have an "=" in the executable name when modifying the environment.RTs -is!%s: variable name contains an "="t s%s=%sRœRRžRŸs "%s=$%s"s )N(RœRRžRŸ(RcthasattrRRgR3t_Runner__prerun_to_shellR:RaR>R^RdR)RbRiR R!RRRtR_R`(RR]t no_prerunRvR$R¡R}((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR:úsV      )   / cCs|S(sÃ!Returns self if self is modifiable, otherwise returns a modifiable copy of self. This is intended to be used to implement unmodifiable subclasses of Runner @returns self((R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRw*scCsËt|tƒstdƒ‚n|jdk r<tdƒ‚n|jdk rZtdƒ‚n|jdk r‡|jdk r‡tdƒ‚n|j ƒ}||_|jdk rÇ|j |jƒd|_n|S(s:!Specifies that this Runner will send its stdout to the other runner's stdin. This will raise MultipleStdout if this Runner's stdout target is already specified, or MultipleStdin if the other's stdin is already specified. @param other the runner to pipe into @returns othersXAttempting to pipe a Runner into something that is not a Runner (likely a syntax error).sFAttempted to pipe more than one process into stdin of the same processs-More than one stdout is detected in prog|progs,More than one stdin is detected in prog|progN( RORRR RaR>R R_R R^RwRŠ(RRPtrother((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR”0s    cCsØ|jdk r&|jj||ƒ|S|jdk rDtdƒ‚nt|tƒsqt|tƒsqt|tƒr}||_nW|r›tt |ƒƒ|_n9t|t ƒrÅtt |ƒdƒ|_nt|ƒ|_|S(sÌ!Specifies that the first Runner in this pipeline takes input from the given file or string specified by stdin. If string=True, then stdin is converted to a string via str(), otherwise it must be a filename or a stream. Raises MultipleStdin if the stdin source is already specified. @param stdin the input file or string @param string if True, stdin is a string. Otherwise, it is a file. @returns selfs*More than one stdin detected in Runner.inpR8N( RaR>RŠR^R RORJR/RMR(RR‹R‰((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRŠPs  cCs||jdk rtdƒ‚nt|tƒri|rNtt|ƒdƒ|_qxtt|ƒdƒ|_nt|ƒ|_|S(sõ!Specifies that this process sends output from its stdout stream to the given file or stream. The stdout object must be a string filename, or a stream. If append=False, and the stdout is a filename, the file will be truncated, if append=True then it is appended. Raises MultipleStdout if the stdout location is already specified @param stdout the stdout file @param append if True, append to the file, otherwise truncate @returns selfs3More than one stdout detected in call to Runner.outR6R7N(R_R>R RORR/RM(RRRZ((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRWks cCs.|jdk rtdƒ‚ntƒ|_|S(s-!Sends stderr to stdout @returns selfs3More than one stderr detected in call to Runner.errN(R`R>R RN(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRYs   cCs‚|jdk rtdƒ‚nt|tƒro|rQtt|ƒdtƒ|_q~tt|ƒdtƒ|_nt|ƒ|_|S(sü!Specifies that this process sends output from its stderr stream to the given file or stream. The stderr object must be a string filename, or a stream. If append=False, and the stderr is a filename, the file will be truncated, if append=True then it is appended. Raises MultipleStderr if the stderr location is already specified. @param stderr the stderr output file @param append if True, append to the file otherwise truncate @returns selfs3More than one stderr detected in call to Runner.errR6R7N(R`R>R RORR/R"RM(RtstderrRZ((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR2Šs  c Csê|jdk r.|jj|d|d|ƒn5|dk rc|jdt|ƒt|jƒfƒn|jdk r™x$|jD]}||d|ƒq|Wn|dk rÛ|jdt|ƒt|ƒt|jdƒfƒni}|jdk rr|jj ƒ\}}}} |dk r$||dt_gentdebugRRcRRgR^RHR_R?R`RbR¢t _impl_addRd( RtpipelineR¬R­R]RmR‰tstreamtsendtclose((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRµ sZ      -            N(-RRRRRnRpRqRtthreadsRrRsR¨R]R{R~RRRQR‡R\RŒRŽRRR‘R“R•R˜RlR>R3R›RSR¢R¤RTR:RwR”R#RŠRWRYR2Rµ(((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRRcsP  X         %                 0    tImmutableRunnercBsÔeZdZd„Zd„Zdd„Zd„Zd„Zd„Z d„Z d„Z d „Z d „Z ed „Zed „Zed „Zd„Zd„Zd„Zd„Zddd„Zd„Zd„ZRS(sÐ!An copy-on-write version of Runner. This subclass of Runner is unmodifiable. It is meant to be used for re-usable exe()-like objects. For example, if one wants an object lsl that runs exe('ls')['-l'] with optional extra arguments, one could do: lsl=ImmutableRunner(Runner('ls')['-l']) and then every time one does run(lsl[argument list]), it generates a new object without modifying the original lsl, ensuring later calls to lsl will have the same effect: lsl['/'] lsl['~'] lsl['/'] # prints the same as the first This is implemented by a copy-on-write method: if a modification is requested, a Runner is returned with the requested modifications.cKsUzDt|_tj||||jdk rCt|jƒ|_nWdt|_XdS(s¹!Creates a new ImmutableRunner. All arguments to this constructor have the same meanings as the Runner constructor. @param args,kwargs passed to Runner.__init__N(R"t_initRRRRaR>R½R#(RRlRm((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRès  cCs|jƒjƒS(s2!Removes all prerun objects. @see prerun()(t _init_runnerRs(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRsõscCs%|dkrt}ntj||ƒS(sH!Creates a deep copy of this runner, except if stream objects are connected to stdin, stdout or stderr. In that case, those same stream objects are still connected. @param typeobj the type of the output object. Do not use this unless you know what you're doing @returns a copy of selfN(R>R½RRR3(RR™((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR3ús cCs |jtƒS(s;!Returns a modifiable version of this object (as a Runner).(R3RR(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRwscCsI|jr|}n |jƒ}|dk s0t‚t|tƒsEt‚|S(s°!Do not call this function: it is an internal implementation function. It returns self if self.__init__ is still being run, otherwise it returns self.runner().N(R¾RwR>R?RORR(RR}((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR¿s    cCs|jƒjƒS(s!Creates a new Runner that is like self in all ways except that it uses the parent process environment. @returns the new Runner(R¿R›(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR›scCs|jƒjƒS(s!Creates a new Runner which is like self in all ways except that it uses an empty environment except for a few critical variables without which most programs cannot run. (Retains PATH, USER, LOGNAME and HOME.) @returns a new Runner(R¿RS(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRSscCs|jƒj|ƒS(s!Returns a new Runner that is like self, except that it cd's to the target directory before running. The directory must already exist before the program starts. @param cd the directory to cd into, which must already exist. @returns the new Runner(R¿R\(RR\((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR\scKs|jƒj|S(sâ!Returns a new Runner that is like self in all ways except that the specified environment variables are set. @param kwargs varname=value arguments of environment variables to set @returns the new Runner(R¿RT(RRm((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRT%scCs|jƒj|ƒS(s¿!Returns a new Runner that is like self in all ways, except that it has been piped into the other Runner. @returns the new Runner @param other the Runner to pipe into.(RwR”(RRP((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR”+scCs|jƒj||ƒS(sÃ!Returns a new Runner that is like self in all ways except that it has a different stdin @param stdin the stdin string or filename @param string if True, stdin is a string(R¿RŠ(RR‹R‰((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRŠ1scCs|jƒj||ƒS(sË!Returns a new Runner that is like self in all ways except with a different stdout. @param stdout the stdout filename @param append if True, append to the file, otherwise truncate(R¿RW(RRRZ((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRW7scCs|jƒj||ƒS(sË!Returns a new Runner that is like self in all ways except with a different stderr. @param stderr the stderr filename @param append if True, append to the file, otherwise truncate(R¿R2(RR«RZ((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR2=scCs|jƒjƒS(sd!Returns a new Runner that is like self in all ways except that stderr is piped into stdout.(R¿RY(R((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRYCscCs|jƒj|ƒS(s¶!Returns a new Runner that is like self in all ways except that a new prerun function has been added. @param arg the new prerun function @sa Runner.prerun()(R¿R](RRy((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR]GscCstj|jƒ|ƒS(s§!Returns a new Runner that is like self in all ways except with new arguments. @param args the new argument or arguments @sa Runner.__getitem__(RRR~R¿(RRl((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR~MscCs|jƒj||ƒS(sÃ!Returns a new Runner that is like self in all ways, except with the specified argument inserted. @param index the index to insert before @param arg the argument to insert(R¿R˜(RR—Ry((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR˜SscCs|jƒj|||ƒS(s0!Creates a Runner object that is a duplicate of this ImmutableRunner, and calls its _gen function. @param pipeline the produtil.pipeline.Pipeline to generate @param logger a logging.Logger for log messages @param next the next Runner in the chain @sa Runner._gen()(RwRµ(RR¸R¬R­((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRµZscCs|jƒ}||_|S(s6!Sets the number of threads requested by this program.(R¿R¼(RRoR;((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRpcs  cCs|jƒ}|`|S(s9!Removes the request for threads. Same as self.threads=1(R¿R¼(RR;((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyRqhs N(RRRRRsR>R3RwR¿R›RSR\RTR”R#RŠRWR2RYR]R~R˜RµRpRq(((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyR½Ós*               (+Rtprodutil.sigsafetytprodutiltiotselectR ttimetfcntlR@tloggingtsignaltprodutil.mpi_impltprodutil.pipelineRRRRt ExceptionRRR R R R R RRRRRRRR%R&R)tobjectR*R/RJRMRNRRR½(((s:/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/prog.pyt(s< l "+ G#ÿÿr