U Ó g·Gã @sdZddddddddd d g Zd d lZd d lZd d lZd d lZd d lZd d lZd a dZ e  e ¡Z d a d ad aGdd„dejƒZdd„Zdd „ZGdd„dejƒZGdd„deƒZdd„ZGdd„dejƒZGdd „d eƒZejejd d fdd„Zdd„Zd ejejejejej ddfdd„Z!d S)a!Configures logging. This module configures logging for stdout, stderr and the jlogfile. It also contains the jlogger, a logger.Logger object that is used to log directly to the jlogfile, and jlogdomain: a string name of the logger domain for the jlogfile.ÚconfigureLoggingÚjloggerÚ jlogdomainÚpostmsgÚMasterLogFormatterÚ JLogFormatterÚstdout_is_stderrÚMasterLogHandlerÚ JLogHandlerÚ set_jlogfileéNÚÚjlogc@seZdZdZdd„ZdS)Ú ThreadLoggerz7!Custom logging.Logger that inserts thread information.c Os@t ¡}dt|jƒt|ƒf}tjj||||||f|ž|Ž} | S)a!Replaces the logging.Logger.makeRecord() with a new implementation that inserts thread information from threading.current_thread() @param name,lvl,fn,lno,msg,args,kwargs Log message information. See the Python logging module documentation for details.z[%s] %s)Ú threadingÚcurrent_threadÚstrÚnameÚloggingÚLoggerÚ makeRecord) ÚselfrZlvlÚfnÚlnoÚmsgÚargsÚkwargsÚctÚx©rú9/lfs/h1/ops/prod/packages/hmon.v3.2.7/ush/produtil/log.pyr0szThreadLogger.makeRecordN)Ú__name__Ú __module__Ú __qualname__Ú__doc__rrrrrr.srcCs t |¡S)z·!Sends the message to the jlogfile logging stream at level INFO. This is identical to: @code jlogger.info(message). @endcode @param message the message to log.)rÚinfo)Úmessagerrrr;scCst |¡|tjd<dS)zÏ!Tells the jlogger to log to the specified file instead of the current jlogfile. Also updates the jlogfile environment variable. The argument must be a filename. @param filename the new jlogfileÚjlogfileN)Ú jloghandlerr ÚosÚenviron)Úfilenamerrrr Es c@s6eZdZdZd dd„Zedd„ƒZdd„Zd d „ZdS) ra'!This is a custom log formatter that inserts the thread or process (logthread) that generated the log message. Also, it always directly calls formatException from format, ensuring that cached information is not used. That allows a subclass (JLogFormatter) to ignore exceptions.NcCstjj|||dd|_dS)zì!MasterLogFormatter constructor @param fmt the log message format @param datefmt the date format @param logthread the thread name for logging @note See the Python logging module documentation for details.)ÚfmtÚdatefmtN)rÚ FormatterÚ__init__Ú _logthread)rr+r,Ú logthreadrrrr.SszMasterLogFormatter.__init__cCs|jdkrtS|jS)z|!The name of the batch thread or process that generated log messages, if the LogRecord does not supply that already.N)r/r0)rrrrr0[s zMasterLogFormatter.logthreadcCsª| ¡|_|j d¡dkr*| ||j¡|_d|jkr@|j|jd<|j|j}d|jkr¦|j dk r¦|  |j ¡}|r¦t |jƒ}t |ƒ  ¡D]}||d<d||j|f}q†|S)a!Replaces the logging.Formatter.format() function. We need to override this due to a "feature" in the Formatter.format: It ignores formatException (never calls it) and caches the exception info, even if the formatter is not supposed to output it. @param record the log record to format @note See the Python logging module documentation for details.z %(asctime)r r0Úexc_infoNr%z%s %s)Ú getMessager%Ú_fmtÚfindÚ formatTimer,ÚasctimeÚ__dict__r0r1ÚformatExceptionÚdictrÚ splitlines)rÚrecordÚsÚeZrec2ÚlinerrrÚformatbs      zMasterLogFormatter.formatcCsdS©z†!Returns nothing to indicate no exception information should be printed. @param ei the exception information to ignoreNr©rÚeirrrr8zsz"MasterLogFormatter.formatException)NNN) r r!r"r#r.Úpropertyr0r?r8rrrrrMs   c@seZdZdZdd„ZdS)rz™!This subclass of MasterLogFormatter does not include exception information in the log file. This is done to prevent cluttering of the log file.cCsdSr@rrArrrr8ƒszJLogFormatter.formatExceptionN)r r!r"r#r8rrrrrsc CslzDt tj ¡¡t tj ¡¡kr(WdStj ¡rBtj ¡rBWdSWn"tk rf}zW5d}~XYnXdS)z²!Returns True if it can determine that stdout and stderr are the same file or terminal. Returns False if it can determine they are not, or if the result is inconclusive.TNF)r(ÚfstatÚsysÚstdoutÚfilenoÚstderrÚisattyÚ Exception)r=rrrrˆs  c@s(eZdZdZdd„Zdd„Zdd„ZdS) raË!Custom LogHandler for the master process of a multi-process job. This is a custom logging Handler class used for multi-process or multi-job batch scripts. It has a higher minimum log level for messages not sent to the jlogfile domain. Also, for every log message, the log file is opened, the message is written and the file is closed. This is done to mimic the postmsg command. Exception information is never sent to the log file.cCs.tj |¡||_||_||_||_||_dS)aZ!MasterLogHandler constructor @param logger The logging.Logger for the master process. @param jlogdomain The logging domain for the jlogfile. @param otherlevels Log level for any extrema to go to the jlogfile. @param joformat Log format for other streams. @param jformat Log format for the jlogfile stream.N)rÚHandlerr.Ú_loggerÚ _otherlevelsÚ _jlogdomainÚ _joformatÚ_jformat)rÚloggerrZ otherlevelsÚjoformatÚjformatrrrr.žs  zMasterLogHandler.__init__cCsbt|jtƒst‚t|jtƒs t‚|j|jkr:|j |¡}n|j|j krJdS|j |¡}|d7}|S)z”!Convert a log record to a string. @note See the Python logging module documentation for details. @returns a string message to printNÚ ) Ú isinstancerOrÚAssertionErrorrPrrNr?ÚlevelnorM©rr;r%rrrÚstringify_record«s   z!MasterLogHandler.stringify_recordcCs&| |¡}|dkrdS|j |¡dS)ú!Write a log message. @param record the log record @note See the Python logging module documentation for details.N)rYrLÚwriterXrrrÚemit»s zMasterLogHandler.emitN)r r!r"r#r.rYr\rrrrr•s c@s eZdZdZdd„Zdd„ZdS)r aŽ!Custom LogHandler for the jlogfile. This is a custom logging Handler class for the jlogfile. It has a higher minimum log level for messages not sent to the jlogfile domain. Also, for every log message, the log file is opened, the message is written and the file is closed. This is done to mimic the postmsg command. Exception information is never sent to the log file.c Csê| |¡}|dkrdSt|jtƒrÚtj |j¡}tj |¡s¶tdƒD]p}zt  |¡WqDt k r²}z>tj |¡r€WY¢,q¶n tj  |¡r‚n|dkr WY¢qD‚W5d}~XYqDXqDt |jdƒ}|  |¡W5QRXn |j  |¡dS)rZNé é Úat)rYrUrLrr(ÚpathÚdirnameÚisdirÚrangeÚmakedirsÚEnvironmentErrorÚexistsÚopenr[)rr;r%Zdirnrr=Úfrrrr\Ìs(       zJLogHandler.emitcCs.t|tƒs$tdt|ƒjt|ƒfƒ‚||_dS)zS!Set the location of the jlogfile @param filename The path to the jlogfile.zPIn JLogHandler.set_jlogfile, the filename must be a string. You passed a %s %s.N)rUrÚ TypeErrorÚtyper ÚreprrL)rr*rrrr ès þÿzJLogHandler.set_jlogfileN)r r!r"r#r\r rrrrr Ãsc Cs˜|dkrt d¡}|dkr.tjtjBtjB}nt|tƒsJtdt |ƒj ƒ‚t|t ƒsftdt |ƒj ƒ‚|dk rŠt|t ƒsŠtdt |ƒj ƒ‚t|t ƒs¤tdt |ƒƒ‚dt |ƒda |  d |t f¡t ||¡}t |d ¡|dk rŠ|  d |t f¡t d ¡}t |d d¡} ||kr,t t |ƒ|¡}t |d ¡tddƒ} t| t|| | ƒ} |  d¡|tjkrt|  |¡ntdƒ‚t ¡ | ¡t t¡adS)aç!Used to split to multiple logging streams. When the Python script splits itself into multiple processes via MPI, this function is called to redirect stdout to stdoutfile, stderr to stderrfile, and produce a new logging stream to the original stderr, with a logging level set to masterlevel. That new logging stream is called the "master log" and will receive any messages at level masterlevel or higher, and any messages sent to the jlogdomain. This can also be used to redirect ONLY stdout, in which case no master logging stream is set up. That is requested by stderrfile=None. @param threadname the name of this process for logging purposes @param stderrfile file to receive stderr @param stdoutfile file to receive stdout @param masterlevel log level to send to master log stream @param openmode integer mode to use when opening files @param logger a logging.Logger for logging errors while splitting the log stream. NÚprodutilz3In mpiRedirect, the openmode must be an int, not a z7In mpiRedirect, the stdoutfile must be a string, not a z?In mpiRedirect, the stderrfile must be a string or None, not a z7In mpiRedirect, the threadname must be a string, not a ú[ú]z(Redirecting stdout to "%s" for thread %séz(Redirecting stderr to "%s" for thread %sér_r zc%(asctime)s.%(msecs)03d %(name)s %(logthread)s (%(filename)s:%(lineno)d) %(levelname)s: %(message)sú%m/%d %H:%M:%SzGTurning on logging of high priority messages to original stderr stream.zno master level)rÚ getLoggerr(ÚO_CREATÚO_WRONLYÚO_APPENDrUÚintrirjr rr0ÚwarningrgÚdup2ÚdupÚfdopenrrÚ masterdomainÚNOTSETÚsetLevelÚ BaseExceptionÚ addHandlerÚ masterlogger) Z threadnameZ stderrfileZ stdoutfileZ threadlevelZ masterlevelZopenmoderQÚfdZolderrfdZolderrÚoformatZ mloghandlerrrrÚ mpi_redirectòsp  ÿÿ ÿÿÿÿ ÿÿÿ   ÿ   ý ÿ   rƒcCs|at |¡adS)N)r{rrrr€)ÚdrrrÚ_set_master_domainHsr…FZmastercCsð|rt tjj¡t|ƒt ¡}|tjkr6| |¡t d¡} t j   dd¡} | dkr`tj   ¡} t| ƒ dd¡ dd¡ dd¡} td| d d ƒ} td| d d ƒ} t d d ¡} tƒröt||ƒ}t tj¡}| | ¡|tjkrê| |¡| |¡n€|dk r6t tj¡}| | ¡|tjkr,| |¡| |¡|dk rvt tj¡}| | ¡|tjkrl| |¡| |¡|dkr¤tt j   dd ¡ƒ}t|ƒdkr¤|}|dk r¶t|ƒntj}t|t|| | ƒa|tjkrât |¡| t¡dS)aÖ!Configures log output to stderr, stdout and the jlogfile Configures log file locations and logging levels for all streams. @note Important notes when choosing levels: * level - sets the global minimum log level. Anything below this level will be discarded regardless of other settings. * jloglevel - this limit is applied before japplevel @param jlogfile path to the jlogfile. Default: use os.environ('jlogfile') if set. Otherwise, stderr. @param level minimum logging level globally. Set to INFO by default. Change this to logging.DEBUG if you're debugging the program. @param jloglevel minimum logging level to send to jlogfile @param japplevel minimum logging level to send to jlogfile from all domains except that specified in jlogdomain. Be careful when changing this as it logs directly to the WCOSS-wide jlogfile in operations. @param eloglevel minimum logging level to send to stderr from ALL logs Set to None to disable stderr logging @param ologlevel minimum logging level to send to stdout from ALL logs Default: logging.NOTSET (no filtering) Set to None to disable stdout logging. @param thread_logger True to include the thread name in log messages. @param masterdomain The logging domain that will send messages to the main log stream for the job, even within individual ranks of mpi-split jobsr&ZjobNú(Ú_ú)ú%z %(asctime)sZ z)-%(levelname)s: %(logthread)s %(message)srqz3-%(name)s: %(levelname)s: %(logthread)s %(message)szU%(asctime)s.%(msecs)03d %(name)s (%(filename)s:%(lineno)d) %(levelname)s: %(message)sr r )rÚsetLoggerClassrlÚlogrr…rrr|r}r(r)ÚgetZ batchsystemZjobnamerÚreplacerr-rÚminÚ StreamHandlerrErHÚ setFormatterrrFÚlenr rr')r&ÚlevelZ jloglevelZ japplevelZ eloglevelZ ologlevelZ thread_loggerr{Úrootr ZjobstrrSrRr‚ZloglevelZ logstreamZ ologstreamZ elogstreamÚvarrrrrMsj&      þÿýý                 )"r#Ú__all__rr(rEÚ tracebackrZprodutil.batchsystemrlr0rrrrr'r€r{rrrr r-rrrrKrr ÚWARNINGÚINFOrƒr…ÚERRORr|rrrrrÚsN þ(   2  .0ý Vù