ó Š÷Îgc @s‰dZddddddddd d g Zd d lZd d lZd d lZd d lZd d lZd d lZd a dZ ej e ƒZ d ad ad adejfd„ƒYZd„Zd„Zdejfd„ƒYZdefd„ƒYZd„Zdejfd„ƒYZd efd„ƒYZejejd d d„Zd„Zd ejejej ejej!e"dd„Z#d S(s!Configures logging. This module configures logging for stdout, stderr and the jlogfile. It also contains the jlogger, a logger.Logger object that is used to log directly to the jlogfile, and jlogdomain: a string name of the logger domain for the jlogfile.tconfigureLoggingtjloggert jlogdomaintpostmsgtMasterLogFormattert JLogFormattertstdout_is_stderrtMasterLogHandlert JLogHandlert set_jlogfileiÿÿÿÿNttjlogt ThreadLoggercBseZdZd„ZRS(s7!Custom logging.Logger that inserts thread information.c OsVtjƒ}dt|jƒt|ƒf}tjj||||||||Ž} | S(s!Replaces the logging.Logger.makeRecord() with a new implementation that inserts thread information from threading.current_thread() @param name,lvl,fn,lno,msg,args,kwargs Log message information. See the Python logging module documentation for details.s[%s] %s(t threadingtcurrent_threadtstrtnametloggingtLoggert makeRecord( tselfRtlvltfntlnotmsgtargstkwargstcttx((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/log.pyR1s '(t__name__t __module__t__doc__R(((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/log.pyR /scCs tj|ƒS(s·!Sends the message to the jlogfile logging stream at level INFO. This is identical to: @code jlogger.info(message). @endcode @param message the message to log.(Rtinfo(tmessage((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/log.pyR<scCstj|ƒ|tjdt |ƒjƒD]'}||d((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/log.pyR5„R (RRRR5(((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/log.pyR€scCstyZtjtjjƒƒtjtjjƒƒkr7tStjjƒrYtjjƒrYtSWntk ro}nXt S(s²!Returns True if it can determine that stdout and stderr are the same file or terminal. Returns False if it can determine they are not, or if the result is inconclusive.( R$tfstattsyststdouttfilenotstderrtTruetisattyt ExceptiontFalse(R:((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/log.pyR‰s0cBs)eZdZd„Zd„Zd„ZRS(sË!Custom LogHandler for the master process of a multi-process job. This is a custom logging Handler class used for multi-process or multi-job batch scripts. It has a higher minimum log level for messages not sent to the jlogfile domain. Also, for every log message, the log file is opened, the message is written and the file is closed. This is done to mimic the postmsg command. Exception information is never sent to the log file.cCsAtjj|ƒ||_||_||_||_||_dS(sZ!MasterLogHandler constructor @param logger The logging.Logger for the master process. @param jlogdomain The logging domain for the jlogfile. @param otherlevels Log level for any extrema to go to the jlogfile. @param joformat Log format for other streams. @param jformat Log format for the jlogfile stream.N(RtHandlerR*t_loggert _otherlevelst _jlogdomaint _joformatt_jformat(RtloggerRt otherlevelstjoformattjformat((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/log.pyR*Ÿs     cCst|jtƒst‚t|jtƒs0t‚|j|jkrW|jj|ƒ}n(|j|j krmdS|jj|ƒ}|d7}|S(s”!Convert a log record to a string. @note See the Python logging module documentation for details. @returns a string message to prints N( t isinstanceRMRtAssertionErrorRNRRLR=tlevelnoRKR+(RR8R!((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/log.pytstringify_record¬s cCs3|j|ƒ}|dkrdS|jj|ƒdS(s!Write a log message. @param record the log record @note See the Python logging module documentation for details.N(RVR+RJtwrite(RR8R!((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/log.pytemit¼s (RRRR*RVRX(((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/log.pyR–s cBs eZdZd„Zd„ZRS(sŽ!Custom LogHandler for the jlogfile. This is a custom logging Handler class for the jlogfile. It has a higher minimum log level for messages not sent to the jlogfile domain. Also, for every log message, the log file is opened, the message is written and the file is closed. This is done to mimic the postmsg command. Exception information is never sent to the log file.cCs|j|ƒ}|dkrdSt|jtƒrtjj|jƒ}tjj|ƒsÜxt dƒD]p}ytj |ƒWqet k rÔ}tjj|ƒr¤Pn*tjj |ƒr¼‚n|dkrÎqen‚qeXqeWnt |jdƒ}|j|ƒWdQXn|jj|ƒdS(s!Write a log message. @param record the log record @note See the Python logging module documentation for details.Ni i tat(RVR+RSRJRR$tpathtdirnametisdirtrangetmakedirstEnvironmentErrortexiststopenRW(RR8R!tdirnRR:tf((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/log.pyRXÍs(  cCsDt|tƒs7tdt|ƒjt|ƒfƒ‚n||_dS(sS!Set the location of the jlogfile @param filename The path to the jlogfile.sPIn JLogHandler.set_jlogfile, the filename must be a string. You passed a %s %s.N(RSRt TypeErrorttypeRtreprRJ(RR&((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/log.pyR és "(RRRRXR (((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/log.pyRÄs c CsM|dkrtjdƒ}n|dkrDtjtjBtjB}n+t|tƒsot dt |ƒj ƒ‚nt|t ƒsšt dt |ƒj ƒ‚n|dk rÒt|t ƒ rÒt dt |ƒj ƒ‚nt|t ƒsút dt |ƒƒ‚ndt |ƒda |jd|t fƒtj||ƒ}tj|d ƒ|dk r:|jd |t fƒtjd ƒ}tj|d d ƒ} ||krµtjt |ƒ|ƒ}ntj|d ƒtddƒ} t| t|| | ƒ} |jdƒ|tjkr| j|ƒn tdƒ‚tjƒj| ƒntjtƒadS(sç!Used to split to multiple logging streams. When the Python script splits itself into multiple processes via MPI, this function is called to redirect stdout to stdoutfile, stderr to stderrfile, and produce a new logging stream to the original stderr, with a logging level set to masterlevel. That new logging stream is called the "master log" and will receive any messages at level masterlevel or higher, and any messages sent to the jlogdomain. This can also be used to redirect ONLY stdout, in which case no master logging stream is set up. That is requested by stderrfile=None. @param threadname the name of this process for logging purposes @param stderrfile file to receive stderr @param stdoutfile file to receive stdout @param masterlevel log level to send to master log stream @param openmode integer mode to use when opening files @param logger a logging.Logger for logging errors while splitting the log stream. tprodutils3In mpiRedirect, the openmode must be an int, not a s7In mpiRedirect, the stdoutfile must be a string, not a s?In mpiRedirect, the stderrfile must be a string or None, not a s7In mpiRedirect, the threadname must be a string, not a t[t]s(Redirecting stdout to "%s" for thread %sis(Redirecting stderr to "%s" for thread %siRYisc%(asctime)s.%(msecs)03d %(name)s %(logthread)s (%(filename)s:%(lineno)d) %(levelname)s: %(message)ss%m/%d %H:%M:%SsGTurning on logging of high priority messages to original stderr stream.sno master levelN(R+Rt getLoggerR$tO_CREATtO_WRONLYtO_APPENDRStintRdReRRR-twarningRatdup2tduptfdopenRRt masterdomaintNOTSETtsetLevelt BaseExceptiont addHandlert masterlogger( t threadnamet stderrfilet stdoutfilet threadlevelt masterleveltopenmodeROtfdtolderrfdtolderrtoformatt mloghandler((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/log.pyt mpi_redirectósX          cCs|atj|ƒadS(N(RsRRjRx(td((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/log.pyt_set_master_domainIstmastercCsË|rtjtjjƒnt|ƒtjƒ}|tjkrQ|j|ƒntjdƒ} t j j ddƒ} | dkr“tj jƒ} nt| ƒjddƒjddƒjddƒ} td| dd ƒ} td| d d ƒ} tjd d ƒ} tƒrit||ƒ}tjtjƒ}|j| ƒ|tjkrY|j|ƒn|j|ƒn´|dk rÃtjtjƒ}|j| ƒ|tjkr³|j|ƒn|j|ƒn|dk rtjtjƒ}|j| ƒ|tjkr |j|ƒn|j|ƒn|dkrbtt j j dd ƒƒ}t|ƒd krb|}qbn|dk rzt|ƒntj}t|t|| | ƒa|tjkrºtj|ƒn|jtƒdS(sÖ!Configures log output to stderr, stdout and the jlogfile Configures log file locations and logging levels for all streams. @note Important notes when choosing levels: * level - sets the global minimum log level. Anything below this level will be discarded regardless of other settings. * jloglevel - this limit is applied before japplevel @param jlogfile path to the jlogfile. Default: use os.environ('jlogfile') if set. Otherwise, stderr. @param level minimum logging level globally. Set to INFO by default. Change this to logging.DEBUG if you're debugging the program. @param jloglevel minimum logging level to send to jlogfile @param japplevel minimum logging level to send to jlogfile from all domains except that specified in jlogdomain. Be careful when changing this as it logs directly to the WCOSS-wide jlogfile in operations. @param eloglevel minimum logging level to send to stderr from ALL logs Set to None to disable stderr logging @param ologlevel minimum logging level to send to stdout from ALL logs Default: logging.NOTSET (no filtering) Set to None to disable stdout logging. @param thread_logger True to include the thread name in log messages. @param masterdomain The logging domain that will send messages to the main log stream for the job, even within individual ranks of mpi-split jobsR"tjobt(t_t)t%s %(asctime)sZ s)-%(levelname)s: %(logthread)s %(message)ss%m/%d %H:%M:%Ss3-%(name)s: %(levelname)s: %(logthread)s %(message)ssU%(asctime)s.%(msecs)03d %(name)s (%(filename)s:%(lineno)d) %(levelname)s: %(message)sR iN(RtsetLoggerClassRgtlogR R†RjRtRuR$R%tgetR+t batchsystemtjobnameRtreplaceRR)Rtmint StreamHandlerRARDt setFormatterRwRBtlenRRR#(R"tlevelt jloglevelt japplevelt eloglevelt ologlevelt thread_loggerRstrootR tjobstrRRRQR‚tloglevelt logstreamt ologstreamt elogstreamtvar((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/log.pyRNsb&   0            !($Rt__all__RR$RAt tracebackR tprodutil.batchsystemRgR-RRjRR+R#RxRsRR RR R)RRRRIRRtWARNINGtINFOR„R†tERRORRtRHR(((s9/lfs/h1/ops/prod/packages/hafs.v2.0.7/ush/produtil/log.pyts>  <  2 .0S