U  g@sdZddlZddlZddlZddlZddlZddlZddlZddlZ ddlm Z m Z ddl Tddlm Z mZmZdddd gZeZGd d d eZdd dZdddZedZddZGdd d eZGdddeZdS)a7!This module provides two different ways to generate Fortran namelist files from HWRFConfig sections: NamelistInserter - given a file, or a multi-line string, finds angle bracket () strings with values from an HWRFConfig section. This is the suggested method for namelists with very few configurable items. Conf2Namelist - given one or more Config sections, find values of the form nlsec.nlkey=value. The nlsec specifies the namelist &nlsec, the nlkey specifies the namelist variable name, and the value is converted to a Python value and then a Fortran namelist value. The Conf2Namelist provides ways to append multiple Conf2Namelist objects' data to form Fortran one-dimensional arrays. This is to support WRF namelists, where one specifies each domain's data in a 1D array. In addition, this module provides two functions to_fortnml and from_fortnml to convert between in-memory Python objects and strings suitable for pasting in a Fortran namelist to achieve the same value in Fortran.N) NoOptionErrorNoSectionError)*) to_datetimeto_datetime_rel to_fraction to_fortnml from_fortnml Conf2NamelistNamelistInserterc@seZdZdZdS)NamelistRecursionz!used to indicate namelist recursion This exception is used only for internal purposes and will never be raised beyond this module.N)__name__ __module__ __qualname____doc__rr:/lfs/h1/ops/prod/packages/hmon.v3.2.7/ush/hwrf/namelist.pyr 'sr cCsbzt||dWStk r\t|}t|dkrB|dddn|}td||fYnXdS)aG!converts a Python object to fortran namelist format Converts the given python data structure to a string suitable for representing in a Fortran namelist. It can handle bool, int, float, string, datetime, fraction and a list (or tuple) of those. A datetime is converted via strftime('"%Y-%m-%d_%H:%M:%S"'). The optional exc_hint is used when raising exceptions to give an idea of what file and line number, or config option name, generated the exception. Conversions: * True ==> "T" * False ==> "F" * int ==> str(py) * float ==> str(py) * fractions.Fraction ==> str(float(py)) * basestring ==> double-quoted string * datetime.datetime ==> WPS/WRF format date stamp: %Y-%m-%D_%H:%M:%S * list or tuple ==> comma-separated list of converted values Anything else will raise an exception. @return the converted value as a string @param py the object to convert @param exc_hint prepended to exception messages when exceptions are raised. Should contain file and line information.exc_hint(r%z...zG%s: cannot convert nested python containers to Fortran namelist values.N)__to_fortnml_implr reprlenNamelistValueError)pyrsrrrr-s  Fcst|tr|rdSdSt|tr(t|St|tr:t|St|tjrRtt|St|trvdtddt|dSt|t j r| dSt|t st|t r|rt ndfdd|DStd t|fd S) al!internal function used to convert python objects to namelist syntax This function does the actual work of converting a Python object to Fortran namelist syntax. Do not call it directly. @param py the object to convert @param recursed have we already recursed? Used to detect nested lists @param exc_hint the exc_hint argument to to_fortnml()TF"""z"%Y-%m-%d_%H:%M:%S"z, csg|]}t|ddqS)T)recursedr)r).0xrrr asz%__to_fortnml_impl..z2%s%s Unable to convert to a fortran namelist valueN) isinstanceboolintstrfloat fractionsFractionresubdatetimestrftimelisttupler joinrr)rr"rrrrrOs*       ra(?ix)\s*(?: (?P[0-9]+[+][0-9]+[/][0-9]+) | "(?P(?:[^"]+|"")+)" | '(?P(?:[^']+|'')+)' | (?P (?:(?:[+-]?[0-9]+\.[0-9]*|[+-]?\.[0-9]+)(?:[eE][+-]?[0-9]+)?) | (?:[+-]?[0-9]+[eE][+-]?[0-9]+) ) | (?P[+-]?[0-9]+) | (?P[a-zA-Z_][a-zA-Z0-9_]+|[a-eg-su-zA-EG-SU-Z_]) | (?Pt|\.true\.) | (?Pf|\.false\.) | (?P[!#].*$) | (?P\s*,\s*) | (?P.) )c Cs~g}d}t|D].}|j}|dks.|dkr>td|fq|dkrNqDq|dkrl|t||q|dkr|t||q|dkr|d q|d kr|dq|d kr|t||q|d kr|t d d||q|dkr|t dd||q|dkr*d }q|dkr|||qt |dkrZ|dSt |dkrvtd|fn|SdS)a!Converts a fortran namelist value to a Python object This is the inverse function for to_fortnml. Given a string from a Fortran namelist value, returns an equivalent python object. Will throw NamelistValueError if an unrecognized object is present, or NamelistRecursion if you send a nested container (such as a list of lists). @param py the string to convert @return the Python objectFNbadz%s: cannot parse namelist valuecommentr(realtrueTfalsefractionZdqcharr!r Zsqcharz'''comma identifierrz%s: does not specify any data) fortnml_parsefinditer lastgrouprappendr(groupr*rr-r.r)routZislistmatchtokrrrr zs>     c@sJeZdZdZedZedZedZedZ ddZ d d d Z dS)r a!Insert config file data into a Fortran namelist file. This class parses an input file that contains a partial namelist, inserting missing values of the format with values from a ConfigParser-like object. The sorts of text follow a specific format: * @ -- insert a string surrounded by double quotes taken from the specified conf variable * @ -- insert a float, taken from conf variable varname * @ -- same as * @ -- insert an integer, taken from conf variable varname * @ -- insert a logical, taken from conf variable varname * @ -- same as @ -- the conf variable is converted to a datetime.datetime using hwrf.numerics.to_datetime, and then to a string of the format "YYYY-MM-DD_HH:MM:SS". If atime is specified to the parse subroutine, then the value is allowed to be a difference relative to the atime (as accepted by hwrf.numerics.to_datetime_rel). @ -- convert the conf variable to a string, and dump its value unquoted. This is used, for example, to have the name of a namelist variable be generated from a conf file. You can also specify angle braces without a type: @ to ask for the variable to be converted by guessing the type that was intended in the conf file. For example: This conf file: @code [conf] var=T,F,T @endcode With this namelist: @code &nl myvar=@ / @endcode will produce: @code &nl myvar=.true., .false., .true. / @endcode As for variables, one can request a subitem of a variable: * varname -- get variable varname * vit[stormname] -- get variable "vit" and then get vit.__getitem__["stormname"] for subscriptable types. This is mainly intended for vitals.z}(?P
(?:[^<]|"(?:[^"]|""")*"|'(?:[^']|'{3})*')*)<(?:(?P[^:]*):)?(?P[^>\[]*)(?:\[(?P[^\]]*)\])?>(?P.*)z*\A(?i)(?:f.*|.false.|n|no|0*[1-9][0-9]*)\Zz\A(?i)(?:t.*|.true.|y|yes|0)\ZzE(?P(?:[^!]|"(?:[^"]|""")*"|'(?:[^']|''')*')*)(?P!.*$)?cCs||_||_dS)z!NamelistInserter constructor

        Creates a new NamelistInserter that will get its data from the
        specified section of the HWRFConfig conf.
        @param conf the hwrf.config.HWRFConfig object to use
        @param section the section to readN)_conf_section)selfconfsectionrrr__init__szNamelistInserter.__init__N
Tc
s4t|trt|dk	rt|}|dk	r2t||}n|dk	r>|}t}|j}	|j}
dfdd}|D]}|	}d7|}
t
j|
}d}|r|
d}|
d}|s||d	qj|}
t|
dkrt
j|
}|r|
d
}|
d}|
d}|
d
}|
d}|r0|
|ks*t|}
nd}
|rD|||dkrRd}|sd|dq|sv|dqt|dkr|d|fq|dkr|d|fqz^||kr||}nF|dk	r|	j|
d|df||d|}n|	j|
d|df|}Wn\tttttfk
rl}z2dk	rLd|f|rTWYqW5d}~XYnX|rz||}|}Wnfttttfk
r}z>dk	rʈjd||t|fdd|r҂WYqW5d}~XYnXz|dkrt|}n|dkrt|}n|dkrt|tr2|}nPt|trzt
j|rRd}n&t
j|rfd}ntd t|fnt|}nN|d!krt |}|dk	rt||}nt|}n|dkrt |}nt|}Wnhtttfk
r<}zBdk	rjd"t||t|fdd|r$WYqW5d}~XYnX|rRd#||f}n|}z(|d$krtt!||d%d&}nt|}Wnjtttfk
r}zDdk	rȈjd'||t|t|fdd|rЂWYqW5d}~XYnX||q||
d}
q|r ||	d	qj|d	qj|"S)(a!Generates the namelist, returning it as a string.

        Reads the config section and file, generating the namelist and
        returning it as a string.

        @param line_iterable  an iterator that iterates over each line of
            the file.
        @param logger optional: a logging.Logger to log messages, or None
            to disable.  Default: None
        @param source  for warning or error messages: the filename.
                       Default: "@"
        @param raise_all set to False to log messages instead of
            raising exceptions.  Raise only one exception at the end.
            Default: True; raise an exception as soon as the first
            error is seen.
        @param atime Optional: the analysis time for conf.timestrinterp
        @param ftime Optional: the forecast time for conf.timestrinterp
        @param kwargs   additional variables and their values for config
                        file accesses.Nrcs dk	rd|fdS)Nz%s:%d: syntax error: %s)warning)whatZilineloggersourcerrsynerrsz&NamelistInserter.parse..synerrr=rcoder5
pretypvarr.restrzno output type specifiedzno variable specifiedz+output type must be one of: bdfilsu, not %szbdfilrsuBDFILRSU*z3invalid type %s specified: only bdfilsu are allowed{})ftimeatimez%s:%d: cannot find variable %sz%s:%d: %s[%s]: %sT)exc_infoZrRfFZiIZbBlLFz%s is not a valid logicalZdDz'%s:%d: cannot convert %s to type=%s: %sz%s[%s]zbdfilrsBDFILRS*:rz1%s:%d: <%s:%s>=%s: error converting to string: %s)#r&r)AssertionErrorrrioStringIOrFrGrstriprr5rDrBwriter	find_ltgt
timestrinterp	strinterpKeyError	TypeError
ValueErrorrrrM	HWRFErrorr*r(r'nlfalsenltruerr	rgetvalue)rHZ
line_iterablerPrQ	raise_allr\r[kwargsrCrIrJrRlineZlinepartmr5rSrUrVrWr.rXvaleZnewvalZtypvalZdvalZfromthisZwritemerrOrparsesT



































zNamelistInserter.parse)NrLTNN)r
rrrr-compilerdrkrlr5rKrtrrrrrs?



c@seZdZdZdZedZedZedZ	dZ
ddZd7d
dZdd
Z
ddZddZddZddZddZd8ddZddZddZddZd d!Zd"d#Zd9d$d%Zd&d'Zd(d)Zd*d+Zd:d,dZd-d.Zd/d0Zd;d1d2Zd3d4Z d5d6Z!d	S)[a-zA-Z_][a-zA-Z0-9_]*)\.)?(?P[a-zA-Z_][a-zA-Z0-9_%]*)\Zz\A(?i)(?:f.*|.false.)\Zz\A(?i)(?:t.*|.true.)\Zz-trait-cCst|j|j|j|jdS)zz!Returns a copy of self.

        Creates a shallow copy of self
        @param other unused; may be removed in the future)section_sortervar_sortersrPnl)r
rvrwrPrx)rHotherrrrcopys
zConf2Namelist.copyNcCs|dkrt}d|_||_|jdkr(t|_tdd|_|dk	rp|D]&\}}	|	D]\}
}||||
<qXqH|dks|dkrdSt|}t	}
|g}t
j}t|dkr|
}||
krq|
||dk	r|d|f|j||dD]\}
}||
}|r6|ddk	r6||d|dt|q|
d	kr~|d
D]0}|}t|dkrJ||
krJ||qJqtd|
r||dt|q|dk	r|d|
||fqqdS)
a!Conf2Namelist constructor

        Creates a Conf2Namelist.
        
        @param conf the HWRFConfig object
        @param section the section to start searching from.  
        @param section_sorter the cmp-like function to use to sort the
            sections when generating the output namelist
        @param var_sorters a dict-like mapping from section name to a
            cmp-like function to use to sort variable names within 
            each section.
        @param logger a logging.Logger object to use to log messages
        @param nl a dict of dicts (or object that acts like that)
            to use to initialize the Conf2Namelist.  Warning: this
            functionality is untested
        @param morevars a dict with additional variables to use when
            expanding strings This is simply passed to conf.items.
            See the HWRFConfig documentation for details.NcSstSN)collectionsdefaultdictrrrrz(Conf2Namelist.__init__..rz$Conf2Namelist now parsing section %s)morevarsrJrWnamelist,z\A[a-zA-Z_][a-zA-Z_0-9%]*\Zz,Conf2Namelist ignoring %s = %s in section %s)	emptydictrvrwcmpr|r}rxitemsr)setr
nlentryrpopadddebugrDrBnl_set_if_unsetr	splitstriprAr-trait_set_if_unset)rHrIrJrvrwrPrxrsecconkeyrrinitialZtouchedZparsemervaluerqZsec2ZtrimrrrrKsZ



zConf2Namelist.__init__cGs |D]}|jt|q|S)z!create namelists if they do not exist

        Ensures that the specified namelist exists.  Returns self.
        @param args list of namelist names)rxr)lower)rHargsrJrrr
nl_sectionDszConf2Namelist.nl_sectioncCszt|tsXt|tjsXt|tsXt|tsXt|tsXt|tjsXtd|j	j
t|f||jt|
t|
<dS)aL!Sets a variable in a namelist

        Sets the value of a namelist's variable.
        @param section the namelist name
        @param var the name of the variable in the namelist
        @param data the value.  This can be a string,
          datetime.datetime, int, float, fractions.Fraction or a list or
          tuple of suchz(%s: invalid type for namelist (value=%s)N)r&r)r/r(r*r1r+r,rh	__class__r
rrxrrHrJrWdatarrrnl_setLs	


zConf2Namelist.nl_setcCs2z|jt||=Wntk
r,YnXdS)z!Removes a variable from a namelist.

        Removes a variable from a namelist 
        @param section the namelist
        @param var the variable to deleteNrxr)rrgrHrJrWrrrnl_del]szConf2Namelist.nl_delcCs.z|jt|=Wntk
r(YnXdS)z,Removes a namelist section from the namelistNrrHrJrrrnl_del_sectgszConf2Namelist.nl_del_sectcCs||jkrdS||j|kS)a9!does this namelist have this variable?

        Determines if the namelist exists and has the variable.

        @return True if the namelist exists and it has the variable, or
          False otherwise
        @param section the string name of the namelist
        @param var     the string name of the variableFrxrrrrnl_havels	
zConf2Namelist.nl_havecCs||jkrdSdS)zWReturns True if the namelist section exists in the namelist
        and False otherwiseTFrrrrrnl_have_sectxs
zConf2Namelist.nl_have_sectcCs^t|}t|}z|j||WStk
rX|dk	rH|YStd||YnXdS)a!get the value of a variable from a namelist

        Gets the value of a variable in a namelist.  If the
        default is supplied and non-None, it will be returned if the
        variable does not exist.  Raises NamelistKeyError if the
        variable does not exist a the default is not provided.
        @param section the namelist name
        @param var the name of the variable in the namelist
        @param default the value to return if the namelist or variable
          do not exist Nzno value given)r)rrxrgNamelistKeyError)rHrJrWdefaultrvrrrnl_get}szConf2Namelist.nl_getcCs8z|||Wn"tk
r2||||YnXdS)aB!Sets the value of a namelist variable if it has no value.

        If the namelist variable has a value, this function does
        nothing.  Otherwise, it is the same as calling nl_set.
        @param section the namelist name
        @param var the variable name
        @param data the value to set, passed to nl_set()N)rrgrrrrrrszConf2Namelist.nl_set_if_unsetccsFt|tst|}||jkr dS|j|D]\}}||fVq.dS)z}!Iterates over variable,value tuples in the given
        namelist.
        @param section the namelist over which to iterateN)r&r)rxr)rHrJrWrrrrnl_eachs

zConf2Namelist.nl_eachccsHtj|jkst|jtjs t|jtjD]\}}||fVq0dS)z3!Iterates over variable,value tuples in the traits.N)r
TRAITrxr_rrHrWrrrr
trait_eachszConf2Namelist.trait_eachcCs|tj||S)z!Sets a trait's value.
        This is the same as nl_set() but sets a trait.  It simply
        passes the special constant Conf2Namelist.TRAIT as the
        namelist
        @param var the name of the trait to set
        @param value the value to set)rr
rrrrr	trait_setszConf2Namelist.trait_setcCs|tj|S)z!Deletes a trait.

        Deletes a trait.  This is the same as calling nl_del passing
        the Conf2Namelist.TRAIT constant as the section.
        @param var the variable to delete.)rr
rrHrWrrr	trait_delszConf2Namelist.trait_delcCs|tj||S)ah!Returns a trait's value.  

        Returns the value of a trait.  If a default is given and
        non-None, returns the default if the trait does not exist.
        This is the same as calling nl_get() passing the
        Conf2Namelist.TRAIT as the section.
        @param var the trait to get
        @param default the default value if the trait is unset)rr
r)rHrWrrrr	trait_gets	zConf2Namelist.trait_getcCs|tj|S)z!Returns True if the trait exists, and False otherwise.

        Determines if a trait is set.  This is the same as passing
        Conf2Namelist.TRAIT as the section argument to nl_have()
        @param var the trait to query)rr
rrrrr
trait_haveszConf2Namelist.trait_havecCs|tj||S)z!Sets the traits value if it does not have one.

        Sets the value of a trait if the trait does not already have a
        value.  This is the same as calling nl_set_if_unset() passing
        the special value Conf2Namelist.TRAIT as the section.)rr
rrrrrrsz Conf2Namelist.trait_set_if_unsetcCst|dkr|Stdd}|jD]|\}}|j|}|D]`\}}|g}|}	|D]@}
|
|||	}|}	t|tst|tr|	|qX|
|qX|||<qBq(|S)aD!create array values by joining multiple Conf2Namelist objects

        Generates a new Conf2Namelist by looping over all arguments in
        this namelist, and appending the other namelists' values in a
        list.  If the other namelist does not have a given value, then
        this namelist's value, or the last namelist that had a value
        for that argument, will be appended.  Variables or namelist
        sections that exist in the other namelists, but not this one,
        will be ignored.

        @param others an iterable object (list, tuple) of Conf2NamelistrN)rrzr
rxrrr&r2r1extendrA)rHZothersrCZsecnameZmydictZoutsecrWrthelistrryZnextvalrrrr3s 

zConf2Namelist.joinc

Cs|dkrtdd}|jD]p\}}|dks4||r|j|}|D]D\}}|dks`|||rFz||}	WqFtk
r|||<YqFXqFq|S)a8!duplicates this object

        Returns a copy of this object, or if other is specified,
        copies this object's contents into the target Conf2Namelist.
        When copying into a target Conf2Namelist, only values that are
        not already in that namelist will be copied.  The copy has its
        own data structures, so modifying the copy will not modify the
        original.  Optionally, you can copy only a subset of this
        object:

        @param section_subset = a callable object that returns True
          for each section to be kept
        @param var_subset = a callable object that returns
          True for each variable to be kept.

        @param other if specified, this must be a Conf2Namelist.
        Instead of creating a new Conf2Namelist, data will be inserted
        into this one.N)r
rxrrg)
rHZsection_subsetZ
var_subsetryrsdZoutdictrWrZjunkrrrrzs

cCstj|jkr|jtj=|S)zb!Removes all traits.
        Deletes the special Conf2Namelist.TRAIT namelist
        @return self)r
rrxrHrrr
remove_traits
s
zConf2Namelist.remove_traitscCs|S)zw!synonym for make_namelist()

        Generates a Fortran namelist as a string.  Equivalent to
        make_namelist().)
make_namelistrrrr__str__szConf2Namelist.__str__c	s"d}|dkr|j}dfddtt|j|D]|jtjkrZ|d7}n|df7}rd}|dk	r|kr|}|dkr|}tjkr|dfdd	tt	|D7}n,|dfd
d	tt	|D7}tjkr|d7}q8|d7}q8|S)
a!generates the namelist as a string

        Returns the stringified namelist for this object, suitable for
        writing to a file for Fortran to read.  Optionally, you can
        provide a sorting for the namelist entries:

        @param section_sorter = a cmp-like function for sorting sections by
          name If absent, cmp is used.
        @param var_sorters = a dict-like mapping of section names to cmp-like
          objects for sorting variables within each section.  If
          absent, self.namelist_sorter(sectionname) is used
        @param morevars a dict of additional variables which override
          values set in self.nlrNcs$dk	r|kr|S|SdSr{r)rW)rrrrgetvar.sz+Conf2Namelist.make_namelist..getvarz
! Traits:
z&%s
rTc	3s6|].}d|t|dtt|fdfVqdS)z!  %s = %s,%s%%%s=rNrr)r#rWrrrr	Asz.Conf2Namelist.make_namelist..c	3s6|].}d|t|dtt|fdfVqdS)z
  %s = %s,rrNrrrrrrEsz

z
/

)
rvsortediterrxkeysr
rnamelist_sorterr3r1)rHrvrwrrCZsorterr)rrrrrrs6







zConf2Namelist.make_namelistcCs&|jdk	r||jkr|j|SddS)ag!return a sorting function for the variables in a namelist

        Returns a cmp function that orders namelist entries for the
        specified namelist section.  See the argument "cmp" of the
        python built-in function sorted() for details.

        @param section the namelist or Conf2Namelist.TRAIT
        @return a cmp-like function sorter(a,b)  NcSs
t||Sr{)r)r$yrrrr~Yrz/Conf2Namelist.namelist_sorter..)rwrrrrrMs	


zConf2Namelist.namelist_sortercCs|dkrtn||_||_|S)a!sets the sorting algorithms for namelists and  variables

        Sets the cmp-like functions for sorting sections and
        variables in each section.  The section_sorter sorts sections.
        The var_sorters is a dict-like mapping from section name to a
        cmp-like function for sorting that section.  If any sorter is
        unspecified, cmp will be used.  See the "cmp" argument of the
        python built-in function sorted() for details.

        @param section_sorter a cmp-like function for sorting
        namelist by namelist name.  If section_sorters is None, 
        cmp is used.
        @param var_sorters a dict-like mapping from namelist name to a
        variable sorter function.  Each variable sorter function must
        be a cmp-like function that compares variable names.  If
        var_sorters is None, then cmp() will be used for all namelists.
        @return selfN)rrvrw)rHrvrwrrrset_sortersZszConf2Namelist.set_sorters)NNNNNNN)N)N)NNN)NNN)"r
rrrZtestnlr-rurrkrlrrzrKrrrrrrrrrrrrrrrr3rrrrrrrrrr
sJ2



>





3
)r)Fr)rr|r-r+r/configparserr`logging
hwrf.numericshwrfrrhwrf.exceptionsrrr__all__dictr	Exceptionrrrrur>r	objectrr
rrrrs 8
"

!