U  gy@sldZddddddddd d d d d dddgZddlZddlZddlZddlZddlZGdddeZGdddeZ GdddeZ GdddeZ GdddeZ Gddde Z GdddeZd d Zd!dZd6d"dZd#dZd$dZd7d'd Zd8d(dZd)dZd*dZd9d,dZd-d Zd.d Zd/d Zd0d1ZGd2d3d3eZGd4ddeZGd5ddeZ dS):a!Time manipulation and other numerical routines. This module implements various simple numerical algorithms, such as partial sorting, time manipulation or fraction-to-date conversions. It also contains two array-like classes that take datetime objects as indices.partial_ordering fcst_hr_minsplit_fraction to_fractionto_datetime_rel to_datetime to_timedelta TimeArrayminutes_seconds_restnearest_datetimeis_at_timestepgreat_arc_disttimedelta_epsilon TimeMappingwithin_dt_epsilonrandint_zeromeanNc@seZdZdZdS) TimeErrorz-!Base class used for time-related exceptions.N__name__ __module__ __qualname____doc__rr>/lfs/h1/ops/prod/packages/hmon.v3.2.7/ush/produtil/numerics.pyrsrc@seZdZdZdS)InvalidTimestepzq!Raised when a timestep is invalid, such as a negative timestep for a situation that requires a positive one.Nrrrrrrsrc@seZdZdZdS) NotInTimespanzT!Raised when a time is outside the range of times being processed by a function.Nrrrrrrsrc@seZdZdZdS)NoNearbyValueszw!Raised when an operation has a set of known times, but another provided time is not near one of those known times.Nrrrrrrsrcs eZdZdZfddZZS)InvalidTimespanzn!Superclass of exceptions relating to groups of one or more distinct times and relationships between them.cs tt||||_||_dS)z! Constructor for InvalidTimespan @param message the string explanation of the problem @param start the start of the timespan @param end the end of the timespanN)superr__init__startend)selfmessager r! __class__rrr(szInvalidTimespan.__init__)rrrrr __classcell__rrr$rrsrc@seZdZdZdS) NoTimespanz=!Raised when a timespan was expected, but none was available.Nrrrrrr'2sr'c@s"eZdZdZdddZddZdS)rau!Sorts a pre-determined list of objects, placing unknown items at a specified location. This class is a drop-in replacement for cmp in sorting routines. It represents a partial ordering of objects by specifying the order of a known subset of those objects, and inserting all unknown objects in a specified location in the that list (at the end, by default) in an order determined by cmp(a,b). Example: @code{.py} p=partial_ordering([3,2,1]) # list is ordered as [3,2,1] with # everything else after that sorted([0,1,2,3,6,4,5],p) # = [3, 2, 1, 0, 4, 5, 6] p(1,-99) # = -1, so -99 goes after 1 since -99 is not # in the partial ordering p(1,3) # = 1, so 3 goes before 1 since 3 is before 1 # in the partial ordering p(5,10) # = -1 since cmp(5,10)=-1 @endcodeNcCsft|_||_||_d}|D]4}||jks>||jks>|d7}q ||j|<|d7}q|jdkrb||_dS)a!partial_ordering constructor. Creates a partial ordering. The subset that is ordered is specified by the ordered iterable "ordered" while the index at which to place unordered values is optionally specified by "unordered", which can be anything that can be compared to an int via less than and greater than. If "unordered" is missing, then all objects not in "ordered" will be placed at the end of any list. To place at the beginning of the list, give unordered=0. To insert between the first and second elements, specify 1, between second and third elements: specify unordered=2, and so on. Specify another tiebreaker key generator with "backup_keygen" (default: original value). @param ordering the ordering of known objects @param unordered Optional: where to put other objects @param unordered_key Optional: the tiebreaker key generator for unordered keys. Default is the original value. rN)dictorder backup_keygen unordered)r"orderingr,r+iobjrrrrLs    zpartial_ordering.__init__cCs<||jkr|j|dfS|jdk r2|j||fS|j|fS)N)r*r+r,)r"arrr__call__{s   zpartial_ordering.__call__)NN)rrrrrr1rrrrr7s /c Cstjd}d}d}t||}t||}t||} t||} |dt|d|} |dt| d|} | | ttdtt|| ddt|t| t|| ddS)a_!Great arc distance between two points on Earth. Calculates the great arc distance in meters between two points using the Haversine method. Uses the local Earth radius at the latitude half-way between the two points. @param xlon1,ylat1 first point, degrees @param xlon2,ylat2 second point, degrees @returns distance in metersgf@g@TXAgˡEr@?g@)mathpifloatsinasinminsqrtcos) xlon1ylat1xlon2ylat2deg2radRequatorflattening_invrlat1rlon1rlat2rlon2Rearth1Rearth2rrrr s     (cCs\||}|jd|jd|jd}t|\}}|dkrH|dksHtt|d}||fS)au!Return forecast time in hours and minutes. Given a forecast datetime.datetime and an analysis datetime.datetime, this returns a tuple containing the forecast hour and minute, rounded to the nearest integer minute. @param time forecast time as a datetime.datetime @param start analysis time as a datetime.datetime @returns a tuple (ihours,iminutes)gtAr2<)daysseconds microsecondsr3modfAssertionErrorround)timer dtfhoursfpartihoursiminutesrrrrs   csttt|}|dkr$tjn|jdkrH ksHtdfddt|D}t|}|dkr|dkrd|d}|| krx||d8<qqxn6|dkrd|d}||kr||d7<qqt|}qht|dkst|S)a!Generates "count" numbers uniformly distributed between -imax and imax, inclusive, with a mean of zero. @param count number of numbers to return @param imax maximum value of any number @param randomizer the random module, or something that looks like itNrzKIn randint_zeromean, imax=%d cannot be negated and fit within a Python int.csg|]} qSrr.0ximaxrandintrr sz$randint_zeromean..r()absintrandomr\ OverflowErrorrangesumrO)countr[ randomizerrandcenicenrrZrrs6   cCs&t|}||}|t|jt|jfS)a!Splits a fraction into components. Splits a fraction.Fraction into integer, numerator and denominator parts. For example, split_fraction(Fraction(13,7)) will return (1,6,7) since 1+6/7=13/7. @returns a tuple (integer,numerator,denominator))r_ numerator denominator)fr.f2rrrrscCstt|tjst|}n:t|tjs*t|}||}t|jd|jd|j}t|tjsdt|}t |t |kS)a$!Returns True if time1 is within epsilon of time2, and False otherwise. @param time1,time2 the times being compared @param epsilon how close they need to be in order to be considered equal. @returns True if the times are within epsilon of each other, False if not@BQ) isinstancedatetimerr fractionsFractionrMrLrKr^)time1time2epsilonrRrrrrs     F c Cs|r t|}|dk rt|}d}d}t|}|D]d}|dkrJt|}|}n t||}|dk rtt||dd|} |dks| |kr| dkr| }|}q0|dkr|dkrtd|ddt|St|S)a!Decides a reasonable epsilon for time equality comparisons. Given an iterable of datetime objects (or anything accepted by to_datetime_rel), computes the minimum time difference between any two adjacent times and divides it by "numerator" (default: 10). The "rel" argument is the relative time when calling to_datetime_rel. If unspecified, it will be the first time seen (in which case that time must be acceptable to to_datetime). The "default" is the return value when all elements in "times" are identical, or when there is only one element. If the default is unspecified and it is needed, then NoTimespan is raised. If sort is specified and True, then the times will be sorted before anything is done (it is False by default). @param times a list of example times for comparison @param rel a reference time to which the times will be compared @param default if too few unique times are found, this is returned If that happens and no default is provided, an exception is raised @param sort if True, sort times @param numerator A measure of how close times should be. The least time difference will be at least numerator times the epsilon @returns an epsilon valueNTnegokrznToo few non-identical times in input, and no default specified. Cannot compute timespan in timedelta_epsilon.)r r!)sortedrr_rrr^rr') timesreldefaultsortrimindiffpriorrQnowdiffrrrr s4 cCsd}|dk rt||}nt|tjrFt|jd|jd|j}nrt|trt d|}|r| dddg\}}}tt |t |t |t |}nt|t rtj|d}|dkrt|}|s|dkstd t|t|f|S) a!Converts an object or two to a fraction. This routine is a wrapper around fraction.Fraction() which accepts additional inputs. Fractions are needed to provide higher precision in time and resolution calculations (and also to match the WRF behavior). The arguments are the same as for the fractions.Fraction constructor, but additional calling conventions are accepted: a single float argument, a single datetime.timedelta object, or a string containing an integer and a fraction. If a float is provided, its denominator is restricted to be no more than 1000000. If the resulting fraction is not larger than 0, then InvalidTimestep is thrown unless negok=True. Examples: @code to_fraction(0.855) # 0.855 seconds to_fraction(33) # 33 seconds to_fraction('12/5') # 2.4 seconds to_fraction('7+1/2') # 7.5 seconds to_fraction(0.0000001) # ERROR! Value of the float() is less # than 1e-6 to_fraction(1,10000000) # Valid as a fraction, even though value # is less than 1e-6 @endcode @param a,b the objects to convert @param negok if True, negative numbers are okay.Nrmrnz<\s*(?P[+-]?\d+)\s*(?P[+-]\d+)\s*/\s*(?P\d+)ipartnumdenrz2to_fraction(%s,%s) resulted in a negative timestep)rqrrrorp timedeltarMrLrKstrrematchgroupsr_r5 from_floatlimit_denominatorrrepr)r0brxresultmr.ndrrrr.s2    &   cCst|tjst|}t|tjr$|St|tjr8||St|trtd|rt|dkrhtj|dSt|dkrtj|dStj|dS|t |S)a!Converts objects to a datetime relative to another datetime. Given a datetime object "rel", converts "d" to a datetime. Object "d" can be anything accepted by to_datetime, or anything accepted as a single argument by to_timedelta. If it is a timedelta, it is added to "rel" to get the final time. @param d the object being converted @param rel the datetime.datetime to which it is to be convertedz7\A(?:\d\d\d\d-\d\d-\d\d \d\d:\d\d:\d\d|\d{10}|\d{12})\Zrv%Y%m%d%H %Y%m%d%H%M%Y-%m-%d %H:%M:%S) rorprrrrrlenstrptimer)rr{rrrr`s       cCst|trt|}t|tjr"|St|trpt|dkrFtj|dSt|dkr`tj|dStj|dSntdt|fdS)a,!Converts the argument to a datetime. If the argument is already a datetime, it is simply returned. Otherwise it must be a string of the format YYYYMMDDHH, YYYYMMDDHHMM, or YYYY-MM-DD HH:MM:SS. @param d the object being converted @returns the resulting datetime.datetime objectrvrrrrzmInvalid value %s for to_datetime. Parameter must be a datetime or a string that matches YYYY-MM-DD HH:MM:SS.N)ror_rrprr ValueErrorr)rrrrr{s     Tc CsNt|tjr|St|tr|dkrztd|}|rd\}}}|}d|krj|ddk rjt|d}d|kr|ddk rt|d}d|kr|ddk rt|d}tj|||d}d|kr|ddk r|dd kr| WS|WSWn,tt t fk r} zW5d} ~ XYnXt |||d } t | \} } tj| tt| d d S) af!Converts an object to a datetime.timedelta. Returns a datetime.timedelta object. If "a" is a timedelta, then that value is returned. If "a" is a string of the format 08:13 or 08:13:12, optionally preceded by a "-" then it is interpreted as HH:MM or HH:MM:SS, respectively (where a "-" negates). Otherwise, the arguments are sent to the to_fraction() function, and the result is converted into a timedelta. @param a object being converted @param b second object, if a time range is sent @param negok if True, negative timespans are allowed @returns a datetime.timedeltaNz(?ix) \A \s* (?P-)? 0* (?P\d+) :0*(?P\d+) (?: :0*(?P\d+(?:\.\d*)?) )? \s*)rrhoursminutesrL)rrrLnegative-rwg.A)rLrM)rorprrrsearch groupdictr5 TypeErrorrAttributeErrorrr3rNr_rP) r0rrxrrrrLmdictrRerkrTrrrrrs@       cCs>t|}t|d}t||d}||d|}|||fS)z!Splits the given fractions.Fraction of seconds into integer minutes, seconds and fractional remainder <=0. @param fraction the fraction to convert, assumed to be in seconds @returns a tuple (minutes,seconds,rest) as integersrJ)rr_)fractionrrLrestrrrr s  cCsFt|}t|}t|}ttt|||}tt|||S)a!Return the nearest datetime.datetime to a target. Given a start time, a target time and a timestep, determine the nearest time not earlier than the target that lies exactly on a timestep. Input start and target can be anything understood by to_datetime, and the timestep can be anything understood by to_fraction. Return value is a datetime.datetime object. @param start the fixed start time of allowed return values @param target the time desired @param timestep the times between allowed return values)rrr_r3ceilrfloor)r targettimestepdstartdtargetrR how_many_dtrrrr s cCs0tt|t|}|t|}|t|dkS)z!Returns True if the target time lies exactly on a timestep, and False otherwise. @param start the fixed start time of allowed return values @param target the time desired @param timestep the times between allowed return valuesgh㈵>)rrr_)r rrspan timestepsrrrr s c Cs|tt|}|dk}|r| }t|\}}}t|d}t|dd}t|d}d|||f} |dkrxd| ||f} | S)aS!Converts a timedelta to a string Converts dt to a string of the format "DD:HH:MM:SS+num/den" * DD - number of days * HH - number of hours * MM - minutes * SS - seconds * num/den - fractional part The to_fraction is used to get the fractional part. @param dt anything convertible to a datetime.timedelta.rrJrIz%02d:%02d:%02dz%s+%d/%d)rrrr_) rRfdtnegr.rrrLrroutrrr str_timedeltas   rc@seZdZdZd-ddZddZddZed d Zed d Z d dZ d.ddZ ddZ ddZ ddZddZddZddZddZdd Zd!d"Zd#d$Zd%d&Zd'd(Zd)d*Zd+d,ZdS)/ TimeContaineraU!Abstract base class that maps from time to objects. This is the abstract base class of any class that represents a mapping from a set of times to a set of data. It provides the underlying implementation of TimeArray and TimeMapping. This is not exported by "from produtil.numerics import *" to prevent accidental use.NcsZ||_t|j}dkr2dg||_dg||_n$dg||_fddt|D|_dS)aV!TimeContainer constructor. Initializes the internal arrays for the given list of times, which must not be empty. Note that strange, potentially bad things will happen if there are duplicate times. Implementation notes: @code self._times[N]: a list of times self._data[N]: the data for each time self._assigned[N]: True if there is data for this time, False otherwise @endcode where N is the length of the input times array. The _data will be filled with init() and _assigned filled with True if init is present and not None, otherwise _assigned will be False. @param times the times to map from @param init a constructor or function that creates initial values for all uninitialized array elementsNFTcsg|] }qSrrrWinitrrr]6sz*TimeContainer.__init__..)_timesr _assigned_datarb)r"rzrNrrrrs   zTimeContainer.__init__cCs|j|st||j|S)zz!Returns the data at the given index, or raises KeyError if no data exists. @param index the desired index)rKeyErrorr)r"indexrrrat_index7s zTimeContainer.at_indexcCs tddS)z!This virtual function should be implemented in a subclass. It should return the index of the time to use for the given time. @param when the time of interestz*TimeContainer.index_of is not implemented.N)NotImplementedError)r"whenrrrindex_of=szTimeContainer.index_ofcCs |jdS)zU!Returns the last time in the array or list of times, even if it has no data.rr"rrrlasttimeDszTimeContainer.lasttimecCs |jdS)zV!Returns the first time in the array or list of times, even if it has no data.rrrrrr firsttimeIszTimeContainer.firsttimecCs*||\}}|j|s t||j|S)z!Returns the item at the latest time that is not later than "when". If there is no data assigned at that time, then KeyError is raised. @param when the time of interest)rrrr)r"riwhenrrrr __getitem__Ns zTimeContainer.__getitem__cCsb||\}}|dk rZtt||ddt|kr6|Stdt|t|tt|fn|SdS)a!Returns a tuple containing the time nearest to "when" without going over, and the index of that time. @param epsilon If specified, raise hwrf.exceptions.NoNearbyValues if no times are near that value. @param when the time of interestNTrwzB%s: nearest value not after is %s, which is not within %s seconds.)rr^rrr)r"rruthenrrrrneartimeWs  zTimeContainer.neartimecCs*|t|\}}|j|s |S|j|S)z!Returns the item at the latest time that is not later than "when." @param when the time of interest @param default If there is no data assigned at that time then the given default is returned.)rrrr)r"rr|rrrrgetks zTimeContainer.getc Cs2||\}}z|j||WSd|j|<XdS)z!Finds the latest time that is not later than "when" in self._times. Assigns "val" to that time. @param when the time of interest @param val the value to assign to that timeTN)rrr __setitem__)r"rvalrrrrrvszTimeContainer.__setitem__ccs.tt|jD]}|j|r|j|VqdS)z!Iterates over all data.Nrbrrrrr"r.rrr__iter__s zTimeContainer.__iter__ccs.tt|jD]}|j|r|j|VqdS)z7!Iterates over data for all known times that have data.Nrrrrr itervaluess zTimeContainer.itervaluesccs.tt|jD]}|j|r|j|VqdS)z(!Iterates over all times that have data.Nrbrrrrrrriterkeyss zTimeContainer.iterkeysccs8tt|jD]$}|j|r|j||j|fVqdS)zw!Iterates over all known times that have data, returning a tuple containing the time and the data at that time.Nrrrrr iteritemss zTimeContainer.iteritemsccs>t|j}t|D]&}||d}|j|r|j|VqdS)zH!Iterates over all known times that have data, in reverse order.r(N)rrrbrr)r"rr.jrrr __reversed__s     zTimeContainer.__reversed__cCs&||\}}d|j|<d|j|<dS)z!Finds the latest time that is not later than "when" and removes the data it is mapped to. Later calls to __getitem__, get, __iter__ and so on, will not return any data for this time. @param when the time of disinterestFN)rrrr"rrrrr __delitem__s zTimeContainer.__delitem__c CsLz||\}}|j|WStjjk rF}z WYdSd}~XYnXdS)z!Finds the latest time that is not later than "when" in self._times. Returns True if there is data mapped to that time and False otherwise. @param when the time of interestFN)rrhwrf exceptionsr)r"rrrrrrr __contains__s  zTimeContainer.__contains__cCs t|jS)z,!Returns the number of times that have data.)rrrrrr__len__szTimeContainer.__len__ccs|jD] }|VqdS)z.!Iterates over all times in this TimeContainerNr)r"trrrrzs zTimeContainer.timesccs.tt|jD]}|j|r|j|VqdS)z@!Iterates over all times in this TimeContainer that map to data.Nrrrrr datatimess zTimeContainer.datatimescs>fdddfddttjD}djj|fS)z0!Returns a string representation of this object.csBj|}|d}j|r4d|tj|fSd|fSdS)Nz %Y%m%d-%H%M%Sz%s:%sz%s:(unassigned))rstrftimerrr)r.rstrrridxstrs    z%TimeContainer.__str__..idxstrz, csg|] }|qSrrrXr.)rrrr]sz)TimeContainer.__str__..z%s(%s))joinrbrrr%r)r"contentsr)rr"r__str__s "zTimeContainer.__str__ccs>t|j}t|D]&}||d}|j|r|j|VqdS)zY!Iterates in reverse order over all times in this TimeContainer that map to data.r(N)rrrbr)r"rr.rrrrdatatimes_reverseds     z TimeContainer.datatimes_reversed)N)N)rrrrrrrpropertyrrrrrrrrrrrrrrrzrrrrrrrrs0         rc@s"eZdZdZdddZddZdS)raL!A time-indexed array that can only handle equally spaced times. This implements an array-like object that uses time as an index. It recognizes only discrete times as specified by a start, an end and a timestep. The end is only used as a guideline: if it does not lie exactly on a timestep, the next timestep end is used. Note that all methods in this class have the same meaning as the built-in list class, but accept times as input, except if specified otherwise. In all cases, the time "index" is anything accepted by to_datetime_rel(...,self.start).Ncst|_t|j_t|_ttjjjd}tjfddt |D|dj_ j_ _ dS)a!TimeArray constructor. @param start,end,timestep Specifies the equally-spaced times. @param init The initializer for data values. This is typically a constructor (dict, list, etc.) or a function that calls a constructor.r(csg|]}jt|qSr)_startrrrRr"rrr]sz&TimeArray.__init__..rN) rrr_endr _timestepr_rrrbr r!r)r"r r!rrrrrrrs $zTimeArray.__init__cCsjt||j}tt||j|j}|dks:|t|jkr\td||j|j f|j ||fS)!Returns the index of the specified time in the internal storage arrays or raises NotInTimespan if the time is not in the timespan. @param when Anything accepted by to_datetime_rel(when,self._start)rz%s: not in range [%s,%s]) rrr_rrrrrctimerrrrrrrs zTimeArray.index_of)Nrrrrrrrrrrrs c@s"eZdZdZdddZddZdS)rad!Maps from an ordered list of times to arbitrary data. A TimeMapping is a mapping from an ordered list of times to a set of data. The full set of possible times is given in the constructor: it is not possible to add new times to the mapping after creation of the TimeMapping. A TimeMapping is more expensive than a TimeArray since every [] lookup (__getitem__) has to do a binary search. However, it is more general since the times do not have to be exactly at an interval. Note that a TimeArray can replace a TimeMapping if a small enough timestep (the greatest common factor) is used since most of the TimeContainer methods only work on the indices that have data. However, if some timespans have dense timesteps and the rest are sparse, there may be significant memory and CPU savings in using a TimeMapping.NcCs*tddt|D}tj|||ddS)a!TimeMapping constructor @param times A list of times, which will be converted with to_datetime and sorted. @param init The initializer for data values. This is typically a constructor (dict, list, etc.) or a function that calls a constructor.cSsg|] }t|qSr)rrWrrrr]"sz(TimeMapping.__init__..rN)rysetrr)r"rzrrrrrszTimeMapping.__init__c Cst|}t|j}|dkr"tdd}|j|}|d}|j|}||krdtdt|t|fn||krz|j||fS||dkr||d}|j|}||kr||}}qz||}}qz|j||fS)rrz6TimeMapping is empty: no data is in an empty timespan.r(z:Time %s is not earlier than first time (%s) in TimeMapping)rrrrr) r"rriearlyearlyilatelateimiddlemiddlerrrr$s2        zTimeMapping.index_of)Nrrrrrr s )N)NNFrv)NF)NT)!r__all__rqr3rprr` Exceptionrrrrrr'objectrr rrrrr rrrrr r r rrrrrrrrsP (M & 1 2 1 E9