U g_@sddlZddlZddlZddlZddlZddlTddlmZmZddlZ ddl Z ddl m m Z ddlZddlmZddlmZGdddeZdS)N)*)datetime timedelta) TimeSlice) Observationc@s8eZdZdZddZddZddZdd Zd d Zd S) USGS_Observationz3 Store one USGS WaterML2.0 data cCsX||_|dr||n8|dr2||n"|drH||n td|dS)ze Initialize the USGS_Observation object with a given filename z.jsonz.xmlz.csvzFATAL ERROR: Unknow file type: N)sourceendswithloadJSON loadWaterML2loadCSV RuntimeError)selfZwaterml2orcsvfilenamerT/lfs/h1/ops/prod/packages/nwm.v3.0.13/ush/usgs_download/analysis/USGS_Observation.py__init__#s      zUSGS_Observation.__init__c Cszt|}t|}|ddd}|dd|_|dddd|_|ddd |_|jd krld }n"|jd kr|d }ntd|jdd |_tt|dD]}|d|d}t|}|dkrt |_ |d|ddd} |d|ddd} |d|d|dd} t j | tjjddt j | tjjddf|_td|D](} |d|d| d} | rb| sb|d|d| d}|d|d| d}d}t|dkr|d|d| dd} t|dkr|d|d| dd}|dkrFd|j t j | tjjdd<nD|t||| |}t|||f|j t j | tjjdd<qbq|j|_|j|_|j|_|j|_|j |_Wn@tk r}z tdt|d|dW5d}~XYnXdS)z Read real-time stream flow data from a given Json file by parsing the JSON file Input: jsonfilename - the Json filename valueZ timeSeriesrZ sourceInfoZsiteNameZsiteCodevariableunitZunitCodeft3/s!?m3/s?FATAL ERROR: Unit  is not known. valuesZ qualifierZ qualifierCodedateTimeNtzinfo qualifiers-999999g~.rzWARNING: parsing JSON error: : skipping ...)openjsonload _stationName _stationID_unitr rangelendict_timeValueQualitydateutilparserparse astimezonepytzutcreplace _obvPeriodisspacecalculateDataQualityfloat obvPeriod stationName stationIDrtimeValueQuality Exceptionstr)rZ jsonfilenameZ json_fileZ json_dataZ timeseriesunitConvertToM3perSecZtsIndexZtsValsZlengthTS qualifier1 beginTimeendTimepoint valuetimerr qualifier2 dataqualityerrrr 3s           zUSGS_Observation.loadJSONc Csz&t|}|}|d}|jdd|_|dj|_tj|dddj t j j dd|_ |d d d d }|d j}|dj}tj| t j j ddtj| t j j ddf|_|d d ddddd}|d|_|jdkr*d} n$|jdkr)rZwaterml2filenameZobvwmlrootZ identifierZ timePeriodrCrDZuomrAZ measurementTSrBrErFrZ pointmetadatarGZmeasurementMetadataQualifiersZ isValueNullrHrIrrrr sf                    zUSGS_Observation.loadWaterML2c CsDt|_t|}t|}|D]}|d}t|d}|ddkr|d}|d}t|dkrn|d} nd} |||| } || f|jt |dd |d d <q d |jt |dd |d d <q |d |_ W5QRXt |j } | d| df|_|j |_| d|_d|_|j|_|j |_|j |_|j|_|j|_dS)z Read real-time stream flow data from a given CSV file by parsing the CSV file Input: csvfilename - the CSV filename Z X_00060_00011rr"ZX_00060_00011_cdrrr!r Ztz_cdz%Y-%m-%d %H:%M:%S %Zr#Zsite_norN)r.r/r&csvZ DictReaderr:splitr-r9rstrptimer*sortedkeysr7r)rPr+r;r<r=rr>) rZ csvfilenameZcsvfilereaderrowZdischstrZ discharger rBrGrHZtimekeysrrrr sL        zUSGS_Observation.loadCSVcCsd}|dks|dkrd}n|rt|dkr|dks:|dkr|dkrHd}q|dkrVd}q|d krdd }q|dkrrd }q|dkrd }qd}n|dks|dkrd }|S) aV Calculate the real-time data quality for given qualifier codes found in the WaterML 2.0 files. Input: value - the stream flow value qualifier1 - the first qualifier qualifier2 - the second qualifier Return: Data quality value ri_APrI2&rrJd)r-)rrrBrGZqualityrrrr9Rs(!z%USGS_Observation.calculateDataQualityN) __name__ __module__ __qualname____doc__rr r r r9rrrrrsY2r)ossystimerVloggingstringrrZdateutil.parserr0r4xml.etree.ElementTreerL ElementTreer'rrrrrrrs