ó V°"`c@@sýdZddlmZddlmZddlmZddlZddlZddlZddlZyIddl m Z ddl m Z ddl m Z mZeZd „ZWnVek rdd lm Z ddlm Z ddlm Z mZeZd „ZnXd d lmZmZmZmZd dlmZd dlmZmZmZdZdZ d„Z!de"fd„ƒYZ#d„Z$d„Z%d„Z&de'e'd„Z(d„Z)de*fd„ƒYZ+de+fd„ƒYZ,de+fd „ƒYZ-dS(!ucModel description Model description is the representation of the various externals included in the model. It processes in input data structure, and converts it into a standard interface that is used by the rest of the system. To maintain backward compatibility, externals description files should follow semantic versioning rules, http://semver.org/ i(tabsolute_import(tunicode_literals(tprint_functionN(tSafeConfigParser(tMissingSectionHeaderError(tNoSectionErrort NoOptionErrorcC@s |jdƒS(u%convert strings into unicode uutf-8(tdecode(ttext((s[/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/externals_description.pytconfig_string_cleaner$s(t ConfigParsercC@s|S(ugPython3 already uses unicode strings, so just return the string without modification. ((R((s[/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/externals_description.pyR 0si(tprintlogt fatal_errort str_to_booltexpand_local_url(texecute_subprocess(t EMPTY_STRtPPRINTERtVERSION_SEPERATORuexternals_descriptionuschema_versioncC@s)tjj|ƒ}dj|ƒ}tj|ƒtdj|ƒƒtjj||ƒ}tjj|ƒs«|j ƒdkrŒdj|ƒ}ndj||ƒ}t |ƒnd}|t j krÒt||ƒ}n4y tƒ}|j|ƒ|}Wntk rnX|dkr%d}t |ƒn|S(uaRead a file containing an externals description and create its internal representation. uIn directory : {0}u+Processing externals description file : {0}unoneuKINTERNAL ERROR: Attempt to read externals file from {0} when not configuredusERROR: Model description file, "{0}", does not exist at path: {1} Did you run from the root of the source tree?uUnknown file format!N(tostpathtabspathtformattloggingtinfoR tjointexiststlowerR tNonetExternalsDescriptiontGIT_SUBMODULES_FILENAMEtread_gitmodules_filet config_parsertreadR(troot_dirt file_nametmsgt file_pathtexternals_descriptiontconfig((s[/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/externals_description.pytread_externals_description_fileBs2          t LstripReadercB@sGeZdZd„Zd„Zdd„Zd„Zd„Zd„ZRS(uHLstripReader formats .gitmodules files to be acceptable for configparsercC@sst|dƒ}|jƒ}WdQXtƒ|_t|ƒ|_d|_x$|D]}|jj|jƒƒqOWdS(Nuri( topent readlinestlistt_linestlent _num_linest_indextappendtlstrip(tselftfilenametinfiletlinestline((s[/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/externals_description.pyt__init__ls   cC@s|jS(u,Return all the lines from this object's file(R-(R3((s[/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/externals_description.pyR+usiÿÿÿÿcC@sWy|jƒ}Wntk r)d}nX|dkrSt|ƒ|krS|d|!S|S(u6Format and return the next line or raise StopIterationui(tnextt StopIterationR.(R3tsizeR7((s[/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/externals_description.pytreadlineys   cC@s d|_|S(uBegin an iterationi(R0(R3((s[/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/externals_description.pyt__iter__…s cC@s=|j|jkrt‚n|jd|_|j|jdS(u+Return the next line or raise StopIterationi(R0R/R:R-(R3((s[/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/externals_description.pyR9Šs cC@s |jƒS(N(R9(R3((s[/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/externals_description.pyt__next__’s( t__name__t __module__t__doc__R8R+R<R=R9R>(((s[/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/externals_description.pyR)js   c C@sãtjƒ}tj|ƒdddg}t|dtƒ}i}|jdƒ}x€|D]x}|rV|d}|djdƒ}t|ƒd krž|d } nd } i|dd 6|d6| d 6||d}|oL||k}|r7|ooˆ||||ƒ}q7q7W|o‚|}nt|t|ƒƒ}|s tdƒt|tƒråˆ||ddƒˆ||ddƒq tdj|t|ƒƒƒtdj|t|ƒƒƒn|S(uqCompare a data structure against a schema and validate all required fields are present. u Unmatched schema and input:uschemauinputu schema = {0} ({1})u input = {0} ({1})(RŸRER­R€R®R R(tschematdatatis_validtin_reftvalidtkey(R³tvalidate_data_struct(s[/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/externals_description.pyRºs(   u(ERROR: source for "{0}" did not validateN(t_source_schemaRRœRR (R3R¡R¸R$((R³Rºs[/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/externals_description.pyRvs # N("R?R@RAR›R—R”RdtNAMER`RaRRbRcR–R•R“R’RR‘t _V1_TREE_PATHt_V1_ROOTt_V1_TAGt _V1_BRANCHt_V1_REQ_SOURCERER»RR8RŠRŽR‹RŒRžR(((s[/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/externals_description.pyRCsL      e < RmcB@seZdZdd„ZRS(u²Create a externals description object from a dictionary using the API representations. Primarily used to simplify creating model description files for unit testing. cC@sžtj|ƒd|_d|_d|_d|_d|_d|_|jƒ|rƒx-|j ƒD]}||kr`||=q`q`Wn|j |ƒ|j ƒdS(u@Parse a native dictionary into a externals description. iiN( RR8RR‚RƒR„R…R†RŠROtupdateRŽ(R3RpRkR¹((s[/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/externals_description.pyR8Âs          N(R?R@RARR8(((s[/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/externals_description.pyRm»sRocB@s8eZdZddd„Zed„ƒZdd„ZRS(u^Create a externals description object from a config_parser object, schema version 1. cC@s„tj|d|ƒd|_d|_d|_t|ƒ\|_|_|_|j ƒ|j |ƒ|j |d|ƒ|j ƒdS(usConvert the config data into a standardized dict that can be used to construct the source objects RliiRkN( RR8RR‚RƒRnR„R…R†RŠt_remove_metadatat _parse_cfgRŽ(R3RpRkRl((s[/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/externals_description.pyR8Üs     cC@s|jtƒdS(u¯Remove the metadata section from the model configuration file so that it is simpler to look through the file and construct the externals description. N(tremove_sectionRe(Rp((s[/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/externals_description.pyRÃìsc C@sctd„}xP|jƒD]B}t|jƒjƒƒ}|rO||krOqni||<||j||j|ƒƒƒi|||j<||jƒj ƒ}x¸|D]°}||j krôt |j |t ƒrWt |||ƒ|||sF       " (+   J %ÿy