U 2g @sdZddlZddlZddlZddlZddlZddlmZddl Z ddZ ddZ e j de e j d e j d e e j d d d Zd dZddZddZddZddZdddZddZedkreejdddS)a# This utility fills in a user-supplied Jinja template from either a YAML file, or command line arguments. The user configuration file and commandline arguments should be YAML-formatted. This script will support a single- or two-level YAML config file. For example: 1. expt1: date_first_cycl: !datetime 2019043000 date_last_cycl: !datetime 2019050100 cycl_freq: !!str 12:00:00 expt2: date_first_cycl: !datetime 2019061012 date_last_cycl: !datetime 2019061212 cycl_freq: !!str 12:00:00 2. date_first_cycl: !datetime 2019043000 date_last_cycl: !datetime 2019050100 cycl_freq: !!str 12:00:00 In Case 1, provide the name of the file and the section title, e.g. expt2, to the -c command line argument. Only provide the name of the file in -c option if it's configured as in Case 2 above. Supported YAML Tags: The script supports additional YAML configuration tags. !datetime Converts an input string formatted as YYYYMMDDHH[mm[ss]] to a Python datetime object !join Uses os.path.join to join a list as a path. Expected behavior: - The template file is required. Script fails if not provided. - Command line arguments in the -u setting override the -c settings. N)metacCstjj||S)z!Uses os to join a list as a path.)ospathjoinconstruct_sequence)loadernoder @/lfs/h1/ops/prod/packages/aqm.v7.0.11/ush/fill_jinja_template.pyr6srcCsR||}t|}|dks"|s4|d}t|dd|d}tj||S)zOConverts a date string with format YYYYMMDDHH[MM[SS]] to a datetime object.) z4 does not conform to input format YYYYMMDDHH[MM[SS]]z %Y%m%d%H%M%Sr)construct_scalarlen isnumeric ValueErrordtdatetimestrptime)rrvalueZval_lenmsgZ date_formatr r r to_datetime=s  rz !datetimeLoaderz!joincCs$tj|s |d}t||S)z>Checks whether a file exists, and returns the path if it does. does not exist!)rrexistsargparseArgumentTypeErrorargrr r r file_existsUs   r!c Cst|dkr$t|d}t|t|d}t|dkrD|dnd}t|d}tj|tjd}W5QRX|rz ||}Wn.tk rd|d |}t|YnX|S) z~ Checks whether the config file exists and if it contains the input section. Returns the config as a Python dict. rz4 arguments were provided for config. Only 2 allowed!rNrrzSection z does not exist in top level of ) rrrr!openyamlload SafeLoaderKeyError)r r file_name section_namefncfgr r r config_exists`s     r-cCs*tj|s |d}t|t|S)z Check to ensure that the provided config file exists. If it does, load it with YAML's safe loader and return the resulting dict. r)rrrrrr% safe_loadrr r r load_config|s   r/cCstj|tjdS)z@Load a dict string safely using YAML. Return the resulting dict.r)r%r&r')r r r r load_strsr0cCsJtjtj|}tj|r2t|tjr2|S|d}t|dS)z Check whether the path to the file exists, and is writeable. Return the path if it passes all checks, otherwise raise an error. z is not a writable path!N) rrabspathdirnamelexistsaccessW_OKrr)r Zdir_namerr r r path_oks  r6cCsztjdd}|jddddtd|jdd d d d |jd ddtd|jdddddtd|jdddddtd||S)z Function maintains the arguments accepted by this script. Please see Python's argparse documenation for more information about settings of each argument. zFill in a Rocoto XML template.) descriptionz-cz--configzhFull path to a YAML user config file, and a top-level section to use (optional).*)helpnargstypez-qz--quiet store_truezSuppress all output)actionr9z-uz --user_configz9Command-line user config options in YAML-formatted string)r9r;z-tz--xml_templatetemplatezFull path to the jinja templateT)destr9requiredr;z-oz--outxmloutxmlz(Full path to the output Rocoto XML file.)rArgumentParser add_argumentr/r0r!r6 parse_args)argvparserr r r rDsJ rDFcCsP|s td|D]*\}}|s6td|dd||||<q|sLtddS)aF Overwrites all values in dest dictionary section with key/value pairs from newdict. Does not support multi-layer update. Turn off print statements with quiet=True. Input: dest A dict that is to be updated. newdict A dict containing sections and keys corresponding to those in dest and potentially additional ones, that will be used to update the dest dict. quiet An optional boolean flag to turn off output. Output: None Result: The dest dict is updated in place. z2**************************************************z Overriding z>20z = N)printitems)r?Znewdictquietkeyrr r r update_dicts rKc Cst|}|jrt|j|_tjt|jd}|j|d}| d}| |}t |}|jdk rj|jni}|j rt||j |jdi}|D]P} || ddkrt| d|jst| dd|| || || <q|jf|} t|jd } | | W5QRXdS) z Loads a Jinja template, determines its necessary undefined variables, retrives them from user supplied settings, and renders the final result. )rN)rINULLz* does not exist in user-supplied settings!z>25z: w)rDconfigr-j2 EnvironmentFileSystemLoaderr>r get_source get_templateparserZfind_undeclared_variablesZ user_configrKrIgetr(rGrenderr$rAwrite) rEZclaenvZtemplate_sourcer>Zparsed_contentZ template_varsr,tvarsvarZ xml_contentsr+r r r fill_jinja_templates*     r\__main__r")F)__doc__rrrsysrjinja2rPrr%rradd_constructorr'r!r-r/r0r6rDrKr\__name__rEr r r r s*)  3 "/