ó V°"`c@@s|dZddlmZddlmZddlmZddlZddlZddlZddlZddl Z ddl m Z ddl m Z ddl mZdd lmZdd lmZmZdd lmZmZe jd krced dƒedje jdƒƒedjejd„e jdd!Dƒƒƒƒed dƒe jdƒned„Zd„Z dS(uÜ Tool to assemble repositories represented in a model-description file. If loaded as a module (e.g., in a component's buildcpp), it can be used to check the validity of existing subdirectories and load missing sources. i(tabsolute_import(tunicode_literals(tprint_functionN(tcreate_externals_description(tread_externals_description_file(tcheck_safe_to_update_repos(t SourceTree(tprintlogt fatal_error(tVERSION_SEPERATORt LOG_FILE_NAMEiiFu*u%ERROR: {0} requires python >= 2.7.x. u*It appears that you are running python {0}cc@s|]}t|ƒVqdS(N(tstr(t.0tx((sN/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/checkout.pys siic C@s d}d}tjd|d|dtjƒ}|jdddd d ƒ|jd d dd ddd dƒ|jdddddtd dƒ|jdddddtd dƒ|jddddddd dƒ|jddddtd dƒ|jd ddd d!ƒ|jd"d#dddtd d$ƒ|jƒ}|jd%d&d'ddd d(ƒ|jd)d&d'dd*dtd d+ƒ|r|j|ƒ}n |jƒ}|S(,u©Process the command line arguments Params: args - optional args. Should only be used during systems testing. Returns: processed command line arguments u; %(prog)s manages checking out groups of externals from revision control based on an externals description file. By default only the required externals are checkout out. Running %(prog)s without the '--status' option will always attempt to synchronize the working copy to exactly match the externals description. u ``` NOTE: %(prog)s *MUST* be run from the root of the source tree it is managing. For example, if you cloned a repository with: $ git clone git@github.com/{SOME_ORG}/some-project some-project-dev Then the root of the source tree is /path/to/some-project-dev. If you obtained a sub-project via a checkout of another project: $ git clone git@github.com/{SOME_ORG}/some-project some-project-dev and you need to checkout the sub-project externals, then the root of the source tree remains /path/to/some-project-dev. Do *NOT* run %(prog)s from within /path/to/some-project-dev/sub-project The root of the source tree will be referred to as `${SRC_ROOT}` below. # Supported workflows * Checkout all required components from the default externals description file: $ cd ${SRC_ROOT} $ ./manage_externals/%(prog)s * To update all required components to the current values in the externals description file, re-run %(prog)s: $ cd ${SRC_ROOT} $ ./manage_externals/%(prog)s If there are *any* modifications to *any* working copy according to the git or svn 'status' command, %(prog)s will not update any external repositories. Modifications include: modified files, added files, removed files, or missing files. To avoid this safety check, edit the externals description file and comment out the modified external block. * Checkout all required components from a user specified externals description file: $ cd ${SRC_ROOT} $ ./manage_externals/%(prog)s --externals my-externals.cfg * Status summary of the repositories managed by %(prog)s: $ cd ${SRC_ROOT} $ ./manage_externals/%(prog)s --status ./cime s ./components/cism ./components/mosart e-o ./components/rtm M ./src/fates e-o ./tools/PTCLM where: * column one indicates the status of the repository in relation to the externals description file. * column two indicates whether the working copy has modified files. * column three shows how the repository is managed, optional or required Column one will be one of these values: * s : out-of-sync : repository is checked out at a different commit compared with the externals description * e : empty : directory does not exist - %(prog)s has not been run * ? : unknown : directory exists but .git or .svn directories are missing Column two will be one of these values: * M : Modified : modified, added, deleted or missing files * : blank / space : clean * - : dash : no meaningful state, for empty repositories Column three will be one of these values: * o : optional : optionally repository * : blank / space : required repository * Detailed git or svn status of the repositories managed by %(prog)s: $ cd ${SRC_ROOT} $ ./manage_externals/%(prog)s --status --verbose # Externals description file The externals description contains a list of the external repositories that are used and their version control locations. The file format is the standard ini/cfg configuration file format. Each external is defined by a section containing the component name in square brackets: * name (string) : component name, e.g. [cime], [cism], etc. Each section has the following keyword-value pairs: * required (boolean) : whether the component is a required checkout, 'true' or 'false'. * local_path (string) : component path *relative* to where %(prog)s is called. * protoctol (string) : version control protocol that is used to manage the component. Valid values are 'git', 'svn', 'externals_only'. Switching an external between different protocols is not supported, e.g. from svn to git. To switch protocols, you need to manually move the old working copy to a new location. Note: 'externals_only' will only process the external's own external description file without trying to manage a repository for the component. This is used for retrieving externals for standalone components like cam and ctsm which also serve as sub-components within a larger project. If the source root of the externals_only component is the same as the main source root, then the local path must be set to '.', the unix current working directory, e. g. 'local_path = .' * repo_url (string) : URL for the repository location, examples: * https://svn-ccsm-models.cgd.ucar.edu/glc * git@github.com:esmci/cime.git * /path/to/local/repository * . NOTE: To operate on only the local clone and and ignore remote repositories, set the url to '.' (the unix current path), i.e. 'repo_url = .' . This can be used to checkout a local branch instead of the upstream branch. If a repo url is determined to be a local path (not a network url) then user expansion, e.g. ~/, and environment variable expansion, e.g. $HOME or $REPO_ROOT, will be performed. Relative paths are difficult to get correct, especially for mixed use repos. It is advised that local paths expand to absolute paths. If relative paths are used, they should be relative to one level above local_path. If local path is 'src/foo', the the relative url should be relative to 'src'. * tag (string) : tag to checkout * hash (string) : the git hash to checkout. Only applies to git repositories. * branch (string) : branch to checkout from the specified repository. Specifying a branch on a remote repository means that %(prog)s will checkout the version of the branch in the remote, not the the version in the local repository (if it exists). Note: one and only one of tag, branch hash must be supplied. * externals (string) : used to make manage_externals aware of sub-externals required by an external. This is a relative path to the external's root directory. For example, if LIBX is often used as a sub-external, it might have an externals file (for its externals) called Externals_LIBX.cfg. To use libx as a standalone checkout, it would have another file, Externals.cfg with the following entry: [ libx ] local_path = . protocol = externals_only externals = Externals_LIBX.cfg required = True Now, %(prog)s will process Externals.cfg and also process Externals_LIBX.cfg as if it was a sub-external. * Lines beginning with '#' or ';' are comments and will be ignored. # Obtaining this tool, reporting issues, etc. The master repository for manage_externals is https://github.com/ESMCI/manage_externals. Any issues with this tool should be reported there. # Troubleshooting Operations performed by manage_externals utilities are explicit and data driven. %(prog)s will always attempt to make the working copy *exactly* match what is in the externals file when modifying the working copy of a repository. If %(prog)s is not doing what you expected, double check the contents of the externals description file or examine the output of ./manage_externals/%(prog)s --status t descriptiontepilogtformatter_classu componentstnargsu*thelpuVSpecific component(s) to checkout. By default, all required externals are checked out.u-eu --externalsu?tdefaultu Externals.cfgu9The externals description filename. Default: %(default)s.u-ou --optionaltactionu store_trueulBy default only the required externals are checked out. This flag will also checkout the optional externals.u-Su--statusu’Output the status of the repositories managed by %(prog)s. By default only summary information is provided. Use the verbose option to see details.u-vu --verboseucountiuŠOutput additional information to the screen and log file. This flag can be used up to two times, increasing the verbosity level each time.u--svn-ignore-ancestryuBy default, subversion will abort if a component is already checked out and there is no common ancestry with the new URL. This flag passes the "--ignore-ancestry" flag to the svn switch call. (This is not recommended unless you are sure about what you are doing.)u --backtraceu>DEVELOPER: show exception backtraces as extra debugging outputu-du--debuguNDEVELOPER: output additional debugging information to the screen and log file.u --loggingtdestu do_logginguDEVELOPER: enable logging.u --no-loggingu store_falseu0DEVELOPER: disable logging (this is the default)(targparsetArgumentParsertRawDescriptionHelpFormattert add_argumenttFalsetadd_mutually_exclusive_groupt parse_args(targsRRtparsert logging_grouptoptions((sN/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/checkout.pytcommandline_arguments(sBÁ       c C@se|jr1tjdtdddddtjƒntjjtj dƒ}tj d|ƒt }|j rrt }ntjjtjƒƒ}t||jƒ}t|d |jƒ}x?|jD]4}||jƒkr»td j||jƒƒq»q»Wt||d |jƒ}td d dƒ|jƒ}tdƒ|jrnxt|jƒƒD]}||j|jƒqMWnÝt|ƒ} | sõx.t|jƒƒD]}||j|jƒq“Wdjd|d|jƒ} tddƒt| ƒtddƒnV|js|j|j|ƒnx*|jD]}|j|j|d|ƒqWtdƒtj d|ƒd|fS(uÍ Function to call when module is called from the command line. Parse externals file and load required repositories or all repositories if the --all option is passed. Returns a tuple (overall_status, tree_status). overall_status is 0 on success, non-zero on failure. tree_status gives the full status *before* executing the checkout command - i.e., the status that it used to determine if it's safe to proceed with the checkout. tfilenametformatu)%(levelname)s : %(asctime)s : %(message)stdatefmtu%Y-%m-%d %H:%M:%StleveliuBeginning of %st componentsuNo component {} found in {}tsvn_ignore_ancestryuChecking status of externals: tenduu¹The external repositories labeled with 'M' above are not in a clean state. The following are two options for how to proceed: (1) Go into each external that is not in a clean state and issue either an 'svn status' or a 'git status' command. Either revert or commit your changes so that all externals are in a clean state. (Note, though, that it is okay to have untracked files in your working directory.) Then rerun {program_name}. (2) Alternatively, you do not have to rely on {program_name}. Instead, you can manually update out-of-sync externals (labeled with 's' above) as described in the configuration file {config_file}. The external repositories labeled with '?' above are not under version control using the expected protocol. If you are sure you want to switch protocols, and you don't have any work you need to save from this directory, then run "rm -rf [directory]" before re-running the checkout_externals tool. t program_namet config_fileu-iFt load_compu %s completed without exceptions.( t do_loggingtloggingt basicConfigR tDEBUGtostpathtbasenametsystargvtinfoRtoptionaltTruetabspathtgetcwdRt externalsRR&tkeysRR#RR'Rtstatustsortedtlog_status_messagetverboseRtcheckout( RR)tload_alltroot_dirt external_datatexternaltcompt source_treet tree_statustsafe_to_updatetmsg((sN/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/checkout.pytmainCsR           (!t__doc__t __future__RRRRR-R0tos.pathR3tmanic.externals_descriptionRRtmanic.externals_statusRtmanic.sourcetreeRt manic.utilsRRtmanic.global_constantsR R t hexversiontprintR#R4tjoint version_infotexittNoneR!RJ(((sN/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/checkout.pyts0      ' ÿ