ó V°"`c@@sdZddlmZddlmZddlmZddlZddlZddlmZm Z ddlm Z dd l m Z dd l mZdd lmZmZdd lmZmZmZdd lmZmZddlmZde fd„ƒYZdS(u,Class for interacting with git repositories i(tabsolute_import(tunicode_literals(tprint_functionNi(t EMPTY_STRtLOCAL_PATH_INDICATOR(tVERBOSITY_VERBOSE(t Repository(tExternalStatus(tExternalsDescriptiontgit_submodule_status(texpand_local_urltsplit_remote_urlt is_remote_url(t fatal_errortprintlog(texecute_subprocesst GitRepositorycB@sßeZdZd„Zd„Zd„Zd(d„Zd„Zd„Z d„Z d„Z d „Z d „Z d „Zd „Zd „Zd(d„Zd„Zd„Zd(d„Zd„Zd„Zd„Zd„Zd„Zed„ƒZed„ƒZed„ƒZed„ƒZed„ƒZed„ƒZ ed„ƒZ!ed„ƒZ"ed„ƒZ#ed „ƒZ$ed!„ƒZ%ed(d"„ƒZ&ed#„ƒZ'ed$„ƒZ(ed%„ƒZ)ed&„ƒZ*ed'„ƒZ+RS()umClass to represent and operate on a repository description. For testing purpose, all system calls to git should: * be isolated in separate functions with no application logic * of the form: - cmd = ['git', ...] - value = execute_subprocess(cmd, output_to_caller={T|F}, status_to_caller={T|F}) - return value * be static methods (not rely on self) * name as _git_subcommand_args(user_args) This convention allows easy unit testing of the repository logic by mocking the specific calls to return predefined results. cC@s)tj|||ƒd|_d|_dS(u4 Parse repo (a XML element). N(Rt__init__tNonet _gitmodulest_submods(tselftcomponent_nametrepo((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyR(s cC@sÈtjj||ƒ}tjj|ƒ}|r=tj|ƒ sD| rZ|j|||ƒn|j|||ƒtjj|tjƒ}tjj|ƒr²||_ t |ƒ|_ nd|_ d|_ dS(u  If the repo destination directory exists, ensure it is correct (from correct URL, correct branch or tag), and possibly update the source. If the repo destination directory does not exist, checkout the correct branch or tag. N( tostpathtjointexiststlistdirt _clone_repot _checkout_refRtGIT_SUBMODULES_FILENAMERR RR(Rt base_dir_patht repo_dir_namet verbosityt recursivet repo_dir_pathtrepo_dir_existstgmpath((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pytcheckout5s     cC@s9|j||ƒtjj|ƒr5|j||ƒndS(u  If the repo destination directory exists, ensure it is correct (from correct URL, correct branch or tag), and possibly update the source. If the repo destination directory does not exist, checkout the correct branch or tag. N(t _check_syncRRRt_status_summary(RtstatR$((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pytstatusKscC@s[|dk rTtjj|tjƒ}tjj|ƒrT||_t|ƒ|_ qTn|jS(N( RRRRRRRRR R(Rt repo_pathR&((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pytsubmodules_fileVs    cC@s@tjƒ}tj|ƒ|j|j||ƒtj|ƒdS(uDPrepare to execute the clone by managing directory location N(Rtgetcwdtchdirt _git_clonet_url(RR R!R"tcwd((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyRes  c C@s t}|jƒ\}}|r-|}t}n|s]|jƒ\}}|r]|}t}q]n|s|jƒ\}}|r|}t}qn|sœd}n|S(uSDetermine the *name* associated with HEAD. If we're on a branch, then returns the branch name; otherwise, if we're on a tag, then returns the tag name; otherwise, returns the current hash. Returns an empty string if no reference can be determined (e.g., if we're not actually in a git repository). u(tFalset_git_current_branchtTruet_git_current_tagt_git_current_hash( Rt ref_foundt branch_foundt branch_namet current_reft tag_foundttag_namet hash_foundt hash_name((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyt _current_refms$    cC@sktjj|ƒs!tj|_nFtjj|dƒ}tjj|ƒsWtj|_n|j||ƒdS(uÑDetermine whether a git repository is in-sync with the model description. Because repos can have multiple remotes, the only criteria is whether the branch or tag is the same. u.gitN( RRRRt STATUS_ERRORt sync_stateRtUNKNOWNt_check_sync_logic(RR*R$tgit_dir((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyR(“s c C@sfd„}tjƒ}tj|ƒ|jƒ\}}|jr—|jtkrX|j}qÝ|jƒ}|sdj|jƒ}qÝdj||jƒ}nF|j r¬|j }n1|j rÁ|j }ndj|j ƒ} t | ƒ|j ƒ|_tj|ƒ|_|tkrtj|_n<|j|ƒ\} } | rCtj|_n||| ƒ|_tj|ƒdS(uÔCompare the underlying hashes of the currently checkout ref and the expected ref. Output: sets the sync_state as well as the current and expected ref in the input status object. cS@s%||krtj}n tj}|S(u3Compare the current and expected ref. (Rt STATUS_OKtMODEL_MODIFIED(R;t expected_refR+((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyt compare_refs²s   uunknown_remote/{0}u{0}/{1}u2In repo "{0}": none of branch, hash or tag are setN(RR.R/R7t_branchR1Rt_determine_remote_nametformatt_hasht_tagt_nameR R@tcurrent_versiontcopytdeepcopytexpected_versionRRRCRBt_git_revparse_commitRG( RR*R$RIR2t_R;RHt remote_nametmsgtrevparse_statustexpected_ref_hash((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyRDªs8            cC@s|jƒ}|jƒ}d}xk|D]c}|jƒ}|sCq%n|jƒ}|djƒ}|djƒ}|j|kr%|}Pq%q%W|S(uˆReturn the remote name. Note that this is for the *future* repo url and branch, not the current working copy! uii(t_git_remote_verboset splitlineststriptsplitR1(Rt git_outputRVtlinetdatatnameturl((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyRKìs     cC@s¡tj|jƒ}t|ƒr-t|ƒ}nt||jƒ}|jdƒ}|d}|d}d}x |D]}|j|dƒ}qoWdj ||ƒ}|S(u~The url specified in the externals description file was not known to git. We need to add it, which means adding a unique and safe name.... The assigned name needs to be safe for git to use, e.g. can't look like a path 'foo/bar' and work with both remote and local paths. Remote paths include but are not limited to: git, ssh, https, github, gitlab, bitbucket, custom server, etc. Local paths can be relative or absolute. They may contain shell variables, e.g. ${REPO_ROOT}/repo_name, or username expansion, i.e. ~/ or ~someuser/. Relative paths must be at least one layer of redirection, i.e. container/../ext_repo, but may be many layers deep, e.g. container/../../../../../ext_repo NOTE(bja, 2017-11) The base name below may not be unique, for example if the user has local paths like: /path/to/my/repos/nice_repo /path/to/other/repos/nice_repo But the current implementation should cover most common use cases for remotes and still provide usable names. u/iÿÿÿÿiþÿÿÿu!@#$%^&*()[]{}\/,;~uu{0}_{1}( RQRRR1R R R ROR]treplaceRL(RRbt repo_namet base_nametunsafe_characterstunsafeRV((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyt_create_remote_names    cC@sbtjƒ}tj|ƒ|jjƒtkrA|j||ƒn|j||ƒtj|ƒdS(uŽCheckout the user supplied reference if is True, recursively initialize and update the repo's submodules N(RR.R/R1R\Rt_checkout_local_reft_checkout_external_ref(Rtrepo_dirR"t submodulesR2((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyR2s   cC@sW|jr|j}n|jr*|j}n |j}|j|ƒ|j|||ƒdS(uCheckout the reference considering the local repo only. Do not fetch any additional remotes or specify the remote when checkout out the ref. if is True, recursively initialize and update the repo's submodules N(RNRJRMt_check_for_valid_reft_git_checkout_ref(RR"Rltref((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyRiAs      cC@s¹|jr|j}n|jr*|j}n |j}|jƒ}|sg|jƒ}|j||jƒn|j|ƒ|j||ƒ|jr¢dj ||ƒ}n|j |||ƒdS(u™Checkout the reference from a remote repository if is True, recursively initialize and update the repo's submodules u{0}/{1}N( RNRJRMRKRht_git_remote_addR1t _git_fetchRmRLRn(RR"RlRoRV((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyRjRs         c C@sÃ|j|ƒ}|j||ƒ}|j|ƒ}|p?|p?|}|spdj|j||jƒ}t|ƒn|r¿|j||ƒ\}}|s¿dj|j|j|ƒ}t|ƒq¿n|S(uŸTry some basic sanity checks on the user supplied reference so we can provide a more useful error message than calledprocess error... u›In repo "{0}": reference "{1}" does not appear to be a valid tag, branch or hash! Please verify the reference name (e.g. spelling), is available from: {2} uIn repo "{0}": tag "{1}" {2}( t _ref_is_tagt_ref_is_brancht _ref_is_hashRLROR1R t_is_unique_tagRN( RRoRVtis_tagt is_branchtis_hashtis_validRWt is_unique_tag((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyRmms cC@sº|j|ƒ}|j||ƒ}|j|ƒ}d}t}|rX| rXd}t}nX|rs|rsd}t}n=| r|rd}t}n!|r¤d}t}n d}t}||fS(uYVerify that a reference is a valid tag and is unique (not a branch) Tags may be tag names, or SHA id's. It is also possible that a branch and tag have the some name. Note: values returned by git_showref_* and git_revparse are shell return codes, which are zero for success, non-zero for error! uuis okulis both a branch and a tag. git may checkout the branch instead of the tag depending on your version of git.uÆis a branch, and not a tag. If you intended to checkout a branch, please change the externals description to be a branch. If you intended to checkout a tag, it does not exist. Please check the name.uXdoes not appear to be a valid tag, branch or hash! Please check the name and repository.(RrRsRtR3R5(RRoRVRvRwRxRWRz((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyRuˆs(        cC@s.t}|j|ƒ}|dkr*t}n|S(uÚVerify that a reference is a valid tag according to git. Note: values returned by git_showref_* and git_revparse are shell return codes, which are zero for success, non-zero for error! i(R3t_git_showref_tagR5(RRoRvtvalue((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyRr´s   cC@sUt}t}|r'|j||ƒ}n|j|ƒ}t}|sH|rQt}n|S(ubVerify if a ref is any kind of branch (local, tracked remote, untracked remote). (R3t_ref_is_remote_brancht_ref_is_local_branchR5(RRoRVt local_brancht remote_branchRw((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyRsÁs  cC@s.t}|j|ƒ}|dkr*t}n|S(u€Verify that a reference is a valid branch according to git. show-ref branch returns local branches that have been previously checked out. It will not necessarily pick up untracked remote branches. Note: values returned by git_showref_* and git_revparse are shell return codes, which are zero for success, non-zero for error! i(R3t_git_showref_branchR5(RRoRwR|((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyR~Ñs   cC@s1t}|j||ƒ}|dkr-t}n|S(u€Verify that a reference is a valid branch according to git. show-ref branch returns local branches that have been previously checked out. It will not necessarily pick up untracked remote branches. Note: values returned by git_showref_* and git_revparse are shell return codes, which are zero for success, non-zero for error! i(R3t_git_lsremote_branchR5(RRoRVRwR|((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyR}ãs   cC@s4t}|j|ƒ\}}|dkr0t}n|S(u+Verify that a reference is a valid commit according to git. This could be a tag, branch, sha1 id, HEAD and potentially others... Note: values returned by git_showref_* and git_revparse are shell return codes, which are zero for success, non-zero for error! i(R3RTR5(RRot is_commitR|RU((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyt_ref_is_commitõs   cC@sLt}|j|ƒ\}}|dkrH|jƒj|ƒrHt}qHn|S(uVerify that a reference is a valid hash according to git. Git doesn't seem to provide an exact way to determine if user supplied reference is an actual hash. So we verify that the ref is a valid commit and return the underlying commit hash. Then check that the commit hash begins with the user supplied string. Note: values returned by git_showref_* and git_revparse are shell return codes, which are zero for success, non-zero for error! i(R3RTR\t startswithR5(RRoRxR+R^((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyRts   cC@sutjƒ}tj|ƒ|jƒ}|j|ƒ}|rItj|_n tj|_|j ƒ|_ tj|ƒdS(u>Determine the clean/dirty status of a git repository N( RR.R/t_git_status_porcelain_v1zt_status_v1z_is_dirtyRtDIRTYt clean_stateRFt_git_status_verboset status_output(RR*R$R2R^tis_dirty((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyR)s    cC@st}|rt}n|S(uÐParse the git status output from --porcelain=v1 -z and determine if the repo status is clean or dirty. Dirty means: * modified files * missing files * added files * removed * renamed * unmerged Whether untracked files are considered depends on how the status command was run (i.e., whether it was run with the '-u' option). NOTE: Based on the above definition, the porcelain status should be an empty string to be considered 'clean'. Of course this assumes we only get an empty string from an status command on a clean checkout, and not some error condition... Could alse use 'git diff --quiet'. (R3R5(R^RŒ((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyR‡+s cC@s5tjdƒ\}}| }|s+d}n||fS(u8Return the full hash of the currently checked-out version. Returns a tuple, (hash_found, hash), where hash_found is a logical specifying whether a hash was found for HEAD (False could mean we're not in a git repository at all). (If hash_found is False, then hash is ''.) uHEADu(RRT(R+R^R>((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyR7Ks  cC@s_dddddg}t|dtdtƒ\}}| }|rO|jƒ}nd}||fS( uDetermines the name of the current branch. Returns a tuple, (branch_found, branch_name), where branch_found is a logical specifying whether a branch name was found for HEAD. (If branch_found is False, then branch_name is ''.) ugitu symbolic-refu--shortu-quHEADtoutput_to_callertstatus_to_calleru(RR5R\(tcmdR+R^R9((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyR4Zs cC@s_dddddg}t|dtdtƒ\}}| }|rO|jƒ}nd}||fS( u Determines the name tag corresponding to HEAD (if any). Returns a tuple, (tag_found, tag_name), where tag_found is a logical specifying whether we found a tag name corresponding to HEAD. (If tag_found is False, then tag_name is ''.) ugitudescribeu --exact-matchu--tagsuHEADRRŽu(RR5R\(RR+R^R<((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyR6ms  cC@s4dddddj|ƒg}t|dtƒ}|S(u‡Run git show-ref check if the user supplied ref is a tag. could also use git rev-parse --quiet --verify tagname^{tag} ugitushow-refu--quietu--verifyu refs/tags/{0}RŽ(RLRR5(RoRR+((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyR{s cC@s4dddddj|ƒg}t|dtƒ}|S(ufRun git show-ref check if the user supplied ref is a local or tracked remote branch. ugitushow-refu--quietu--verifyurefs/heads/{0}RŽ(RLRR5(RoRR+((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyRŒs cC@s.dddd||g}t|dtƒ}|S(urRun git ls-remote to check if the user supplied ref is a remote branch that is not being tracked ugitu ls-remoteu --exit-codeu--headsRŽ(RR5(RoRVRR+((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyR‚—s  cC@sUdddddj|dƒg}t|dtdtƒ\}}|jƒ}||fS( ubRun git rev-parse to detect if a reference is a SHA, HEAD or other valid commit. ugitu rev-parseu--quietu--verifyu{0}^{1}u{commit}RŽR(RLRR5R\(RoRR+R^((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyRT¢s   cC@s+dddddg}t|dtƒ}|S(uüRun git status to obtain repository information. This is run with '--untracked=no' to ignore untracked files. The machine-portable format that is guaranteed not to change between git versions or *user configuration*. ugitustatusu--untracked-files=nou --porcelainu-zR(RR5(RR^((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyR†¯s cC@s"ddg}t|dtƒ}|S(uERun the git status command to obtain repository information. ugitustatusR(RR5(RR^((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyRŠ½s cC@s%dddg}t|dtƒ}|S(uERun the git remote command to obtain repository information. ugituremoteu --verboseR(RR5(RR^((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyRZÅscC@s@|dkrtj}ntjj|tjƒ}tjj|ƒS(u‘Return True iff the repository at (or the current directory if is None) has a '.gitmodules' file N(RRRRRRR(R$tfname((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pythas_submodulesÍs     cC@s‡dddg}d}|j||gƒ|tkrStdjdj|ƒƒƒnt|ƒ|dk rƒtj|ƒt|ƒndS(uDRun git clone for the side effect of creating a repository. ugitucloneu--quietu {0}u N( RtextendRRRLRRRR/(RbR!R"Rtsubcmd((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyR0ßs    cC@s#ddd||g}t|ƒdS(uJRun the git remote command for the side effect of adding a remote ugituremoteuaddN(R(RaRbR((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyRpîscC@s#dddd|g}t|ƒdS(uKRun the git fetch command for the side effect of updating the repo ugitufetchu--quietu--tagsN(R(RVR((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyRqõscC@saddd|g}|tkr=tdjdj|ƒƒƒnt|ƒ|r]tj|ƒndS(uÂRun the git checkout command for the side effect of updating the repo Param: ref is a reference to a local or remote object in the form 'origin/my_feature', or 'tag1'. ugitucheckoutu--quietu {0}u N(RRRLRRRt_git_update_submodules(RoR"RlR((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyRnüs   cC@sftjjtjƒrbdddddg}|tkrUtdjdj|ƒƒƒnt |ƒndS( uaRun git submodule update for the side effect of updating this repo's submodules. ugitu submoduleuupdateu--initu --recursiveu {0}u N( RRRRRRRRLRR(R"R((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyR” s  N(,t__name__t __module__t__doc__RR'R+RR-RR@R(RDRKRhRRiRjRmRuRrRsR~R}R„RtR)t staticmethodR‡R7R4R6R{RR‚RTR†RŠRZR‘R0RpRqRnR”(((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyRsR    &  B  0     ,           (R—t __future__RRRRQRtglobal_constantsRRRt repositoryRtexternals_statusRtexternals_descriptionRR tutilsR R R R RRR(((sT/gpfs/hps/nco/ops/nwpara/hiresw.v8.0.1/sorc/manage_externals/manic/repository_git.pyts