Robbie Cannings

Resolving git submodule path problems

Robbie Cannings -
Clone a repo containing submodules

In this example I'll clone the terraform-docker-registry repo. The repo contains a few submodules, although the paths (on the git server) have changed, therefore a clone fails.

[user@linux_workstation ~]$ git clone -c http.sslVerify=false --recursive ssh://git@git:7999/myproject/terraform-docker-registry.git
Cloning into 'terraform-docker-registry'...
remote: Counting objects: 128, done.
remote: Compressing objects: 100% (99/99), done.
remote: Total 128 (delta 50), reused 0 (delta 0)
Receiving objects: 100% (128/128), 14.19 KiB | 0 bytes/s, done.
Resolving deltas: 100% (50/50), done.
Submodule 'modules/terraform-aws-common' (https://stash_server/scm/myproject/terraform-aws-common.git) registered for path 'modules/terraform-aws-common'
Submodule 'modules/terraform-intermediate' (https://stash_server/scm/myproject/terraform-intermediate.git) registered for path 'modules/terraform-intermediate'
Submodule 'provision/puppet/modules/docker' (https://stash_server/scm/myproject/puppet-docker.git) registered for path 'provision/puppet/modules/docker'
Submodule 'provision/puppet/modules/stdlib' (https://stash_server/scm/myproject/puppetlabs-stdlib.git) registered for path 'provision/puppet/modules/stdlib'
Cloning into 'modules/terraform-aws-common'...
remote: Counting objects: 160, done.
remote: Compressing objects: 100% (149/149), done.
remote: Total 160 (delta 41), reused 0 (delta 0)
Receiving objects: 100% (160/160), 21.88 KiB | 0 bytes/s, done.
Resolving deltas: 100% (41/41), done.
Submodule path 'modules/terraform-aws-common': checked out '3de51bdabdcbee2da0567381d54680b33558a968'
Cloning into 'modules/terraform-intermediate'...
remote: Counting objects: 3, done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
Submodule path 'modules/terraform-intermediate': checked out '8999cbbbafd3ef9012d745b455316cde1a082358'
Cloning into 'provision/puppet/modules/docker'...
remote: Counting objects: 191, done.
remote: Compressing objects: 100% (156/156), done.
remote: Total 191 (delta 55), reused 0 (delta 0)
Receiving objects: 100% (191/191), 22.42 KiB | 0 bytes/s, done.
Resolving deltas: 100% (55/55), done.
fatal: reference is not a tree: 99e0ab8cff49e5f63b714dd60fbb9a130e950733
Cloning into 'provision/puppet/modules/stdlib'...
remote: Counting objects: 7707, done.
remote: Compressing objects: 100% (3346/3346), done.
remote: Total 7707 (delta 3675), reused 7586 (delta 3602)
Receiving objects: 100% (7707/7707), 1.51 MiB | 0 bytes/s, done.
Resolving deltas: 100% (3675/3675), done.
Submodule path 'provision/puppet/modules/stdlib': checked out '990e1d757549a9c792cf5f7113e4d6bcd592ae3d'
Unable to checkout '99e0ab8cff49e5f63b714dd60fbb9a130e950733' in submodule path 'provision/puppet/modules/docker'
Remove faulty submodule path

I've still checked out the intiial repo (containing the submodule path references), so I can fix these.
We need to remove both the directory containing the bad path, as well as the .git directory containing the git metadata

[user@linux_workstation ~]$ cd terraform-docker-registry/
[user@linux_workstation terraform-docker-registry]$ git rm --cached provision/puppet/modules/docker/
rm 'provision/puppet/modules/docker'
[user@linux_workstation terraform-docker-registry]$ git rm -f .git/modules/provision/puppet/modules/docker/
Add the correct path
[user@linux_workstation terraform-docker-registry]$ git submodule add https://github.com/rcannings/new-submodule.git .git/modules/provision/puppet/modules/docker/
[user@linux_workstation terraform-docker-registry]$ git status
# On branch master
# Changes to be committed:
#   (use "git reset HEAD <file>..." to unstage)
#
#       deleted:    provision/puppet/modules/docker
#
[user@linux_workstation terraform-docker-registry]$ git commit -m "removed bad submodule path"
[master 77ea624] removed bad submodule path
1 file changed, 1 deletion(-)
delete mode 160000 provision/puppet/modules/docker
Push changes and try clone again

Looks like there are still some bad paths to fix up. These only show up one at a time, so we just repeat the process until they are all resolved.

[user@linux_workstation terraform-docker-registry]$ git push origin master
Counting objects: 9, done.
Compressing objects: 100% (3/3), done.
Writing objects: 100% (5/5), 409 bytes | 0 bytes/s, done.
Total 5 (delta 2), reused 0 (delta 0)
To ssh://git@git:7999/~user/terraform-docker-registry.git
   294d7d8..77ea624  master -> master
[user@linux_workstation terraform-docker-registry]$ cd ..

Remove the current checkout so I can clone again

[user@linux_workstation ~]$ rm -rf terraform-docker-registry
[user@linux_workstation ~]$ git clone -c http.sslVerify=false --recursive ssh://git@git:7999/~user/terraform-docker-registry.git
Cloning into 'terraform-docker-registry'...
remote: Counting objects: 133, done.
remote: Compressing objects: 100% (102/102), done.
remote: Total 133 (delta 52), reused 0 (delta 0)
Receiving objects: 100% (133/133), 14.48 KiB | 0 bytes/s, done.
Resolving deltas: 100% (52/52), done.
Submodule 'modules/terraform-aws-common' (https://stash_server/scm/myproject/terraform-aws-common.git) registered for path 'modules/terraform-aws-common'
Submodule 'modules/terraform-intermediate' (https://stash_server/scm/myproject/terraform-intermediate.git) registered for path 'modules/terraform-intermediate'
Submodule 'provision/puppet/modules/stdlib' (https://stash_server/scm/myproject/puppetlabs-stdlib.git) registered for path 'provision/puppet/modules/stdlib'
Cloning into 'modules/terraform-aws-common'...
remote: Counting objects: 160, done.
remote: Compressing objects: 100% (149/149), done.
remote: Total 160 (delta 41), reused 0 (delta 0)
Receiving objects: 100% (160/160), 21.88 KiB | 0 bytes/s, done.
Resolving deltas: 100% (41/41), done.
Submodule path 'modules/terraform-aws-common': checked out '3de51bdabdcbee2da0567381d54680b33558a968'
Cloning into 'modules/terraform-intermediate'...
remote: Counting objects: 3, done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
Submodule path 'modules/terraform-intermediate': checked out '8999cbbbafd3ef9012d745b455316cde1a082358'
Cloning into 'provision/puppet/modules/stdlib'...
remote: Counting objects: 7707, done.
remote: Compressing objects: 100% (3346/3346), done.
remote: Total 7707 (delta 3675), reused 7586 (delta 3602)
Receiving objects: 100% (7707/7707), 1.51 MiB | 0 bytes/s, done.
Resolving deltas: 100% (3675/3675), done.
Submodule path 'provision/puppet/modules/stdlib': checked out '990e1d757549a9c792cf5f7113e4d6bcd592ae3d'
[user@linux_workstation ~]$ cd terraform-docker-registry/
[user@linux_workstation terraform-docker-registry]$ git add  https://stash_server/scm/myproject/puppet-docker.git provision/puppet/modules/docker/
fatal: pathspec 'https:/stash_server/scm/myproject/puppet-docker.git' did not match any files

And repeat

[user@linux_workstation terraform-docker-registry]$ git submodule add  https://stash_server/scm/myproject/puppet-docker.git provision/puppet/modules/docker/
Cloning into 'provision/puppet/modules/docker'...
remote: Counting objects: 191, done.
remote: Compressing objects: 100% (156/156), done.
remote: Total 191 (delta 55), reused 0 (delta 0)
Receiving objects: 100% (191/191), 22.42 KiB | 0 bytes/s, done.
Resolving deltas: 100% (55/55), done.
[user@linux_workstation terraform-docker-registry]$ git status
# On branch master
# Changes to be committed:
#   (use "git reset HEAD <file>..." to unstage)
#
#       new file:   provision/puppet/modules/docker
#
[user@linux_workstation terraform-docker-registry]$ git commit -m "added submodule back in"
[master 303b041] added submodule back in
1 file changed, 1 insertion(+)
create mode 160000 provision/puppet/modules/docker
[user@linux_workstation terraform-docker-registry]$ git push origin master
Counting objects: 9, done.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (5/5), 445 bytes | 0 bytes/s, done.
Total 5 (delta 2), reused 0 (delta 0)
To ssh://git@git:7999/~user/terraform-docker-registry.git
   77ea624..303b041  master -> master
[user@linux_workstation terraform-docker-registry]$ cd ..

(I had to fix quite a few paths, so now skipping ton the end)

[user@linux_workstation ~]$ rm -rf terraform-docker-registry
[user@linux_workstation ~]$ git clone -c http.sslVerify=false --recursive ssh://git@git:7999/myproject/terraform-docker-registry.git
Cloning into 'terraform-docker-registry'...
remote: Counting objects: 154, done.
remote: Compressing objects: 100% (121/121), done.
remote: Total 154 (delta 65), reused 0 (delta 0)
Receiving objects: 100% (154/154), 16.14 KiB | 0 bytes/s, done.
Resolving deltas: 100% (65/65), done.
Submodule 'modules/terraform-aws-common' (https://stash_server/scm/myproject/terraform-aws-common.git) registered for path 'modules/terraform-aws-common'
Submodule 'modules/terraform-intermediate' (https://stash_server/scm/myproject/terraform-intermediate.git) registered for path 'modules/terraform-intermediate'
Submodule 'provision/puppet/modules/docker' (https://stash_server/scm/myproject/puppet-docker.git) registered for path 'provision/puppet/modules/docker'
Submodule 'provision/puppet/modules/stdlib' (https://stash_server/scm/myproject/puppetlabs-stdlib.git) registered for path 'provision/puppet/modules/stdlib'
Submodule 'provision/puppet/modules/systemd' (https://stash_server/scm/myproject/puppet-systemd.git) registered for path 'provision/puppet/modules/systemd'
Cloning into 'modules/terraform-aws-common'...
remote: Counting objects: 160, done.
remote: Compressing objects: 100% (149/149), done.
remote: Total 160 (delta 41), reused 0 (delta 0)
Receiving objects: 100% (160/160), 21.88 KiB | 0 bytes/s, done.
Resolving deltas: 100% (41/41), done.
Submodule path 'modules/terraform-aws-common': checked out '3de51bdabdcbee2da0567381d54680b33558a968'
Cloning into 'modules/terraform-intermediate'...
remote: Counting objects: 3, done.
remote: Compressing objects: 100% (2/2), done.
remote: Total 3 (delta 0), reused 0 (delta 0)
Unpacking objects: 100% (3/3), done.
Submodule path 'modules/terraform-intermediate': checked out '8999cbbbafd3ef9012d745b455316cde1a082358'
Cloning into 'provision/puppet/modules/docker'...
remote: Counting objects: 191, done.
remote: Compressing objects: 100% (156/156), done.
remote: Total 191 (delta 55), reused 0 (delta 0)
Receiving objects: 100% (191/191), 22.42 KiB | 0 bytes/s, done.
Resolving deltas: 100% (55/55), done.
Submodule path 'provision/puppet/modules/docker': checked out '0668f9f202347cf3301607596dc5f1b412b63553'
Cloning into 'provision/puppet/modules/stdlib'...
remote: Counting objects: 7707, done.
remote: Compressing objects: 100% (3346/3346), done.
remote: Total 7707 (delta 3675), reused 7586 (delta 3602)
Receiving objects: 100% (7707/7707), 1.51 MiB | 0 bytes/s, done.
Resolving deltas: 100% (3675/3675), done.
Submodule path 'provision/puppet/modules/stdlib': checked out '990e1d757549a9c792cf5f7113e4d6bcd592ae3d'
Cloning into 'provision/puppet/modules/systemd'...
remote: Counting objects: 48, done.
remote: Compressing objects: 100% (29/29), done.
remote: Total 48 (delta 10), reused 0 (delta 0)
Unpacking objects: 100% (48/48), done.
Submodule path 'provision/puppet/modules/systemd': checked out '161a690ab107c0fcdf88d2dae3a3dccaa3e74c82'