Mark asked me to follow up with a specific proposal and instructions on how we would use git submodules for external puppet modules. Here we go! In the examples below, lines that start with 'otto@localhost:~/puppet#' are assumed to be commands run on your cloned working copy of operations/puppet, and those that start with 'root@sockpuppet:~/puppet#' are run on, um, sockpuppet.

Adding a new puppet module as a git submodule

Adding the submodule to operations/puppet

First up is adding a new puppet module. 'git submodule add' will modify the .gitmodules file, and also take care of cloning the remote into the local directly you specify.

 otto@localhost:~/puppet# git submodule add https://gerrit.wikimedia.org/r/analytics/puppet-cdh4 modules/cdh4

git status shows the modified .gitmodules file, as well as a new 'file' at modules/cdh4. This new file is a pointer to a specific commit in the puppet-cdh4 repository.

 otto@localhost:~/puppet# git status
 # On branch master
 # Changes to be committed:
 #   (use "git reset HEAD <file>..." to unstage)
 #
 #	modified:   .gitmodules
 #	new file:   modules/cdh4
 #

Commit the changes and post them to gerrit for review.

 otto@localhost:~/puppet# git commit && git-review

Updating the submodule on the puppetmaster

Once reviewed and merged in gerrit, we need to update the cloned copy of the puppet repository on sock puppet. Fetch from the remote and make sure you know what you are about to merge into sockpuppet's production.

 root@sockpuppet:~/puppet# git fetch origin/production && git diff HEAD origin/production
 root@sockpuppet:~/puppet# git merge origin/production

Now that we've got the latest production commit, initialize any new submodules:

 root@sockpuppet:~/puppet# git submodule init

And update them so that they clone from upstream.

 root@sockpuppet:~/puppet# git submodule update

This checks out the submodule repository at exactly the commit when we added it. Even if new changes are pushed to the submodule's upstream repositories, those changes will not show up in our parent repository unless someone manually makes it so. Here's how to make it so:

Pulling changes from an already initialized submodule

Updating the commit SHA locally

To update a submodule with upstream changes:

 otto@localhost:~/puppet# cd modules/cdh4

We need to check out the master branch, since the git submodule add and update commands check out a headless commit.

 otto@localhost:~/puppet# git checkout master

Now pull from the upstream. You are also welcome to checkout a specific commit here if you like. For example, you could checkout an older commit in the puppet-cdh4 module if we wanted to roll it back.

 otto@localhost:~/puppet# git pull

Now that the cloned module is at the commit we want, we need to commit the change and push it to gerrit for review:

 otto@localhost:~/puppet# cd ../..
 otto@localhost:~/puppet# git status
 # On branch master
 # Changes not staged for commit:
 #   (use "git add <file>..." to update what will be committed)
 #   (use "git checkout -- <file>..." to discard changes in working directory)
 #
 #	modified:   modules/cdh4 (new commits)
 #

 otto@localhost:~/puppet# git add modules/cdh4
 otto@localhost:~/puppet# git commit && git-review

Updating the puppetmaster working copy to the new commit

Once this commit is approved and merged in gerrit, we can go back to sockpuppet and update the submodule:

 root@sockpuppet:~/puppet# git fetch origin/production && git diff HEAD origin/production
 root@sockpuppet:~/puppet# git merge origin/production
 root@sockpuppet:~/puppet# git submodule update
 Submodule path 'modules/cdh4': checked out '3e61fd67fc424b234e4548d7822d59c897c0d8b3'

Ta da!

So, for the most part, unless you are intentionally working with submodules yourself, you won't have to think about it. The same puppet+gerrit workflow that we are using now will continue to work as is.

User:Ottomata/Git Submodules and Puppet