GitMeta - Clone/update many Git repositories using Meta repos
# use the command line interface gitmeta-update gitmeta-repo-loc meta.gmf /local/git/repo/dir
GetMeta allows you to work on dozens of git repositories hosted on different servers, and update all of your local copies with a single command. It defines a new syntax, called GMF (git meta format), to configure many different remote git repository locations and provides a script,
gitmeta-update, to create local copies of all of these repos or updates them if they already exist. This is useful to
For example, if you want to follow the Perl core developers on perl5.git.perl.org and also the Log4perl project on Github, simply put these lines into a new file
# myrepos.gmf # Log4perl project - git://github.com/mschilli/log4perl.git # Perl core development - git://perl5.git.perl.org/perl.git
Then, if you run
gitmeta-update myrepos.gmf ~/my-git-repos
the script will create clones of theses repos in the directory
~/my-git-repos (it will ask to create it if it doesn't exist yet) or update them if they're already cloned but out-of-date:
Updating git://perl5.git.perl.org/perl.git ... Updating git://github.com/mschilli/log4perl.git ...
If you look your local
~/my-git-repos directory, you now have clones of both projects, ready to use:
$ ls ~/my-git-repos log4perl perl
Having the meta configuration
myrepos.gmf local on your box is nice for testing, but it's much more powerful to store it in a new repo somewhere on the Net, e.g.
github.com/mschilli/gitmeta-test (this actually exists for your testing pleasure). Now, wherever you are, simply call
gitmeta-update git://github.com/mschilli/gitmeta-test \ myrepos.gmf ~/my-git-repos
and the script will go out and fetch the git meta configuration from github, process each entry, and create or update the corresponding repositories in your local git repo directory (
If you want to follow all repositories of a given user (like yourself) on Github, or you want to clone all repositories in a given directory on your hosting service, it would be tiresome to constantly update your .gmf file when you create new repositories or remove retired ones.
This is why GitMeta offers additional modules to automate this:
Expands to all git repos of a given user on Github. Put
# All github projects of user 'mschilli' - type: Github user: mschilli
in your .gmf file (note the peculiar YAML syntax requiring indentation and an empty - line to define an array entry referencing a hash) then
gitmeta-update will fetch a list of all Github projects of user
user and add them to the processing list, before it starts cloning/updating those repos.
Expands to all git repos in the given directory on a server via ssh. If you put
# All projects in directory 'projects' # on some host via git/SSH - type: SshDir host: email@example.com dir: projects
in your .gmf, then
gitmeta-update will fetch a list of all repositories in the given directory on the given host and add them to the processing list. Requires ssh keys to be set up or you'll be prompted for your password.
You guessed it: You can refer to other .gmf files in other gitmeta repos, which
gitmeta-update will dutifully follow. If you put
- type: GMF repo: firstname.lastname@example.org:git/gitmeta gmf_path: privdev.gmf
in your .gmf file,
gitmeta-update will fetch the .gmf file
privdev.gmf from email@example.com:git/gitmeta, process its directives and add the results to the processing list.
This mechanism allows you to group repos into several meta repos and retrieve them separately or combined. For example, if you have your private repositories in
firstname.lastname@example.org:git/gitmeta/priv.gmf and your public repos in
email@example.com:git/gitmeta/pub.gmf, you can write a .gmf file that fetches them all at once:
# all.gmf - type: GMF repo: firstname.lastname@example.org:git/gitmeta gmf_path: priv.gmf - type: GMF repo: email@example.com:git/gitmeta gmf_path: pub.gmf
If you put that file in firstname.lastname@example.org:git/gitmeta as well, all you need to do is run
gitmeta-update email@example.com:git gitmeta/all.gmf ~/local-dir
to get all repos created/updated.
To combine all of the above, let's say that you're following several projects on Github.com, another set of git repositories located on a private hosting service, the perl core development on
perl5.git.perl.org, and another git meta definition in another gitmeta repo. You define the following .gmf file (in YAML format):
# gitmeta.gmf # Perl core development - git://perl5.git.perl.org/perl.git # All github projects of user 'mschilli' - type: Github user: mschilli # All projects in directory 'projects' # on some host via git/SSH - type: SshDir host: firstname.lastname@example.org dir: projects # Private Project via git/SSH - email@example.com:git/private-project.git # Another .gmf file somewhere in another gitmeta repo - type: GMF repo: firstname.lastname@example.org:git/gitmeta gmf_path: privdev.gmf
Note that in order to update a local repository, gitsync will fetch the remote changes, but won't merge them into the local clone. This is because there might be merge conflicts and when updating dozens of repos in one quick run, you don't want to be interrupted to resolve a conflicted merge.
So, in git parlance,
gitsync performs a
git fetch, not a
git pull. The updates will therefore be available in your locally defined remote branches, and if you want to merge them into the local branches, you need to run a
git merge (you don't need an Internet connection for that, so you can do this later), e.g. use
git merge origin/master
to merge the changes in the 'master' branch of the 'origin' remote into the local branch you're currently on (presumably 'master' as well).
Make sure that 'git' is in your PATH.
This module was first published in the German edition of Linux Magazin in August 2010:
An English translation is available here:
Copyright 2010-2011 by Mike Schilli, all rights reserved. This program is free software, you can redistribute it and/or modify it under the same terms as Perl itself.
2010, Mike Schilli <email@example.com>