Run git pull over all subdirectories

Question

How can I update multiple git repositories from their shared parent's directory without cd'ing into each repo's root directory? I have the following which are all separate git repositories (not submodules):

/plugins/cms
/plugins/admin
/plugins/chart

I want to update them all at once or at least simplify my current workflow:

cd ~/plugins/admin
git pull origin master
cd ../chart
git pull

etc.

The same question [answered](https://stackoverflow.com/a/48790388/2192488) for `hg mercurial`. — Serge Stroobandt, Feb 14 '18 at 15:13
`find . -name .git -print -execdir git pull \;` is OK. `-print` will echo the current dir. — DawnSong, Dec 29 '18 at 09:18
See also, with Git 2.30 (Q4 2020), the [new `git for-each-repo` command] (https://stackoverflow.com/a/65766304/6309) — VonC, Jan 17 '21 at 21:50
https://github.com/earwig/git-repo-updater — *“gitup is a tool for updating multiple git repositories at once. It is smart enough to handle several remotes, dirty working directories, diverged local branches, detached HEADs, and more. It was originally created to manage a large collection of projects and deal with sporadic internet access. gitup should work on macOS, Linux, and Windows. You should have the latest version of git and either Python 2.7 or Python 3 installed.”* — sideshowbarker, Aug 13 '21 at 04:08

leo · Accepted Answer · 2013-01-26T22:01:07.490

433

Run the following from the parent directory, plugins in this case:

find . -type d -depth 1 -exec git --git-dir={}/.git --work-tree=$PWD/{} pull origin master \;

To clarify:

find . searches the current directory
-type d to find directories, not files
-depth 1 for a maximum depth of one sub-directory
-exec {} \; runs a custom command for every find
git --git-dir={}/.git --work-tree=$PWD/{} pull git pulls the individual directories

To play around with find, I recommend using echo after -exec to preview, e.g.:

find . -type d -depth 1 -exec echo git --git-dir={}/.git --work-tree=$PWD/{} status \;

Note: if the -depth 1 option is not available, try -mindepth 1 -maxdepth 1.

edited Jan 26 '13 at 22:01

answered Sep 19 '12 at 13:00

leo

7,518
1
24
25

Thanks @batandwa for your remark about the use of -d. – leo Jan 26 '13 at 22:02
22

find: warning: you have specified the -depth option after a non-option argument -type, but options are not positional (-depth affects tests specified before it as well as those specified after it). Please specify options before other arguments. – Yuri Astrakhan May 01 '13 at 06:33
40

I used `find . -maxdepth 1 -type d -print -execdir git --git-dir={}/.git --work-tree=$PWD/{} pull origin master \;` to output the name of the folder before doing the pull, to get rid of the warning and to only run the pull on subfolders. – Rystraum May 15 '13 at 06:36
5

replacing 'pull origin master' with `fetch origin master:master` tells git to explicitly update your 'master' branch with origin's master branch. This **will not do a merge**, any commits to master will be lost if you do this. – ThorSummoner Jun 24 '14 at 22:18
103

since git 1.8.5 it is possible to replace --git-dir and --work-tree by the -C option, see [this question](http://stackoverflow.com/questions/5083224/git-pull-while-not-in-a-git-directory). -- I'm using `find . -mindepth 1 -maxdepth 1 -type d -print -exec git -C {} pull \;` – Zarat Mar 11 '15 at 11:06
Didnt´t work unfortunately: "find: warning: you have specified the -depth option after a non-option argument -type, but options are not positional (-depth affects tests specified before it as well as those specified after it). Please specify options before other arguments. find: paths must precede expression: 1 Usage: find [-H] [-L] [-P] [-Olevel] [-D help|tree|search|stat|rates|opt|exec] [path...] [expression] " – Zsolt Szilagyi Sep 14 '16 at 14:44
5

@ZsoltSzilagy as mentioned by @Rystraum you can use `-maxdepth 1` instead of `-depth 1` – jamiebarrow Dec 09 '16 at 07:41
Note that when using this solution for bare repositories you have to omit --work-tree and leave only --git-dir, i.e. `find . -type d -depth 1 -exec echo git --git-dir={}/ remote update \;` – mdziob Jan 09 '17 at 15:03
1

I use this to avoid hidden folders `find . -not -path '*/\.*' -mindepth 1 -maxdepth 1 -type d -print -exec git -C {} pull \;` – alayor Jun 18 '18 at 01:39
Can we make each `git pull` run in parallel? – Bruno Finger Sep 20 '18 at 08:00
Also all the spaces in this line are sensitive I found out. – Alper Sep 27 '18 at 08:44
After following some of the comments that explain that depth has changed to maxdepth, I get an error 'missing argument to -exec'. – user1445967 May 02 '19 at 19:44
Thank you for this answer, this is what I came up with: find . -type d -name ".git" \ -a -exec echo {} \; \ -exec git --git-dir={} --work-tree=$PWD/{}/../ status \; \ -exec git --git-dir={} --work-tree=$PWD/{}/../ pull \; \ -exec git --git-dir={} --work-tree=$PWD/{}/../ fetch -ap \; – 4F2E4A2E Feb 17 '20 at 10:05
depending on how you use, you might add --rebase: `find . -type d -depth 1 -exec git --git-dir={}/.git --work-tree=$PWD/{} pull --rebase origin master \;` – srikanth Nutigattu Nov 22 '20 at 15:48

score 296 · Answer 2 · answered Mar 16 '15 at 08:51

296

ls | xargs -I{} git -C {} pull

To do it in parallel:

ls | xargs -P10 -I{} git -C {} pull

answered Mar 16 '15 at 08:51

Dmitry Mitskevich

4,946
2
15
8

62

Nice! I've put it as an alias in my .gitconfig: `all = "!f() { ls | xargs -I{} git -C {} $1; }; f"` Now I can do `git all pull`, `git all "checkout master"` etc. – borisdiakur Jul 13 '15 at 08:39
10

Cleaned up a bit, will search all directories recursively for only git repos, and will strip out colors in case you have ls aliased `ls -R --directory --color=never */.git | sed 's/\/.git//' | xargs -P10 -I{} git -C {} pull` – cchamberlain Jul 15 '15 at 04:18
Here's the git config command to add it as an alias: git config --global alias.all '!f() { ls | xargs -I{} git -C {} $1; }; f' – Simon de Lorean May 10 '16 at 10:27
9

I smashed some of the answers together to create this for git on macOS that filters on folder that contain a .git folder, and lets you run arbitrary commands like `git all fetch --prune`: `git config --global alias.all '!f() { ls -R -d */.git | sed 's,\/.git,,' | xargs -P10 -I{} git -C {} $1; }; f'` – Courtney Faulkner Dec 21 '16 at 23:10
@CourtneyFaulkner This looks like it assumes a .git directory is available where the command is run. Am I right? Seems different from the original question where just the subdirectories contain .git dirs. – AWrightIV Dec 22 '16 at 17:14
4

@AWrightIV actually, `ls -R -d */.git` is returning a filtered list of the directories within the current folder that contain a `.git` directory. That way, when I run something like `git all fetch`, it only executes against subfolders that have .git folders. It's an answer to the original question, but it tries to be a bit more efficient by not assuming all the subdirectories are git repos. – Courtney Faulkner Jan 03 '17 at 17:32
16

A slight improvement over borisdiakur command, to avoid running on `.` and to have a printed list of on which directory it's running at each instant: `git config --global alias.all '!f() { ls -R -d */.git | xargs -I{} bash -c "echo {} && git -C {}/../ $1"; }; f'` – Jose Luis Blanco Dec 06 '17 at 07:51
@borisdiakur Is there a trick to run the pull in parallel in all directories ? – Sankar Jul 17 '18 at 04:36
I'm getting xargs: unknown option -- C – David S. Feb 04 '21 at 12:19
2

If you want to avoid non git directories it will be ```bash ls -R -d */.git | cut -d'.' -f1 | xargs -I{} git -C {} pull ``` – Joel Deleep Mar 23 '21 at 09:07
1

Why this answer isn't on top? – scorpion Jan 19 '22 at 15:44
1

this answer is the best: `ls | xargs -P10 -I{} git -C {} pull` – gurkan Apr 01 '23 at 21:53

score 120 · Answer 3 · answered Feb 05 '15 at 15:40

120

A bit more low-tech than leo's solution:

for i in */.git; do ( echo $i; cd $i/..; git pull; ); done

This will update all Git repositories in your working directory. No need to explicitly list their names ("cms", "admin", "chart"). The "cd" command only affects a subshell (spawned using the parenthesis).

answered Feb 05 '15 at 15:40

Ingo Blechschmidt

1,439
1
10
11

Exactly what I was looking for – GameSalutes Feb 15 '17 at 21:06
This one has the advantage of displaying which repository it is dealing with, usefeul when something goes wrong (missing branch, un-available remote..) – Victor Lamoine Jun 14 '18 at 13:51
This worked great. I added a crontab -e file to run this every 5 minutes and it appears to do exactly what I was hoping for. – trench Jul 06 '18 at 10:42
You inspired me. Many thanks. – DawnSong Dec 29 '18 at 09:48
1

I like this solution because it only pulls on sub-directories that are a git repo, thx! – Cory Robinson Jun 02 '19 at 22:16
2

Alternate using `git -C`: `for i in */.git; do git -C $i pull; done` – Jon-Eric Jul 07 '20 at 17:11
Is there any way to automate typing in the username and password for each subfolder repo? – palacetrading Apr 29 '21 at 13:19
@colinodowd: Yes :-) This is independent of this one-liner; you'll have to tell git about the credentials in each individual case. How this is done depends on whether you're accessing your repos using https or ssh. – Ingo Blechschmidt Apr 29 '21 at 23:09
@IngoBlechschmidt Thanks Ingo, I figured it out through another SO thread :-) I love how automated my process is now! – palacetrading May 02 '21 at 15:57
This is the most straightforward solution IMO. Just add a '&' to the end of git pull to make it async so you're not waiting for each pull to complete before looping to the next item: `for i in */.git; do ( echo $i; cd $i/..; git pull &) ; done` – Saturn K Jan 20 '23 at 17:49
Simply, easy, working and easily configurable. THX! – jabujavi Mar 22 '23 at 09:06

score 56 · Answer 4 · edited Oct 09 '21 at 16:50

56

Actually, if you don't know if the subfolders have a git repo or not, the best would be to let find get the repos for you:

find . -type d -name .git -exec git --git-dir={} --work-tree=$PWD/{}/.. pull origin master \;

The PowerShell equivalent would be:

Get-ChildItem -Recurse -Directory -Hidden -Filter .git | ForEach-Object { & git --git-dir="$($_.FullName)" --work-tree="$(Split-Path $_.FullName -Parent)" pull origin master }

edited Oct 09 '21 at 16:50

evandrix

6,041
4
27
38

answered Nov 15 '16 at 03:24

Gildas

728
6
5

1

This works with my macOS. – Jin Kwon Dec 06 '16 at 14:15
Ensure all your git repos are on master before executing this as written. Otherwise you may be unintentionally merging master into your current branch. – Jacob Archambault May 05 '21 at 15:40
1

Thank you for the powershell version, still works with PS 7.10 – Jean-Sébastien Gervais May 20 '21 at 15:33
For PS 2.0 this works: ```gci | where {$_.Attributes -match'Directory'} | foreach { write-host $_.fullname; push-location $_; & git pull; & cd ..}``` – jlo Jun 29 '21 at 17:07
Note - I could not find a `hidden` parameter, but `Force` seemed to work, as in `Get-ChildItem -Path "C:\Repos" -Recurse -Directory -Force -Filter ".git" -Depth 1` – sommmen Aug 13 '21 at 07:22
Is there a way to have it act on either master or main based on whichever is used? – ss7 Mar 11 '22 at 22:16

score 33 · Answer 5 · answered Jan 29 '16 at 09:29

33

I use this one:

find . -name ".git" -type d | sed 's/\/.git//' |  xargs -P10 -I{} git -C {} pull

Universal: Updates all git repositories that are below current directory.

answered Jan 29 '16 at 09:29

Liudvikas

396
3
4

1

It's dangerous to use `--git-dir` without `--work-tree` which is why the `-C` shortcut was created. I just messed up my home directory because of this. I would recommend just doing `find . -maxdepth 8 -type d -name .git | xargs -P8 -I{} git -C {}/../ fetch --all` – jaksco Apr 12 '23 at 08:29

score 16 · Answer 6 · edited Apr 27 '17 at 10:00

16

None of the top 5 answers worked for me, and the question talked about directories.

This worked:

for d in *; do pushd $d && git pull && popd; done

edited Apr 27 '17 at 10:00

phuclv

37,963
15
156
475

answered Jul 10 '16 at 12:37

Ryan

171
1
4

1

For Windows, see my answer here: https://stackoverflow.com/a/51016478/207661. It's very similar to above. – Shital Shah Jun 25 '18 at 04:50
The accepted answer used to work for me, but it quit sometime in the last year. I finally decided to look for another solution and found your answer. Thanks for sharing. It works perfectly. – Barry Jun 08 '20 at 16:18
no need to push and pup, just use git's `-C` option. – Travis Stevens Mar 11 '21 at 05:02
Another option is to use a subshell: `for d in *; do (cd $d && git pull --ff-only); done` This has an advantage over -C as this approach is universally applicable to programs which don't have such option. Note that `--ff-only` is good to have in case of automatic updates; I use it to update vim plugins if native vim package management is used. – nponeccop Jun 14 '21 at 23:46

Jamie Wong · Answer 7 · 2010-08-16T21:46:30.427

14

This should happen automatically, so long as cms, admin and chart are all parts of the repository.

A likely issue is that each of these plugins is a git submodule.

Run git help submodule for more information.

EDIT

For doing this in bash:

cd plugins
for f in cms admin chart
do 
  cd $f && git pull origin master && cd ..
done

edited Aug 16 '10 at 21:46

answered Aug 16 '10 at 20:45

Jamie Wong

18,104
8
63
81

No, sorry you misunderstood. Each of those directories are a separate git repository. `/plugins` is not a repository – Petah Aug 16 '10 at 21:06
Ahhh. My mistake. Will give you the bash solution in a minute. – Jamie Wong Aug 16 '10 at 21:44
There you go. If you want to return to the parent directory, just run another `cd ..` afterwards. – Jamie Wong Aug 16 '10 at 21:47
Or use `pushd` and `popd` or put the group of commands in a subshell (when the subshell exits, you'll be left in the original directory). `(cd dir; for ... done)` – Dennis Williamson Aug 16 '10 at 22:13
Cool, that works. Now is there a way to pass the ssh password to the bash script and have the script pass it to git/ssh instead of prompting every time for my password? – Petah Aug 16 '10 at 23:10
5

Out of curiousity - why are aren't you using ssh keys instead? – Jamie Wong Aug 17 '10 at 03:56

score 12 · Answer 8 · answered Aug 17 '15 at 23:27

The mr utility (a.k.a., myrepos) provides an outstanding solution to this very problem. Install it using your favorite package manager, or just grab the mr script directly from github and put it in $HOME/bin or somewhere else on your PATH. Then, cd to the parent plugins folder shared by these repos and create a basic .mrconfig file with contents similar to the following (adjusting the URLs as needed):

# File: .mrconfig
[cms]
checkout = git clone 'https://<username>@github.com/<username>/cms' 'cms'

[admin]
checkout = git clone 'https://<username>@github.com/<username>/admin' 'admin'

[chart]
checkout = git clone 'https://<username>@github.com/<username>/chart' 'chart'

After that, you can run mr up from the top level plugins folder to pull updates from each repository. (Note that this will also do the initial clone if the target working copy doesn't yet exist.) Other commands you can execute include mr st, mr push, mr log, mr diff, etc—run mr help to see what's possible. There's a mr run command that acts as a pass-through, allowing you to access VCS commands not directly suported by mr itself (e.g., mr run git tag STAGING_081220015). And you can even create your own custom commands that execute arbitrary bits of shell script targeting all repos!

mr is an extremely useful tool for dealing with multiple repos. Since the plugins folder is in your home directory, you might also be interested in vcsh. Together with mr, it provides a powerful mechanism for managing all of your configuration files. See this blog post by Thomas Ferris Nicolaisen for an overview.

score 12 · Answer 9 · answered Nov 06 '15 at 00:17

12

Most compact method, assuming all sub-dirs are git repos:

ls | parallel git -C {} pull

answered Nov 06 '15 at 00:17

cmcginty

113,384
42
163
163

1

can't find command `parallel`. reference? – balanceglove2 Feb 02 '22 at 04:44
@LeonTepe This tool is usually included with `moreutils` package. – cmcginty Feb 03 '22 at 06:17

score 11 · Answer 10 · answered Nov 10 '17 at 08:27

My humble construction that

shows the current path (using python, convenient and just works, see How to get full path of a file?)
looks directly for .git subfolder: low chance to emit a git command in a non-git subfolder
gets rid of some warnings of find

as follow:

find . \
    -maxdepth 2 -type d \
    -name ".git" \
    -execdir python -c 'import os; print(os.path.abspath("."))' \; \
    -execdir git pull \;

Of course, you may add other git commands with additional -execdir options to find, displaying the branch for instance:

find . \
    -maxdepth 2 -type d \
    -name ".git" \
    -execdir python -c 'import os; print(os.path.abspath("."))' \; \
    -execdir git branch \;
    -execdir git pull \;

Not sure why there's net downvotes on this answer. This was the most helpful, imo, since I could go to a greater max depth and not keep hitting non-git repos. I used this to run git gc on all all the repositories in my "developer" repo. — Harshita Gupta, Mar 20 '19 at 20:50
I also like this answer as it's easy to read and very easy to add multiple commands. — Top Cat, Sep 13 '19 at 11:41

score 5 · Answer 11 · answered Apr 03 '15 at 09:58

You can try this

find . -type d -name .git -exec sh -c "cd \"{}\"/../ && pwd && git pull" \;

Also, you can add your customized output by adding one more && argument like.

find . -type d -name .git -exec sh -c "cd \"{}\"/../ && pwd && git pull && git status" \;

score 5 · Answer 12 · edited Apr 27 '17 at 10:00

5

gitfox is a tool to execute command on all subrepos

npm install gitfox -g
g pull

edited Apr 27 '17 at 10:00

phuclv

37,963
15
156
475

answered Jan 13 '17 at 06:57

eqfox

97
1
1

what's `g`? not everyone has the same alias as yours – phuclv Apr 27 '17 at 10:00
1

@LưuVĩnhPhúc gitfox installs itself under the alias "g" for some reason (though the help message says "gitfox"). Personally I do not think it's a command important enough to claim such a shortcut but ah well. It does the job, though. – Stoffe Jul 17 '17 at 09:57
@LưuVĩnhPhúc Check the source repo for the usage https://github.com/eqfox/gitfox – thuanle Mar 06 '18 at 02:10

score 5 · Answer 13 · answered Sep 11 '17 at 14:59

5

I combined points from several comments and answers:

find . -maxdepth 1 -type d -name .git -execdir git pull \;

answered Sep 11 '17 at 14:59

Nick

4,901
40
61

score 1 · Answer 14 · answered Apr 06 '16 at 13:10

1

I use this

for dir in $(find . -name ".git")
do cd ${dir%/*}
    echo $PWD
    git pull
    echo ""
    cd - > /dev/null
done

Github

answered Apr 06 '16 at 13:10

P Pang

254
3
15

score 0 · Answer 15 · edited May 23 '17 at 12:02

0

Original answer 2010:

If all of those directories are separate git repo, you should reference them as submodules.

That means your "origin" would be that remote repo 'plugins' which only contains references to subrepos 'cms', 'admin', 'chart'.

A git pull followed by a git submodule update would achieve what your are looking for.

Update January 2016:

With Git 2.8 (Q1 2016), you will be able to fetch submodules in parallel (!) with git fetch --recurse-submodules -j2.
See "How to speed up / parallelize downloads of git submodules using git clone --recursive?"

edited May 23 '17 at 12:02

Community

1
1

answered Aug 16 '10 at 21:26

VonC

1,262,500
529
4,410
5,250

1

See also http://stackoverflow.com/questions/1979167/git-submodule-update and http://stackoverflow.com/questions/1030169/git-easy-way-pull-latest-of-all-submodules: `git submodule foreach git pull` can also be of interest. – VonC Aug 16 '10 at 21:31
Note: to clarify, '`plugins`', which is not a git repo at the moment, should be made one, as a parent git repo for the submodules. – VonC Aug 16 '10 at 21:33

score -1 · Answer 16 · answered Apr 13 '15 at 16:48

-1

If you have a lot of subdirs with git repositories, you can use parallel

ls | parallel -I{} -j100 '
  if [ -d {}/.git ]; then
    echo Pulling {}
    git -C {} pull > /dev/null && echo "pulled" || echo "error :("
  else
     echo {} is not a .git directory
  fi
'

answered Apr 13 '15 at 16:48

cphyc

440
5
16

Great one. I modified a bit to suit my style. [pastebin](http://pastebin.com/U7SMszkP) – maricn Jun 09 '15 at 09:38

Run git pull over all subdirectories

16 Answers16

Linked