Add destination folder when batch cloning git repositories using shell script

Question

I'm trying to clone a set of repositories from GitHub. It works but I can't work out how to add a destination folder with xargs. This is what I currently have

curl "https://api.github.com/orgs/$org/repos?access_token=$authToken" |
  grep -e 'clone_url*' | 
  grep -w clone_url | 
  grep -o '[^"]\+://.\+.git' |
  xargs -L1 git clone

Except I want the cloned repositories to go into a another directory ./my_folder. I've tried changing the last line:

xargs -L1 git clone "./my_folder"

But this does not work. I can't work out how to add on the extra argument using xargs. The normal way would be to just add the destination folder on to the end of the command like in this answer but I can't quite work out how to do this with my code.

Can you try doing `xargs -L1 -I {} git clone {} "./my_folder"` — Inian, Jun 15 '20 at 05:51
Thank you @Inian! That works perfectly. Do you want to make that an answer and I'll mark as correct? — Muon, Jun 15 '20 at 05:53
@alaniwi: Please feel free to add an answer on that with your observations. — Inian, Jun 15 '20 at 06:13
I've now added a separate answer (so have removed my comments above). — alani, Jun 15 '20 at 06:23

score 1 · Accepted Answer · answered Jun 15 '20 at 06:20

The problem here is that the destination folder for each git clone operation needs to be a separate subdirectory of my_folder, whose name would probably be something like the basename part of the git URL. This is probably hard to achieve using xargs, so I suggest in place of the xargs an explicit loop:

... | while read repo_url ; do git clone $repo_url my_folder/$(basename $repo_url) ; done

This to go at the end of the pipeline after the curl and grep commands.

You could of course strip off the .git if preferred, by replacing $(basename $repo_url) with

$(basename $repo_url | sed 's/\.git$//')`

Putting it all together for convenience:

curl "https://api.github.com/orgs/$org/repos?access_token=$authToken" | \
  grep -e 'clone_url*' | \
  grep -w clone_url | \
  grep -o '[^"]\+://.\+.git' | \
  while read repo_url
  do 
      git clone $repo_url my_folder/$(basename $repo_url | sed 's/\.git$//')
  done

Add destination folder when batch cloning git repositories using shell script

1 Answers1