4

I have a private github repository (Server) with another private repository (Shared) as a submodule. Since they're both set to private, and github doesn't allow sharing deploy keys - when I try to run submodule update I get the following error:

ERROR: Repository not found. fatal: Could not read from remote repository.

Please make sure you have the correct access rights and the repository exists.

It works fine if I'm using a github-wide SSH key on my desktop, but I obviously don't want my server to have access to all the repositories on my account - so I need to use deploy keys.

How can I update submodules using github deploy keys?

Jon
  • 305
  • 3
  • 20
  • 45

1 Answers1

7

The actions/checkout issue 183 proposes a few options:

For instance (to be tested)

What do you think about being able to specify multiple ssh keys. For example:

ssh-key: |
 ${{ secrets.my_main_repo_deploy_key }}
 ${{ secrets.my_other_repo_deploy_key }}

I think if one doesnt work, it will fallback and try the next. To be clear, this currently won't work - would need to update the action to support it.

Or:

I already use a deploy key to pull in a python dependency from another private repository 'B' into the build of our repository 'A'.

For this to work, I set up a private key via a configured secret in 'A' and the respective public key in 'B' and use the following step:

     - name: Setup access via public/private key.
       # Below command requires the FOOBAR_PRIVATE_KEY to be configured via github repository secrets.
       # Also the key's public part must be added to the foobar repository deploy keys.
       # A private/public key pair without password (required in this case) can be generated with ssh-keygen.
       # This part is used for the git/foobar part in requirements.txt
       run: |
         mkdir ~/.ssh
         echo "${{ secrets.FOOBAR_PRIVATE_KEY }}" > ~/.ssh/id_rsa
         chmod 600 ~/.ssh/id_rsa

Also:

On GitHub the problem is, that we need one key for each submodule. AFAIK the idea in PR #190 will only allow one key for all submodules.
But we need multiple if we have multiple private submodules.

You have a full workaround in "Using private git submodules in GitHub CI" from Maximilian Ehlers.

The OP jon adds in the comments

I got it working: I wasn't putting the "-----BEGIN OPENSSH PRIVATE KEY-----" and the END in the secret; adding those fixed it.

Note: trying to replicate that on your workstation, using multipe SSH keys, will mean using ~/.ssh/config in order to reference those keys under different Host entries.

See "Enable Multiple SSH Key for GitHub on Windows 10" as an example.

Note: the issue 116 "private submodule checkout fails" now (July 2022) includes as an alternatives:

This solution works when you want to keep flexibility of URL repos and still use GitHub Actions with Deploy Keys to access private submodules:

  - name: Checkout
    uses: actions/checkout@v3

  - name: Clone Submodule
    run: |
        mkdir -p $HOME/.ssh
        echo '${{ secrets.SUBMODULE_REPO_DEPLOY_KEY }}' > $HOME/.ssh/ssh.key
        chmod 600 $HOME/.ssh/ssh.key
        export GIT_SSH_COMMAND="ssh -i $HOME/.ssh/ssh.key"
        git submodule set-url <path-to-submodule> git@github.com:<organization/submodule>.git
        git submodule update --init --recursive
        git submodule set-url <path-to-submodule> https://github.com/<organization/submodule>.git
        unset GIT_SSH_COMMAND
VonC
  • 1,262,500
  • 529
  • 4,410
  • 5,250
  • I keep getting "Load key "/home/runner/.ssh/id_rsa": invalid format", and when I try to run "more $HOME/.ssh/id_rsa" after adding it, it just prints out a few lines of ***'s. I followed the "Using private git submodules in GitHub CI" tutorial exactly. – Jon Mar 10 '22 at 15:38
  • @Jon Try again after re-generating (and registering) a new set of keys with `ssh-keygen -t rsa -P ""` (assuming you are not using your existing one for anything else). If it still does not work, switch format with `ssh-keygen -t rsa -P "" -m PEM` (the [old format](https://stackoverflow.com/a/53645530/6309)) – VonC Mar 10 '22 at 15:42
  • Thanks! I got it working, I wasn't putting the "-----BEGIN OPENSSH PRIVATE KEY-----" and the END in the secret, adding those fixed it. Thank you so much – Jon Mar 10 '22 at 15:54
  • @Jon Great! Well done. I have included your comment in the answer for more visibility. – VonC Mar 10 '22 at 16:01
  • Now how do I go about pulling changes to the submodule? Because pulling changes on my server doesn't update the folder the shared module is in – Jon Mar 10 '22 at 16:09
  • @Jon `git submodule update --recursive --remote`, if the submodule follows a branch. See https://stackoverflow.com/a/18799234/6309 – VonC Mar 10 '22 at 16:14
  • Followed the instructions and it doesn't work. Even started fresh on my server and followed everything again, and now the shared submodule folder is empty and the command you mentioned doesn't do anything. Git pull says already up to date too. When I look at the shared submodule repo directly on github, my recent commits are shown. But when I view the submodule's folder on github in the server's repository, it's behind several commits. This is true for another repo that shares that submodule, so I'm not sure if that's just how github works or not – Jon Mar 10 '22 at 16:27
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/242818/discussion-between-jon-and-vonc). – Jon Mar 10 '22 at 16:31