1

Consider the following Git repositories:

  • controller
  • modules
  • common

Both controller and modules depend on code within common. As such, common is used as a git submodule within controller and modules.

Based on multiple answers given on SO (e.g., How do I "commit" changes in a git submodule? [duplicate], Updating a Git Submodule With Local Changes), I understand that the "ideal" workflow for making changes to common would be:

  1. Make changes to common repository, commit and push
  2. Update both controller and modules as necessary

However, this seems extremely cumbersome as I need to test the code within common in combination with controller or modules, which would require that I (manually) copy the changes within common to the respective submodules within controller and modules.

What am I missing here? Is it possible to update the common submodules within the repos controller and modules with my local common repository to spare me from any copy-paste work?

John Doe
  • 11
  • 1
  • If `common` is a real Git submodule you don't to "*(manually) copy the changes … within `controller` and `modules`*" — you just do `git submodule update --remote` and this brings changes from `common` original repository to one of the superprojects. Run the command (+ `git commit common`) two times in both superprojects and you're done. No manual labor at all. – phd Nov 25 '22 at 20:58
  • 1
    @phd I think the point is that we don't want to have to push a change that hasn't even been tested with `controller` and `modules` – LLSv2.0 Nov 25 '22 at 20:59
  • @phd Thank you for your help. Well, my problem is that the changes within `common` have not yet been pushed to remote. Basically, when I work on `controller` or `modules` I want to have the local changes of the `common` repository in the respective submodules. – John Doe Nov 25 '22 at 21:02
  • @LLSv2.0 Absolutely. This is the problem that I am facing. – John Doe Nov 25 '22 at 21:03
  • Submodules — even when they're submodules attached to a superproject — still are separate repositories. You don't obliged to push to a single remote. You can have additional remote `testing` (or whatever) to which you push new untested changes from `common` and from which you pull these changes into `controller` and `modules` (still not manually, just `cd common && git pull testing && cd superproject && git commit common && make test`). After testing you can push `common` to its `origin`. Everything in this pipeline should be scripted with Git aliases or shell scripts to avoid manual typing. – phd Nov 25 '22 at 21:07
  • @phd Thank you for the suggestion. Just to make sure that I understand this correctly: Every user on our team would have to create a remote repository for `common`, add this additional remote and then during development/testing the changes will be pushed to this remote. Probably we will have to setup a file listener to execute this pipeline on every file change to make this a smooth experience. – John Doe Nov 25 '22 at 22:00
  • No, just one additional repository and many `git remote add testing URL` commands. Not a file listener — Git hooks either in the `common` repository or in `testing`. – phd Nov 25 '22 at 22:03
  • @phd Alright, thanks, I will consult the documentation. I thought that this would mean that users on the team cannot develop/test their changes in parallel (because they push to the same remote). And I thought about file listeners because users will not be happy to initiate this pipeline on every file change when they develop. – John Doe Nov 25 '22 at 22:11
  • "*…users on the team cannot develop/test their changes in parallel (because they push to the same remote).*" They can push different branches. The same branchy workflow like Git workflow, Github workflow, any multibranch workflow. Finally merge every finished branch into `master` and push to the stable `common` remote (`origin`). – phd Nov 25 '22 at 22:18

1 Answers1

0

... this seems extremely cumbersome as I need to test the code within common in combination with controller or modules ...

That's correct, and it is more cumbersome, especially if your testing involves pushing to some hosting site that does the CI build.

It is possible to automate most or even all of this. Unfortunately, the details on how you go about automating it depend on your hosting and CI systems.

This kind of thing is why I always prefer to be able to build and test locally, in isolation. But if you can't, you just have to bite the bullet, as the saying goes, and dig into what options are available in your non-local build-and-test systems, and choose one of the many possibilities for structuring your test scaffolds.

torek
  • 448,244
  • 59
  • 642
  • 775
  • Thanks for the insights. When I mentioned "testing" in my question, I actually meant the usually testing as part of development. So it surprises me that the workflow is that "complex" (although it can be automated as you stated). Before submodules we just had symbolic links to `common` (which obviously has disadvantages) but at least we only had to push to remote when we "completed" a task and not repeatedly during development/testing. – John Doe Nov 26 '22 at 07:17
  • Also, how would the workflow look if the remote is not available? Say, I am on a 10 hour flight without Internet connection and need to work on a feature. Do I have to setup a git server on my notebook and refer to this as my `remote`? It just seems that I am making things unnecessarily complicated. – John Doe Nov 26 '22 at 08:11
  • Again, that's why I prefer a system that can be tested in isolation: you could set up all the components on your laptop and run the tests while in flight. This all goes back to Leslie Lamport's observation that a distributed system is one in which the failure of a computer you didn't even know existed can make it impossible for you to get your own work done locally... – torek Nov 26 '22 at 08:18
  • If you do have everything locally, you can use tricks like `git worktree add` to create working trees in various locations *instead of* bothering with submodules, while you build and test. Once everything works, you remove the added working trees, and make a submodule commit. – torek Nov 26 '22 at 08:25
  • Thanks! This really sounds like the best approach to work directly with the local versions and then just push the "final" submodule code to remote. – John Doe Nov 26 '22 at 08:34