How do I Re-root a git repo to a parent folder while preserving history?

Question

I have a git repo in /foo/bar/baz with a large commit history and multiple branches.

I now want /foo/qux to be in the same repo as /foo/bar/baz, which means that I need them both to be in a repo rooted at /foo. However, I want to preserve history of changes that I've made to /foo/bar/baz.

I first thought of git format-patch followed by apply, but commit messages aren't preserved.

So,

I need to reroot the repo

(1) to an arbitrarily higher ancestor directory (2) while preserving my commit history by making it look like I've been comitting to /foo/bar/baz all along

Answer 1

What you want is git filter-branch, which can move a whole repository into a subtree, preserving history by making it look as if it's always been that way. Back up your repository before using this!

Here's the magic. In /foo/bar, run:

git filter-branch --commit-filter '
    TREE="$1";
    shift;
    SUBTREE=`echo -e 040000 tree $TREE"\tbar" | git mktree`
    git commit-tree $SUBTREE "$@"' -- --all

That will make the /foo/bar repository have another 'bar' subdirectory with all its contents throughout its whole history. Then you can move the entire repo up to the foo level and add baz code to it.

Update:

Okay, here's what's going on. A commit is a link to a "tree" (think of it as a SHA representing a whole filesystem subdirectory's contents) plus some "parent" SHA's and some metadata link author/message/etc. The git commit-tree command is the low-level bit that wraps all this together. The parameter to --commit-filter gets treated as a shell function and run in place of git commit-tree during the filter process, and has to act like it.

What I'm doing is taking the first parameter, the original tree to commit, and building a new "tree object" that says it's in a subfolder via git mktree, another low-level git command. To do that, I have to pipe into it something that looks like a git tree i.e. a set of (mode SP type SP SHA TAB filename) lines; thus the echo command. The output of mktree is then substituted for the first parameter when I chain to the real commit-tree; "$@" is a way to pass all the other parameters intact, having stripped the first off with shift. See git help mktree and git help commit-tree for info.

So, if you need multiple levels, you have to nest a few extra levels of tree objects (this isn't tested but is the general idea):

git filter-branch --commit-filter '
    TREE="$1"
    shift
    SUBTREE1=`echo -e 040000 tree $TREE"\tbar" | git mktree`
    SUBTREE2=`echo -e 040000 tree $SUBTREE1"\tb" | git mktree`
    SUBTREE3=`echo -e 040000 tree $SUBTREE2"\ta" | git mktree`
    git commit-tree $SUBTREE3 "$@"' -- --all

That should shift the real contents down into a/b/bar (note the reversed order).

Update: Integrated improvements From Matthew Alpert's answer below. Without -- --all this only works on the currently-checked out branch, but since the question is asking about a whole repo, it makes more sense to do it this way than branch-by-branch.

Answer 2

Rather than create a new repository, move what's in your current repository into the right place: create a new directory bar in your current directory and move the current content in (so your code is in /foo/bar/bar). Then create a baz directory next to your new bar directory (/foo/bar/baz). mv /foo /foo2; mv /foo2/bar /foo; rmdir /foo2 and you're done :).

Git's rename tracking means that your history will still work and Git's hashing of content means that even though you've moved things around, you're still referencing the same objects in the repository.

Answer 3

This specifically answers "how do I move my git repo up one or more directories and make it look like it was always that way?"

With the advent of git >= 2.22.0 git filter-repo can be leveraged to rewrite history to appear as if that parent directory had always been part of it.

This is the same thing that @Walter-Mundt's answer accomplishes using git filter-branch, but is simpler and not as fragile to execute.

Note that these days git filter-repo is advertised by git filter-branch itself as the safer alternative.

---

So, given that your repo lives in /foo/bar/baz and you want to move it up to /foo

First, to prevent any changes to the files in the workspace while history is being rewritten, temporarily turn the repository into a so-called "bare" one like this:

cd /foo/bar/baz
git config --local --bool core.bare true

The actual history rewriting can now be done directly in the .git directory itself:

cd ./.git
git filter-repo --path-rename :bar/baz/

This will rewrite the repo's complete history as if every path has always had bar/baz/ prepended to it (which they would have had, had the repo's root been two levels up). The actual files are untouched by this operation because this is a bare repository now.

To wrap up, turn it un-bare again, move the .git directory up to its designated position, and reset:

git config --local --bool core.bare false
cd ..
mv ./.git ../..
cd ../..
git reset

I think, the git reset cancels the after-effects of the repository having been turned bare and back again. Try a git status before doing git reset to see what I mean.

A final git status should now prove that all is well, modulo some new untracked files in /foo/qux to deal with.

CAVEAT - if you try the above on an un-cloned repository, git filter-repo will refuse to do its magic unless you --force it to... Have a backup at the ready and consider yourself warned.

Answer 4

I had a solution no one seems to have said yet:

What I specifically needed was to include files from the parent directory in my repository (effectively moving the repo up one directory).

I achieved this via:

• move all files (except for .git) into a new subdirectory with the same name. And tell git about it (with git mv)
• move all files from the parent directory into the now empty (except for .git/) current directory and tell git about it (with git add)
• commit the whole thing into the repo, which hasn't moved (git commit).
• move the current directory up one level in the directory hierarchy. (with command line jiggery-pokery)

I hope this helps the next guy to come along -- I'm probably just having a brainless day, but I found the answers above over-elaborate and scary (for what I needed.) I know this is similar to Andrew Aylett's answer above, but my situation seemed a little different and I wanted a more general view.

Answer 5

Most common solution

In most normal situations, git looks at all files relatively to its location (meaning the .git directory), as opposed to using absolute file paths.

Thus, if you don't mind having a commit in your history which shows that you have moved everything up, there is a very simple solution, which consists in moving the git directory. The only slightly tricky thing is to make sure git understands that the files are the same and that they only moved relatively to him :

# Create sub-directory with the same name in /foo/bar
mkdir bar

# Move everything down, notifying git :
git mv file1 file2 file3 bar/

# Then move everything up one level :
mv .git ../.git
mv bar/* .
mv .gitignore ../

# Here, take care to move untracked files

# Then delete unused directory
rmdir bar

# and commit
cd ../
git commit

The only thing to be careful, is to correctly update .gitignore when moving to the new directory, to avoid staging unwanted files, or forgetting some.

Bonus solution

In some settings, git manages to figure out by itself that files have been moved when it sees new files that are exactly the same as deleted files. In that case, the solution is even simpler :

mv .git ../.git
mv .gitignore ../.gitignore

cd ../
git commit

Again, be careful with your .gitignore

Answer 6

This adds to Walter Mundt's accepted answer. I would have rather commented on his answer, but I don't have the reputation.

So Walter Mundt's method works great, but it works only for one branch at a time. And after the first branch, there may be warnings that require -f to force the action through. So to do this for all branches at once, simply add "-- --all" to the end:

git filter-branch --commit-filter '
    tree="$1";
    shift;
    subtree=`echo -e 040000 tree $tree"\tsrc" | git mktree`
    git commit-tree $subtree "$@"' -- --all

And to do this for specific branches, add their names to the end instead, although I can't imagine why you would change the directory structure of only some of the branches.

Read more about this in the man page for git filter-branch. However, please notice the warning about possible difficulties pushing after using such a command. Just make sure you know what you're doing.

I'd appreciate more input on any potential problems with this method.

Answer 7

1 addition to the accepted answer, which helped me get it to work: when I put the listed text in a shell script, for some reason the -e was kept. (quite likely because I am too thick to work with shell scripts)

when I removed the -e , and moved the quotes to encompass everything, it worked. SUBTREE2=echo "040000 tree $SUBTREE1 modules" | git mktree

note that there is a tab between $SUBTREE1 and modules , which is the same \t that -e should interpret.

Answer 8

You could create a git repo in foo and reference both baz and bar through git submodules.

Then both bar and baz coexist with their full history preserved.

---

If you really want only one repo (foo), with both bar and baz history in it, then some grafts technique or subtree merge strategy are in order.