File disappeared after git rebase attempt

Question

I tried to do git rebase, and in the process lost a file. Not a very important file, but I'd like to understand what happened.

I wanted to merge branch 'master' into branch 'learning' using rebase command.

Here is a summary of my actions:

Did git rebase master, got a conflict, resolved it, tried to continue rebase, failed again with more conflicts, added/commited an untracked file, failed rebase again, tried rebase --skip, failed, aborted rebase, finally did git merge, which was successful.

The untracked file (test.py) I committed during rebase is now gone.

Here is the detailed record of my actions:

$git checkout learning
Switched to branch 'learning'
Your branch is up to date with 'origin/learning'.

$git rebase master
First, rewinding head to replay your work on top of it...
Applying: first commit: changed mnist example trainer
Using index info to reconstruct a base tree...
A       examples/mnist-convnet.py
Falling back to patching base and 3-way merge...
Auto-merging examples/basics/mnist-convnet.py
Applying: added print tensor callback to run after each epoch
Using index info to reconstruct a base tree...
A       examples/mnist-convnet.py
Falling back to patching base and 3-way merge...
Auto-merging examples/basics/mnist-convnet.py
Applying: trying to make callback for mnist-visualization
Using index info to reconstruct a base tree...
A       examples/mnist-convnet.py
A       examples/mnist-visualizations.py
Falling back to patching base and 3-way merge...
Auto-merging examples/basics/mnist-visualizations.py
Auto-merging examples/basics/mnist-convnet.py
Applying: still trying to convert visualization into a callback
Using index info to reconstruct a base tree...
A       examples/mnist-visualizations.py
Falling back to patching base and 3-way merge...
Auto-merging examples/basics/mnist-visualizations.py
Applying: misc changes to visualization
Using index info to reconstruct a base tree...
A       examples/mnist-visualizations.py
Falling back to patching base and 3-way merge...
Auto-merging examples/basics/mnist-visualizations.py
Applying: added VisualizeTensors callback
Using index info to reconstruct a base tree...
A       examples/mnist-visualizations.py
Falling back to patching base and 3-way merge...
Auto-merging examples/basics/mnist-visualizations.py
Applying: first commit: changed mnist example trainer
Using index info to reconstruct a base tree...
A       examples/mnist-convnet.py
Falling back to patching base and 3-way merge...
No changes -- Patch already applied.
Applying: added print tensor callback to run after each epoch
error: Failed to merge in the changes.
Using index info to reconstruct a base tree...
A       examples/mnist-convnet.py
Falling back to patching base and 3-way merge...
Auto-merging examples/basics/mnist-convnet.py
CONFLICT (content): Merge conflict in examples/basics/mnist-convnet.py
Patch failed at 0008 added print tensor callback to run after each epoch
The copy of the patch that failed is found in: .git/rebase-apply/patch

Resolve all conflicts manually, mark them as resolved with
"git add/rm <conflicted_files>", then run "git rebase --continue".
You can instead skip this commit: run "git rebase --skip".
To abort and get back to the state before "git rebase", run "git rebase --abort".

$git status
rebase in progress; onto 4142b9e7
You are currently rebasing branch 'learning' on '4142b9e7'.
  (fix conflicts and then run "git rebase --continue")
  (use "git rebase --skip" to skip this patch)
  (use "git rebase --abort" to check out the original branch)

Unmerged paths:
  (use "git reset HEAD <file>..." to unstage)
  (use "git add <file>..." to mark resolution)

        both modified:   examples/basics/mnist-convnet.py

Untracked files:
  (use "git add <file>..." to include in what will be committed)

        examples/basics/test.py

no changes added to commit (use "git add" and/or "git commit -a")

$git add examples\basics\mnist-convnet.py

$git status
rebase in progress; onto 4142b9e7
You are currently rebasing branch 'learning' on '4142b9e7'.
  (all conflicts fixed: run "git rebase --continue")

Changes to be committed:
  (use "git reset HEAD <file>..." to unstage)

        modified:   examples/basics/mnist-convnet.py

Untracked files:
  (use "git add <file>..." to include in what will be committed)

        examples/basics/test.py

$git commit -am "fixing mnist-convnet conflict"
[detached HEAD 2835adcc] fixing mnist-convnet conflict
 1 file changed, 2 insertions(+), 1 deletion(-)

$git status
rebase in progress; onto 4142b9e7
You are currently rebasing branch 'learning' on '4142b9e7'.
  (all conflicts fixed: run "git rebase --continue")

Untracked files:
  (use "git add <file>..." to include in what will be committed)

        examples/basics/test.py

nothing added to commit but untracked files present (use "git add" to track)

$git rebase --continue
Applying: added print tensor callback to run after each epoch
No changes - did you forget to use 'git add'?
If there is nothing left to stage, chances are that something else
already introduced the same changes; you might want to skip this patch.

Resolve all conflicts manually, mark them as resolved with
"git add/rm <conflicted_files>", then run "git rebase --continue".
You can instead skip this commit: run "git rebase --skip".
To abort and get back to the state before "git rebase", run "git rebase --abort".

$git status
rebase in progress; onto 4142b9e7
You are currently rebasing branch 'learning' on '4142b9e7'.
  (all conflicts fixed: run "git rebase --continue")

Untracked files:
  (use "git add <file>..." to include in what will be committed)

        examples/basics/test.py

nothing added to commit but untracked files present (use "git add" to track)

$git add examples\basics\test.py

$git commit -am "added test"
[detached HEAD 9f32ce6d] added test
 1 file changed, 23 insertions(+)
 create mode 100644 examples/basics/test_FP16_vs_FP32.py

$git status
rebase in progress; onto 4142b9e7
You are currently rebasing branch 'learning' on '4142b9e7'.
  (all conflicts fixed: run "git rebase --continue")

nothing to commit, working tree clean

$git rebase --continue
Applying: added print tensor callback to run after each epoch
No changes - did you forget to use 'git add'?
If there is nothing left to stage, chances are that something else
already introduced the same changes; you might want to skip this patch.

Resolve all conflicts manually, mark them as resolved with
"git add/rm <conflicted_files>", then run "git rebase --continue".
You can instead skip this commit: run "git rebase --skip".
To abort and get back to the state before "git rebase", run "git rebase --abort".

$git rebase --skip
Applying: trying to make callback for mnist-visualization
error: Failed to merge in the changes.
Using index info to reconstruct a base tree...
A       examples/mnist-convnet.py
A       examples/mnist-visualizations.py
Falling back to patching base and 3-way merge...
CONFLICT (modify/delete): examples/mnist-visualizations.py deleted in HEAD and modified in 
trying to make callback for mnist-visualization. Version trying to make 
callback for mnist-visualization of examples/mnist-visualizations.py left in tree.
Auto-merging examples/basics/mnist-convnet.py
CONFLICT (content): Merge conflict in examples/basics/mnist-convnet.py
Patch failed at 0009 trying to make callback for mnist-visualization
The copy of the patch that failed is found in: .git/rebase-apply/patch

Resolve all conflicts manually, mark them as resolved with
"git add/rm <conflicted_files>", then run "git rebase --continue".
You can instead skip this commit: run "git rebase --skip".
To abort and get back to the state before "git rebase", run "git rebase --abort".

$git status
rebase in progress; onto 4142b9e7
You are currently rebasing branch 'learning' on '4142b9e7'.
  (fix conflicts and then run "git rebase --continue")
  (use "git rebase --skip" to skip this patch)
  (use "git rebase --abort" to check out the original branch)

Unmerged paths:
  (use "git reset HEAD <file>..." to unstage)
  (use "git add/rm <file>..." as appropriate to mark resolution)

        both modified:   examples/basics/mnist-convnet.py
        deleted by us:   examples/mnist-visualizations.py

no changes added to commit (use "git add" and/or "git commit -a")

$git rebase --abort

$git status
On branch learning
Your branch is up to date with 'origin/learning'.

nothing to commit, working tree clean

$git merge master
Merge made by the 'recursive' strategy.
 ...                                   
 78 files changed, 880 insertions(+), 539 deletions(-)

$dir examples\basics

 Directory of \Code\tensorpack\examples\basics

03/01/2018  03:19 PM    <DIR>          .
03/01/2018  03:19 PM    <DIR>          ..
03/01/2018  03:19 PM             6,846 cifar-convnet.py
03/01/2018  03:19 PM             5,994 mnist-convnet.py
02/14/2018  12:58 PM             5,828 mnist-tflayers.py
02/12/2018  07:14 PM             3,488 mnist-tfslim.py
03/01/2018  03:01 PM             6,624 mnist-visualizations.py
02/12/2018  07:14 PM             4,166 svhn-digit-convnet.py
02/15/2018  03:43 PM    <DIR>          train_log
               6 File(s)         32,946 bytes
               3 Dir(s)  86,909,169,664 bytes free

As you can see, test.py which used to be in examples\basics is gone.

So, I'd like to understand two things: why did git rebase fail with conflicts but git merge went through fine? And what happened to the untracked file that I committed during rebase operation?

Am I right to assume that `test.py` was untracked at the start of the rebase? In that case, it would have been lost at `rebase --abort`, since that would have reset the tree to how it was before the rebase. Normally, of course, that wouldn't touch untracked files that are unrelated, but since you had added `test.py` during the rebase process, it wasn't untracked at the point of doing the abort. — Dolda2000, Mar 02 '18 at 00:21
Yes. That looks like a bug in git implementation. Is there a way to recover it now? — MichaelSB, Mar 02 '18 at 00:33
That doesn't sound like an implementation bug to me. You made a commit with that file, and then effectively reverted that commit. It makes sense that that removes the file. You can probably recover the file by looking at the reflog for that branch (in `.git/logs/refs`), which lists the commits that branch has gone through in real-time order, and getting it out of a commit in which it existed. — Dolda2000, Mar 02 '18 at 05:03
If I made a commit, then reverted that commit, I would get my committed file back to its unstaged state. That's what happens when I do git reset HEAD~, and that's what I expect to happen when rebase --abort decides to revert the commit I made during rebase. Deleting the file is clearly not what should be expected, and looks like a bug. If there is some git design limitation behind this mess, then git should not let me do a commit during rebase. — MichaelSB, Mar 02 '18 at 20:04
`rebase --abort` doesn't do an ordinary `git reset`, but a `git reset --hard`. If it didn't, you'd also have all the other rebase intermediate results uncommitted as well, which isn't normally what you'd want. — Dolda2000, Mar 03 '18 at 02:35

score 1 · Accepted Answer · edited Jun 20 '20 at 09:12

First, that does not merge master into learning: a git rebase master will replay learning on top of master.

Second, if at any point you added/committed test.py, you can find it back in the reflog. See "Query git reflog for all commits to a specific file".

git rev-list --all --  test.py

Third, during a rebase, you are not supposed to commit, only to add, then git rebase --continue.

Another case: "git rebase BASE BRANCH" rebased/updated the tip of BRANCH and checked it out, even when the BRANCH is checked out in a different worktree.

This has been corrected with Git 2.26 (Q1 2020),

See commit b5cabb4, commit df126ca (23 Feb 2020) by Eric Sunshine (sunshineco).
^{(Merged by Junio C Hamano -- gitster -- in commit a0ab37d, 05 Mar 2020)}

rebase: refuse to switch to branch already checked out elsewhere

^{Reported-by: Mike Hommey}
^{Signed-off-by: Eric Sunshine}

The invocation "git rebase <upstream> <branch>" switches to before performing the rebase operation.

However, unlike git switch, git checkout, and git worktree which all refuse to switch to a branch that is already checked out in some other worktree, git rebase switches to <branch> unconditionally.

Curb this careless behavior by making git rebase also refuse to switch to a branch checked out elsewhere.

OK, I think I understand your first two points. Regarding the third one - what exactly did I do wrong, which prevented git rebase --continue? Should I not have done git commit -am "fixing mnist-convnet conflict" ? — MichaelSB, Mar 02 '18 at 19:35
@MichaelSB the commit: no commit is needed. Only add + rebase --continue. — VonC, Mar 02 '18 at 19:44
why did test.py got deleted? if rebase --abort reverted my commit of that file, it should have made it unstaged, just like git reset HEAD~ would do. — MichaelSB, Mar 02 '18 at 20:10
@MichaelSB Not sure, but at least, if it was committed, it can be found and restored. — VonC, Mar 02 '18 at 20:13

File disappeared after git rebase attempt

1 Answers1

`rebase`: refuse to switch to branch already checked out elsewhere

File disappeared after git rebase attempt

1 Answers1

rebase: refuse to switch to branch already checked out elsewhere

`rebase`: refuse to switch to branch already checked out elsewhere