I have two branches, let's call them project/branch-A and project/branch-B. In branch A I have a directory, let's say parent/dirI/dirII/dirIII . 
In branch B I have parent/dirI/dirIII where I have moved dirIII up one level and remove dirII. Now when I am on branch A and go 
git checkout branch-B
I still can see dirII and dirIII with all the correct contents in dirIII. This does not change, again when I am currently on branch-B and go
git checkout branch-A
I get the same stuff again: both dirII and dirIII presents in the lower level of dirI.
What is happening?
Have I done something wrong? Or is this how git supposed to work?
 
    