To understand what is happening you need to consider the fact that the default vertical alignment is baseline which is creating the first output.
Here is a better illustration to see the baseline:
.box {
  position:relative;
}
.box:before {
  content:"";
  position:absolute;
  bottom:14px;
  height:1px;
  background:blue;
  left:0;
  right:0;
}
<div style="border: 1px solid red;" class="box">
  Text on left
  <span><button style="height:40px;">test</button></span>
</div>
 
 
As you can clearly see both text are aligned the same and the button is somehow defining the baseline. Your text isn't centred like you may think.
Increase the font-size to see what happen:
.box {
  position:relative;
}
.box:before {
  content:"";
  position:absolute;
  bottom:14px;
  height:1px;
  background:blue;
  left:0;
  right:0;
}
<div style="border: 1px solid red;" class="box">
  <span >Text on left</span>
  <span><button style="height:40px;">test</button></span>
</div>
<div style="border: 1px solid red;" class="box">
  <span style="font-size:20px;">Text on left</span>
  <span><button style="height:40px;">test</button></span>
</div>
<div style="border: 1px solid red;" class="box">
  <span style="font-size:30px;">Text on left</span>
  <span><button style="height:40px;">test</button></span>
</div>
 
 
We still have the same baseline logic and not a centring logic.
Another example where the button is smaller in height
.box {
  position:relative;
}
.box:before {
  content:"";
  position:absolute;
  bottom:9px;
  height:1px;
  background:blue;
  left:0;
  right:0;
}
<div style="border: 1px solid red;" class="box">
  <span style="font-size:40px;">Text on left</span>
  <span><button style="height:20px;">test</button></span>
</div>
 
 
Still the same baseline thing.
You can change this behavior by changing the vertical alignment.
<div style="border: 1px solid red;" class="box">
  Text on left
  <span><button style="height:40px;vertical-align:bottom">test</button></span>
</div>
<div style="border: 1px solid red;" class="box">
  Text on left
  <span><button style="height:40px;vertical-align:top">test</button></span>
</div>
<div style="border: 1px solid red;" class="box">
  Text on left
  <span><button style="height:40px;vertical-align:sub">test</button></span>
</div>
 
 
Now if you float the button to the right, it will become a block level element and will no more affect the alignment of text that's why you obtain the following output:
<div style="border: 1px solid red;" class="box">
  Text on left
  <span><button style="height:40px;float:right;">test</button></span>
</div>
 
 
Even if you clear the float to fix the height issue the text will remain on the top and more precisely aligned on the baseline inside its line. 
.box {
  overflow:auto;
}
<div style="border: 1px solid red;" class="box">
  Text on left
  <span><button style="height:40px;float:right;">test</button></span>
</div>
 
 
Now centring the text inside the red block is a common job and you can find a lot of question related to this. One solution like stated in the other answer is to set line-height which will simply increase the size of the line and move the baseline of the text around the center.
Some related questions to get more examples and detail:
Vertical-align aligns everything else except self
Vertical align not working on inline-block
Why is this inline-block element pushed downward?
Another important thing to notice is how the button is moved outside its span container. Add border to see this:
.box {
  overflow:auto;
}
<div style="border: 1px solid red;" class="box">
  Text on left
  <span style="border:2px solid blue;"><button style="height:40px;">test</button></span>
</div>
<div style="border: 1px solid red;" class="box">
  Text on left
  <span style="border:2px solid blue;"><button style="height:40px;float:right;">test</button></span>
</div>
 
 
Another related question to understand why: Can you float to the right of a span?