Like Rob Sedgwick said in his answer this is just the way the control looks in IE, and styling it is not really allowed.
But… you can cheat: make the file input disappear, and create your own fake input. Then redirect the relevant events using JS.
HTML
<div class="input-group">
  <span class="input-group-btn">
    <button class="btn btn-default" type="button">
      <i class="icon-upload-alt"></i> Upload
    </button>
  </span>
  <input id="fileField" class="form-control" name="fileField" type="file" />
  <span class="form-control form-control-overlay" id="fileFieldOverlay">Choose file</span>
</div>
CSS
.form-control[type="file"] {
  margin-bottom: -100%;
  opacity: 0;
}
.form-control-overlay {
  /* style, if you want */
  cursor: pointer;
}
Javascript
var fileFieldEl = document.getElementById("fileField");
var fileFieldOverlayEl = document.getElementById("fileFieldOverlay");
// On change of file selection, update display
fileFieldEl.onchange = function(ev) {
  // remove file path; it's a fake string for security
  fileFieldOverlayEl.innerText = ev.target.value.replace(/^.*(\\|\/)/, '');
};
// Redirect clicks to real file input
fileFieldOverlayEl.onclick = function() {
  fileFieldEl.click();
};
Run the code: http://jsbin.com/alESiBo/16/edit