Seems like to be some kind of a JS conflict between .tabs() and .checkboxradio()!!!
And it is about the background-position of the icon located in that nice .png.

It is used as a background and its background-position setted to 0% 0% via JS...
Which correspond to the top-left icon.
And you want the one in the 5th column, 9th row.
So the position for this one is -5 * 16px  and -9 * 16px which is, in clear:
"background-position":"-64px -144px"
It took time to nail that!
It absolutely has to be fixed by JS, since a slight delay is needed to override the glitch.
1ms seems to be enougth to apply the "bug fix".
    (That really is the right term!).
$(document).ready(function(){
  $( function() {
    $("#guestsData").tabs();
    $(".checkbox").checkboxradio();
  });
  setTimeout(function(){
    var target = $(document).find("#test1").prev().find("span").first();
    
    console.log(target.css("background-position"));
    target.css({"background-position":"-64px -144px"});
    console.log(target.css("background-position"));
  },1);
});
div {
  margin: 20px;
}
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.2.1/jquery.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/jqueryui/1.12.1/jquery-ui.min.js"></script>
<link href="https://cdnjs.cloudflare.com/ajax/libs/jqueryui/1.12.1/jquery-ui.css" rel="stylesheet"/>
<div>
  <label for="test0">This is checked and works</label>
  <input type="checkbox" class="checkbox" id="test0" value="yes" checked />
</div>
<div id="guestsData">
  <ul>
    <li><a href="#guest-0">Tab 0</a></li>
    <li><a href="#guest-1">Tab 1</a></li>
    <li><a href="#guest-2">Tab 2</a></li>
    <li><a href="#guest-3">Tab 3</a></li>
  </ul>
  <div id="guest-0">
    <div>
      <label for="test1">This is checked and works fine when a <b>BUG FIX</b> is applied!</label>
      <input type="checkbox" class="checkbox" id="test1" value="yes" checked />
    </div>
    <div>
      <label for="test2">This is unchecked</label>
      <input type="checkbox" class="checkbox" id="test2" value="yes" />
    </div>
  </div>
  <div id="guest-1">
    Tab 1 content
  </div>
  <div id="guest-2">
    Tab 2 content
  </div>
  <div id="guest-3">
    Tab 3 content
  </div>
</div>
 
 
EDIT!
I just found a better way to fix that for all checkboxes in tabs.
   (And be able to uncheck them!!)
Only a function that will run on document ready... And two classes are needed.
$(document).ready(function(){
  $( function() {
    $("#guestsData").tabs();
    $(".checkbox").checkboxradio();
  });
  setTimeout(function(){
    
    // Fixes the checkbox at checked state on load 
    var target = $(document).find(".ui-tabs .ui-checkboxradio-icon.ui-icon-check")
    // Apply the bugfix.
    target.addClass("bugfix");
    // Handler to "unfix" now...
    target.on("click",function(){
      $(this).toggleClass("unchecked")
    });
  },1);
  
});
div {
  margin: 20px;
}
.bugfix{
  background-position:-64px -144px !important;
}
.bugfix.unchecked{
  background-position:100% 100% !important;
}
<script src="https://cdnjs.cloudflare.com/ajax/libs/jquery/3.2.1/jquery.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/jqueryui/1.12.1/jquery-ui.min.js"></script>
<link href="https://cdnjs.cloudflare.com/ajax/libs/jqueryui/1.12.1/jquery-ui.css" rel="stylesheet"/>
<div>
  <label for="test0">This is checked and works</label>
  <input type="checkbox" class="checkbox" id="test0" value="yes" checked />
</div>
<div id="guestsData">
  <ul>
    <li><a href="#guest-0">Tab 0</a></li>
    <li><a href="#guest-1">Tab 1</a></li>
    <li><a href="#guest-2">Tab 2</a></li>
    <li><a href="#guest-3">Tab 3</a></li>
  </ul>
  <div id="guest-0">
    <div>
      <label for="test1">This is checked and works fine when a <b>BUG FIX</b> is applied!</label>
      <input type="checkbox" class="checkbox" id="test1" value="yes" checked />
    </div>
    <div>
      <label for="test2">This is unchecked</label>
      <input type="checkbox" class="checkbox" id="test2" value="yes" />
    </div>
  </div>
  <div id="guest-1">
    Tab 1 content
  </div>
  <div id="guest-2">
    Tab 2 content
  </div>
  <div id="guest-3">
    Tab 3 content
  </div>
</div>