There several ways to do so. I recommend using JQuery:
Let's suppose you have the html code of your side bar in sidebar.html file.
Then, in any of the html files you want to include, you should create a div for containing it and load the sidbar.html content in it:
<html> 
  <head> 
    <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.12.4/jquery.min.js"></script> <!-- ...or whereever you have jquery stored -->
    <script> 
    $(function(){
      $("#sidebar").load("sidebar.html"); 
    });
    </script> 
  </head> 
  <body> 
     <!-- other content here -->
     <div id="sidebar"></div>
     <!-- other content here -->
  </body> 
</html>