There's several issues here. Firstly, you're missing quotes around the value you're setting. Secondly you're using attr('value') instead of val().
However the bigger issue is that val() will not decode the entity you're setting. In order to achieve that you will need to use a <button> element then set its html(), like this:
$("#degree-toggle").html('℉');
<script src="https://ajax.googleapis.com/ajax/libs/jquery/2.1.1/jquery.min.js"></script>
<button type="button" id="degree-toggle"></button>
 
 
Also note that neither <button> nor <input type="button"> elements have a checked attribute.