jQuery's .each() takes a callback function and applies it to each element in the jQuery object.
Imagine something like this:
$('a.ui-icon-cart').click(function(){
  $(this).closest('li').clone().appendTo('#cart ul').each(function() {
    $(this).find('h5').remove(); 
    $(this).find('img').css({'height':'40px', 'width':'40px'});
    $(this).find('li').css({'height':'60px', 'width':'40px'});
  });
});
You could also just store the result and work on it instead:
$('a.ui-icon-cart').click(function(){
  var $new = $(this).closest('li').clone().appendTo('#cart ul')
  $new.find('h5').remove(); 
  $new.find('img').css({'height':'40px', 'width':'40px'});
  $new.find('li').css({'height':'60px', 'width':'40px'});
});
I would also suggest that instead of mofiying the CSS like that you just add a class to your cloned li like this:
$(this).closest('li').clone().addClass("new-item").appendTo('#cart ul');
Then setup some styles like:
.new-item img, .new-item li { height: 40px; width: 40px; }
.new-item h5 { display: none }