To get a button with only an image showing by default, then a different image on hover, I tried having an icon set in the editor and playing around with the onSelected, onActive, etc. but naturally, it didn't work.
What did work is inspired from JosephFarrish's answer and goerge's.
For the particular push button, I have 2 images:


My solution for a specific QPushButton is:
QPushButton {
border-image: url(:/icons/ic-explore);
background-repeat: no-repeat;
width: 32px;
height: 32px;
}
QPushButton:hover {
border-image: url(:/icons/ic-explore-hover);
background-repeat: no-repeat;
}
as you can see, the ic-explore and ic-explore-hover are added to my resource file as shown below:

where the actual icons are in the root project folder, in a folder named icons. The prefix for the icons is given by :/icons/ and this coincidentally happens to be the same name as the icons folder name.
Note: with the CSS that I set the width and height of the QPushButton.