I can confirm I experienced the same issue in Linux 19.3 XFCE edition 5 years after the original question.
The solution was to completely remove the ~/.config/xfce4 directory.
Apparently this folder holds all the session customizations. When you ran 'startx', it completely overwrote mint's default settings with base-xfce's default settings.
Once you delete this directory and reboot, linux mint reverts to its 'out of the box' look. It also automatically generates a fresh ~/.config/xfce4 directory to take its place, with mint's default 'factory' settings. The xfce-default login screen may persist after the first reboot, but rebooting a second time takes care of that as well.
The only downside to this solution, of course, is that any customizations you may have made are also gone now too. E.g., my panel has moved from the top back to the bottom again, as is the factory setting. Nothing tragic, but still a minor annoyance if you'd tweaked your desktop heavily compared to the original.
If this is a deal breaker for you, and you're feeling adventurous, you might try and make a backup of the ~/.config/xfce4 directory before deleting it, and try comparing your backed up configurations with the newly generated ones (e.g. using xxdiff), and try to figure out which files are safe to copy over and which are not.
Hope this helps some future poor shmuck facing the same problem :)
PS. Obviously, I suspect most people who bump into this question will do so after having ran 'startx' already. However, if you haven't yet and are considering doing so to get an emergency graphical interface up, the easiest thing to do is to back up your ~/.config/xfce4 directory first, so that you can safely restore it later.
This is the kind of session appearence i had and want to have now:
