I found the answer to this question, but that only works for EXT* filesystems, there's an answer also for NFS (even though the question tagged ext4). However I'm using XFS so neither of the answers works for me.
There was a comment on the accepted answer that it only works for EXT* and it was suggested that one should ask a new question instead of commenting an old. So here's the new question: how do I find out the last time a XFS file system was mounted?
I'm using Ubuntu, but I think it shouldn't differ between distributions.