The Set interface is a collection that is not considered to be ordered. Although that's not explicit at the class level, it's all over the methods of the class, e.g. iterator():
Returns an iterator over the elements in this set. The elements are returned in no particular order (unless this set is an instance of some class that provides a guarantee).
In order to have a first and last, you need a order or an index, and Set has neither. As many other people have pointed out already, you can use SortedSet if you want an order.
TreeSet implements SortedSet, so you can certainly modify your supplied code to:
SortedSet<Integer> treeSet = new TreeSet<>();
treeSet.last();