This is because class instances and functions are reference types. Ints, structs, and everything else are value types. When you pass a reference type into a function as a parameter, you are already going to be referencing that instance. When you pass a value type as a parameter, the function gets a copy of that variable (by default), so inout is usually (see edit) only needed if you want to alter a value type from inside of a function.
Altering a class instance without & or inout:

More details
When you create a reference type var t = myClass(), you're really creating a variable t that is a pointer to a myClass instance in memory. By using an ampersand &t in front of a reference type, you are really saying "give me the pointer to the pointer of a myClass instance"
More info on reference vs value types: https://stackoverflow.com/a/27366050/580487
EDIT
As was pointed out in the comments, you can still use inout with reference types if you want to alter a pointer, etc, but I was trying to shed light on the general use case.
Below is an example of sorting an array inside of a function:
