Your min/2 and max/2 are indeterminate, so there are alternatives to be looked for. When you evaluate
min_max(2,5,X,Y).
You see
X = 2,
Y = 5
and it pauses, waiting for input from you. If you then press ., it will succeed. If, however, you press ; it searches for the alternatives, and finding none, fails.
You need to modify your min/2 andmax/2 to be determinate, something like this:
min(X,Y,Z) :- ( X < Y -> Z = X ; Z = Y ) .
max(X,Y,Z) :- ( X > Y -> Z = X ; Z = Y ) .
[The -> operator (implication) effectively acts as a 'soft cut', eliminating alternatives.]
Once you do that, it will work as you might expect, without pausing for your input (as you've eliminated the choice points):
13 ?- min_max(2,5,X,Y).
X = 2,
Y = 5.
14 ?-
It might help if you think of the your prolog program as something of a tree with the choice points as branches (and solutions as leaf nodes). The prolog engine runs over that tree trying to find solutions.
In your original program, your min_max/4 has 4 possible solutions (2 for min/2 and 2 for max/2), even though there is just one "functional" solution. On backtracking, the prolog engine trys to find alternative solutions and finding none, fails.
By re-writing min/2 and max/2 to eliminate the choice points, we are effectively pruning the tree as we descend it, leaving us with a single solution.