First of all, you can't reference to a second row using a rownum = 2 condition. You can either select first two rows by specifying a where rownum < 3 condition, or you may wrap it in another query and reference your rownum as an ordinary column from over there.
Then, you can't reference a column alias in a where clause of a subquery this alias was assigned.
You can either bring it one level up:
SELECT instmax
FROM (SELECT instmax, rownum r
FROM (SELECT instmax
FROM pswlinstmax
ORDER BY instmax DESC NULLS LAST)
)
WHERE r = 2;
or just avoid this reference
-- this will return first two rows
SELECT instmax
FROM (SELECT instmax, rownum r
FROM (SELECT instmax
FROM pswlinstmax
ORDER BY instmax DESC NULLS LAST)
WHERE rownum < 3
);