The ./configure trick will work for some applications. Many applications, however, encode the prefix into themselves and then use it at run time to find supporting files. They also need the ability to load their shared library files (although the LD_LIBRARY_PATH environment variable can be set to help with that). The result of this, however, is that the portable applications will likely not be very portable -- they will depend upon being at the specified prefix. This will probably work in many places (most modern distros mount media at /media/DISKNAME), but it does prevent the application from being truly portable and location-independent.
The net answer, therefore, is that making portable Linux applications, while not impossible, is rife with subtleties. I suspect that this is a major contributing factor to why it isn't done near as much as it is on Windows. Additionally, public computing facilities (where the bulk of the portable application benefit arises) typically do not run Linux. A notable exception is university labs, but you can usually install some extra software in your home directory there.