There are actually three types of JMP instructions; short, near and far (long).
A short JMP is the relative JMP that you refer to. It is encoded as a two bytes; the actual JMP and the number of bytes +/- relative to the current IP.
A near jump allows you to jump within the current "segment" (using real mode terms) or within the currently selected memory area in the CS selector.
A long or Far JMP additionally includes a selector (or segment in real mode)
You can look up the timings for yourself. The biggest difference related to time is caused by the different numbers of bytes that must be read to accomplish the JMP.