You probobly don't want to mail on any exception.  There are lots of code in the JDK that actaully depend on exceptions to work normally.  What I presume you are more inerested in are uncaught exceptions.  If you are catching the exceptions you should handle notifications there.
In a desktop app there are two places to worry about this, in the event-dispatch-thread (EDT) and outside of the EDT.  Globaly you can register a class implementing java.util.Thread.UncaughtExceptionHandler and register it via java.util.Thread.setDefaultUncaughtExceptionHandler.  This will get called if an exception winds down to the bottom of the stack and the thread hasn't had a handler set on the current thread instance on the thread or the ThreadGroup.
The EDT has a different hook for handling exceptions.  A system property 'sun.awt.exception.handler' needs to be registerd with the Fully Qualified Class Name of a class with a zero argument constructor.  This class needs an instance method handle(Throwable) that does your work.  The return type doesn't matter, and since a new instance is created every time, don't count on keeping state.
So if you don't care what thread the exception occurred in a sample may look like this:
class ExceptionHandler implements Thread.UncaughtExceptionHandler {
  public void uncaughtException(Thread t, Throwable e) {
    handle(e);
  }
  public void handle(Throwable throwable) {
    try {
      // insert your e-mail code here
    } catch (Throwable t) {
      // don't let the exception get thrown out, will cause infinite looping!
    }
  }
  public static void registerExceptionHandler() {
    Thread.setDefaultUncaughtExceptionHandler(new ExceptionHandler());
    System.setProperty("sun.awt.exception.handler", ExceptionHandler.class.getName());
  }
}
Add this class into some random package, and then call the registerExceptionHandler method and you should be ready to go.