You could assume they are in increasing ordinal() order. I would add this a method on Size.
protected enum Size {
    XS, S, M, L, XL;
    static final Size[] VALUES = values();
    public Size incrementSize() { return VALUES[ordinal()+1]; }
    public Size decrementSize() { return VALUES[ordinal()-1]; }
}
Note: I wouldn't assume that XS is after XL, but rather you get an error (though not a very clear one)
Note: every time you call values() it creates a new array. It has to do this because the array is mutable and you might change it.  I highly recommend saving a copy and avoid calling values() each time.
You can make the error messages clearer by overriding those methods.
protected enum Size {
    XS {
        public Size decrementSize() { throw new UnsupportedOperationException("No smaller size"); }
    }, 
    S, 
    M, 
    L, 
    XL {
        public Size incrementSize() { throw new UnsupportedOperationException("No larger size"); }
    };
    static final Size[] VALUES = values();
    public Size incrementSize() { return VALUES[ordinal()+1]; }
    public Size decrementSize() { return VALUES[ordinal()-1]; }
}