Don't declare it abstract; declare a private constructor, so no one, not even a subclass, can instantiate an instance of your utility class.
You can declare your class final, although if all constructors are private, then no one will be able to subclass it anyway.
To borrow the idea from Pshemo's comment in another answer, throw a RuntimeException in the constructor to prevent reflection's setAccessible method in AccessibleObject from allowing instantiation:
public class MyUtility
{
   private MyUtility()
   {
      throw new RuntimeException("Instantiation of MyUtility is not allowed!");
   }
   public static void utilityMethod()
   {
      // Your utility method here.
   }
}