A quick google search returned me this from Appache hadoop project. Copying from there:
(Apache License, Version 2.0):
private static DecimalFormat oneDecimal = new DecimalFormat("0.0");
  /**
   * Given an integer, return a string that is in an approximate, but human 
   * readable format. 
   * It uses the bases 'k', 'm', and 'g' for 1024, 1024**2, and 1024**3.
   * @param number the number to format
   * @return a human readable form of the integer
   */
  public static String humanReadableInt(long number) {
    long absNumber = Math.abs(number);
    double result = number;
    String suffix = "";
    if (absNumber < 1024) {
      // nothing
    } else if (absNumber < 1024 * 1024) {
      result = number / 1024.0;
      suffix = "k";
    } else if (absNumber < 1024 * 1024 * 1024) {
      result = number / (1024.0 * 1024);
      suffix = "m";
    } else {
      result = number / (1024.0 * 1024 * 1024);
      suffix = "g";
    }
    return oneDecimal.format(result) + suffix;
  }
It uses 1K = 1024, but you can adapt this if you prefer. You also need to handle the <1024 case with a different DecimalFormat.