PBKDF2 with bouncycastle in Java

Question

I'm trying to securely store a password in a database and for that I chose to store its hash generated using the PBKDF2 function. I want to do this using the bouncy castle library but I don't know why I cannot get it to work by using the JCE interface... The problem is that generating the hash in 3 different modes:
1. using the PBKDF2WithHmacSHA1 secret key factory provided by sun
2. using the bouncy castle api directly
3. using the bouncy castle through JCE
results in 2 distinct values: one common to the first two and one for the third.

Here is my code:

    //Mode 1

    SecretKeyFactory factory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA1");
    KeySpec keyspec = new PBEKeySpec("password".toCharArray(), salt, 1000, 128);
    Key key = factory.generateSecret(keyspec);
    System.out.println(key.getClass().getName());
    System.out.println(Arrays.toString(key.getEncoded()));

    //Mode 2

    PBEParametersGenerator generator = new PKCS5S2ParametersGenerator();
    generator.init(PBEParametersGenerator.PKCS5PasswordToUTF8Bytes(("password").toCharArray()), salt, 1000);
    KeyParameter params = (KeyParameter)generator.generateDerivedParameters(128);
    System.out.println(Arrays.toString(params.getKey()));

    //Mode 3

    SecretKeyFactory factorybc = SecretKeyFactory.getInstance("PBEWITHHMACSHA1", "BC");
    KeySpec keyspecbc = new PBEKeySpec("password".toCharArray(), salt, 1000, 128);
    Key keybc = factorybc.generateSecret(keyspecbc);
    System.out.println(keybc.getClass().getName());
    System.out.println(Arrays.toString(keybc.getEncoded()));
    System.out.println(keybc.getAlgorithm());

I know that PBKDF2 is implemented using HMAC SHA1 so that is why i chose as algorithm in the last method the "PBEWITHHMACSHA1" which i took from the bouncy castle java docs.

The output is the following:

com.sun.crypto.provider.SunJCE_ae
[-53, 29, 113, -110, -25, 76, 115, -127, -64, 74, -63, 102, 75, 81, -21, 74]
[-53, 29, 113, -110, -25, 76, 115, -127, -64, 74, -63, 102, 75, 81, -21, 74]
org.bouncycastle.jce.provider.JCEPBEKey
[14, -47, -87, -16, -117, -31, 91, -121, 90, -68, -82, -31, -27, 5, -93, -67, 30, -34, -64, -40]
PBEwithHmacSHA

Any ideas?

Answer 1

In short, the reason for the difference is that PBKDF2 algorithm in modes #1 and #2 uses PKCS #5 v2 scheme 2 (PKCS5S2) for iterative key generation, but the BouncyCastle provider for "PBEWITHHMACSHA1" in mode #3 uses the PKCS #12 v1 (PKCS12) algorithm instead. These are completely different key-generation algorithms, so you get different results.

More detail on why this is so and why you get different sized results is explained below.

First, when you're constructing a JCE KeySpec, the keyLength parameter only expresses "a preference" to the provider what key size you want. From the API docs:

Note: this is used to indicate the preference on key length for variable-key-size ciphers. The actual key size depends on each provider's implementation.

The Bouncy Castle providers don't appear to respect this parameter, judging from the source of JCEPBEKey, so you should expect to get a 160-bit key back from any BC provider which uses SHA-1 when using the JCE API.

You can confirm this by programmatically accessing the getKeySize() method on the returned keybc variable in your test code:

Key keybc = factorybc.generateSecret(keyspecbc);
// ...
Method getKeySize = JCEPBEKey.class.getDeclaredMethod("getKeySize");
getKeySize.setAccessible(true);
System.out.println(getKeySize.invoke(keybc)); // prints '160'

Now, to understand what the "PBEWITHHMACSHA1" provider corresponds to, you can find the following in the source for BouncyCastleProvider:

put("SecretKeyFactory.PBEWITHHMACSHA1", 
    "org.bouncycastle.jce.provider.JCESecretKeyFactory$PBEWithSHA");

And the implementation of JCESecretKeyFactory.PBEWithSHA looks like this:

public static class PBEWithSHA
    extends PBEKeyFactory
{
    public PBEWithSHA()
    {
        super("PBEwithHmacSHA", null, false, PKCS12, SHA1, 160, 0);
    }
}

You can see above that this key factory uses the PKCS #12 v1 (PKCS12) algorithm for iterative key generation. But the PBKDF2 algorithm that you want to use for password hashing uses PKCS #5 v2 scheme 2 (PKCS5S2) instead. This is why you're getting different results.

I had a quick look through the JCE providers registered in BouncyCastleProvider, but couldn't see any key generation algorithms that used PKCS5S2 at all, let alone one which also uses it with HMAC-SHA-1.

So I guess you're stuck with either using the Sun implementation (mode #1 above) and losing portability on other JVMs, or using the Bouncy Castle classes directly (mode #2 above) and requiring the BC library at runtime.

Either way, you should probably switch to 160-bit keys, so you aren't truncating the generated SHA-1 hash unnecessarily.

Answer 2

I found a BC Crypto-Only method (actually from the cms package of BC) which works to produce a UTF-8 based password encoding. This way I can generate KDF output which is compatible to

http://packages.python.org/passlib/lib/passlib.hash.cta_pbkdf2_sha1.html#passlib.hash.cta_pbkdf2_sha1

private byte[] calculatePasswordDigest(char[] pass, byte[] salt, int iterations)
    throws PasswordProtectionException
{
    try
    {
        /* JCE Version (does not work as BC uses PKCS12 encoding)
        SecretKeyFactory kf = SecretKeyFactory.getInstance("PBEWITHHMACSHA1","BC");
        PBEKeySpec ks = new PBEKeySpec(pass, salt, iterations,160);
        SecretKey digest = kf.generateSecret(ks);
        return digest.getEncoded();
        */
        PKCS5S2ParametersGenerator gen = new PKCS5S2ParametersGenerator();
        gen.init(PBEParametersGenerator.PKCS5PasswordToUTF8Bytes(pass), salt, iterations);
        byte[] derivedKey = ((KeyParameter)gen.generateDerivedParameters(160)).getKey();
        return derivedKey;
    }
    catch(Exception e)
    {
        LOG.error("Failed to strengthen the password with PBKDF2.",e);
        throw new PasswordProtectionException();
    }
}

Answer 3

PBKDF2WithHmacSHA1 is already supported in BouncyCastle 1.60

https://www.bouncycastle.org/specifications.html Password Hashing and PBE

Test passed with OpenJDK Runtime Environment 18.9 (build 11.0.1+13):

    Security.addProvider(new BouncyCastleProvider());

    String password = "xrS7AJk+V6L8J?B%";
    SecureRandom rnd = new SecureRandom();
    int saltLength = 16;
    int keyLength = 128;
    int iterationCount = 10000;

    byte[] salt = new byte[saltLength];
    rnd.nextBytes(salt);

//SunJCE
    SecretKeyFactory factorySun = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA1", "SunJCE");
    KeySpec keyspecSun = new PBEKeySpec(password.toCharArray(), salt, iterationCount, keyLength);
    SecretKey keySun = factorySun.generateSecret(keyspecSun);
    System.out.println(keySun.getClass().getName());
    System.out.println(Hex.toHexString(keySun.getEncoded()));

//BouncyCastle  
    SecretKeyFactory factoryBC = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA1", "BC");
    KeySpec keyspecBC = new PBEKeySpec(password.toCharArray(), salt, iterationCount, keyLength);
    SecretKey keyBC = factoryBC.generateSecret(keyspecBC);
    System.out.println(keyBC.getClass().getName());
    System.out.println(Hex.toHexString(keyBC.getEncoded()));

    Assert.assertArrayEquals(keySun.getEncoded(), keyBC.getEncoded());

The output is:

com.sun.crypto.provider.PBKDF2KeyImpl
e9b01389fa91a6172ed6e95e1e1a2611
org.bouncycastle.jcajce.provider.symmetric.util.BCPBEKey
e9b01389fa91a6172ed6e95e1e1a2611