Bouncy Castle vs Java default RSA with OAEP

Martin Dow picture Martin Dow · May 11, 2018 · Viewed 9.3k times · Source

Can someone explain to me why this code throws javax.crypto.BadPaddingException: Decryption error on the final line when it's decrypting the key?

// Given an RSA key pair...
KeyPairGenerator keyGen = KeyPairGenerator.getInstance("RSA");
keyGen.initialize(2048);
KeyPair keyPair = keyGen.genKeyPair();
PrivateKey privateKey = keyPair.getPrivate();
PublicKey publicKey = keyPair.getPublic();

// ... and an AES key:
KeyGenerator keyGenerator = KeyGenerator.getInstance("AES");
keyGenerator.init(256);
SecretKey aesKey = keyGenerator.generateKey();

// When I encrypt the key with this Bouncy Castle cipher:
Cipher encryptionCipher = Cipher.getInstance("RSA/NONE/OAEPWithSHA256AndMGF1Padding", "BC");
encryptionCipher.init(Cipher.ENCRYPT_MODE, publicKey);
byte[] encryptedKey = encryptionCipher.doFinal(aesKey.getEncoded());

// Then trying to decrypt the key with this cipher...
Cipher decryptionCipher = Cipher.getInstance("RSA/ECB/OAEPWITHSHA-256ANDMGF1PADDING");
decryptionCipher.init(Cipher.DECRYPT_MODE, privateKey);
// ... throws `javax.crypto.BadPaddingException: Decryption error` here:
decryptionCipher.doFinal(encryptedKey);

Is the following statement from https://stackoverflow.com/a/27886397/66722 also true for RSA with OAEP?

"RSA/ECB/PKCS1Padding" actually doesn't implement ECB mode encryption. It should have been called "RSA/None/PKCS1Padding" as it can only be used to encrypt a single block of plaintext (or, indeed a secret key). This is just a naming mistake of Sun/Oracle.

If so, I would expect these transformations to be equivalent and my test above to pass. The same padding has been specified in both, so why the BadPaddingException?

Either way, I would appreciate a layman's explanation of what the difference is.

Answer

President James K. Polk picture President James K. Polk · May 11, 2018

For similar Stackoverflow questions with more information please see Maarten Bodewes answers to this and this.

The "mode" part of the transformation string has no effect. The problem is different defaults used by different providers. This is unfortunate and very definitely suboptimal. Should we blame Sun/Oracle? I have no opinion beyond being dissatisfied with the result.

OAEP is a fairly complicated construction with two different hash functions as parameters. The Cipher transform string lets you specify one of these, which you have specified as SHA-256. However, the MGF1 function also is parameterized by a hash function which you cannot specify in the cipher transformation string. The Oracle provider defaults to SHA1 whereas the BouncyCastle provider defaults to SHA-256. So, in effect, there is a hidden parameter that is critical for interoperability.

The solution is to specify more fully what these hidden parameters are by supplying an OAEPParameterSpec to the Cipher.init(...) method as in the following example:

Cipher encryptionCipher = Cipher.getInstance("RSA/NONE/OAEPWithSHA256AndMGF1Padding", "BC");
OAEPParameterSpec oaepParameterSpec = new OAEPParameterSpec("SHA-256", "MGF1",
                MGF1ParameterSpec.SHA256, PSource.PSpecified.DEFAULT);
encryptionCipher.init(Cipher.ENCRYPT_MODE, publicKey, oaepParameterSpec);
// ...
// ...
// ...
Cipher decryptionCipher = Cipher.getInstance("RSA/ECB/OAEPWITHSHA-256ANDMGF1PADDING");
oaepParameterSpec = new OAEPParameterSpec("SHA-256", "MGF1",
                MGF1ParameterSpec.SHA256, PSource.PSpecified.DEFAULT);
decryptionCipher.init(Cipher.DECRYPT_MODE, privateKey, oaepParameterSpec);

The first one is effectively a no-op, because those are already the defaults for Bouncycastle.