Java server self-signed certificate + client certificate and SSL - connection reset

Domchi picture Domchi · Feb 1, 2011 · Viewed 11.6k times · Source

(I've already asked the similar question and it turns out that my client key wasn't getting loaded, but I only got one exception further so I'm posting another question.)

I'm connecting to a web service which was used before successfully, however now they've changed hostname and sent me two .pem files; one is CA, and other is my new client certificate.

(I'm using Java 1.5, Spring + Spring Web Services with Apache httpclient, but I suspect my problem is with certificates, keys and SSL itself.)

I've imported both .pem files, as well as host's .crt which I exported from Firefox into my cacerts. However, I'm obviously doing something wrong since I get this exception:

org.springframework.ws.client.WebServiceIOException: I/O error: Connection reset; nested exception is java.net.SocketException: Connection reset
Caused by: 
java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(SocketInputStream.java:168)
    at com.sun.net.ssl.internal.ssl.InputRecord.readFully(InputRecord.java:284)
    at com.sun.net.ssl.internal.ssl.InputRecord.readV3Record(InputRecord.java:396)
    at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:348)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:720)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1025)
    at com.sun.net.ssl.internal.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:619)
    at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:59)
    at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:65)
    at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:123)
    at org.apache.commons.httpclient.methods.EntityEnclosingMethod.writeRequestBody(EntityEnclosingMethod.java:502)
    at org.apache.commons.httpclient.HttpMethodBase.writeRequest(HttpMethodBase.java:1973)
    at org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:993)
    at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:397)
    at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:170)
    at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:396)
    at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:324)
    at org.springframework.ws.transport.http.CommonsHttpConnection.onSendAfterWrite(CommonsHttpConnection.java:83)
    at org.springframework.ws.transport.AbstractWebServiceConnection.send(AbstractWebServiceConnection.java:42)
    at org.springframework.ws.client.core.WebServiceTemplate.sendRequest(WebServiceTemplate.java:547)
    at org.springframework.ws.client.core.WebServiceTemplate.sendAndReceive(WebServiceTemplate.java:405)
    at org.springframework.ws.client.core.WebServiceTemplate.doSendAndReceive(WebServiceTemplate.java:358)
    at org.springframework.ws.client.core.WebServiceTemplate.sendSourceAndReceiveToResult(WebServiceTemplate.java:304)
    at org.springframework.ws.client.core.WebServiceTemplate.sendSourceAndReceiveToResult(WebServiceTemplate.java:289)
    ...

When I turn on SSL logging with System.setProperty("javax.net.debug", "all"), I see that server certificate is accepted and then this happens after or somewhere during client key exchange:

setting up default SSLSocketFactory
use default SunJSSE impl class: com.sun.net.ssl.internal.ssl.SSLSocketFactoryImpl
class com.sun.net.ssl.internal.ssl.SSLSocketFactoryImpl is loaded
keyStore is : D:\AdriaticaCentral\.metadata\.plugins\org.eclipse.wst.server.core\tmp0\wtpwebapps\AdriaticaCentralOnlineServer\WEB-INF\classes\keystore
keyStore type is : jks
keyStore provider is : 
init keystore
init keymanager of type SunX509
***
found key for : ypsilonclient
chain [0] = [
[
  Version: V1
  Subject: [email protected], CN=enxi.norrisdata.net-ca, OU=Certificate Authority, O=ypsilon.net ag, L=Frankfurt, C=DE
  Signature Algorithm: SHA1withRSA, OID = 1.2.840.113549.1.1.5

  Key:  Sun RSA public key, 1024 bits
  modulus: 103786554737956184369138386227517475430156404603922533481712260490997247291004352385079204978431207687092828117962473600295977103686791448953158848873575487907656378655168840104433047747570602454550203304683174555325033654946526304210710782190667961616217273402229863778090825217190222869236148684215668636483
  public exponent: 65537
  Validity: [From: Fri Mar 26 13:14:36 CET 2010,
               To: Mon Mar 23 13:14:36 CET 2020]
  Issuer: [email protected], CN=enxi.norrisdata.net-ca, OU=Certificate Authority, O=ypsilon.net ag, L=Frankfurt, C=DE
  SerialNumber: [    94778886 f4ca92c2]

]
  Algorithm: [SHA1withRSA]
  Signature:
0000: 86 EE 6C 03 20 76 E5 0C   C7 1D E5 44 60 C0 D0 40  ..l. v.....D`..@
0010: 02 96 EE 05 39 31 E8 5A   FE F4 72 7B 9B CC E7 0F  ....91.Z..r.....
0020: 97 E6 41 7E EC E3 65 C5   A2 B0 41 61 93 B4 48 EE  ..A...e...Aa..H.
0030: DE 44 76 94 C1 48 E4 05   96 C2 0A 9B 1C 94 1B 85  .Dv..H..........
0040: 96 9F F3 00 D3 AC B7 95   C5 2C D5 ED 52 FA D7 79  .........,..R..y
0050: A1 10 BB CB A4 BD 30 08   51 71 50 EE DC 60 88 AD  ......0.QqP..`..
0060: 31 6E 88 D9 97 F3 8B 5B   01 B3 80 B2 B2 06 62 FB  1n.....[......b.
0070: DE A4 74 87 D9 2A 2B 2F   AF 31 22 97 4A F6 B8 9F  ..t..*+/.1".J...

]
***
trustStore is: D:\AdriaticaCentral\.metadata\.plugins\org.eclipse.wst.server.core\tmp0\wtpwebapps\AdriaticaCentralOnlineServer\WEB-INF\classes\cacerts
trustStore type is : jks
trustStore provider is : 
init truststore
adding as trusted cert:
  Subject: [email protected], CN=http://www.valicert.com/, OU=ValiCert Class 2 Policy Validation Authority, O="ValiCert, Inc.", L=ValiCert Validation Network
  Issuer:  [email protected], CN=http://www.valicert.com/, OU=ValiCert Class 2 Policy Validation Authority, O="ValiCert, Inc.", L=ValiCert Validation Network
  Algorithm: RSA; Serial number: 0x1
  Valid from Sat Jun 26 02:19:54 CEST 1999 until Wed Jun 26 02:19:54 CEST 2019

adding as trusted cert:
  Subject: [email protected], CN=enxi.norrisdata.net, OU=enxi.norrisdata.net, O=ypsilon.net ag, L=Frankfurt, C=DE
  Issuer:  [email protected], CN=enxi.norrisdata.net-ca, OU=Certificate Authority, O=ypsilon.net ag, L=Frankfurt, C=DE
  Algorithm: RSA; Serial number: 0x2
  Valid from Fri Mar 26 11:37:00 CET 2010 until Mon Mar 23 11:37:00 CET 2020

adding as trusted cert:
  Subject: [email protected], OU=TC TrustCenter Class 3 CA, O=TC TrustCenter for Security in Data Networks GmbH, L=Hamburg, ST=Hamburg, C=DE
  Issuer:  [email protected], OU=TC TrustCenter Class 3 CA, O=TC TrustCenter for Security in Data Networks GmbH, L=Hamburg, ST=Hamburg, C=DE
  Algorithm: RSA; Serial number: 0x3eb
  Valid from Mon Mar 09 12:59:59 CET 1998 until Sat Jan 01 12:59:59 CET 2011

adding as trusted cert:
  Subject: [email protected], CN=enxi.norrisdata.net-ca, OU=Certificate Authority, O=ypsilon.net ag, L=Frankfurt, C=DE
  Issuer:  [email protected], CN=enxi.norrisdata.net-ca, OU=Certificate Authority, O=ypsilon.net ag, L=Frankfurt, C=DE
  Algorithm: RSA; Serial number: 0x94778886f4ca92c2
  Valid from Fri Mar 26 13:14:36 CET 2010 until Mon Mar 23 13:14:36 CET 2020

[unimportant certificates snipped]

adding as trusted cert:
  Subject: OU=VeriSign Trust Network, OU="(c) 1998 VeriSign, Inc. - For authorized use only", OU=Class 1 Public Primary Certification Authority - G2, O="VeriSign, Inc.", C=US
  Issuer:  OU=VeriSign Trust Network, OU="(c) 1998 VeriSign, Inc. - For authorized use only", OU=Class 1 Public Primary Certification Authority - G2, O="VeriSign, Inc.", C=US
  Algorithm: RSA; Serial number: 0x4cc7eaaa983e71d39310f83d3a899192
  Valid from Mon May 18 02:00:00 CEST 1998 until Wed Aug 02 01:59:59 CEST 2028

init context
trigger seeding of SecureRandom
done seeding SecureRandom
instantiated an instance of class com.sun.net.ssl.internal.ssl.SSLSocketFactoryImpl
http-8080-Processor25, setSoTimeout(90000) called
http-8080-Processor25, setSoTimeout(90000) called
%% No cached client session
*** ClientHello, TLSv1
RandomCookie:  GMT: 1296423943 bytes = { 233, 32, 138, 106, 31, 235, 174, 62, 53, 252, 155, 255, 248, 43, 255, 58, 99, 70, 232, 17, 220, 98, 42, 40, 101, 157, 26, 113 }
Session ID:  {}
Cipher Suites: [SSL_RSA_WITH_RC4_128_MD5, SSL_RSA_WITH_RC4_128_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_DES_CBC_SHA, SSL_DHE_RSA_WITH_DES_CBC_SHA, SSL_DHE_DSS_WITH_DES_CBC_SHA, SSL_RSA_EXPORT_WITH_RC4_40_MD5, SSL_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA, SSL_DHE_DSS_EXPORT_WITH_DES40_CBC_SHA]
Compression Methods:  { 0 }
***
http-8080-Processor25, WRITE: TLSv1 Handshake, length = 73
http-8080-Processor25, WRITE: SSLv2 client hello message, length = 98
http-8080-Processor25, READ: TLSv1 Handshake, length = 74
*** ServerHello, TLSv1
RandomCookie:  GMT: 1296423943 bytes = { 201, 241, 99, 38, 140, 0, 132, 20, 231, 186, 165, 243, 178, 143, 146, 172, 108, 161, 126, 74, 70, 56, 138, 165, 39, 99, 254, 173 }
Session ID:  {1, 78, 15, 139, 52, 55, 227, 34, 190, 155, 208, 146, 92, 216, 197, 173, 214, 218, 238, 194, 255, 48, 34, 171, 219, 162, 231, 250, 183, 158, 235, 63}
Cipher Suite: SSL_RSA_WITH_RC4_128_MD5
Compression Method: 0
***
%% Created:  [Session-1, SSL_RSA_WITH_RC4_128_MD5]
** SSL_RSA_WITH_RC4_128_MD5
http-8080-Processor25, READ: TLSv1 Handshake, length = 1378
*** Certificate chain
chain [0] = [
[
  Version: V1
  Subject: [email protected], CN=enxi.norrisdata.net, OU=enxi.norrisdata.net, O=ypsilon.net ag, L=Frankfurt, C=DE
  Signature Algorithm: SHA1withRSA, OID = 1.2.840.113549.1.1.5

  Key:  Sun RSA public key, 1024 bits
  modulus: 105158323961649143261675059370957210288137897982882368398075567460896421730512351351129218695072925445303830065152794594929017968110838209795249871435238567060656353603426816451022832577131638028495007888967083020723809918589055189033188525472465535607293377867184162059586888049098196531889988723950292830313
  public exponent: 65537
  Validity: [From: Fri Mar 26 11:37:00 CET 2010,
               To: Mon Mar 23 11:37:00 CET 2020]
  Issuer: [email protected], CN=enxi.norrisdata.net-ca, OU=Certificate Authority, O=ypsilon.net ag, L=Frankfurt, C=DE
  SerialNumber: [    02]

]
  Algorithm: [SHA1withRSA]
  Signature:
0000: 3A F3 91 84 EA B1 CF 28   7B 52 EC 50 34 56 CB A5  :......(.R.P4V..
0010: 22 B2 3C 62 9B 8C 45 30   BE 89 C6 8C D5 CD D0 4C  ".<b..E0.......L
0020: 0A 92 3C AB C6 72 5C 7E   A4 4B 12 B5 3D 90 6F D1  ..<..r\..K..=.o.
0030: 8D 23 8F FE 46 9E D5 15   BA 8D 32 12 79 86 D8 42  .#..F.....2.y..B
0040: A9 AF 95 3A 58 D6 F0 1C   C9 44 B7 AB 78 F8 0E 16  ...:X....D..x...
0050: E5 B1 30 29 56 D5 C1 4F   06 D2 5C 9B 7F 61 22 7D  ..0)V..O..\..a".
0060: 6C EB C5 7C 02 8B D4 3B   3B 66 20 55 72 2D 1B F1  l......;;f Ur-..
0070: 3A 28 3F 10 80 BC 9F 46   DA 0E 8F DC 53 0E 0B 85  :(?....F....S...

]
chain [1] = [
[
  Version: V1
  Subject: [email protected], CN=enxi.norrisdata.net-ca, OU=Certificate Authority, O=ypsilon.net ag, L=Frankfurt, C=DE
  Signature Algorithm: SHA1withRSA, OID = 1.2.840.113549.1.1.5

  Key:  Sun RSA public key, 1024 bits
  modulus: 103786554737956184369138386227517475430156404603922533481712260490997247291004352385079204978431207687092828117962473600295977103686791448953158848873575487907656378655168840104433047747570602454550203304683174555325033654946526304210710782190667961616217273402229863778090825217190222869236148684215668636483
  public exponent: 65537
  Validity: [From: Fri Mar 26 13:14:36 CET 2010,
               To: Mon Mar 23 13:14:36 CET 2020]
  Issuer: [email protected], CN=enxi.norrisdata.net-ca, OU=Certificate Authority, O=ypsilon.net ag, L=Frankfurt, C=DE
  SerialNumber: [    94778886 f4ca92c2]

]
  Algorithm: [SHA1withRSA]
  Signature:
0000: 86 EE 6C 03 20 76 E5 0C   C7 1D E5 44 60 C0 D0 40  ..l. v.....D`..@
0010: 02 96 EE 05 39 31 E8 5A   FE F4 72 7B 9B CC E7 0F  ....91.Z..r.....
0020: 97 E6 41 7E EC E3 65 C5   A2 B0 41 61 93 B4 48 EE  ..A...e...Aa..H.
0030: DE 44 76 94 C1 48 E4 05   96 C2 0A 9B 1C 94 1B 85  .Dv..H..........
0040: 96 9F F3 00 D3 AC B7 95   C5 2C D5 ED 52 FA D7 79  .........,..R..y
0050: A1 10 BB CB A4 BD 30 08   51 71 50 EE DC 60 88 AD  ......0.QqP..`..
0060: 31 6E 88 D9 97 F3 8B 5B   01 B3 80 B2 B2 06 62 FB  1n.....[......b.
0070: DE A4 74 87 D9 2A 2B 2F   AF 31 22 97 4A F6 B8 9F  ..t..*+/.1".J...

]
***
Found trusted certificate:
[
[
  Version: V1
  Subject: [email protected], CN=enxi.norrisdata.net, OU=enxi.norrisdata.net, O=ypsilon.net ag, L=Frankfurt, C=DE
  Signature Algorithm: SHA1withRSA, OID = 1.2.840.113549.1.1.5

  Key:  Sun RSA public key, 1024 bits
  modulus: 105158323961649143261675059370957210288137897982882368398075567460896421730512351351129218695072925445303830065152794594929017968110838209795249871435238567060656353603426816451022832577131638028495007888967083020723809918589055189033188525472465535607293377867184162059586888049098196531889988723950292830313
  public exponent: 65537
  Validity: [From: Fri Mar 26 11:37:00 CET 2010,
               To: Mon Mar 23 11:37:00 CET 2020]
  Issuer: [email protected], CN=enxi.norrisdata.net-ca, OU=Certificate Authority, O=ypsilon.net ag, L=Frankfurt, C=DE
  SerialNumber: [    02]

]
  Algorithm: [SHA1withRSA]
  Signature:
0000: 3A F3 91 84 EA B1 CF 28   7B 52 EC 50 34 56 CB A5  :......(.R.P4V..
0010: 22 B2 3C 62 9B 8C 45 30   BE 89 C6 8C D5 CD D0 4C  ".<b..E0.......L
0020: 0A 92 3C AB C6 72 5C 7E   A4 4B 12 B5 3D 90 6F D1  ..<..r\..K..=.o.
0030: 8D 23 8F FE 46 9E D5 15   BA 8D 32 12 79 86 D8 42  .#..F.....2.y..B
0040: A9 AF 95 3A 58 D6 F0 1C   C9 44 B7 AB 78 F8 0E 16  ...:X....D..x...
0050: E5 B1 30 29 56 D5 C1 4F   06 D2 5C 9B 7F 61 22 7D  ..0)V..O..\..a".
0060: 6C EB C5 7C 02 8B D4 3B   3B 66 20 55 72 2D 1B F1  l......;;f Ur-..
0070: 3A 28 3F 10 80 BC 9F 46   DA 0E 8F DC 53 0E 0B 85  :(?....F....S...

]
http-8080-Processor25, READ: TLSv1 Handshake, length = 14
*** CertificateRequest
Cert Types: RSA, DSS, Type-64, 
Cert Authorities:
*** ServerHelloDone
matching alias: ypsilonclient
*** Certificate chain
chain [0] = [
[
  Version: V1
  Subject: [email protected], CN=enxi.norrisdata.net-ca, OU=Certificate Authority, O=ypsilon.net ag, L=Frankfurt, C=DE
  Signature Algorithm: SHA1withRSA, OID = 1.2.840.113549.1.1.5

  Key:  Sun RSA public key, 1024 bits
  modulus: 103786554737956184369138386227517475430156404603922533481712260490997247291004352385079204978431207687092828117962473600295977103686791448953158848873575487907656378655168840104433047747570602454550203304683174555325033654946526304210710782190667961616217273402229863778090825217190222869236148684215668636483
  public exponent: 65537
  Validity: [From: Fri Mar 26 13:14:36 CET 2010,
               To: Mon Mar 23 13:14:36 CET 2020]
  Issuer: [email protected], CN=enxi.norrisdata.net-ca, OU=Certificate Authority, O=ypsilon.net ag, L=Frankfurt, C=DE
  SerialNumber: [    94778886 f4ca92c2]

]
  Algorithm: [SHA1withRSA]
  Signature:
0000: 86 EE 6C 03 20 76 E5 0C   C7 1D E5 44 60 C0 D0 40  ..l. v.....D`..@
0010: 02 96 EE 05 39 31 E8 5A   FE F4 72 7B 9B CC E7 0F  ....91.Z..r.....
0020: 97 E6 41 7E EC E3 65 C5   A2 B0 41 61 93 B4 48 EE  ..A...e...Aa..H.
0030: DE 44 76 94 C1 48 E4 05   96 C2 0A 9B 1C 94 1B 85  .Dv..H..........
0040: 96 9F F3 00 D3 AC B7 95   C5 2C D5 ED 52 FA D7 79  .........,..R..y
0050: A1 10 BB CB A4 BD 30 08   51 71 50 EE DC 60 88 AD  ......0.QqP..`..
0060: 31 6E 88 D9 97 F3 8B 5B   01 B3 80 B2 B2 06 62 FB  1n.....[......b.
0070: DE A4 74 87 D9 2A 2B 2F   AF 31 22 97 4A F6 B8 9F  ..t..*+/.1".J...

]
***
*** ClientKeyExchange, RSA PreMasterSecret, TLSv1
Random Secret:  { 3, 1, 110, 20, 216, 88, 174, 234, 11, 164, 154, 148, 54, 171, 55, 181, 52, 238, 214, 252, 168, 169, 18, 121, 177, 216, 220, 143, 238, 36, 200, 90, 23, 216, 108, 223, 141, 204, 89, 1, 87, 183, 19, 114, 250, 78, 84, 76 }
http-8080-Processor25, WRITE: TLSv1 Handshake, length = 833
SESSION KEYGEN:
PreMaster Secret:
0000: 03 01 6E 14 D8 58 AE EA   0B A4 9A 94 36 AB 37 B5  ..n..X......6.7.
0010: 34 EE D6 FC A8 A9 12 79   B1 D8 DC 8F EE 24 C8 5A  4......y.....$.Z
0020: 17 D8 6C DF 8D CC 59 01   57 B7 13 72 FA 4E 54 4C  ..l...Y.W..r.NTL
CONNECTION KEYGEN:
Client Nonce:
0000: 4D 46 DC 07 E9 20 8A 6A   1F EB AE 3E 35 FC 9B FF  MF... .j...>5...
0010: F8 2B FF 3A 63 46 E8 11   DC 62 2A 28 65 9D 1A 71  .+.:cF...b*(e..q
Server Nonce:
0000: 4D 46 DC 07 C9 F1 63 26   8C 00 84 14 E7 BA A5 F3  MF....c&........
0010: B2 8F 92 AC 6C A1 7E 4A   46 38 8A A5 27 63 FE AD  ....l..JF8..'c..
Master Secret:
0000: DE 21 44 E2 E9 3B E8 1E   EE 64 D3 44 B2 41 D6 F8  .!D..;...d.D.A..
0010: 06 67 95 7B 4C 8C D3 DB   AC C4 85 1E 35 67 30 1A  .g..L.......5g0.
0020: 36 F2 15 EE 5E 1D 3F 67   35 74 4F 0B 0B EE 02 92  6...^.?g5tO.....
Client MAC write Secret:
0000: 9E AF AB 0F D1 71 21 ED   0B B5 BB 65 12 F2 F9 0A  .....q!....e....
Server MAC write Secret:
0000: BD 17 61 C4 3F FE 61 8D   85 EF 5A E9 2D 8E 06 CD  ..a.?.a...Z.-...
Client write key:
0000: C0 0D 6C 01 63 74 1D E6   53 04 92 BC 6D 12 A6 8F  ..l.ct..S...m...
Server write key:
0000: 32 B4 99 5C 37 A2 83 67   78 09 95 55 C8 63 72 6F  2..\7..gx..U.cro
... no IV for cipher
*** CertificateVerify
http-8080-Processor25, WRITE: TLSv1 Handshake, length = 134
http-8080-Processor25, WRITE: TLSv1 Change Cipher Spec, length = 1
*** Finished
verify_data:  { 47, 74, 83, 184, 225, 220, 176, 197, 212, 45, 72, 182 }
***
http-8080-Processor25, WRITE: TLSv1 Handshake, length = 32
http-8080-Processor25, handling exception: java.net.SocketException: Connection reset
http-8080-Processor25, SEND TLSv1 ALERT:  fatal, description = unexpected_message
http-8080-Processor25, WRITE: TLSv1 Alert, length = 18
http-8080-Processor25, Exception sending alert: java.net.SocketException: Connection reset by peer: socket write error
http-8080-Processor25, called closeSocket()
http-8080-Processor25, called close()
http-8080-Processor25, called closeInternal(true)
http-8080-Processor25, called close()
http-8080-Processor25, called closeInternal(true)
http-8080-Processor25, called close()
http-8080-Processor25, called closeInternal(true)

Why does my connection keep resetting and how can I troubleshoot this?

Answer

Domchi picture Domchi · Feb 9, 2011

Problem solved.

I did this:

openssl pkcs8 -topk8 -nocrypt -outform der -in clientkey.pem -out clientkey.der

But I didn't do this:

openssl x509 -outform der -in clientkey.pem -out clientkey.cer

Both files need to be imported into keystore through Java, not keytool. I was importing only the clientkey.der.

Turns out you have to separately import client key and server certificate in keystore; I wasn't aware that converting .pem to .der didn't export attached server certificate as well.