Java HTTPS client certificate authentication

Question

I'm fairly new to HTTPS/SSL/TLS and I'm a bit confused over what exactly the clients are supposed to present when authenticating with certificates.

I'm writing a Java client that needs to do a simple POST of data to a particular URL. That part works fine, the only problem is it's supposed to be done over HTTPS. The HTTPS part is fairly easy to handle (either with HTTPclient or using Java's built-in HTTPS support), but I'm stuck on authenticating with client certificates. I've noticed there's already a very similar question on here, which I haven't tried out with my code yet (will do so soon enough). My current issue is that - whatever I do - the Java client never sends along the certificate (I can check this with PCAP dumps).

I would like to know what exactly the client is supposed to present to the server when authenticating with certificates (specifically for Java - if that matters at all)? Is this a JKS file, or PKCS#12? What's supposed to be in them; just the client certificate, or a key? If so, which key? There's quite a bit of confusion about all the different kinds of files, certificate types and such.

As I've said before I'm new to HTTPS/SSL/TLS so I would appreciate some background information as well (doesn't have to be an essay; I'll settle for links to good articles).

Answer 1

Finally managed to solve all the issues, so I'll answer my own question. These are the settings/files I've used to manage to get my particular problem(s) solved;

The client's keystore is a PKCS#12 format file containing

1. The client's public certificate (in this instance signed by a self-signed CA)
2. The client's private key

To generate it I used OpenSSL's pkcs12 command, for example;

openssl pkcs12 -export -in client.crt -inkey client.key -out client.p12 -name "Whatever"

Tip: make sure you get the latest OpenSSL, not version 0.9.8h because that seems to suffer from a bug which doesn't allow you to properly generate PKCS#12 files.

This PKCS#12 file will be used by the Java client to present the client certificate to the server when the server has explicitly requested the client to authenticate. See the Wikipedia article on TLS for an overview of how the protocol for client certificate authentication actually works (also explains why we need the client's private key here).

The client's truststore is a straight forward JKS format file containing the root or intermediate CA certificates. These CA certificates will determine which endpoints you will be allowed to communicate with, in this case it will allow your client to connect to whichever server presents a certificate which was signed by one of the truststore's CA's.

To generate it you can use the standard Java keytool, for example;

keytool -genkey -dname "cn=CLIENT" -alias truststorekey -keyalg RSA -keystore ./client-truststore.jks -keypass whatever -storepass whatever
keytool -import -keystore ./client-truststore.jks -file myca.crt -alias myca

Using this truststore, your client will try to do a complete SSL handshake with all servers who present a certificate signed by the CA identified by myca.crt.

The files above are strictly for the client only. When you want to set-up a server as well, the server needs its own key- and truststore files. A great walk-through for setting up a fully working example for both a Java client and server (using Tomcat) can be found on this website.

Issues/Remarks/Tips

1. Client certificate authentication can only be enforced by the server.
2. (Important!) When the server requests a client certificate (as part of the TLS handshake), it will also provide a list of trusted CA's as part of the certificate request. When the client certificate you wish to present for authentication is not signed by one of these CA's, it won't be presented at all (in my opinion, this is weird behaviour, but I'm sure there's a reason for it). This was the main cause of my issues, as the other party had not configured their server properly to accept my self-signed client certificate and we assumed that the problem was at my end for not properly providing the client certificate in the request.
3. Get Wireshark. It has great SSL/HTTPS packet analysis and will be a tremendous help debugging and finding the problem. It's similar to -Djavax.net.debug=ssl but is more structured and (arguably) easier to interpret if you're uncomfortable with the Java SSL debug output.

4. It's perfectly possible to use the Apache httpclient library. If you want to use httpclient, just replace the destination URL with the HTTPS equivalent and add the following JVM arguments (which are the same for any other client, regardless of the library you want to use to send/receive data over HTTP/HTTPS):

   -Djavax.net.debug=ssl
   -Djavax.net.ssl.keyStoreType=pkcs12
   -Djavax.net.ssl.keyStore=client.p12
   -Djavax.net.ssl.keyStorePassword=whatever
   -Djavax.net.ssl.trustStoreType=jks
   -Djavax.net.ssl.trustStore=client-truststore.jks
   -Djavax.net.ssl.trustStorePassword=whatever